Discrete Probability Spaces
3
26
Discrete Probability Spaces
In this chapter we introduce the term discrete probability
space. Discrete probability spaces are characterized by a nite
or countably innite, i.e. discrete basic space. The main
concept is the probability measure for which σ additivity
is demanded. With help of the probability function a probability measure can uniquely be dened on a discrete basic
space.
With the motivations given in Chapter 3 as a background we present the denition of a discrete probability space.
3.1 Denition
The pair (Ω, P(Ω), P ) is called a discrete probability space, if
Ω is a non-empty, nite or countably innite (i.e.
a discrete) set, and
P : P(Ω) → R is a mapping from the power set
P(Ω) of Ω to the real numbers with the following
properties:
(3.1.1)
P (A) ≥ 0
(A ⊂ Ω)
(Non-negativity)
Discrete Probability Spaces
(3.1.2)
(3.1.3)
P (Ω) = 1
tion)
27
(Normaliza-
for every sequence (An ) of pairwise disjoint sets from P(Ω),
X X
P
An =
P (An )
(σ additivity)
n∈N
n∈N
(read: sigmaadditivity)
Ω is called the basic space or the sample space,
P(Ω) the system of events, P a (discrete) probability measure, where, if we want to be more precise,
the basic space will be mentioned too, i.e.: (discrete)
probability measure over Ω.
In the following, Ω will be either Z or such suitable
subsets of Z as N, Nn , N0 or N0n , or cartesian products
of these sets, although parts of the theory can be formulated for abstract discrete basic spaces.
3.2 Remarks
3.2.1 The subsets of the basic space Ω are called
events. P (A) is understood as the probability of the event A or in the language of
the random experiment which serves as an intuitive background as the probability that a
Discrete Probability Spaces
28
randomly selected element ω ∈ Ω belongs to
the set A, i.e. ω ∈ A.
3.2.2 Note that P is not a mapping of the set Ω
but of the set P(Ω) to R; i.e. it is the subsets
(not the points) of Ω to which the real numbers are assigned. Therefore P is called a set
function.
3.2.3 If for A ∈ P(Ω) either
P (A) = 0 or P (A) = 1 ,
A is called a (P-)null set or a (P -)one set,
respectively.
∅ is always a null set, and Ω is a one set.
At the moment we are content with a rst example of
a probability measure, the point probability measure.
This provides typical examples for non-trivial null and
one sets.
3.3 Point probability measure
Let ω ∈ Ω. Then δω : P(Ω) → [0; 1] with
(
1, if ω ∈ A
δω (A) :=
(A ⊂ Ω)
/A
0, if ω ∈
Discrete Probability Spaces
29
is called the point probability measure in ω ; this
is also referred to as the Diracmeasure in ω .
Evidently the whole mass is concentrated in ω . If ω ∈
A for A ⊂ Ω, then δω (A) has the value 1, i.e. A is a one
/ A
set (under this point probability measure). If ω ∈
then δω (A) becomes 0 and A is a null set (under this
point probability measure).
Convince yourself that δω is a probability measure; especially consider the σ additivity.
3.4 Corollary
3.4.1
P (∅) = 0,
3.4.2
P (A + B) = P (A) + P (B) (A, B ⊂
Ω , A ∩ B = ∅),
(Additivity of P )
3.4.3
P (A ) = 1 − P (A)
(A ⊂ Ω),
3.4.4
P (B \ A) = P (B) − P (A)
(A, B ⊂
Ω , A ⊂ B),
3.4.5
A ⊂ B =⇒ P (A) ≤ P (B)
(A, B ⊂
Ω)
3.4.6
P (A) ≤ 1
(A ⊂ Ω).
c
Proof:
Discrete Probability Spaces
3.4.1
30
Because of (3.1.2) and (3.1.3) we have
1 = P (Ω) = P (Ω+∅+ . . . ) = P (Ω)+∞·P (∅) ,
3.4.2
which implies ∞ · P (∅) = 0; together with
(1.5.3) we have P (∅) = 0.
By (3.1.3), 3.4.1, and (1.5.3),
P (A+B) = P (A+B+∅+ . . . ) = P (A)+P (B)+0+ . . . = P (A
3.4.3
(Evidently in the case of a nite number of
non-empty sets σ additivity of P implies the
additivity of P .)
By (3.1.2) and 3.4.2 we have
1 = P (Ω) = P (A + Ac ) = P (A) + P (Ac ) .
3.4.4
For A ⊂ B we have B = (B \ A) + A and
therefore, by 3.4.2
P (B) = P (B \ A) + P (A) .
3.4.5
3.4.6
This is a consequence of 3.4.4 together with
(3.1.1).
This result is a special case of 3.4.5 with B :=
Ω.
Discrete Probability Spaces
31
3.5 Remark
The probability measure has been introduced as a mapping P : P(Ω) → R, i.e. it assigns a real number to
every subset of Ω.
But the probability measure P is already
determined
if the probability values P {ω} are known for
all ω ∈ Ω. From the σ additivity of P and the countability of Ω it follows that for all A ⊂ Ω
X
X
(3.5.1) P (A) = P
{ω} =
P {ω} .
ω∈A
ω∈A
(Note that the sum in the right hand term of (3.5.1)
can consist of either a nite or a countably innite
sum; also see the proof of 3.4.2.)
3.6 Denition
Let Ω be a non-empty, discrete set.
3.6.1 A function w : Ω → [0; 1] isPcalled probability function (on Ω), if
ω∈Ω w(ω) = 1.
In the case that Ω is countable, summa-
tion means to calculate the value of the
respective non-negative series.
Discrete Probability Spaces
32
3.6.2 Let P be a probability measure over Ω. Then
the mapping w : Ω → [0; 1] dened by
w(ω) := P {ω}
(ω ∈ Ω)
is called the probability function of P .
3.7 Theorem
A probability function w : Ω → [0; 1] uniquely denes
a probability measure P over Ω, and
X
(3.7.1)
P (A) =
w(ω)
(A ⊂ Ω) .
ω∈A
Proof:
We can restrict the proof to establishing the σ additivity
of P , particularly since the other two properties of a
probability measure, the non-negativity and the normalization, are fullled already.
Let (An ) be a sequence of pairwise disjoint sets in Ω.
Then the permutation theorem for series with nonnegative terms implies that
X X
X
XX
P (An ) .
P
An =
w(ω) =
w(ω) =
n∈N
ω∈
P
An
n∈N ω∈An
n∈N
Discrete Probability Spaces
33
In particular, from (3.7.1) and the permutation theorem for series with nonnegative terms it follows that
P (A) is uniquely determined for all A ⊂ Ω, i.e. it does
not depend on the order of summation.
3.8 Discrete probability measures over uncountable basic spaces
Occasionally it is convenient to consider a discrete
probability measure over an uncountable basic
space, e.g. over R. This leads, for instance in con-
nection with probability functions, to the problem of
a summation with uncountably many terms.
If P is a discrete probability measure over an uncountable basic space Ω0 , this simply means that there exists
a countable set Ω, Ω ⊂ Ω0 , as an basic space over which
P is dened.
Within a summation process we take into account
even without explicit mention only those terms
that are indicated by elements of Ω; terms that
are indicated by elements of Ω0 \ Ω are omitted. Of
course, the set Ω can be replaced by any convenient
countable superset D ⊂ Ω0 of Ω.