4 Discrete Probability Spaces
In this chapter we introduce the term discrete probability space. Discrete probability spaces are characterized by a finite or countably infinite, i.e. discrete basic space. The main concept is the probability measure for which σ-additivity
is demanded. With help of the probability function a probability measure can
uniquely be defined on a discrete basic space.
With the motivations given in Chapter 3 as a background we present the
definition of a discrete probability space.
4.1 Definition
The pair (Ω, P ) is called a discrete probability space, if
Ω is a non-empty, finite or countably infinite (i.e. a discrete) set, and
P : P(Ω) → R is a mapping from the power set P(Ω) of Ω to the
real numbers with the following properties:
(4.1.1)
(4.1.2)
(4.1.3)
P (A) ≥ 0
(A ⊂ Ω)
P (Ω) = 1
for every sequence (An )
X
P
An =
n∈N
(Non-negativity)
(Normalization)
of pairwise disjoint sets from P(Ω),
X
P (An )
(σ-additivity)
n∈N
(read: sigma-additivity)
Ω is called the basic space or the sample space, P a (discrete) probability measure, where, if we want to be more precise, the basic space will
be mentioned too, i.e.: (discrete) probability measure over Ω. In the
following, Ω will be either Z or such suitable subsets of Z as N, Nn , N0 or
N0n , or cartesian products of these sets, although parts of the theory can be
formulated for abstract discrete basic spaces.
4.2 Remarks
4.2.1
4.2.2
In the following, the adjective “discrete” will be omitted if it
is not required for emphasis or differentiation.
In the sequel the set Ω is called the basic space only.
20
4.2.3
4.2.4
4.2.5
4 Discrete Probability Spaces
The subsets of the basic space Ω are called events. P (A) is
understood as the probability of the event A or – in the language
of the random experiment which serves as an intuitive background –
as the probability that a randomly selected element ω ∈ Ω belongs
to the set A, i.e. ω ∈ A.
Note that P is not a mapping of the set Ω but of the set P(Ω) to R;
i.e. it is the subsets (not the points) of Ω to which the real numbers
are assigned. Therefore P is called a set function.
If for A ∈ P(Ω) either
P (A) = 0 or P (A) = 1,
A is called a (P-)null set or a (P -)one set, respectively.
∅ is always a null set, and Ω is a one set.
At the moment we are content with a first example of a probability measure, the point probability measure. This provides typical examples for nontrivial null and one sets.
4.3 Example (Point probability measure)
Let ω ∈ Ω. Then δω : P(Ω) → [0; 1] with
(
1, if ω ∈ A,
δω (A) :=
(A ⊂ Ω)
0, if ω ∈
/A
is called the point probability measure in ω; this is also referred to as
the Dirac-measure in ω.
Evidently the whole mass is concentrated in ω. If ω ∈ A for A ⊂ Ω,
then δω (A) has the value 1, i.e. A is a one set (under this point probability
measure). If ω ∈
/ A then δω (A) becomes 0 and A is a null set (under this
point probability measure).
Convince yourself that δω is a probability measure; especially consider the
σ-additivity.
4.4 Corollary
4.4.1
P (∅) = 0,
4.4.2
P (A + B) = P (A) + P (B)
(Additivity of P )
4.4.3
P (Ac ) = 1 − P (A)
4.4.4
P (B \ A) = P (B) − P (A)
(A, B ⊂ Ω, A ∩ B = ∅),
(A ⊂ Ω),
(A, B ⊂ Ω, A ⊂ B),
4.6 Definition
4.4.5
A ⊂ B =⇒ P (A) ≤ P (B)
4.4.6
P (A) ≤ 1
21
(A, B ⊂ Ω)
(A ⊂ Ω).
Proof:
4.4.1:
Because of (4.1.2) and (4.1.3) we have
1 = P (Ω) = P (Ω + ∅ + · · · ) = P (Ω) + ∞ · P (∅),
4.4.2:
which implies ∞ · P (∅) = 0; together with (1.5.3) we have P (∅) = 0.
By (4.1.3), 4.4.1, and (1.5.3),
P (A+B) = P (A+B+∅+· · · ) = P (A)+P (B)+0+· · · = P (A)+P (B).
4.4.3:
(Evidently in the case of a finite number of non-empty sets σadditivity of P implies the additivity of P .)
By (4.1.2) and 4.4.2 we have
1 = P (Ω) = P (A + Ac ) = P (A) + P (Ac ).
4.4.4:
For A ⊂ B, we have B = (B \ A) + A and therefore, by 4.4.2,
P (B) = P (B \ A) + P (A).
4.4.5:
4.4.6:
This is a consequence of 4.4.4 together with (4.1.1).
This result is a special case of 4.4.5 with B := Ω.
4.5 Remark
The probability measure has been introduced as a mapping P : P(Ω) → R,
i.e. it assigns a real number to every subset of Ω.
But the probabilitymeasure P is already determined if the probability values P { ω } are known for all ω ∈ Ω. From the σ-additivity
of P and the countability of Ω it follows that for all A ⊂ Ω
X
X
{ω} =
P {ω} .
(4.5.1)
P (A) = P
ω∈A
ω∈A
(Note that the sum in the right hand term of (4.5.1) can consist of either a
finite or a countably infinite sum; also see the proof of 4.4.2.)
4.6 Definition
Let Ω be a non-empty, discrete set.
4.6.1
AP
function w : Ω → [0; 1] is called probability function (on Ω),
if ω∈Ω w(ω) = 1. In the case that Ω is countable, summation
means to calculate the value of the respective non-negative
series.
22
4.6.2
4 Discrete Probability Spaces
Let P be a probability measure over Ω. Then the mapping w : Ω →
[0; 1] defined by
w(ω) := P { ω }
(ω ∈ Ω)
is called the probability function of P .
4.7 Theorem
A probability function w : Ω → [0; 1] uniquely defines a probability measure P
over Ω, and
X
(4.7.1)
P (A) =
w(ω)
(A ⊂ Ω).
ω∈A
Proof:
We can restrict the proof to establishing the σ-additivity of P , particularly
since the other two properties of a probability measure, the non-negativity
and the normalization, are fulfilled already.
Let (An ) be a sequence of pairwise disjoint sets in Ω. Then the permutation theorem for series with non-negative terms implies that
X
X X
X
X
w(ω) =
w(ω) =
P (An ).
P
An =
n∈N
P
ω∈ An
n∈N ω∈An
n∈N
In particular, from (4.7.1) and the permutation theorem for series with nonnegative terms it follows that P (A) is uniquely determined for all A ⊂ Ω,
i.e. it does not depend on the order of summation.
4.8 Discrete probability measures over uncountable basic spaces
Occasionally it is convenient to consider a discrete probability measure
over an uncountable basic space, e.g. over R. This leads, for instance in
connection with probability functions, to the problem of a summation with
uncountably many terms.
If P is a discrete probability measure over an uncountable basic space Ω 0 ,
this simply means that there exists a countable set Ω, Ω ⊂ Ω 0 , as an basic
space over which P is defined.
Within a summation process we take into account – even without explicit mention – only those terms that are indicated by elements of
Ω; terms that are indicated by elements of Ω 0 \ Ω are omitted. Of course, the
set Ω can be replaced by any convenient countable superset D ⊂ Ω 0 of Ω.
S
4.9 Exercise
Let (Ω, P ) be a probability space and B ⊂ Ω with P (B) > 0. Show that
4.10 Exercise
P (A | B) :=
P (A ∩ B)
P (B)
23
(A ⊂ Ω)
defines a probability measure P ( . | B) over Ω.
Evidently P ( . | B) is concentrated on B, i.e. B is a one set with respect
to P ( . | B). On the other hand Ω \ B is a null set.
4.10 Exercise
Show that
S
P (A ∪ B) + P (A ∩ B) = P (A) + P (B).
http://www.springer.com/978-3-540-14913-2