Lectures 6-10: Axioms of Probability Theory Let X be a set. We denote by 2X the set of all subsets of X . Definition A collection of subsets F ⊆ 2X is called an algebra if I ∅, X ∈ F I A ∈ F ⇒ Ac ∈ F I A, B ∈ F ⇒ A ∪ B ∈ F 1 / 16 Algebras of sets Exercises: 1. Check that A∪B c = Ac ∩ B c and use this fact to show that if F is an algebra then A, B ∈ F ⇒ A ∩ B ∈ F. 2. For any set X , show that the collections F = {∅, X } and F = 2X are algebras. 3. Let X = R. Show that the collection F consisting of finite unions of intervals (closed, open or half-open) is an algebra. 2 / 16 Pre-measure Definition A function P from an algebra of sets F ⊆ 2X to the set of non-negative real numbers R≥0 is called a pre-measure if for any disjoint sets A, B ∈ F (A ∩ B = ∅) one has P(A ∪ B) = P(A) + P(B). Exercises: 1. Let X be any set and A ⊂ X . Take any numbers x, y ≥ 0. Show that F = {∅, A, Ac , X } is an algebra. Check that P : F → R≥0 given by P(∅) = 0, P(A) = x, P(Ac ) = y , P(X ) = x + y is a pre-measure. 3 / 16 2. Show that for any pre-measure P on an algebra F one has I I P(∅) = 0 A ⊂ B ⇒ P(A) ≤ P(B) 3. Let F ⊆ 2X be any algebra and x ∈ X . Show that the function δx : F → R≥0 given by ( 0, x ∈ /A δx (A) = 1, x ∈ A is a pre-measure. 4. Show that for any finite collection of pairwise disjoint sets A1 , . . . An ∈ F, Ai ∩ Aj = ∅ one has P( n [ i=1 Ai ) = n X P(Ai ) . i=1 4 / 16 σ-algebras Definition An algebra of sets F ⊆ 2X is called a σ-algebra if A1 , A2 , . . . ∈ F ⇒ ∞ [ Ai ∈ F. i=1 Definition For any collection of subsets X ⊆ 2X , the σ-algebra generated by X is defined as \ σ(X ) = F X ⊆F ⊆2X where the intersection is taken over all σ-algebras of subsets of X containing the collection X . 5 / 16 Exercise: Show that σ(X ) in the above definition is indeed a σ-algebra. We see that σ(X ) is the minimal σ-algebra which contains all sets in X . Example: Let X = R and X is a collection of all intervals. Then σ(X ) is called the Borel σ-algebra. Exercise: Show that Borel σ-algebra contains all open and closed sets. 6 / 16 Measure Definition A pre-measure on a σ-algebra F is called a measure if A1 , A2 , . . . ∈ F, Ai ∩ Aj = ∅ ⇒ P( ∞ [ Ai ) = i=1 ∞ X P(Ai ). i=1 While pre-measure is an additive function of sets, a measure is a σ-additive function of sets. Exercise: Show that for a measure P and any countable collection of sets A1 , A2 , . . . ∈ F one has P( ∞ [ i=1 Ai ) = lim P( n→∞ n [ Ai ) . i=1 7 / 16 Main theorem: extension of a pre-measure A pre-measure P on an algebra of sets F is called a measure if it is σ-additive in the following sense: A1 , A2 , . . . ∈ F, Ai ∩ Aj = ∅ , ⇒ P( ∞ [ i=1 ∞ [ Ai ∈ F i=1 ∞ X Ai ) = P(Ai ). i=1 Theorem For any measure P on an algebra F there exists a unique extension of P to a measure on σ(F). 8 / 16 Extension of a measure Examples: 1. Let F be the algebra of finite unions of subintervals in X = [a, b] and λ be the length function, i.e. [ X λ [ci , di ] = (di − ci ) i i Then one can extend the length uniquely to all Borel sets: for every A ∈ σ(F) the length λ(A) is well-defined. 9 / 16 Extension of a measure 2. Let F be the algebra of finite unions of subintervals in X = [a, b] and p(x) be a non-negative integrable function on [a, b]. Then we define [ XZ P [ci , di ] = i i di p(x)dx . ci This is a measure, and therefore it has the uniqueRextension to Borel sets. Now, for any A ∈ σ(F) we can write A p(x)dx meaning the value P(A) of the extension. 10 / 16 Extension of a measure 3. Let F be the algebra of finite unions of rectangles in X = [a, b] × [c, d] and α be the area function on F. This is a measure, and therefore it has the unique extension to Borel sets. Now, for any A ∈ σ(F) the area α(A) is well defined. 4. (product of measures) Let Pi be a measure on Fi ⊆ 2Xi for i = 1, 2. Then [ X P( Aj × Bj ) = P1 (Aj )P2 (Bj ) j j is a measure and extends to the σ-algebra generated by finite unions of products of sets from F1 and F2 respectively. This σ-algebra is denoted F1 × F2 and P is denoted P1 × P2 . For example, α = λ × λ. 11 / 16 Probability measures Definition A measure P : F → R≥0 on a σ-algebra F ⊆ 2X is called a probability measure if P(X ) = 1. In the above examples: R p(x)dx P(A) = R A X p(x)dx is a probability measure on Borel subsets of X = [a, b]. P(A) = α(A) α(A) = α(X ) (b − a)(d − c) is a probability measure on Borel subsets of X = [a, b] × [c, d]. 12 / 16 Axioms of probability theory Recall that a probabilistic experiment is an experiment which I can be repeated infinitely many times; I has a well-defined set of possible outcomes, which we denote by Ω. Subsets A ⊆ Ω are called events, and we want to have a probability law which assigns to an event A its probability P(A). 13 / 16 Axioms of probability theory Definition A probability space is a triple (Ω, F, P) where I Ω is a set; I F ⊆ 2Ω is a σ-algebra of subsets of Ω; I P : F → R≥0 is a probability measure, i.e. we have P(Ω) = 1. 14 / 16 Examples: 1. Consider an interval [a, b] (a = −∞, b = ∞ are possible) and let p(x) be a non-negative integrable function on [a, b] such Rb that a p(x)dx = 1. Then we have a probability space: I I I Ω = [a, b]; F is the RBorel σ-algebra; P(A) = A p(x)dx. Probability measures constructed by means of integration are often called distributions. p(x) is called the density function of the distribution. 1 satisfies properties of a Exercise: Check that p(x) = b−a density function on a the finite interval [a, b]; this is called the uniform distribution on [a, b]. Exercise: Let λ > 0. Check that p(x) = λe −λx is a density function on [0; +∞]; this is the exponential distribution with parameter λ. 15 / 16 2. (Discrete models) Let Ω = {w1 , w2 , w3 , . . . } be a countable set. Then for anyPsequence of numbers {p1 , p2 , p3 , . . . } such that pi ≥ 0 and i pi = 1 we can define a probability measure on F = 2Ω by X P(A) = pi . i:wi ∈A In the case when Ω is a finite set, one can define uniform 1 . In this case we probability law by taking pi all equal to |Ω| have X 1 |A| = . P(A) = |Ω| |Ω| i:wi ∈A 16 / 16