統計理論 Chapter 1: Probability Theory 1.1 Basics of Probability Theory
Definition 1: The set S of all possible outcomes
of an experiment is called the sample space of the
experiment.
Definition 2: Any subset of sample space S , is
called an event.
Definition 3: A collection of subsets of S ,
denoted by W , is called a sigma algebra, if it
satisfies the following properties:
(a)
f ÎW.
C (b) If A Î W , then A Î W .
¥
(c) If A 1 , A 2 , L Î W , then U i = 1 A i Î W .
Definition 4: Let W be a sigma algebra on a
given sample space S . A probability function P is a non-negative function with domain W such
that
(a) P ( A ) ³ 0 for all A Î W .
(b) P ( S ) = 1 .
(c) If A 1 , A 2 , L Î W , and A i Ç A j = f for any i ¹ j , then
¥
P ( U ¥i =1 A i ) = å i =1 P ( A i ) .
Example 1: Assume that S = { s 1 , s 2 , L } is a
countable sample space, and sigma algebra W is
defined on S . Let { p 1 , p 2 , L } be a sequence of
non-negative numbers such that
å
¥
p = 1 . For
i =1 i any A Î W , define
P ( A ) = å
{ i : s i Î A } p i .
Then P is a probability function on W .
Theorem 1: Let P be a probability function, and A and B be any sets in W . Then
(a) P (f ) = 0 .
(b) 0 £ P ( A ) £ 1 .
C
(c) P ( A ) = 1 - P ( A ) .
C
(d) P ( A Ç B ) = P ( A ) - P ( A Ç B ) .
(e) P ( A È B ) = P ( A ) + P ( B ) - P ( A Ç B ) .
(f) If A Í B , then P ( A ) £ P ( B ) .
Theorem 2: Let P be a probability function, and
E 1 , E 2 , L be any partition of the sample space Then
¥
P ( A ) = å i =1 P ( A Ç E i ) .
Theorem 3: Let A 1 , A 2 , L be any events, then
(a) (Boole Inequality)
¥
P ( U ¥i =1 A i ) £ å i =1 P ( A i ) .
(b) (Bonferroni Inequality) n P ( I n i = 1 A i ) ³ å i =1 P ( A i ) - ( n - 1 ) .
S .
1.2 Conditional Probability and
Independence
Definition 5: Let A and B be any events in S ,
and P ( B ) > 0 , then the conditional probability
of A given B , denoted by P ( A | B ) , is P ( A | B ) =
P ( A Ç B ) P ( B ) .
Theorem 5: (Bayes＇Rule) Let E 1 , E 2 , L be any
partition of the sample space S , and A be any
event. Then
P ( E i | A ) = P ( A | E i ) P ( E i ) å
¥
P ( A | E k ) P ( E k ) k =1 Theorem 6: The following statements are
equivalent:
(a) A and B are independent.
c (b) A and B are independent.
c (c) A and B are independent.
c c (d) A and B are independent.
Definition 6: A sequence of events A 1 , A 2 , L, A n are mutually independent if for any subsequence A i 1 , A i 2 , L, A i k , we have
k æ k ö
P çç I A i j ÷÷ = Õ P ( A i j ) è j =1 ø j =1 1.3 Random Variables
Definition 7: A random variable is a real value
function defined on a sample space S .
Definition 8: The cumulative distribution
function (cdf) of a random variable X ,denoted
by F X (x ) , is defined as F X ( x ) = P X ( X £ x ) , for all x .
Theorem 7: A non-negative function F (x ) is a cdf
of a random variable X if and only if it satisfy
the following conditions: F ( x ) = 0 and lim F ( x ) = 1 . (a) x lim
® - ¥
x ® ¥
(b) F (x ) is a non-decreasing function of x .
(c) F (x ) is right continuous, i.e. lim F ( x ) = F ( x 0 ) , for every x ® x 0 +
x 0 Definition 9: A r.v. X is discrete if F X (x ) is a
step function of x .
A r.v. X is continuous if F X (x ) is a continuous function of x .
Definition 10: The probability mass function (pmf)
of a discrete r.v. X is defined as f X ( x ) = P ( X = x ) , for every x .
Definition 11: The probability density function
(pdf), denoted by f X (x ) , of a continuous r.v. X is the function that satisfies
F X ( x ) = ò
x -¥
f X ( t ) dt , for every x .
Theorem 8: A function f X (x ) is a pdf (or pmf) of
a r.v. X if and only if
(a) f X ( x ) ³ 0 for all (b) å
x .
f X ( x ) = 1 ( pmf ) or ò
¥
-¥
f X ( x ) dx = 1 ( pdf ) .