Theoretical Probability Distributions
October 26, 2006
Theoretical Probability
Distributions
Bernoulli Process
• Two outcomes
• Probability is fixed across outcomes
• Trials are independent
Probability Tree for a Bernoulli
Process
H
S
H
⅔
¼
¾
Hc
S
(
¾
)
Hc HHHc
¾
¼
¼
H HHH
¼
H HHcH
¼
¾
Hc
H
Hc
¼
Hc HHcHc
H HcHH
¾
¼
Hc HcHHc
¾
Hc HcHcHc
1 of 3 H @ 1/64
3 of 2 H @ 3/64
3 of 1 H @ 9/64
1 of 0 H @ 27/64
H HcHcH
( ) ( )
[ ( )]
P HH c H c = P(H ) ⋅ P H c ⋅ P H c = [P(H )] ⋅ P H c
1
2
1
Theoretical Probability Distributions
October 26, 2006
Probability of Any Number (k)
of Successes
P(E ) = p k (1 − p )
n−k
• E is the event of getting k successes in
any order
• k is number of successes
• p is probability of success on one trial
• 1-p Is probability of failure on one trial
• n is number of trials
Permutation Rule
Number of ways of getting k outcomes
on n trials:
⎛n⎞
n!
⎜⎜ ⎟⎟ =
⎝ k ⎠ k! (n − k )!
⎛10 ⎞ 10! 10 ⋅ 9 ⋅ 8 ⋅ 7 ⋅ 6 ⋅ 5 ⋅ 4 ⋅ 3 ⋅ 2 ⋅1
⎜⎜ ⎟⎟ =
=
7
(7 ⋅ 6 ⋅ 5 ⋅ 4 ⋅ 3 ⋅ 2 ⋅1)(3 ⋅ 2 ⋅1)
7
!
3
!
⎝ ⎠
Binomial Distribution Formula
⎛ n⎞
n−k
P( X = k ) = ⎜⎜ ⎟⎟ p k (1 − p )
⎝k⎠
where:
⎛n⎞
n!
⎜⎜ ⎟⎟ =
⎝ k ⎠ k! (n − k )!
2
Theoretical Probability Distributions
October 26, 2006
Binomial Table
Binomial Example
5
0
5! ⎛ 1 ⎞ ⎛ 4 ⎞ ⎛ 1 ⎞
p(5) =
⎜ ⎟ ⎜ ⎟ =⎜ ⎟
5!⋅0! ⎝ 5 ⎠ ⎝ 5 ⎠ ⎝ 5 ⎠
4
1
5! ⎛ 1 ⎞ ⎛ 4 ⎞
p(4) =
⎜ ⎟ ⎜ ⎟
4!⋅1! ⎝ 5 ⎠ ⎝ 5 ⎠
3
2
2
3
5! ⎛ 1 ⎞ ⎛ 4 ⎞
p(3) =
⎜ ⎟⎜ ⎟
3!⋅2! ⎝ 5 ⎠ ⎝ 5 ⎠
p(2) =
5! ⎛ 1 ⎞ ⎛ 4 ⎞
⎜ ⎟ ⎜ ⎟
2!⋅3! ⎝ 5 ⎠ ⎝ 5 ⎠
p(1) =
5! ⎛ 1 ⎞ ⎛ 4 ⎞
⎜ ⎟⎜ ⎟
1!⋅4! ⎝ 5 ⎠ ⎝ 5 ⎠
1
⎛4⎞
p(0 ) = ⎜ ⎟
⎝5⎠
pdf
cdf
.0003
.0003
.0064
.0067
.0512
.0579
.2048
.2627
.4096
.6723
.3277
1.0000
5
4
5
Poisson Distribution
λk e − λ
P( X = k ) =
k!
where:
λ = mean number of occurrences
e = natural constant (2.71828)
n
⎛1+ n ⎞
⎛ 1⎞
e = lim⎜
⎟ = lim⎜1 − ⎟
n →∞
n→∞
n
⎝
⎠
⎝ n⎠
−n
3
Theoretical Probability Distributions
October 26, 2006
Poisson Distribution Example
P( X = k ) =
λk e − λ
k!
.20 ⋅ e −.2 1 ⋅ .81872
P(0 ) =
=
= .81873
0!
1
P(1) =
λ=
.21 ⋅ e −.2 .2 ⋅ .81872
=
= .16375
1!
1
P(2 ) =
.2 ⋅ e
2!
2
− .2
=
73
= .2
365
p(≥ 3) = 1 − p(≤ 2 )
.04 ⋅ .81872
= .01637
2
P(0 ) + P(1) + P(2 ) = .99886
Binomial
P(≥ 3) = 1 − .99886 = .0014
vs.
p = .05
Poisson
λ = 25 ⋅ .05 = 1.25
25!
.055.9520
5!20!
25 ⋅ 24 ⋅ 23 ⋅ 22 ⋅ 21
=
1.12 × 10 −7
5 ⋅ 4 ⋅ 3 ⋅ 2 ⋅1
= .00595 ≈ .006
p (5) =
1.255 ⋅e −1.25
5!
3.052 ⋅ .2865
=
120
= .00728 ≈ .007
p (5) =
Multinomial Distribution
P(n1 , n2 , ... nk ,)
P(n1 , n2 , ... nk ,) =
n!
p1n1 p2n2 ... pknk
n1!n2 ! ... nk !
3
3
3
1
10! ⎛ 1 ⎞ ⎛ 1 ⎞ ⎛ 1 ⎞ ⎛ 1 ⎞
P(3,3,3,1) =
⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ = .01602
3!⋅3!⋅3!⋅1! ⎝ 4 ⎠ ⎝ 4 ⎠ ⎝ 4 ⎠ ⎝ 4 ⎠
4
Theoretical Probability Distributions
October 26, 2006
Binomial, p=.4, n=10
0.30
0.25
0.20
0.15
0.10
0.05
0.00
0
1
2
3
4
5
6
7
8
9
10
Poisson, λ=1.4
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
6
7
Expected Value and
Variance of a
Discrete Distribution
k
μ X = ∑ X i p( X i )
i =1
k
σ X2 = ∑ ( X i − μ X )2 p( X i )
i =1
5
Theoretical Probability Distributions
October 26, 2006
Mean and Variance of Binomial
μbinoimial = n ⋅ p
2
σ binomial
= n ⋅ p ⋅ (1 − p )
Expected Value and
Variance of a
Continuous Distribution
b
μ X = ∫ X p( X i ) dX
a
b
σ X2 = ∫ ( X i − μ X )2 p( X i )
a
Normal Distribution
1 ⎛ x−μ ⎞
⎟
σ ⎠
− ⎜
1
p(x ) =
e 2⎝
σ 2π
2
6
Theoretical Probability Distributions
October 26, 2006
Features of the Normal Distribution
0.45
0.40
34.13%
0.35
0.30
0.25
0.20
47.73%
0.15
0.10
0.05
0.00
-4
-3
-2
-1
0
1
2
3
4
standard deviations
Standard Normal
Distribution
Z=
x−μ
σ
1 − 12 z 2
p(z ) =
e
2π
Standard Normal Table
7
Theoretical Probability Distributions
October 26, 2006
68-95-99.7 Rule
• Approximately 68% of observations fall
within one standard deviation of the mean
• Approximately 95% of observations fall
within two standard deviation of the mean
• Approximately 99.7% of observations fall
within three standard deviation of the mean
Normal Approximation for
Binomial
np > 5 and n(1 − p ) > 5
2
σ binomial
= n ⋅ p ⋅ (1 − p )
μbinoimial = n ⋅ p
Example: 10 heads on 15 flips, p=.1509
σ = 15 ⋅ .5 ⋅ (1 − .5) = 1.9365
μ = 15 ⋅ .5 = 7.5
Z=
10 − 7.5
= 1.29
1.9365
P ( Z ≥ 1.29) = .0985
Correction for Continuity
P (# H ≥ 10)
P (# H < 10) = P(# H ≤ 9)
0.45
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
0
Z=
1
2
3
4
5
9.5 − 7.5
= 1.033
1.9365
6
7
8
9
10 11 12 13 14 15
P ( Z ≥ 1.033) = .151
8
Theoretical Probability Distributions
October 26, 2006
Proportions vs. Binomial
p=
1
k
n
for k successes on n trials
snew = b s
xnew = a + bx
1
n
1
n
μ P = μbinoimial = ⋅ n ⋅ p = p
1
n
σ P = σ binomial =
1
n ⋅ p ⋅ (1 − p )
n ⋅ p ⋅ (1 − p ) =
=
n
n2
p ⋅ (1 − p )
n
Chi Square Distribution
χν2 = Z12 + Z 22 + ... + Zν2
ν is called “degrees of freedom
( )
E χν2 = ν
χ12 = Z 2
t distribution
tν =
Z
⎛ χν
⎜⎜
⎝ν
2
⎞
⎟⎟
⎠
1
2
E (tν ) = 0
lim tν = normal(0,1)
ν →∞
9
Theoretical Probability Distributions
October 26, 2006
F Distribution
F(ν1 ,ν 2
χ (2ν ) ν 1
)= 2
χ (ν ) ν 2
1
2
F(1,ν 2 ) = tν22
FT-Bush, Three Distributions
0
20
40
60
Feeling Thermometer: Bush
80
100
Percent
0
5
0
0
5
5
Percent
10
Percent
10
10
15
15
15
Sample Means
20
One Sample
20
The Population
0
20
40
60
Feeling Thermometer: Bush
80
100
0
20
40
60
Mean of Feeling T hermometer: Bush
80
Sample Distribution of Sample
Mean of Normal Distribution
If X and Y are normally distributed then any linear
combination of X and Y is also normally distributed
N=20
N=10
N=1
N=5
N=3
N=2
10
Theoretical Probability Distributions
October 26, 2006
Sampling Distribution of
Sample Mean of a Uniform
Distribution
Fraction
.398942
.166667
.16658
.12454
.10011
.11501
.07187
.003367 0
1 2.5
1
4.5
6
mean
var1
x
Central Limit Theorem
The sampling distribution of a sample mean of
X approaches normality as the sample size
gets large regardless of the distribution of X.
The mean of this sampling distribution is
μX and the standard deviation (standard
error) is σX/√n
The sampling distribution of any linear
combination of N random variables
approaches normality as N gets large.
Sampling Distribution of
Sample Mean of an Arbitrary
Distribution
Fraction
.398942
.33215
.29019
.23086
.16436
.11922
.5
.003367 0
1 1
3
mean
x
11
Theoretical Probability Distributions
October 26, 2006
Small Samples
SErf =
σX =
σ Pˆ =
N = population size
N −n
N −1
σX
n
n = sample size
N −n
N −1
p ⋅ (1 − p ) n
n
N −n
N −1
12