Continuous Distributions
Density Curve
Percent f(x)
Chapter 3 Continuous Distributions
A smooth curve that fit
the distribution
Densityy
function, f (x)
3.2 Random Variables of the
Continuous Type
Test scores
10 20 30 40 50 60 70 80 90 100 110
1
Continuous Distribution
Density Curve
Probability Density Function (p.d.f.) of a random
variable X of continuous type with a space S is an
integrable function, f(x), that satisfying the following
conditions:
Percent f(x)
A smooth curve that fit
the distribution
Probability
y
Density
Function, f (x)
Shows the
probability
density for all
values of x.
2
Test scores
10 20 30 40 50 60 70 80 90 100 110
1. f (x) ≥ 0, x ∈ S,
2. ∫S f(x) dx = 1, (Total area under curve is 1.)
b
3. For a and b in S, P(a < X < b) = ∫a f ( x)dx
f(x)
Probability density is not probability!
Use a mathematical model to describe the variable.
3
Meaning of Area Under Curve
P(72 ≤ X ≤ 86) = P(72 < X < 86)
86) = P(72 < X ≤ 86)
f(x) = P(72 ≤ X <86)
X (Height)
P(X = 72) = 0, density is not probability.
4
Example: What percentage of the
distribution is in between 72 and 86?
P(72 ≤ X ≤ 86)
72 86
x
Meaning of Area Under Curve
Example: What percentage of the
distribution is in between 72 and 86?
f(x)
a b
5
72 86
X (Height)
6
1
Continuous Distributions
Example:
If the density function of a continuous distribution is
for 0 < x < 0.5
⎧ 8x ,
f ( x) = ⎨
elsewhere
⎩ 0 ,
Find the proportion of values in this distribution that is less than 1/4.
The area under the f (x) between 0 and 1/4 =
∫
1/4
8 x dx
Review of Calculus
1
∫ x dx = n + 1 x
n
n +1
1
∫ x dx = 3 + 1 x
+c
3
3+1
+c =
1 4
x +c
4
0
x2
=8
2
1/ 4
= 4x
1/ 4
2
= 4(1/4)2 – 4(0)2 = 4/16 = 1/4.
0
0
∫e
f (x) = 8x is a line with
a slope 8 and passes
through (0,0)
f (x) = y
ax
dx = ∫ e u
=
x
0
0.5
1
du
a
u = ax, du = adx
∫e
1 u
e +c
a
2x
dx =
1 2x
e +c
2
7
Example:
If the density function of a continuous distribution X, waiting time
between arrivals of cars at a intersection, is
x
8
Example:
If the density function of a continuous distribution X, waiting time
between arrivals of cars at a intersection, is
x
1 −
f ( x) = e 5 , for x > 0
5
1 −
f ( x) = e 5 , for x > 0
5
Find the probability that the waiting time (in seconds) till the next
arrival of car at this intersection is more than 3 seconds.
Find the probability that the waiting time (in seconds) till the next
arrival of car at this intersection is less than 3 seconds.
The area under the f (x) and x > 3 =
b
∫
∞
3
x
5
1
e dx
5
The area under the f (x) below 3 =
(−e − b / 5 ) − (−e −3 / 5 ) = 0 − (−0.5488) = 0.5488
= lim(−e − x / 5 ) = blim
→∞
b →∞
3
= ( −e
−x/ 5
3
) = (−e
−3 / 5
0
f (x) = y
) − ( −e
−0 / 5
∫
3
0
x
1 -5
e dx
5
) = 1 − e −3 / 5 = 1 − 0.5488 = 0.4522
f (x) = y
.2
.2
x
0
x
3
0
9
Cumulative Distribution Function
The cumulative distribution function (c.d.f. or
distribution function, d.f.) of a continuous
random variable is defined as
x
F ( x) = P ( X ≤ x) = ∫ f (t )dt
f ( x) =
1
θ
−
x
e θ , for x > 0
Find the distribution function.
function
x
−∞
= ( −e
• F(− ∞) = 0, F(∞) = 1
• P(a < X < b) = F(b) – F(a)
• F′(x) = f(x), if derivative exists
10
Example:
If the p.d.f. of a continuous distribution X, waiting
time between arrivals of cars at a intersection, is,
where θ is a constant parameter of this distribution,
F ( x) = ∫
−∞
3
f (t ) dt
−t / θ
=∫
x
1
θ
) = 1 − e− x /θ
0
-
t
e θ dt
x
0
11
−∞ < x < 0
⎧ 0,
F ( x) = ⎨
− x /θ
⎩1 − e , 0 < x < ∞
12
2
Continuous Distributions
Measure of Center for a
Continuous Distribution
Measure of Spread for a
Continuous Distribution
The mean value (expected value) of a
continuous random variable (distribution)
X, denoted by μX or just μ (or E[X]) is
defined as
∞
μ X = ∫ x ⋅ f ( x) dx
The variance of a continuous random
variable (distribution) X, denoted by σX2
or just
j σ2 (or
( Var[X])
[ ]) is
i defined
d fi d as
∞
σ 2 = E[( X − μ ) 2 ] = ∫ ( x − μ ) 2 ⋅ f ( x)dx
−∞
The standard deviation of X is
−∞
σ = σ 2 = E[( X − μ ) 2 ]
13
14
Example:
If the density function of a continuous distribution is
for 0 < x < 0.5
⎧ 8x ,
f ( x) = ⎨
elsewhere
⎩ 0 ,
Find the mean and variance of this distribution
Moment Generating Function
for a Continuous Distribution
The moment generating function, if
exists, of a continuous random variable
(di ib i ) X, denoted
(distribution)
d
d bby M(t)
( ) is
i
defined as
The mean is
∞
0.5
−∞
0
μ = E ( X ) = ∫ x ⋅ f ( x) dx = ∫
∞
M (t ) = E[e tX ] = ∫ etx f ( x)dx, − h < t < h
−∞
=∫
0.5
0
x ⋅ 8 x dx = 8
x3
3
0.5
0
=
x ⋅ f ( x) dx
8
1
(.125 − 0) =
3
3
15
Example:
If the density function of a continuous distribution is
for 0 < x < 0.5
⎧ 8x ,
f ( x) = ⎨
elsewhere
⎩ 0 ,
Find the mean and variance of this distribution
The Percentile
The (100p)th percentile (quantile of order p)
is the number πp such that the area under f(x) to
the left of πp is p.
The variance is σ 2 = Var ( X ) = E[ X 2 ] − ( E[ X ]) 2
μ = E[ X 2 ] =
=
∫
0.5
0
∫
∞
−∞
x 2 ⋅ f ( x ) dx =
x4
x ⋅ 8 x dx = 8
4
2
0 .5
0
∫
0.5
0
p=∫
x 2 ⋅ f ( x ) dx
8
1
= (. 0625 − 0 ) =
4
8
M ″(0) M
πp
2
′(0)2
f ( x) dx = F (π p )
p
2
2
πp
−∞
1 ⎛1⎞
1
σ = E[ X ] − ( E[ X ]) = − ⎜ ⎟ =
8 ⎝3⎠
72
2
16
17
Median is the 50th percentile.
18
3
Continuous Distributions
Median of a distribution
Example:
If the density function of a continuous
distribution is
What is c?
for 0 < x < 0.5
⎧ 8x ,
f ( x) = ⎨
⎩ 0 ,
∫
Median is c such that
c
8 x dx = .5
0
c
∫ 8 x dx = 8
0
c
x2
2
= .5
0
⇒ 4c 2 − 0 = .5
elsewhere
⇒ c 2 = .5 / 4
Find the median of this distribution.
f (x) = y
1
= .354
8
⇒c=
x
0
0.5
19
20
Percentile
Example:
If the density function of a continuous
distribution is
c
0
What is c?
for 0 < x < 0.5
elsewhere
⎧ 8x ,
f ( x) = ⎨
⎩ 0 ,
∫
25th percentile is c such that
c
x2
∫ 8 x dx = 8 2
0
8 x dx = .25
c
= .25
0
⇒ 4c 2 − 0 = .25
⇒ c 2 = .25 / 4
Find the 25th percentile of this distribution.
f (x) = y
⇒c=
1
= .25
16
x
0
0.5
21
22
Examine Distribution with
Quantile-Quantile Plot
Sample Quantile
Let y1 ≤ y2 ≤ ...≤ yn be the order statistics
associated with the sample x1, x2, ···, xn , then
yr is called the sample quantile of order
r
100r
as well as
percentile.
n +1
n +1
r − .5
Other
options:
n
Example: 1, 3, 7, 8, 9, 13
n = 6, and the value 8, y4
r − .375
n + .25
,
is the [4/(6+1)]th quantile of the distribution of the
sample, i.e. 0.571 sample quantile or 57.1 percentile.
23
Theoretical
Distribution,
Theoretical
Distribution,
πp
πp
Sample, yr
Good model for the
observations
Sample, yr
Not a good model for
the observations
24
4
Continuous Distributions
Uniform Distribution
Chapter 3 Continuous Distributions
The continuous random variable X has a
uniform distribution if its p.d.f. is equal to a
constant on its support. If the support is the
interval [a, b], then its p.d.f. is
f ( x) =
3.3 The Uniform and Exponential
Distributions
1
,
b−a
a ≤ x ≤ b.
It is usually denoted as U(a, b).
f(x)
1
b−a
25
Uniform Distribution
0
a
b
1
F(x)
0
a
b
26
Uniform Distribution
The mean, variance, and m.g.f. of a
continuous random variable X that has a
uniform distribution are:
Example: Let X be U(0,10), find the mean and
the variance, and the m.g.f. of X.
a+b
(b − a ) 2
,
σ2 =
,
2
12
⎧ e tb − eta
⎪
, t ≠ 0,
M (t ) = ⎨ t (b − a)
⎪⎩ 1,
t = 0.
μ=
0 + 10
(10 − 0) 2 25
= 5,
= ,
σ2 =
2
12
3
⎧ e10t − 1
⎪
M (t ) = ⎨ 10t , t ≠ 0,
⎪⎩ 1,
t = 0.
μ=
Pseudo-Random Number Generator on most computers U(0, 1)
27
Exponential Distribution
Exponential Distribution
The mean, variance, and m.g.f. of a
continuous random variable X that has an
exponential distribution are:
The continuous random variable X has an
exponential distribution if its p.d.f. is
⎧ 1 − x /θ
⎪
x
f ( x) = ⎨ θ
⎪⎩ 0 ,
,
28
for 0 < x
μ =θ,
elsewhere
M (t ) =
where θ is the mean of the distribution.
* X can be the waiting time until next success in a Poisson process.
29
σ 2 = θ 2,
1
,
1 − θt
t<
1
θ
−∞ < x < 0
⎧ 0,
− x /θ
⎩1 − e , 0 ≤ x < ∞
c.d.f. ⇒ F ( x) = ⎨
30
5
Continuous Distributions
Exponential Distribution
Exponential Distribution
Example: Let X have an exponential distribution
with a mean of 30, what is the first quartile of
this distribution?
−∞ < x < 0
⎧ 0,
F ( x) = ⎨
− x / 30
, 0≤ x<∞
⎩1 − e
F (π .25 ) = 1 − e
−π .25 / θ
= .25 ⇒ e
−π .25 / θ
= .75
⇒ −π.25 /θ = ln(.75) ⇒ π.25 = − θ ·ln(.75)
{θ = 30} ⇒ π.25 = − 30·ln(.75) ⇒ π.25 = 8.63
πp = −θ · ln(1 – p)
31
Exponential Distribution
f(w) =
1
θ
F(w) = P(W ≤ w) = 1 – P(W > w)
= 1 – P(W > w)
= 1 – P(no success in [0,w])
= 1 – e-λw ⇒ d.f. of exponential distribution.
F ′(w) = f(w) = λe-λw ⇒ p.d.f. of exponential distribution.
1
θ
e − x /θ
32
Exponential Distribution
Let W be the waiting time until next
success in a Poisson process in which the
average number of success in unit
interval is λ, then, λ = 1/θ, where θ is
the average waiting time until next
success, for w ≥ 0
F(w) = 1 – e-λw =1 – e-w/θ
Let W be the waiting time until next success in
a Poisson process in which the average number
of success in unit interval is λ, then, for w ≥ 0
Suppose that number of arrivals of
customers follows a Poisson process with a
mean of 10 per hour. What is the probability
that the next customer will arrive within 15
minutes? (15 min. = 0.25 hour)
P(X ≤ x) = F (x) = 1 – e-x/θ
⇒ d.f. of exponential
distribution.
e − w / θ ⇒ p.d.f. of exponential distribution.
P(X ≤ 0.25) = F (0.25) = 1 – e-0.25/θ
λ = 10 ⇒ θ = 0.1
F (0.25) = 1 – e-0.25/0.1 = .918
33
34
Gamma Distribution
Chapter 3 Continuous Distributions
The continuous random variable X has a
Gamma distribution, Gamma(α, θ ). if its
p.d.f. is
p
1
⎧
x α −1e − x / θ ,
for 0 ≤ x
⎪
f ( x ) = ⎨ Γ (α )θ α
⎪⎩
0 ,
elsewhere.
3.4 The Gamma and Chi-square
Distributions
Gamma Function: Γ(n) = (n – 1)!
* X can be the waiting time until α−th success in a Poisson process.
35
36
6
Continuous Distributions
Gamma Distribution
Graphs of Gamma Distributions
θ =4
α = 1/4
θ = 5/6
θ =1
θ =2
θ =3
α=1
α=2
α=3 α=4
0
10
20
The mean, variance, and m.g.f. of a
continuous random variable X that has an
Gamma distribution are:
α=4
30
0
10
μ = αθ ,
M (t ) =
θ=4
20
30
σ 2 = αθ 2 ,
1
(1 − θ t )
α
,
t<
1
θ
•Gamma(1, θ ) ⇒ Exponential Distribution
•Gamma(2, r/2 ) ⇒ Ch-square with d.f. = r.
37
38
Special Notation χ2α (r)
Let X be a random that has Chi-square distribution with
degrees of freedom r,
P[ X ≥ χα2 (r )] = α
Tail area
Probability of X
greater than or
equal to χ2α(r) is α.
χ2α(r)
P[ X ≤ χ12−α ( r )] = α
Example: Find χ2.05(3) = ?7.815
39
40
Try This!
• Find χ2.01(7) = ?
• Find χ2.025(4) = ?
• Find the 10th percentile from a χ2
distribution with degrees of freedom 6.
Chapter 3 Continuous Distributions
3.5 Distributions of Functions of
Random Variable
- Random number generation
41
42
7
Continuous Distributions
Methods for Generating
Non-Uniform RN’s
A Good Generator
Xi = (16,807 Xi-1) mod (231 – 1)
•
•
•
•
Starts with a Seed number
• Simple
• Widespread use
• Long cycle length 231 – 2 (all number
besides 1 and 231 – 1 can be generated.)
• Ui = Xi /(231 – 1), Ui ~ U(0,1)
CDF Inversion
T
Transformations
f
ti
Accept/Reject Methods
…
43
CDF Inversion
44
Proof:
Theorem: Let U have a distribution that is
U(0,1). Let F(x) have the properties of a
distribution function of the continuous type with
F(a)
( ) = 0 and F(b)
( ) = 1,, and suppose
pp
that F(x)
( ) is
strictly increasing on the support a < x < b,
where a and b could be −∞ and ∞,
respectively. Then the random variable X
defined by X = F −1(U) is a continuous
random variable with distribution function F(x).
45
Generate random numbers from
Exponential distribution, θ = 10
Let X = F −1(U) , the distribution function of X is
P (X ≤ x ) = P [ F−1(U) ≤ x ],
a< x<b.
Since F ((x)) is strictlyy increasing,
g, { F−1((U)) ≤ x } is
equivalent to { U ≤ F (x) } and hence
P ( X ≤ x ) = P [ U ≤ F (x) ],
a< x<b.
But U is U(0, 1); so P ( U ≤ u ) = u for 0 < u <
1, and accordingly,
P ( X ≤ x ) = P [ U ≤ F (x) ] = F (x) , 0 ≤ F(x) < 1.
That is the distribution function of X is F(x).
46
Try this!!!
• F(x) = 1 − e−x/10 and x =F−1(y) = −10·ln(1− y)
• Use uniform U(0, 1) random number generator to
generate random numbers y1, y2, ..., yn .
• The exponentially distributed random numbers xi's
would be xi = −10·ln(1− yi) , i = 1, ..., n.
• Therefore, if the uniform random number generator
generates a number 0.1514, then 1.6417 = −10·ln(1−
0.1514) would be a random observation from
exponential distribution with θ = 10.
47
An U(0,1) random number generator has
generated a value of 0.26.
1. Use the CDF Inversion method to convert this
U(0,1) random number to simulate an observation
from an exponential distribution with θ = 12.
xi = F−1(yi) = −12·ln(1− yi)
2. Use CDF Inversion method to generate a random
number from the continuous random variable that has
the following p.d.f.
for 0 < x < 0.5
⎧ 8x ,
f ( x) = ⎨
0
,
elsewhere
48
⎩
8