EC381/MN308
Probability and Some Statistics
Lecture 9 - Outline
1. Functions of Random Variables
Yannis Paschalidis
2. A Unified View of Discrete and Continuous Random Variables
[email protected], http://ionia.bu.edu/
Dept. of Manufacturing Engineering
Dept. of Electrical and Computer Engineering
Center for Information and Systems Engineering
1
EC381/MN308 – 2007/2008
3.5 Functions of Continuous RVs
Discrete-to-Discrete Transformations
A random variable X is transformed by a specified function g(X) into a
random variable:
If X is a discrete RV taking on values x1, x2, … , xn, …,
then Y must also be a discrete RV with values y1 = g(x1),
y2 = g(x2), … , yn = g(xn), …
Y = g(X) .
What are the statistical properties of Y?
discrete
→ discrete
continuous → continuous
Some of these values may be the same, i.e., g(xi) = g(xj)
for some values of i and j and this will, of course, be
reflected in the probabilities associated with Y
continuous → mixed
The PMF of Y is readily obtained via:
continuous → discrete
Types of Transformations:
In any of these cases, the expected
value of Y, or of functions of Y, can be
readily determined, but the PMF/ PDF
are harder to determine
pY(yj) = P{Y = yj} = ∑ pX(xi),
where the sum is over all i such that g(xi) = yj .
E [Y2] = E [g2(X)]
Var[Y] = E [g2(X)] – {E [g(X)]}2
EC381/MN308 – 2007/2008
3
2
g(x) = n, for n – 0.5 < x ≤ n + 0.5,
n = integer
EC381/MN308 – 2007/2008
–1.5 –0.5
In
0.5
1.5
x
2.5
–1
–2
PY(n) = P [Y = n]
yn
y2
1
–2.5
g(x)
y1
g(x)
g(•) is a staircase function
If the range of X can be partitioned into a (countable) set of
intervals I1, I2, … In, … such that g(x) has a constant
value yj for all x ∈ Ij,
Then Y is a discrete RV taking values y1, y2, …, yn, …
with probabilities P{X ∈ I1}, P{X ∈ I2}, … , P{X ∈ In}, …,
respectively as shown in the figure below:
respectively,
I2
4
EC381/MN308 – 2007/2008
Example: A quantizer function
Continuous-to-Discrete Transformations
I1
2
EC381/MN308 – 2007/2008
= P [n – 0.5 < X ≤ n + 0.5]
= area under PDF fX(x)
from x = n – 0.5 to x = n + 0.5
x
= FX(n + 0.5) – FX(n – 0.5)
5
EC381/MN308 – 2007/2008
fX(x)
x
6
1
Example: A quantizer function transforms an exponential
Continuous RV into a Geometric Discrete RV
If X is an exponential RV with parameter λ and
its derivative g′(x) is not zero on any interval (this restriction simply avoids
“flat spots” in g(•) that would give rise to mixed or discrete RVs, although
Y = ⎡X⎤ (smallest integer greater than or equal to x)
allowing g′(x) = 0 for some values of x is acceptable),
Then Y is a geometric random variable with parameter
p = 1 – exp(–λ) since ∫01 λ exp(–λx) dx = 1 – exp(–λ) , so
Then, Y = g(X) is a continuous RV.
P
Procedure
d
tto determine
d t
i th
the PDF off Y,
Y given
i
th
the PDF off X
PY(n) = [1 – exp(–λ)]•[exp(–λ)] n–1 for n ≥ 1
1) Sketch the function y = g(x) so that you can determine the range of values of X
that correspond to any range of values of Y
g(x)
y
2
1
0
1/λ
x
2) Determine the CDF of Y in terms of the CDF of X:
PY(n)
0
1
2
x
y
n
1 2 3
7
EC381/MN308 – 2007/2008
Special Case: Y = g(X) is a continuous monotonic increasing
function for the entire range of X. Given fX(x), determine fY(y).
y
P [y ≤ Y ≤ y + Δy] = P [x ≤ X ≤ x + Δx]
fY ( y ) Δy = f X ( x ) Δx
dy
fY ( y )
= fX ( x ).
dx
dy
= f ( x ).
dx X
Example:
Range x
x
x2
8
Y = c X + d,
y
where c > 0
g(x)
x
x
Δx
Since g(X) is continuous and monotonic, we have
fY ( y )
dy
= fX ( x )
dx
X is an exponential RV: f X ( x ) = λ exp( −λ x ), x ≥ 0
x1
EC381/MN308 – 2007/2008
Given fX(x), determine fY(y).
Δy
So, for any monotonic function: fY ( y )
= (d/dy)FY(y)
Special Case: Linear Transformations
g(x)
Similarly, for a monotonic decreasing function
−fY ( y )
3) Differentiate to get the PDF: fY(y)
Range y
fX(x)
λ
Continuous-to-Continuous Transformations
If X is a continuous RV and g(x) is a continuous function of x such that
and Y = X 2 ,
dy
= f X ( x ) ⇒ fY ( y ) c = f X ( x )
dx
fY ( y ) =
X ≥0
dy
= f X ( x ) ⇒ fY ( y ) ⋅ 2 x = f X ( x )
dx
f ( x ) fX y
λ
=
=
exp −λ y
fY ( y ) = X
2x
2 y
2 y
1 ⎛y−d⎞
f
c X ⎜⎝ c ⎟⎠
fY ( y )
( )
EC381/MN308 – 2007/2008
fY ( y ) =
If d = 0, then,
(
)
9
1 ⎛ y⎞
f
c X ⎜⎝ c ⎟⎠
Example: Linear Transformation of a Gaussian RV
Thus, the function fY(y) has the same shape as fX(x), but is stretched out
by a factor of c (if c > 1) or squeezed down by a factor of c (if c < 1)
If X is a Gaussian RV, N(μ, σ 2), then Y = cX + d
is also a Gaussian RV, N(cμ+d, c2σ 2).
If the horizontal scale expands by a factor c, the vertical scale must
compress by the same factor to maintain the area at unity
c=2
fX(x)
If d ≠ 0, then
as before:
fY ( y ) =
Proof:
fY(y)
x
y
1 ⎛y−d⎞
f
c X ⎜⎝ c ⎟⎠
i.e., fY(y) is a scaled version of fX(x) that is translated rightward by d.
EC381/MN308 – 2007/2008
10
EC381/MN308 – 2007/2008
11
fX ( x ) =
⎡ ( x − μ )2 ⎤
exp ⎢ −
⎥
2σ 2 ⎦
σ 2π
⎣
fY ( y ) =
1
1 ⎛ y−d⎞
f
c X ⎜⎝ c ⎟⎠
2
⎡ ⎛⎛ y − d ⎞
⎞ ⎤
⎢ ⎜⎜
⎟−μ⎟ ⎥
⎡ ( y − d − μ c )2 ⎤
1 1
1
⎢ ⎝⎝ c ⎠
⎥
⎠
⎢
⎥
fY ( y) =
exp ⎢−
⎥ = c σ 2π exp ⎢ −
2σ 2
2c 2σ 2
c σ 2π
⎥
⎣
⎦
⎢
⎥
⎣⎢
⎦⎥
EC381/MN308 – 2007/2008
= N(cμ + d, c2σ 2)
12
2
3.6 Conditional Probability for General RVs
Continuous-to-Mixed Transformations
If X is a continuous RV and g(x) is a
continuous function of x with zero
derivative on one or more intervals
then Y = g(X) is a mixed (or hybrid)
RV
Review: for random events
g(x)
y1
0
fX(x)
FY(y)
A
x
A−B
A∩B
B
x
CDF
0
y1
y
y1
y
fY(y)
PDF/
PMF
0
EC381/MN308 – 2007/2008
13
Review: for Discrete RVs:
14
EC381/MN308 – 2007/2008
Conditional Probability for Arbitrary RVs
Conditional CDF:
Special case: B ⊂ SX
p. 83
Conditional PDF:
PX|B(x) is a probability mass function with the usual properties:
•Special case: B ⊂ SX
–Truncate and rescale!
15
EC381/MN308 – 2007/2008
Example: Exponential RV truncated and rescaled
16
EC381/MN308 – 2007/2008
3.7 Unified View of Continuous & Discrete RVs
Discrete RV
X is exponential, parameter λ = 0.5;
CDF
B = {X > 6} (we know the RV > 6)
Continuous RV
FX(x) = P [X ≤ x]
CDF FX(x) = P [X ≤ x]
1
1
P [xk]
0
P(B) = e-3 since area
under fX(x) from x = 6 to
x = ∞ = FX(∞) – FX(6) =
1 – FX(6) = e-0.5 x 6 = e-3
x
xk
0
x
PDF fX(x) = (d/dx) FX(x)
PMF
P [xk]
For exponential RVs (and also Geometric RVs): residual life Xr is
exponentially distributed, i.e.,
xk
P{Xr > b | X > a} = P {X > b}, so that these RVs are “memoryless.”
EC381/MN308 – 2007/2008
17
x
x
Let’s bring both of these under the same view.
EC381/MN308 – 2007/2008
18
3
PDF versus PMF
Define PDF functions for Discrete RVs using the
concept of the impulse (delta) function
• Discrete RV defines a set of point masses on the axis, with total mass
=1
– PMF PX(x) = mass at x = probability that x occurs
PDF = fX(x) = (d/dx) FX(x) for both discrete & continuous RVs
• Continuous RV defines a spread of the total unity probability mass
along the axis
Differentiate the “stairs” function using the unit step function
– There is no probability mass at any point
u(x) and the delta function δ(x):
• PDF of a continuous RV tells the density of the mass at each point
u(x)
– PDF is measured in units of probability mass/length
1
– PDF is analogous to mass density, charge density, etc.
0
x
0
x
δ (x)
19
EC381/MN308 – 2007/2008
Properties of the Impulse Function
Unit impulse looks like a probability density!
• The unit impulse is a “generalized function”
• It is nonnegative
– In principle it’s not a proper function, as it has an infinite value
– But, we can integrate against it
– From a mathematician’s perspective, it is a “distribution” or
“generalized function”
• It integrates to 1:
• So, we will use it to describe PDFs for RVs with CDFs
that are discontinuous
• Given any well-behaved function f(.) (e.g., bounded and
measurable),
– Discrete RVs
– Mixed (or hybrid) RVs
21
EC381/MN308 – 2007/2008
Random Variable
Discrete RV
Continuous RV
22
EC381/MN308 – 2007/2008
Unified View of the Expectation of a Function of an RV
∞
E [g ( X )] =
Both: CDF FX(x) = P [X ≤ x]
1
20
EC381/MN308 – 2007/2008
∫ g ( x )f
X
( x )d x
for both continuous & discrete
−∞
1
If the RV is discrete:
P [xk]
f X ( x ) = ∑ P [ xk ]δ ( x − xk )
k
0
xk
x
0
∞
E [g ( X )] =
x
∫ g ( x )∑ P [ x ]δ ( x − x
k
−∞
Both: PDF fX(x) = (d/dx) FX(x)
k
)d x
k
∞
= ∑ P [ xk ] ∫ g ( x )δ ( x − xk )d x
P [xk]δ(x-xk)
k
−∞
= ∑ P [ xk ]g ( xk )
k
xk
x
fX ( x ) = ∑ P[ xk ]δ ( x − xk )
i.e., E [g ( X )] = ∑ g ( xk )P [ x k ] as it should
x
k
k
EC381/MN308 – 2007/2008
23
EC381/MN308 – 2007/2008
24
4
Example
Mixed RVs
X is discrete RV, with P [X = 1] = 0.6, P [X = 2] = 0.3, P [X = 3] = 0.1
Determine the PDF and second moment E [X 2].
Example (admittedly somewhat artificial):
CDF (use unit step function):
• Experiment:
FX ( x ) = 0.6u ( x − 1) + 0.3u ( x − 2 ) + 0.1u ( x − 3 )
– 2 cashiers. Cashier 1 is “sloppy,” cashier 2 is “meticulous.” Meticulous
one services customers exactly on the minutes.
– Pick cashier 1 with probability 1/3 and cashier 2 with probability 2/3.
– If cashier 1 is picked, generate delay X as an exponential RV with
parameter λ = 0.5.
0
If cashier
hi 2 iis picked,
i k d generate d
delay
l X as geometric
i
RV with parameter p = 0.5.
Differentiate to get PDF (use delta function):
f X ( x ) = 0.6δ ( x − 1) + 0.3δ ( x − 2 ) + 0.1δ ( x − 3 )
Compute E [X2] using old way (PMF):
• What is CDF of X?
– Event A: cashier 1 picked.
– Event B: cashier 2 picked.
– Use Total Probability Theorem.
Compute via integral of PDF using new way:
EC381/MN308 – 2007/2008
25
EC381/MN308 – 2007/2008
26
CDF
PDF
EC381/MN308 – 2007/2008
27
5