1
Transformations of One
Random Variable: Mapping
of Probability Distributions
There are several ways obtain basic information about a transformation: examine the
mathematics, or look at a sketch, and carefully determine the endpoints. The moments,
mean,variance, standard deviation, also provide
We will examine the impact of the ZMNL (zero- useful information.
memory non-linear transformation) on the disAgain, given the y = g(x), determine what
tribution of a continuous RV x mapping to a
values y can take on, given g(x) and the pdf
continuous RV y. To define notation, let the
f (x). When we need a specific input distribuZMNL be the function g(x)
tion, assume the exponential model for x holds.
y = g(x)
(1)
The PDF Method. First, take the case
where
y = g(x) is an increasing continuous funcg(x) = Ax + B, g(x) = Ax2 , g(x) = 1/x, . . .
tion. Then, it has an inverse, call it h(y) =
(2) −1
g (y). To find the impact of the mapping g
These will correspond to some useful mapping of
on the probability, note that when y = g(x)
the RV. For example, we may linearly change the
units (sec to msec), or non-linearly map power
p(y)dy = P [y ∈ (y, y + dy]]
(10)
to dB, g(x) = 10 log10 x.
Typical functions may be
= P [x ∈ (x, x + dx]] = f (x)dx
(11)
To establish notation, let the input RV x have
(pdf, cdf, edf) = (f (x), F (x), F (x)). The output Equivalently, with g 0 (x) = dy/dx,
RV y has (pdf, cdf, edf) = (p(x), P (x), P (x)).
In this way, I avoid having to typeset subscripts,
f (x) such as F (x) = Fx (x), etc.
p(y) = 0
(12)
g (x) x=h(y)
The prototypical, but not only, example will
be the exponential x ∼ EXP(1/a) on x ∈ [0, ∞),
Now if g is decreasing, instead of increasing,
f (x)dx := P [x ∈ (x, x + dx]]
(3) some thought shows that the result is (with the
important absolute values taken)
f (x) = a exp(−ax), x ≥ 0
(4)
F (x) := P [x ≤ x]
F (x)
=
1 − exp(−ax), x ≥ 0
F (x) := P [x > x]
F (x)
=
exp(−ax), x ≥ 0
(5)
(6)
(7)
p(y)|dy| = P [y ∈ (y, y + dy]]
(13)
= P [x ∈ (x, x + dx]] = f (x)|dx|
(14)
f (x) p(y) = 0
|g (x)| (8)
(15)
x=h(y)
You can sketch these exponential functions.
This is the key result of the pdf method.
The mean value, or location, 1/a,
Z
The use of h(x) assumes that a single value of
1
E[x] = xf (x)dx =
(9) x gives rise to a single value of y, but we may
a
1
also have a multiple values of x giving rise to a
The Conditional CDF Method. Many
single y,
times g(x) is defined using different mathematical expressions over different domains. An exFor example, y = x2
ample might be
Then we need to sum over all the ways in which
such a y can arise, since y = g(x) has multiple
solutions, x1 , x2 , . . .. When y = x2 , both x1 =
g(x) = |x|2 , |x| ≤ A
(21)
√
√
+ y and x2 = − y are solutions.
2
g(x) = C = A , |x| > A
(22)
p(y)|dy| = P [y ∈ (y, y + dy]]
=
X
P [x ∈ (xk , xk + dxk ]]
More generally let,
(16)
g(x) = gk (x), when x ∈ Ik
(17)
(23)
k
where ∪Ik = (−∞, +∞) is a partition (disjoint
This is the general result, needed when g is intervals Ik ). For each x ∈ Ik we have a single
many-to-one. Note that in the equation above expression for g = gk , Recall that, in general for
event A = {x ∈ A}, A ∈ R = {x ∈ (−∞, +∞)}
y = g(x).
and partition Bk = {x ∈ Bk }, ∪k Bk = R
P [A] = P [A ∩ B] =
X
P [A ∩ Bk ]
(24)
The CDF Method. The ”pdf-method” can
k
be tedious to apply to obtain p(y). We need to
compute g 0 (x)|x=h(y) . Instead, we can use the Now each term is simpler because of the extra
condition; that the input RV x is contained in a
”cdf method.” With RVs x, y, y = g(x),
interval where g(x) is described by the simpler
0
P [y ≤ y] = P [g(x) ≤ y]
(18) gk (x). The simpler gk have derivatives gk and
inverses hk .
When the inverse exists, and g is increasing,
The final result can also be described in terms
of conditional probabilities,
P (y) = P [y ≤ y] = P [x ≤ h(y)] = F (h(y))
(19)
To find the accompanying pdf, differentiate using
the Leibniz rule (the chain rule)
P [A] = P [A ∩ B] =
P [A|Bk ]P [Bk ]
(25)
k
1.1
d
p(y) =
P (y) = h0 (y)f (h(y))
dy
X
(20)
Examples
Example 1. Take the specific case, y = g(x) =
ex . This is positive and increasing and for x ≥ 0,
. This is simpler when the inverse is easy to y ≥ 1 The inverse to g is x = log(y), and g 0 (x) =
obtain, and directly yields probabilities, that is, ex . Putting it together,
the cdf P (y). The CDF builds probabilistic in
p(y) = f (x)e−x x=log(y)
(26)
tuition, and is often faster than the pdf-method.
2
or
The EDF (EDF = CCDF := 1-CDF) is
p(y) = ay −a−1 , y ∈ [1, ∞)
(27)
P (y) = exp(−ay b ), y > 0
(31)
This is a power-law pdf on [1, ∞). Check that p
integrates to 1. The pdf of power-laws decrease
The Weibull pdf arises in life testing, or reliabilmuch slower than exponentials. That should be
ity.
clear, since y = ex increases rapidly, and so y =
In communications, the most common case
ex is much larger than x.
takes b = 2, the Rayleigh Distribution, often with
variable r,
Example 2. What about the case y = g(x) =
r
−x? Here, |g 0 | = 1, but h(y) = −x. This flips
p(r) = 2 exp(−r2 /2σ 2 ), r ≥ 0
(32)
σ
the domain of the RV about the axis. Plugging
into the formula gives
The CDF of the Rayleigh can be found in closed
p(y) = f (−y), y ≤ 0
(28) -form,
2
2
P (r) = e−r /2σ
(33)
which is an exponential on the negative axis.
Here I have used r = y and σ 2 = 1/a.
Recall that a function (here a pdf), p(y) is
symmetric about y = 0 when
Example 4.
The Inverse CDF Transp(y) = p(−y)
form Next, we consider what appears to be a
Since the left hand side is the pdf of RV y, the strange choice of g, but is actually quite comright hand side is the pdf of RV −y. We get the mon. Later, we will also specialize to the case of
result that if p(y) is symmetrical about y = 0, x ∼ UNIF[0, 1]. First, let’s examine what hapthen
pens when a RV x with pdf f (x) is transformed
y ∼ −y
by its own CDF,
The ∼ reads: is distributed as.
g(x) = F (x).
(34)
Example 3. A ZMNL (zero-memory transfor- Clearly, y = g(x) is bounded, y ∈ [0, 1]. Then,
mation) creates new pdfs. For example, against by the CDF method,
starting with the now unit mean exponential
P [y ≤ y] = P [g(x) = F (x) ≤ y]
(35)
x ∼ EXP(1), f (x) = e−x U (x)
(29)
Solving, since the inverse exists,
Consider ZMNL g with inverse h, g(x) =
P [y ≤ y] = P [x ≤ F −1 (y)]
(36)
(x/a)1/b , h(y) = axb . Since x > 0, y > 0, and
we take a, b > 0. Then, we obtain the Weibull
Of course, we have the CDF of x, therefore,
Distribution
p(y) = aby b−1 exp(−ay b ), y > 0
P [y ≤ y] = F (F −1 (y)) = y
(30)
3
(37)
We need to recognize the result, if the CDF of
a RV is linear, the PDF is constant, and the RV
is uniformly distributed.
If x has CDF F , the y = F (x) is UNIF(0, 1).
But this relationship can be used in either direction!
If RV y is UNIF (0, 1), the RV x = F −1 (y)
has CDF F (x).
This is important, it tells us how to simulate (or generate synthetically), iid (independent and identically distributed) random samples from distribution F . Simply put iid uniform
samples into the transformation,
x = F −1 (u)
(38)
where u is UNIF(0, 1), generated in MATLAB
by
>> u = RAND(L, 1); %Lx1vector
(39)
To apply this to the exponential distribution,
take
1
x = g(u) = − log(1 − u)
(40)
a
This can be simplified slightly, since u ∼ 1 − u
to
1
x = g(u) = − log(u)
a
(41)
Here u is UNIF(0, 1), and x has PDF
ae−ax , x > 0
4