Simulation of Random Variables


In many fields of science and engineering, computer simulation is used to study
random phenomena in nature and the performance of an engineering system in a
noisy environment. For example, we may study through computer simulation the
performance of a communication receiver. Sometimes a probability model may
not be analytically tractable and computer simulation is used to calculate
probabilities.
The heart of all these applications is that it is possible to simulate a random
variable with an empirical CDF or PDF that fits well with the theoretical CDF or
PDF.
Generation of Random Numbers
Generation of random numbers means producing a sequence of independent random
numbers with a specified CDF or PDF. All the random number generators rely on a
routine to generate random numbers with the uniform PDF. Such routine is of vital
importance because the quality of the generated random numbers with any other
distribution depends on it. By the quality of the generated random numbers, we mean
how closely the empirical CDF or PDF fits the true one.
There are several algorithms to generate U [0 1] random numbers. Note that these
algorithms generate random number by a reproducible deterministic method. These
numbers are pseudo random numbers because they are reproducible and the same
sequence of numbers repeats after some period of count specific to the generating
algorithm. This period is very high and a finite sample of data within the period appears
to be uniformly distributed. We will not discuss about these algorithms. Software
packages provide routines to generate such numbers.
Method of Inverse transform
Suppose we want to generate a random variable X with a prescribed distribution
function FX ( x). We have observed that the random variable Y , defined by
Y  FX ( X ) ~ U [ 0, 1 ].. Thus given U [ 0, 1 ]. random number Y ,
the inverse transform X  FX1 (Y ) will have the CDF FX ( x).
The algorithmic steps for the inverse transform method are as follows:
1. Generate a random number from Y ~ U [0, 1]. Call it y. .
2. Compute the value x such that FX ( x) =y.
3. Take x to be the random number generated.
Example Suppose, we want to generate a random variable with the PDF f X ( x) given
by
2
0 x3
 x
f X ( x)   9
0
otherwise
The CDF of X is given by
0
1

FX ( x)   x 2
9
1
x<0
0 x3
x>3
FX(x)
1
x2 /9
x
3
Therefore, we generate a random number y from the U [0, 1] distribution and set
FX ( x)  y.
We have
1 2
x y
9
 x= 9y
Example Suppose, we want to generate a random variable with the exponential
distribution given by f X ( x)   e  x   0, x  0. Then
FX ( x)  1  e  x
Therefore given y, we can get x by the mapping
1  e  x  y
x  
log e 1  y 

Since 1  y is also uniformly distributed over [0, 1], the above expression can be
simplified as,
log y
x  

Generation of Gaussian random numbers
Generation of discrete random variables
We observed that the CDF of a discrete random variable is also U [0, 1] distributed.
Suppose X is a discrete random variable with the probability mass function
pX ( xi ), i  1, 2,..n. Given Y  FX ( x), the inverse mapping is defined as shown in the
Fig. below.
Y  FX ( X )
y
xk  FX1 ( y )
X
Thus if FX ( xk 1 )  y  FX ( xk ), then
FX1 ( y )  xk
The algorithmic steps for the inverse transform method for simulating discrete random
variables are as follows:
1. Generate a random number from Y ~ U [0, 1]. Call it y.
2. Compute the value xk such that FX ( xk 1 )  y  FX ( xk ),
3. Take xk to be the random number generated.
Example
Generation of Bernoulli random numbers
Suppose we want to generate a random number from X ~ Br ( p ). Generate y
from the U [0, 1] distribution. Set
0 for y  1  p
x
1 otherwise
y
1 p
0
1
x
Method of Rejection
Inverting the distribution function is a difficult task unless we have a closed-form
mathematical expression. This makes it difficult to generate random numbers using the
inversion method. The rejection method can be used in such a situation.
Suppose we want to generate random variable Y with the PDF fY ( y ). To use the
rejection method, we have to find another random variable X with the PDF f X ( x) and
a constant c such that
f X ( x)
c0
fY ( x )
x  X .
Before giving a justification for the method, let us state the steps of the rejection method
Steps of the Rejection method
Step 1
Generate number x with the PDF f X ( x) and a random number u from U ~ U [0 1].
Step 2
If u 
cfY ( x)
output Y  x . Else go to step 1.
f X ( x)
We now prove that the random variable Ygenerated by the rejection method has the PDF
fY ( y ).
P({ X  x}/{U 
cfY ( X )
}) 
fX (X )
cfY ( X )
})
fX (X )
cf ( X )
P({U  Y
})
fX (X )
P({ X  x,U 
cfY ( X )
})
fX (X )
cf ( X )
P({U  Y
})
fX (X )
P({ X  x,U 

Now
P({ X  x,U 
x cf (u )
x
cfY ( X )
})   Y
f X (u )du  c  fY (u )du
 f (u )

fX (X )
X
and
P({U 
 cf ( x )

cfY ( X )
})   Y
f X ( x)dx  c  fY ( x)dx  c
 f ( x )

fX (X )
X
 P({ X  x}/{U 
x
cfY ( X )
})   fY (u )du

fX (X )
Example Suppose we want to apply the Rejection method to simulate the random
variable Y with
3 2
 y
fY ( y )   8
0
0 y2
otherwise
We can choose X ~ U [0, 2] and c  3 so that
fY ( y )
3
f X ( y)
0 y2
We can now apply the rejection method to simulate Y .
Testing the closeness of the random data to a theoretical distribution
Once we have the generated random data, we have to test how closely they fit the
theoretical distribution. We can do so by comparing the frequencies observed in the
random data to the expected frequencies of the theoretical distribution.
Histogram of data
The histogram is a graphical plot used to describe the behaviour of random data. The
range of the data is split into equal-sized bins called class intervals and the number of
data that fall into each bin (frequency) is counted. The histogram is obtained by plotting
the relative frequency (frequency normalized by the total number of data points) versus
the class interval. Thus the histogram has


Horizontal axis: Class intervals
Vertical axis: Relative frequency
Fig. shows the true PDF of a random variable X ~ N (0,1) and the frequency plot of the
generated N (0,1) random numbers. The plot shows that the relative frequency plot
approximately matches the Gaussian PDF.