IE 303
Discrete-Event Simulation
LECTURE 6: RANDOM NUMBER GENERATION
Review of the Last Lecture
 Continuous Distributions









Uniform distributions
Exponential distributions and “memoryless property”
Gamma Distributions
Erlang Distributions
Weibull Distribution
Normal Distribution
Triangular Distribution
Lognormal Distribution
Truncated Normal Distribution
 Convolution
 Maximum Likelihood Estimation
Outline of Lecture 6
 Random Number Generation in Simulation
 Definition of Pseudo Random Numbers
 Linear Congruent Method
 Random Number Streams
 Test for Random Numbers
 Tests for Uniformity
Kolmogorov, Smirnov Tests
 Chi-square Tests


Test for Statistical Independence from History
Outline
In this lecture we will learn how to generate random numbers and statistical
tests for measuring ‘randomness’.
Why do we learn this??
Suppose you start your IE career at a moderately large manufacturing company and
you are faced with a design of parallel assembly lines problem.
However your company cannot afford buying ARENA or Flexim (commercial license
costs 32500€ per PC, 32000$ for 2 PC). Your boss suggest you two options:
a) You might be allowed for 2 or 3 weeks to build your own program (in Java or
C++) which solves the problem using discrete event simulation.
b) He is ok with buying a cheaper simulation software package which has never
been used by someone else, i.e. unknown reliability of results!
What to do??
Empirical Distribution
5
 An empirical distribution, either discrete or continuous, is a
distribution whose parameters are the observed values in a sample
of data.
 This is in contrast to parametric distribution families such as the
Exponential(λ), Normal(μ,σ) or Poisson(λ), which are characterized
by specifying a small number of parameters such as the mean, the
variance, the rate of activity, etc.
 An empirical distribution may be used when it is impossible or
unnecessary to establish that a random variable has any particular
parametric distribution.
 One advantage of an empirical distribution is that nothing is
assumed beyond the observed values in the sample; however, this is
also a disadvantage because the sample might not cover the entire
range of possible values.
03-Nov-16
Empirical Distribution
6
Example:
Customers at a local
restaurant arrive at
lunchtime in groups
of from one to eight
persons. The number
of persons per party
in the last 300 groups
has been observed;
the results, the
relative frequencies
are summarized in
the tables below. An
histogram and the
empirical cdf are
depicted as well.
03-Nov-16
Empirical Distribution
7
03-Nov-16
Random Number Generation
 Random numbers are a necessary basic ingredient in the
simulation of all discrete systems.
 Most computer languages have a subroutine or function
that will generate a random number.
 Similarly, simulation languages generate random
numbers that are used to generate event times and other
random variables.
 Any random number generation should satisfy two
properties of random numbers:


Uniformity
independence
Random Number Generation
 Random numbers should come from Uniform(0,1)
and each random number should be independent
from its history?
But why????


Heads: Tom -> Harry $1
Tails: Tom <- Harry $1
 Coin tosses should be fair (the
same probability of having heads
or tails) – uniformity.
 New games should be independent
of the previous results – statistical
independence.
30
25
20
15
10
5
0
-5
-10
-15
-20
-25
1
8
15
22
29
36
43
50
57
64
71
78
85
92
99
Lets revisit our coin tossing game
between Harry and Tom. Charlie,
the referee of the game,
successively tosses a fair coin.
Harry's Winnings
Random Number Generation
Pseudo Random Numbers
 Random numbers generated by computers are called pseudo
random numbers.
 “Pseudo” means false, so false random numbers are being
generated!
 In this instance, “pseudo” is used to imply that the very act of
generating random numbers by a known method removes the
potential for true randomness.
 If the method is known, the set of random numbers can be
replicated. But then these numbers are not truly random.
 Whereas in computer simulation of discrete-event systems
replicability is actually a desired feature.
Random Number Generation
 In the generation of pseudo-random numbers certain problems or
errors can occur.
 These errors, or departures from ideal randomness, are all related
to the two properties: Uniformity and independence. Some
examples of which are:




The generated numbers might not be uniformly distributed.
The generated numbers might be discrete-valued instead of continuous-valued.
The mean and/or the variance of the generated numbers might be too high or too
low.
There might be dependence. The following are examples: – autocorrelation
between numbers; – numbers successively higher or lower than adjacent
numbers; – several numbers above the mean followed by several numbers below
the mean.
 There are statistical tests to detect such departures from uniformity
and independence. Once a generator (a method or a routine) is
found to be defective, it will be dropped in favor of an acceptable
generator.
Random Number Generation
 Desired Qualities of a Pseudo-Random Number Generator
 The routine should be fast. A simulation experiment could require
many millions of random numbers. The generator needs to be
computationally efficient.
 The routine should be portable to different programming languages.
It is desirable that the simulation program will produce the same
results wherever it is executed.
 The routine should have a sufficiently long cycle.
 The random numbers should be replicable.
 The generated random numbers should closely approximate the
ideal statistical properties of uniformity and independence.
Linear Congruent Method
 The linear congruent method produces integers, X1, X2,
... between 0 and m – 1 by following a recursive
relationship:
 Random number is
 The initial value X0 is called the seed, a is called the
multiplier, c is called the increment, and m is called the
modulus.
Linear Congruent Method
 The selection of the values for a, c, m and X0 drastically
affects the statistical properties and the cycle length.
 Variations of this method are quite common in the
computer generation of random numbers.
 Example
Use X0 = 27, a = 17, c = 43, and m = 100. Here, the integer
values generated will all be between 0 and 99 because of
the value of the modulus. Also, notice that random integers
are being generated, are discrete rather than continuous.
These random integers should appear to be uniformly
distributed on the integers 0 to 99.
Linear Congruent Method
0.9
Generated random numbers…
There is sth wrong….
0.8
0.7
0.6
0.5
0.4
0.3
a) Theoretically the numbers generated
assume values only from the set
{0, 1/m, 2/m, ... , (m-1)/m}
because each Xi is an integer in the set
{0, 1, 2, ... , m-1}. Hence obtained
random numbers follow a discrete
distribution.
0.2
0.1
0
0
5
10
15
20
0.9
0.8
0.7
0.6
0.5
b) These numbers has a period of 5!
0.4
0.3
0.2
So what???
0.1
0
0
5
10
15
20
Linear Congruent Method
 Discreteness can be overcome if the modulus m is a very
large integer.
 Values such as m=231–1 and m=248 are common use in
generators appearing in many simulation languages.
 It is desirable the values assumed by Ri , i = 1, 2, ..., leave
no large gaps on [0,1].
 Also notice that in practical applications, the generator
should have the largest possible period. Maximal period
should be achieved by the proper choice of a, c, m, and
X0.
Linear Congruent Method
 Three choices:
 For m a power of 2, say m = 2b, and c ≠ 0, the longest possible
period is P = m = 2b, which is achieved whenever c is relatively
prime to m (that is the common factor of c and m is 1) and a =
1 + 4k, where k is an integer.
 For m a power of 2, say m = 2b, and c = 0, the longest possible
period is P = m /4 = 2b–2, which is achieved if the seed X0 is
odd and if the multiplier, a, is given by a = 3 + 8k or a = 5 +
8k, for some k = 0, 1, ...
 For m a prime number and c = 0, the longest possible period is
P=m– 1, which is achieved whenever the multiplier, a, has the
property that the smallest integer k such that ak – 1 is divisible
by m is k=m–1.
Linear Congruent Method
 Example
Find the period of the generator for a = 13,
c=0, m=26 = 64, and X0 = 1, 2, 3, and 4.
The solution is given in the table on the right.
When the seed is 1 or 3, the sequence has
period 16. However, a period of length 8 is
achieved when the seed is 2 and a period
of length 4 occurs when the seed is 4.
In this example the maximal period is given
by the second rule given above, m = 24,
and c = 0, the longest possible period is P
= 64 /4 = 16. As mentioned before, the
maximal period is achieved when the seed
is odd and a has the form given by 5 + 8k.
When the seed is 1, the density is
4/64=0.0625.
Note that modulus operation is very efficient
when the modulo is a power of 2 which is
another consideration when selecting a, c,
and m.
Linear Congruent Method
 Example
Different values have been extensively tested. The values for a, c,
and m have been selected to ensure that the characteristics
desired in a generator are most likely to be achieved. By
changing X0, the user can control the repeatibility of the stream.
Let a = 75 = 16807, m = 231 –1 = 2,147,483,647 (a prime number),
and c = 0. These choices satisfy the conditions that insure a
period of P = m – 1, (well over 2 billion). Further, specify the
seed X0 = 123,457. The first few numbers generated are as
follows:
Random Number Streams
 The seed for a linear congruential random number
generator is the integer value X0 that initializes the
random-number sequence. Since the sequence of
integers X0, X1, X2, ... , XP ,produced by a generator
repeats, any value in the sequence could be used to
“seed” the generator.
 For a linear congruential generator, a random
number stream is nothing more than a convenient
way to refer to starting seed taken from the
sequence X0, X1, X2, ... , XP.
Tests for Random Numbers
 Generated numbers should be tested if they satisfy
the two property of random numbers:


Uniformity
Independence
 To this end, we use hypothesis tests. Uniformity is
tested with


Kolmogorov Smirnov
Chi-square test
Whereas independence is tested with

Autocorrelation test
Tests for Random Numbers
Hypothesis statements in these tests are
Uniformity
Independence
The level of significance denoted by α satisfies:
α= P{reject H0 | H0 is true}
which means that the critical value for the test statistic is
selected such that the probability of rejecting the null
hypothesis, H0, equals α.
Tests for Random Numbers
 For a reliable decision on the acceptance of a random
number generator, statistical tests must be done on a
large number of samples, say M.
 If the number of rejected samples significantly
exceeds the expected number of rejected tests, aM,
then we shall discard the given random number
generator as the random numbers generated by it
do not possess the property which we are testing
for.
Kolmogorov Smirnov Test
Kolmogorov Smirnov Test
END OF LECTURE 6
Next Lecture: Statistical Tests (Chp. 7),
@LMF 208 (Computer Lab)
Kolmogorov Smirnov Test
Kolmogorov Smirnov Test
Chi-Square Test
Chi-Square Test
Random Number Streams