Random Number Generators
Why do we need random variables?
•random components in simulation
→ need for a method which generates numbers that are random
•examples
–interarrival times
–service times
–demand sizes
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
2
terminology
•Random Numbers
–stochastic variable that meets certain conditions
–numbers randomly drawn from some known distribution
–It is impossible to “generate” them with the help of a digital computer.
•Pseudo-random Numbers
–numbers which seem to be randomly drawn from some known
distribution
–may be generated with the help of a digital computer
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
3
techniques for generating random variables
–ten-sided die (each throw generates a decimal)
–throwing a coin n times
•get a binary number between 0 and 2^n-1
–other physical devices
–observe substances undergoing atomic decay
•e.g., Caesium-137, Krypton-85,. . .
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
4
techniques for generating pseudo-random variables
•numbers are generated using a recursive formula
–ri is a function of ri-1, ri-2, …
–relation is deterministic – no random numbers
–if the mathematical relation not known and well chosen it is practically
impossible to predict the next number
•In most cases we want the generated random numbers to simulate a
uniform distribution over (0, 1), that is a U(0, 1)-distribution
–there’re many simple techniques to transform uniform (U(a, b))
samples into samples from other well-known distributions
1
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
0
1
0
1
5
terminology
•seed
–random number generators are usually initialized with a starting
number which is called the “seed”
–different seeds generates different streams of pseudorandom numbers
–the same seed results in the same stream
•cycle length
–length of the stream of pseudorandom numbers without repetition
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
6
methods for generating pseudo-random variables
•all methods usually generate uniformly distributed pseudorandom
numbers in the interval [0,1]
–midsquare method (outdated)
–congruential method (popular)
–shift register (Tausworthe generator)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
7
midsquare method
•arithmetic generator
–proposed by von Neumann and Metropolis in 1940s
•algorithm
–start with an initial number Z0 with m digits (seed) [m even]
–square it and get a number with 2m digits (add a zero at the beginning
if the number has only 2m-1 digits)
–obtain Z1 by taking the middle m digits
–to be in the interval [0,1) divide the Zi by 10m
–etc….
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
8
midsquare method: example
i
0
1
2
3
4
5
6
7
…
Zi
7182
5811
7677
9363
6657
3156
9603
2176
…
Ui
0.7182
0.5811
0.7677
0.9363
0.6657
0.3156
0.9603
0.2176
…
(Z0 = 7182, m = 4)
Zi2
51581124
33767721
58936329
87665769
44315649
9960336
92217609
4734976
…
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
9
midsquare method (cont.)
•advantages
–fairly simple
•disadvantage
–tends to degenerate fairly rapidly to zero
•example: try Z0 = 1009
–not random as predictable
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
10
Linear Congruential Generator (LCG)
•LCGs
–first proposed by Lehmer (1951).
–produce pseudorandom numbers such that each single number
determines its successor by means of a linear function followed by a
modular reduction
–to be in the interval [0,1) divide the Zi by m
–Z0
seed
a
multiplier (integer)
–c
increment (integer)
m
modulus (integer)
•Variations are possible
–combinations of previous numbers instead of using only the last value
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
11
a=5
c=3
m = 16
Z0 = 7
LCG: example
i
0
1
2
3
4
5
6
7
…
Zi
7
6
1
8
11
10
5
12
…
Ui
0.4375
0.375
0.0625
0.5
0.6875
0.625
0.3125
0.75
…
Z1 = (5*Z0 + 3) mod 16 = 38 mod 16 = 6
Z2 = (5*Z1 + 3) mod 16 = 33 mod 16 = 1
Z3 = (5*Z2 + 3) mod 16 = 8 mod 16 = 8
…
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
12
properties of LCGs
•parameters need to be chosen carefully
–nonnegative
–0 < m
a<m
c<m
Z0 < m
•disadvantages
–not random
–if a and m are properly chosen, the Uis will “look like” they are
randomly and uniformly distributed between 0 and 1.
–can only take rational values 0, 1/m, 2/m, … (m-1)/m
–looping
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
13
LCG (cont.)
•looping
–whenever Zi takes on a value it has had previously, exactly the same
sequence of values is generated, and this cycle repeats itself endlessly.
–length of sequence = period of generator
–period can be at most m
•if so: LCG has full period
–LCG has full period iff
•the only positive integer that divides both m and c is 1
•if q is a prime number that divides m then q divides a-1
•if 4 divides m, then 4 divides a-1
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
14
Linear Feedback Shift Register Generators (LFSR)
•developed by Tausworthe (1965)
–related to cryptographic methods
–operate directly on bits to form random numbers
•linear combination of the last q bits
–ci
constants (equal to 0 or 1, cq = 1)
–in most applications only two of the cj coefficients are nonzero
–execution can be sped up
•modulo 2 equivalent to exclusive-or
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
15
LFSR (cont.)
•form a sequence of binary integers W1, W2, …
–string together l consecutive bits
–consider them as numbers in base 2
•transform them into Uis
•maximum period
2q -1
–if l is relatively prime to 2q-1 LFSR has full period
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
16
LFSR : example
•r = 3
q=5
b1 = b 2 = b 3 = b 4 = b 5 = 1
l=4
I
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
15
17
18
bi
1
1
1
1
1
0
0
0
1
1
0
1
1
1
0
1
0
1
i
1
2
3
Wi (base 2)
1111
1000
1101
Wi (base 10)
15
8
13
Ui
0.9375
0.5
0.8125
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
17
Testing RNGs
Empirical Tests
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
18
random number generators
•methods presented so far…
–completely deterministic
–Uis appear as if they were U(0,1)
•test their actual quality
–how well do the generated Uis resemble values of true IID U(0,1)
•different kinds of tests
–empirical tests
•based on actual Uis produced by RNG
–theoretical tests
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
19
Chi-Square test (with all parameters known)
•checks whether the Uis appear to be IID U(0,1)
–divide [0,1] into k subintervals of equal length (k > 100)
–generate n random variables: U1, U2, .. Un
–calculate fj
(number of Uis that fall into jth subinterval)
–calculate test statistic
–for large n χ2 will have an approximate chi-square distribution with k-1
df under the null hypothesis that the Uis are IID U(0,1)
–reject null hypothesis at level ® if
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
20
Generating Random Variates
Generating Random Variates
•General Approach
–Inverse Transformation
–Composition
–Convolution
•Generating Continuous Random Variates
–Uniform U(a,b)
–Exponential
–m-Erlang
–Gamma, Weibull
–Normal
–etc…
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
22
Inverse Transformation
•generate continuous random variate X
–distribution function F (continuous, strictly increasing)
•0 < F(x) < 1
•if x1 < x2 then 0 < F(x1) · F(x2) < 1
–inverse of F: F-1
•algorithm
–generate U ~ U(0,1)
–return X = F-1(U)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
23
Inverse Transformation (cont.)
F(x)
1
U1
U2
X2
0
x
X1
•returned value X has desired distribution F
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
24
Inverse Transformation (cont.)
•create X according to exponential distribution with mean ¯
•in order to find F-1 set
take natural
logarithm (base e)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
25
Inverse Transformation for discrete variates
•distribution function
•probability mass function
•algorithm
–generate U ~ U(0,1)
–determine smallest possible integer i such that U · F(xi)
–return X = xi
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
26
Composition
•applies then the distribution function F can be expressed as a
convex combination of other distribution functions F1, F2, ..
–we hope to be able to sample from the Fj’s more easily than from the
original F
–if X has a density it can be written as
•algorithm
–generate positive random integer J such that P(J = j) = pj
–Return X with distribution function FJ
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
27
Composition (example)
•double-exponential distribution (laplace distribution)
–can be rewritten as
–where IA(x) is the indicator function of set A
•f(x) is a convex combination of
–f1(x) = ex I(-1, 0) and f2(x) = e-x I[0, 1)
–p1 = p2 = 0.5
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
28
Convolution
•desired random variable X can be expressed as sum of other
random variables that are IID and can be generated more readily
then X directly
•algorithm
–generate Y1, Y2, … Ym IID each with distribution function G
–return X = Y1 + Y2 +  + Ym
•example: m-Erlang random variable X with mean ¯
–sum of m IID exponential random variables with common mean ¯/m
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
29
Generate Continuous Variates
•Uniform distribution X(a,b)
X = a + (b-a)U
•Exponential (mean ¯)
X = - ¯ ln (1-U)
or
X = -¯ ln U
•m-Erlang
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
30