New Randomness Tests Using Random Walk
Ali Doğanaksoy, Çağdaş Çalık, Fatih Sulak, Meltem Sönmez Turan
Institute of Applied Mathematics,
Middle East Technical University, Ankara, Turkey,
{aldoks, e110870, e112437, msonmez}@metu.edu.tr
Abstract. In testing the security of cryptographic primitives, statistical
randomness tests play an important role. In this study, we concentrated
on statistical properties of one dimensional random walks. The probabilities related to excursions, height and expansion of random walks are
given and based on these probabilities, three new randomness tests are
proposed.
Keywords: Random Walk, Randomness tests.
1
Introduction
Statistical randomness tests play an important role in testing the security of
cryptographic primitives especially synchronous stream ciphers and block ciphers. Various independent statistical test suites are available in the literature.
Knuth [1] presented several empirical tests including frequency, serial, gap, poker,
coupon collector’s, permutation, run, maximum-of-t, collision, birthday spacings
and serial correlation. Another test suite DIEHARD [2] consists of 18 different,
independent statistical tests. Also, Crypt-XS suite [3] which was developed in the
Information Security Research Centre at Queensland University of Technology
consists of frequency, binary derivative, change point, runs, sequence complexity
and linear complexity tests. Lastly, NIST [4] statistical test suite that has been
used as an evaluation tool in the AES selection process consists of 16 tests.
Typically, a statistical randomness test is defined by a test statistic whose
distribution is either known, or well approximated. Then the randomness property of a given sequence is evaluated using these theoretical distributions by
hypothesis testing. These tests return a p-value ranging from 0 to 1. It is conventionally assumed that the p-values less than 0.01 are obtained from non-random
resources.
In this study, we developed three new randomness test based on random walk
which is widely used in economics, physics and mathematics. The definition of a
random walk can be given as a stochastic process that consists of a sequence of
changes each of whose magnitude or direction is determined by chance. By using
random walks, it is possible to analyze a binary sequences deeply. There are
some randomness tests available in the literature [4] based on random walks and
most of these tests are applied to long sequences since approximate distributions
are used while evaluating. In this study, we calculate exact probability values
for fixed length sequences. Therefore, it is convenient to evaluate randomness of
short sequences by these tests.
In the following chapter, a preliminary information is available. In section 3,
calculations of the necessary probabilities are given. In section 4, the randomness
tests are defined and finally in the last section discussion and conclusion is given.
2
Preliminaries
Consider a particle on the real line located at a point k ∈ Z. At each step, this
particle moves either to k+1 or k−1 by equal probabilities. The motion described
by this particle is known as a one dimensional random walk. This concept can
be generalized by various ways: A motion in 2, 3 or higher dimension, a motion
concerning different probabilities to different directions, etc. In this work we
consider only the one dimensional random walk with equal probabilities.
Consider a binary sequence {an } = a1 , a2 , . . . aN and let bi = 1 − 2ai with
bi ∈ {−1, 1}, for i = 1, 2, . . . , N . It is obvious that {bn } describes a walk uniquely
and such a walk is random if P r(ai = 0) = P r(ai = 1) = 1/2.
A walk can be represented in the plane as follows: bn is identified with the
point (xn , yn ) where xn = n, yn = yn−1 + bn , for n ≥ 1 and x0 = y0 = 0.
Then the path (x0 , y0 ), (x1 , y1 ), . . . , (xn , yn ) represents the walk defined by the
sequence {bn }.
PnIt is seen that the sequence yn is in fact the sequence of partial sums sn =
i=1 bi of bi . The points at which the path intersects the x-axis correspond to
sn = 0. The part of the sequence between two such successive points is referred
to an excursion. Length of an excursion starting at bi and ending at bj is j − i.
Note that in such a case, si = sj = 0 and sk 6= 0 for i < k < j.
The maximum value of |sn | is defined as the height of a random walk and the
expansion of a random walk is defined as the difference between the maximum
and the minimum value of sn .
3
Calculations on Random Walk
The probability distribution for the length of the random excursions, P (l), is
given by
P (l) =
Ck−1
22k−1
0
if l = 2k
otherwise
1 2i
where Ci is the Catalan number defined as i+1
i .
Let Pk be the probability of a random excursion to be of length greater than
k. Obviously,
k
X
P2k = 1 −
P (2i)
i=1
and P2k = P2k+1 .
Given a sequence of even length N , to compute the probability P (N, k) of
having exactly k excursions in the sequence, two cases are considered:
– The sequence is balanced. This means that the sequence consists of k complete excursions. If N/2 = p1 + p2 + . . . + pk is a partition of N/2 into k
positive integers p1 , . . . , pk we have N = 2p1 +2p2 +. . .+2pk . Then the probability of having k successive excursions of respective lengths 2p1 , . . . , 2pk is
P (2p1 ).P P
(2p2 ) . . . P (2pk ). Consequently, the required probability is given by
the sum
P (2p1 ) ∧ P (2pk ) where the summation ranges over all ordered
partitions (p1 , . . . , pk ) of N/2 into k positive integers.
– The sequence is not balanced. This means that the sequence consists of k − 1
(not excluding the possibility that k − 1 = 0) complete excursions of total
length N − 2m and an incomplete (last) one of length 2m (m > 0). In this
PN/2
P
case the required probability is m=1 PN −m P (2p1 )∧P (2pk−1 ) where the
inner summation runs over all ordered partitions p1 , . . . , pk−1 of N/2 − m
into k − 1 positive integers.
Given a random walk of length B. Let n be the weight of the sequence and
let m = B − n. We represent this walk in a m × n grid as a path starting from
the bottom left corner, (stepping one unit up for a 0 term and stepping one
unit right for a 1 in the sequence) and ending at the top right corner. A path
which intersects the inclined line shown in the figure corresponds to a random
walk with Smax ≥ r. The number of such paths is known to be equal to m+n
m−r ,
provided r > |m − n|.
Let us investigate a more general case. We want to find the number of paths
starting from A, finishing at B and not cutting the two given lines, that is a
random walk with m ones and n zeros and taking values between [−s, r]. To find
this number we employ the principle of inclusion-exclusion. The required number
can be found by subtracting the paths cutting the above and below lines from all
paths and adding the paths cutting both above and below lines and continuing
like this. Thus the probability, P r(max < r, min < s|m, n) is
P∞ m+n
m+n
m+n
m+n
m+n
− i=1 m−ir−(i−1)s
+ n−(i−1)r−is
− m−ir−is
− n−ir−is
m
2m+n
for r, s > |m − n| and 0, otherwise.
If r = s, then
(
P r(max < r|m, n) =
(m+n
m )−
P∞
i=1
m+n
m+n
{[(m−ir
)+(n−ir
)](−1)i+1 }
2m+n
0
if r > |m − n|,
otherwise.
For a fixed block length B = m + n, to define the Random Walk Height
and Random Walk Expansion tests, we calculate the above probabilities for all
possible values of (m, n) and (r, s), where r + s represents the expansion.
4
Test Descriptions
We define three tests to decide the randomness of a sequence considering statistical distributions based on certain properties of the random walk; (i) Random
Walk Excursion Test, (ii) Random Walk Height Test, (iii) Random Walk Expansion Test. As their names suggest, these tests consider the number of excursions,
the height and the expansion of the sequence.
To test a binary sequence {an } = a1 , a2 , . . . , aN for randomness, we first
convert {an } into the sequence {bn } by setting bi = 1 − 2i so that bi ∈ {−1, 1},
i = 1, 2, . . . , N . To apply a test to {bn } we partition the sequence {bn } into subsequences of a certain length (B = 64, 128,
. . . , etc). Then to each subsequence
we assign a test value ej , (j = 1, 2, . . . , N
B ). The pseudocode of the tests are
given in Algorithm 4.1, 4.2, 4.3.
Algorithm 4.1: Random Walk Excursion Test(B, an )
for i ← 1 to N
do
bi = 1 − 2ai ; for j ← 1 to N
B
do
ej = 0;




for i ← 1 to B




do


Pi

si = k=1 b(j−1)B+k ;

for i ← 1 to B





do



if si = 0


then ej = ej + 1;
Apply χ2 of Goodness of F it test to ej values;
return (p − value)
Algorithm 4.2: Random Walk Height Test(B, an )
for i ← 1 to N
do
bi = 1 − 2ai ; for j ← 1 to N
B
do
ej = 0;




for i ← 1 to B




do


Pi

si = k=1 b(j−1)B+k ;

for i ← 1 to B





do



if |si | > ej


then ej = |si |;
Apply χ2 of Goodness of F it test to ej values;
return (p − value)
Algorithm 4.3: Random Walk Expansion Test(B, an )
for i ← 1 to N
do
bi = 1 − 2ai ; for j ← 1 to N
B
do

ej = 0, max = 0, min = 0;




for i ← 1 to B





 do Pi



 si = k=1 b(j−1)B+k ;


for i ← 1 to B
do



if si > max






then max = si ;




if
− si > min







then
min = −si ;


ej = max + min;
Apply χ2 of Goodness of F it test to ej values;
return (p − value)
5
Discussion and Conclusion
In this study, we proposed three new randomness tests based on one dimensional random walks namely, (i) Random Walk Excursion Test, (ii) Random
Walk Height Test, (iii) Random Walk Expansion Test. Firstly, we calculated the
exact probability distributions for length of excursion, heights and expansion
of random walks. Then, to apply χ2 Goodness of Fit test, we generated boxes
with approximately equal probabilities for block lengths of 16, 32, 64, 128 and
256 bits. These probabilities are given in the Tables 1, 2, 3.
The apparent benefit of using the proposed tests rises from the fact that
no approximations have been used. Therefore, they can be used to test short
sequences. The relation between these tests can be summarized as following.
The sequences that pass from excursion test may fail from the height and expansion tests, therefore excursion test do not dominate the others. Also, the
sequences with too much or too low number of excursions, it is likely that they
fail from height and expansion tests. As a future work, we plan to concentrate
on correlation between these tests by making some experiments.
References
1. D. E. Knuth. Seminumerical Algorithms, volume 2 of The Art of Computer Programming. Addison-Wesley, 1981.
2. G. Marsaglia. DIEHARD Statistical Tests. http://stat.fsu.edu/ geo/diehard.html.
3. Information
Security
Institute.
Crypt-X,
1998.
http://www.isi.qut.edu.au/resources/cryptx/.
4. A. Rukhin, J. Soto, J. Nechvatal, M. Smid, E. Barker, S. Leigh, M. Levenson,
M. Vangel, D. Banks, A. Heckert, J. Dray, and S. Vo. A statistical test suite for
random and pseudorandom number generators for cryptographic applications. 2001.
http://www.nist.gov.
6
Appendix
In the following tables, necessary probability and interval values for the randomness tests are presented. These intervals are determined in a manner such that
each interval has approximately equal probability.
Box
Box
Box
Box
Box
B=16
B=32
B=64
B=128
B=256
Interval Pr. Interval Pr. Interval Pr. Interval Pr. Interval Pr.
1
0
0.196 0-1 0.279 0-1 0.198 0-2 0.210 0-3 0.198
2
1
0.196 2-3 0.261 2-3 0.192 3-5 0.200 4-7 0.189
3
2
0.183 4-5 0.210 4-5 0.173 6-8 0.177 8-12 0.206
4
3
0.157 6-16 0.247 6-8 0.206 9-12 0.184 13-19 0.211
5 4-8 0.266
9-32 0.228 13-64 0.227 20-128 0.193
Table 1. Interval and probability values of Random Walk Excursion Test for
block lengths of 16,32, 64, 128 and 256 bits.
Box
Box
Box
Box
Box
B=64
B=128
B=256
B=512
B=1024
Interval Pr. Interval Pr. Interval Pr. Interval Pr. Interval Pr.
1 0-6 0.252 0-8 0.180 0-12 0.196 0-17 0.180 0-25 0.196
2 7-8 0.227 9-11 0.241 13-16 0.230 18-22 0.205 26-32 0.203
3 9-10 0.182 12-14 0.209 17-20 0.194 23-28 0.214 33-40 0.200
4 11-13 0.173 15-18 0.183 21-26 0.195 29-36 0.195 41-52 0.204
5 14-64 0.163 19-128 0.185 27-256 0.232 37-512 0.203 53-1024 0.195
Table 2. Interval and probability values of Random Walk Height Test for block
lengths of 64,128, 256, 512 and 1024 bits.
Box
Box
Box
Box
Box
B=32
B=64
B=128
Interval Pr. Interval Pr. Interval Pr.
1 0-6 0.307 0-8 0.188 0-12 0.196
2
7
0.166 9-10 0.234 13-14 0.165
3 8-9 0.266 11-12 0.212 15-17 0.235
4 10-32 0.259 13-15 0.206 18-21 0.214
5
16-64 0.158 22-128 0.187
Table 3. Interval and probability values of Random Walk Expansion Test for
block lengths of 32, 64 and 128 bits.