JOURNAL OF MATHEMATICAL PHYSICS
VOLUME 40, NUMBER 6
JUNE 1999
Random walk with memory
Ryszard Rudnicki
Institute of Mathematics, Polish Academy of Sciences, Staromiejska 8, 40-013 Katowice, Poland
and Institute of Mathematics, Silesian University, Bankowa 14, 40-007 Katowice, Poland
Marek Wolfa)
Institute of Theoretical Physics, University of Wrocław,
PI. Maxa Borna 9, PL-50-204 Wrocław, Poland
~Received 16 June 1998, accepted for publication 17 November 1998!
A reinforced random walk on the d-dimensional lattice is considered. It is shown
that this walk is equivalent to an iterated function system ~IFS!. Criteria for the
existence of limit cycles are given. Numerical results and conjectures about the
quantitative behavior of the walk are stated. © 1999 American Institute of Physics. @S0022-2488~99!00905-6#
I. INTRODUCTION
There are a large number of different modifications and variants of the usual symmetrical
random walk ~RW!.1–3 Let us mention only Levy flights, biased diffusions, self-avoiding walk
~SAW for short!, etc. Let us confine ourselves to the random walks on the discrete lattices. In
SAW a walking particle is choosing its trajectory in such a way that it does not step down onto the
already visited site. If a particle runs into such a node that all neighboring sites were already
visited, it stops. In Ref. 4 the interacting RW was discussed in which the parameter 0, p,` has
influenced probabilities of visiting a given site and p51 corresponds to usual RW. For p→` this
RW goes on into the SAW.
In 1987 Coppersmith and Diaconis5 introduced reinforced random walk ~RRW!. This walk,
opposite to SAW, prefers earliest visited paths. Pemantle6 discussed a related process on trees and
proved that this process is equivalent to a random walk in a random environment. He also gave
criteria for transience and recurrence of RRW. Davis7 considered a variety of types of RRW on the
integers Z. One of them was RRW of sequence type. This process is defined in the following way.
Let w k be an increasing sequence of non-negative numbers. Let (X n ) be a random motion on Z.
If some interval was traversed k-times, then its weights is w k . If X n 5i, then the probability that
X n11 5i21 or X n11 5i11 is proportional to the weights at time n of the intervals (i21,i) and
(i,i11). Davis proved that the moving point visits a finite number of integers and eventually
oscillates between two adjacent integers if and only if ( `k50 w 21
k ,`. This result was generalized
to RRW sequence type on the d-dimensional lattice by Sellke.8
In this paper we consider another type of reinforced random walk on the d-dimensional lattice.
The random point moves according to the following reinforcement convention. Let the moving
point be found at time t5n at a certain point APZn . Let p 1 ,..., p N be the probabilities of
choosing one of the adjacent points A 1 ,...,A N . Assume that we choose the point A i 0 . If after some
time the moving point returns to A, then the probabilities that at the next step it can be found at the
adjacent points are equal to p 18 ,...,p N8 . The values of p 18 ,..., p N8 depend on the previous values
p 1 ,..., p N and i 0 . We assume that the probability of choosing a given path will increase when it
was already traversed and probabilities of remaining paths emanating from a given site will
decrease. In other words, the fact that some sites were already visited will be remembered. The
memory of passing particular edges will be encoded in the change of probabilities. At some time
a!
Electronic mail: [email protected]
0022-2488/99/40(6)/3072/12/$15.00
3072
© 1999 American Institute of Physics
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
3073
the probability of going in some direction from a given site will reach almost 1, while probabilities
to go in other directions will be practically zero. It will result in closed paths: a random walker
will oscillate between a few sites with practically zero probability to escape from such a limiting
cycle. We will treat such a final behavior as stopping of the random walk.
Our walk is not Markovian because the probability of choosing any direction changes in time.
If we extend the phase space by adding the distributions of probabilities of passing particular
edges, we obtain a Markov process which is also an iterated function system.9 In Refs. 10 and 11
were introduced self-attracting diffusions: processes attracted by their own trajectories. It is interesting that these processes and our walk have similar features. For example, self-attracting diffusions are not Markovian, but jointly with their occupation measures are Markov processes. Moreover, their trajectories converge almost surely.
The paper is organized as follows. First we define our random walk and the notion of the limit
cycle is introduced. Next we prove the theorem that this walk reaches the limit cycle. In Sec. IV
the result of the Monte Carlo simulations for a particular ‘‘memorizing’’ function are presented.
These computer experiments allow us to make some conjecture about the quantitative behavior of
some characteristics of the walk.
II. MATHEMATICAL MODEL
A. Description of the random walk with memory
Let Z denote the set of all points of the d-dimensional Euclidean space which have integer
coordinates, i.e., Z is the d-dimensional lattice. A point moves randomly over this lattice. It starts
at point 05~0,...,0!. If at time t5n the moving point can be found at a certain point x
5(x 1 ,...,x d ), then at the time t5n11 it can be found at one of the N52 d adjacent points y
5(y 1 ,...,y d ), where y i 5x i 11 or y i 5x i 21 for i51,2,...,d. By K we denote the set of all
possible ‘‘steps’’ during the walk, i.e.,
K5 $ ~ x,y ! PZ3Z: u y i 2x i u 51
for i51,...,d % .
If z5(x,y)PK, then the points x and y are, respectively, the beginning and the end of the step z.
Let S5 $ 1,21 % d be the set of all N steps to the nearest neighbors. If xPZ, sPS, and y5x1s,
then (x,y)PK.
At time t50 the probability of choosing of any adjacent point equals 2 2d . During the walk
the point ‘‘memorizes’’ its path in the following way. Assume that the moving random walker can
be found at some time t5n at a certain point xPZ. Let p x,x1s , sPS, be the probabilities that at
the time t5n11 it can be found at one of the adjacent points x1s, sPS. Assume that we choose
the point x1s 0 to shift the particle from the point x. If after some time the moving point returns
to x, then the probabilities that at the next step it can be found at the adjacent points are equal to
8
8
p x,x1s
, sPS. The numbers p x,x1s
are related to the previous values p x,x1s in the following way.
8 ) sPS, and
Let px 5(p x,x1s ) sPS , px8 5(p x,x1s
H
P5 px P @ 0, 1 # S:
(
sPS
J
p x,x1s 51 .
Then
p8x 5 f s 0 ~ px ! ,
~1!
where f s 0 :P→P is a continuous function.
If zPK and z5(x,y), we will often write p z instead of p x,y .
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
3074
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
B. Iterated function system
The state of the random walk with memory is described at any time t by the position of the
moving point and the probability p z for every zPK. Let P denote the set of all admissible
distributions of probabilities p z , zPK, i.e.,
H
P5 pP @ 0, 1 # K:
(
sPS
p x,x1s 51
J
for each xPZ .
Then the phase space is the set X5Z3P. Since px 5(p x,x1s ) sPS for every xPZ, we have P
5PZ and X5Z3PZ.
Now, we define an iterated function9 system on the phase space X. It consists of N transformations T s :X→X, sPS. These transformations are defined as follows. Let xPZ and sPS be
given. Then T s (x, p)5(x1s, p 8 ), where
pz8 5
H
pz ,
if z 1 Þx,
f s ~ pz ! ,
if z 1 5x,
for each z5(z 1 ,z 2 )PK. If (x, p) is the state of the random walk at a time t and if the next position
of the moving point is x1s, then T s (x,p) is the next state of the random walk. The probability
that at a point x 5(x, p) we choose the transformation T s is equal p s ( x )5p x,x1s .
C. Markov process on X
Now we construct a Markov process corresponding to the iterated function system given in
Sec. II B.
The phase space X is a metric space with some metric r defined as follows. The set K is
countable, that is, K5 $ z 1 ,z 2 ,z 3 ,... % , where $ z n % nPN , z n PK, is the sequence of all possible steps.
If xPX, yPX, then x5(u,p z 1 , p z 2 ,...) and y5(u 8 ,p z8 ,p z8 ,...), where u, u 8 PZ and p z k ,p z8
1
2
k
P @ 0, 1 # for each kPN. The metric r is given by
`
r ~ x,y ! 5 u u2u 8 u 1
(
k51
2 2k u p z k 2 p z8 u .
k
Let B be the s-algebra of Borel subsets of X. For any xPX and APB we set
I ~ x,A ! 5 $ sPS:T s ~ x ! PA % ,
P ~ x,A ! 5
(
sPI ~ x,A !
p s~ x ! .
Then P(x,A) is a transition probability function, i.e.,
~a!
~b!
for each xPX the function A° P(x,A) is a probabilistic measure and
for each APB the function x° P(x,A) is B-measurable.
Since the space X is s-compact, there exists a homogeneous Markov process $ j n % `n50 which
corresponds to the transition function P(x,A). 12 It means that we have some probability space ~V,
A, Prob! and a sequence $ j n % `n50 of random elements j n :V→X such that the sequence j n is a
Markov process and
Prob ~ j n11 PA u j n 5x ! 5 P ~ x,A !
for each xPX, APB, n>0. Since the initial state of the system is x 0 5(0,22d ,22d ,...), we
assume that j 0 5x 0 .
We assume that the probability space ~V, A, Prob! is complete, i.e., if A is a measurable set
and Prob (A)50, then every subset of A is measurable.
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
3075
D. Limit cycle
A sequence (u 0 ,u 1 ,...,u m21 ) of different elements of Z is called a cycle if there exists a
sequence (s 0 ,s 1 ,...,s m21 ) of elements of S such that u k11 5u k 1s k for k50,...,m21, where
u m 5u 0 . Let P 0 :X→Z be the operator given by P 0 (u,p)5u. We say that a sequence (x n ) `n50 of
elements of X has a limit cycle if there exist a cycle (u 0 ,u 1 ,...,u m21 ) and an integer n 0 >0 such
that for every n>n 0 we have P 0 (x n )5u k , where k5n(mod m).
Let c 0 (t)5l and c n (t)5 c + c n21 (t).
Now we can formulate the following theorem.
Theorem 1: Let c:@0, 1#→@0, 1# be a continuous nondecreasing function such that c~1!51
and
`
(
n50
F S DG
12 c n
1
N
,`.
~2!
We assume that there exists a continuous function c:@0, 1#→@0, 1# such that
f s,s ~ px ! > c ~ p x,x1s !
~3!
for sPS, where f s,s is the s-th coordinate of f s . Then there exists a measurable subset V 0 ,V
such that Prob (V 0 )51 and for each v PV 0 the sequence $ j n ( v ) % `n50 has a limit cycle.
The proof of Theorem 1 is given in Sec. III.
Remark 1: Let P z , zPK, be the operator P z :X→ @ 0, 1 # given by P z (u,p)5 p z . Assume that
the sequence $ j n ( v ) % `n50 has the limit cycle (u 0 ,u 1 ,...u m21 ). Then from Theorem 1 it follows
that limn→` P z „j n ( v )…51 for each z5(u k ,u k11 ), k50,...,m21.
Remark 2: If c:@0, 1#→@0, 1# is a continuous nondecreasing function such that c (x).x for
xP(0,1) and c8~1!.1, then c satisfies ~2!
III. PROOF OF THEOREM 1
A. Boundedness of trajectories
The thread of the proof of Theorem 1 is as follows. First we check that almost all paths are
bounded. From this it follows that a point performing a random walk returns infinitely often to
some points of the lattice Z. Then we show that if a point uPZ is visited infinitely often, then
after some time the random walker chooses a fixed adjacent point to u. This implies that the
random walk has a limit cycle.
Let h n ( v )5P 0 „j n ( v )…. Then the random variable h n describes the position of the moving
point at time t5n.
Proposition 1: For almost all v the sequence $ h n ( v ) % is bounded.
We precede the proof of Proposition 1 with the following lemmas.
Lemma 1: Let w (t)5P `n50 c n (t). Then w (t).0 for t>1/N and
lim w ~ t ! 51.
~4!
t→1
Proof: Since c is a nondecreasing function, from ~2! it follows that
w~ t !>w
SD
1
,`
N
for tP
F G
1
,1 .
N
~5!
Let «.0 be given. Since w (1/N).0 there is an integer n 0 such that
`
)
n5n
`
c ~ t !>
n
0
)
n5n
cn
0
SD
1
.12«
N
for tP
F G
1
,1 .
N
~6!
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
3076
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
The function c is continuous and c~1!51. This implies that there is d.0 such that
n 0 21
)
n50
c n ~ t ! .12«
for tP @ 12 d ,1# .
~7!
From ~6! and ~7! it follows that w (t).(12«) 2 for tP @ 12 d , 1 # . Consequently, limt→1 w (t)51.h
Lemma 2: Let xPZ and yPZ be two adjacent points. Denote by A the event that the point x
is visited for the first at time t5n 0 and the point y was not visited earlier. Let
B5 $ v PV: h n 0 12i21 ~ v ! 5y,
h n 0 12i ~ v ! 5x,
for i51,2,...% .
Then the conditional probability Prob (B u A) satisfies
F S DG
Prob ~ B u A ! > w
1
N
2
.
Proof: Let B 0 5A and
B k 5 $ v PV: h n 0 12i21 ~ v ! 5y,
h n 0 12i ~ v ! 5x,
for i51,2,...,k % .
If v PB k ùA, then at each time n 0 <t,n 0 12k and at each visit at x we have chosen the next
point y and at each visit at y we have chosen the next point x. Let s5y2x and s 8 5x2y. Then
k
Prob ~ h n 0 12k11 5y u B k ùA ! 5 f s,s
SD
1
,
N
Prob ~ h n 0 12k12 5x u B k ùAù $ h n 0 12k11 5y % ! 5 f s 8 ,s 8
k
SD
1
,
N
where f ks is the kth iterate of f s and f ks,s is the sth coordinate of f ks . Since c is a nondecreasing
function, from ~3! we obtain
f ks,s
S D X S DC X S DC
SD SD
1
1
> c f k21
s,s
N
N
k
f s 8 ,s 8
> c 2 f k22
s,s
1
N
>¯> c k
SD
1
,
N
1
1
>ck
.
N
N
Consequently,
F S DG
Prob ~ B k11 u B k ùA ! > c k
From ~8! it follows that
1
N
2
F) S DG
k
Prob ~ B k11 ùA ! >Prob ~ A !
i50
ci
~8!
.
1
N
2
.
~9!
If k→`, then we obtain
F S DG
Prob ~ BùA ! >Prob ~ A ! w
1
N
2
,
and finally
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
F S DG
Prob ~ B u A ! > w
1
N
3077
2
.0.
h
Proof of Proposition 1: For xPRd we set
i x i 5max$ u x i u :i51,...,d % .
For k51,2,..., we define
C k 5 $ v PV:supi h n v !i >k % .
n
According to Lemma 2,
F S DG
Prob ~ C k11 u C k ! <12 w
1
N
2
.
~10!
Indeed, let n 0 ( v ) be the first time such that i h n 0 ( v ) i 5k. If x5 h n 0 and y is an adjacent point to
x such that i y i 5k11, then with probability p> @ w (1/N) # 2 the moving point visits only x and y at
any time t.n 0 . This implies ~10!. From ~10! it follows that
H F S DG J
Prob ~ C k ! < 12 w
1
N
2 k21
.
~11!
Let C5ù `k51 C k . Since a trajectory h n ( v ) is unbounded if and only if v PC, from ~11! it follows
that almost all trajectories are bounded.
h
B. Stabilization of directions
From Proposition 1 it follows that almost all trajectories are bounded. Consequently, the
moving point visits some points of the lattice Z infinitely often. Let a point xPZ be given. By A
we denote the event that the point x is visited infinitely often. For any v PA we denote by
$ k n ( v ) % `n51 successive times of visits at point x. Let x n ( v ) be the adjacent point to x visited at
time t5k n ( v )11. We show that for almost every v PA there exists a point y( v )PZ such that
x n ( v )5y( v ) for n.n 0 ( v ).
The process of choosing the adjacent points can be described as an iterated function system
( f s ) sPS on the space P and the probability that at the point px PP we choose the transformation f s 0
equals p x,s 0 . Indeed, let us assume that we visit the point x and let px 5(p x,s ) sPS be the distribution of probability of choosing adjacent points x1s, sPS. If we choose the point x1s 0 , then at
the next visit at x, p8x 5 f s 0 (px ) is the new distribution of probability of choosing adjacent points.
Since x is a given point we will write p instead of px and p s instead of p s,x .
Let $ z n % `n51 be a homogeneous Markov process on the phase space P corresponding to the
iterated function system ( f s ) sPS . The transition probability function for the process ( z n ) is given
by the formula
P ~ p,A ! 5
( ps ,
sPI
where I5 $ sPS: f s ~ p! PA % .
~12!
Denote by x1s n ( v ) the adjacent point chosen at time t5n. Then z 1 5(1/N,...,1/N), z n11
5 f s n ( z n ), and
Prob ~ s n 5s u z n 5p! 5 p s .
~13!
for pPP, n51,2,..., and sPS.
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
3078
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
Proposition 2: For almost every v there is s( v )PS and an integer n 0 ( v ) such that s n ( v )
5s( v ) for n.n 0 ( v ).
We precede the proof of Proposition 2 by a lemma. Let
Pse 5 $ pPP:p s >« % ,
Pe 5 ø Pse .
sPS
Lemma 3: Let P n (p,A) be the n-step transition probability function. Then for every d,1 and
pPP we have
lim P n ~ p,Pd ! 51.
~14!
n→`
Proof: Let «,1 be given. First we check that for pPP« we have
P ~ p,Pc ~ « ! ! >«.
~15!
Indeed, if pPP« , then for some s we have pPPs« and ps >«. Consequently, f s (p)PPcs («) and
inequality ~15! follows immediately from ~12!. From ~15! it follows that
P i11 ~ p,Pc i11 ~ « ! ! >
E
Pc i ~ « !
P ~ q,Pc i11 ~ « ! ! P i ~ p,dq! > c i ~ « ! P i ~ p,Pc i ~ « ! ! ,
which gives
n21
P n ~ p,Pc n ~ « ! ! >
)
c k~ « ! > w ~ « !
k50
~16!
for pPP« .
If «>1/N, then limn→` c n («)51. From ~16! it follows that for every d,1, «>1/N, and pPP«
we have
lim inf P n ~ p,Pd ! > w ~ « ! .
~17!
n→`
If pPP, then pPP1/N and, consequently,
lim infP n ~ p,Pd ! > w
n→`
SD
1
.
N
~18!
Since
P n1m ~ p,Pd ! 5
E
P
P n ~ q,Pd ! P m ~ p,dq! 5
E
Pd
P n ~ q,Pd ! P m ~ p,dq! 1
the inequalities ~17! and ~18! imply
lim infP n1m ~ p,Pd ! > w ~ d ! P m ~ p,Pd ! 1 w
n→`
SD
X
E
P\Pd
P n ~ q,Pd ! P m ~ p,dq! ,
S DC
1 m
1
P ~ p,P\Pd ! 5 w ~ d ! 2 w
N
N
P m ~ p,Pd ! 1 w
SD
1
.
N
~19!
Set
a ~ d ! 5lim infP n ~ p,Pd ! .
n→`
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
J. Math. Phys., Vol. 40, No. 6, June 1999
Then from ~19! it follows
R. Rudnicki and M. Wolf
X
a~ d !> w~ d !2w
S DC
1
N
a~ d !1w
SD
1
.
N
3079
~20!
Since a~d! is a nonincreasing function there exists the limit limd →1 a ( d )5 a 0 . According to
Lemma 1, limd →1 w ( d )51. A passage to the limit d→1 in inequality ~20! gives
X S DC S D
a 0 > 12 w
1
N
1
.
N
a 01 w
From ~21! we conclude that a 0 >1 and ~14! holds.
Proof of Proposition 2: Let d P(1/N,1) be a given number. Since
~21!
h
Prob ~ z n PPd ! 5 P n21 ~ z 1 ,Pd ! ,
from Lemma 3 it follows that there exists n 0 such that
Prob ~ z n 0 PPd ! > d .
~22!
Let A5 $ v : z n 0 PPd % and A s 5 $ v : z n 0 PPsd % . Then A5ø sPSA s and the sets A s , sPS, are pair
disjoint. From ~13! it follows that
m
Prob ~ s n 0 11 5s,...,s n 0 1m 5s u A s ! >
)
i50
c i~ d ! > w ~ d ! .
~23!
Let
B s 5 $ v :s n ~ v ! 5s
for n>n 0 ~ v ! % ,
B5 ø B s .
sPS
Inequality ~23! implies that
Prob ~ B s u A s ! > w ~ d !
and consequently
Prob ~ B s ! > w ~ d ! Prob ~ A s ! .
~24!
The sets A s , sPS, are pair disjoint and the sets B s , sPS, are pair disjoint. From ~22! and ~24! we
obtain
Prob ~ B ! > w ~ d ! Prob ~ A ! > w ~ d !~ 12 d ! .
Letting d→0 we have Prob (B)51, which completes the proof.
h
C. Existence of the limit cycles
Now we are ready to complete the proof of Theorem 1.
According to Proposition 1 almost every trajectory is bounded and goes through some point
u 0 PZ infinitely often. Let D be a bounded subset of Z and u 0 PZ be a given point. Denote by A 0
the subset of V which consists of all vPV such that the trajectory $ h n ( v ) % is contained in D and
goes through u 0 infinitely often. According to Proposition 2, for almost every v PA 0 there exists
s( v )PS such that the random walker going through u 0 chooses the direction s( v ) for sufficiently
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
3080
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
TABLE I. The numbers RW of the cycle for a few values of a.
2
4
6
8
10
12
14
16
18
0.8
0.82
0.84
0.86
0.88
0.9
0.92
0.94
9554
397
38
7
0
0
1
0
0
9555
410
29
5
0
0
1
0
0
9592
363
38
3
1
1
0
0
0
9640
329
23
5
0
0
0
0
0
9645
316
27
6
3
1
0
0
0
9633
343
19
1
2
0
0
0
0
9723
247
17
4
0
0
0
0
1
8973
199
10
6
2
0
0
2
0
large times. Now, we can divide the set A 0 into N disjoint subsets B 1 ,...,B N in such a way that in
each set B k the step (u 0 ,u 1 ) is determined uniquely. Denote one of these sets by A 1 . Then we can
divide the set A 1 into N subsets related to the next step (u 1 ,u 2 ), etc. After some steps the moving
point returns to u 0 and in this way we obtain a limit cycle.
h
IV. NUMERICAL SIMULATIONS
A. Details of the studied models
The theorem proved in previous sections gives only general information about the RW with
memory. To gain some insights into the more quantitative characteristics of RW with memory we
have performed Monte Carlo simulations for the function
f s ~ x ! 5 c a ~ x ! 5x @ 22 a 2 ~ 12 a ! x # ,
~25!
where a is a parameter from the interval ~0,1!. Since for a51 the function c a (x) is equal to the
identical mapping, we expect that for a→1, the RW with memory will tend to the usual symmetrical RW. In particular, for a'1 it should never fall into the limiting cycle. In other words
some critical slowing down in reaching the limit cycle will occur for a'1.
FIG. 1. The plot of the histograms of the number of steps performed by random walkers before the stop of the process for
a sample of the values of a.
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
3081
FIG. 2. The plot of the dependence of the parameter r in ~26! obtained from the moments of actual data. Remarkably, this
figure suggests that r does not depend on a.
B. Obtained results
We have performed the simulations only on the two-dimensional, square lattice of the size
130031300. In each node of the lattice we have stored probabilities of making a step in one of the
four directions. Initially all p z were set to be equal 41. The random walker started from the origin
of the lattice and after each step the probabilities were updated according to Eq. ~1!. A given
particular simulation of the RW was finished when one p z reached the value of 0.999 or else the
total number of steps was equal to 2 000 000. We have imposed the periodic boundary conditions
on the RW and we recorded the facts of crossing by RW the edges of the torus. There were rare
cases of such events, most of them occurred, of course, for larger values of a. We have performed
simulations for a in the range ~0.8, 0.94!. In the subrange ~0.8, 0.91! a was changed with the step
Da50.01, while in the subinterval ~0.91, 0.94! with the step Da50.001, because the number of
steps performed by random walker before the stop was increasing very rapidly with growing a.
We did not continue to larger values of a, because the number of steps needed to stop the RW was
too large. For each a there were 10 000 separate random walks performed. We have stored the
number of steps N at which for the first time one of the probabilities reached p z 50.999. The path
of RW falls into the cycle ~see Sec. II D! and the length of the limiting trajectory was also stored.
Table I gives a sample of this data for a few values of a for the length of the cycle 2, 4,...,
18—larger cycles have occurred very randomly. The numbers in this table do not sum up to
10 000 because some RW had limit cycles larger than 18, and for large a rare samples did not fall
into the limit cycle in less than 2 000 000 steps. These lengths of cycles do not follow the Poisson
distribution and we do not have any conjecture describing these numbers.
The numbers of steps, for each a, varied considerably from one sample RW to another. For
example, for a50.8 there was a RW which stabilized after N min563 steps, while the largest
number of steps was N max54114. This gap between smallest and largest number of steps needed
to stop RW increased with a, for example, for a50.93 the minimal and maximal number of steps
before RW stopped was N min51034 and N max5590 176, respectively. We claim that the number
of steps N is governed by the gamma distribution with parameters r, b:
g r, b ~ x ! 5 b r x r21 e 2 b x /G ~ r ! ,
~26!
where G(r) is a generalization of the factorial:
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
3082
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
FIG. 3. The plot of the dependence of the parameter b in ~26! obtained from the moments of actual data.
G~ r !5
E
`
0
e 2t t r21 dt.
~27!
In Fig. 1 we present plots of the histograms of the number of steps for a few of values of a.
The size of the bins was 1000, so the y axis gives the number of random walks with the number
of steps in the range (10003k,10003k11000). In Figs. 2 and 3 the values of the fitted parameters r and b for all investigated values of a are shown. Remarkably, the parameter r takes values
around 1.72 and it seems not to depend on a. It is probably linked with the special choice of the
function ~25!.
Despite the large fluctuation of N between different realizations of RW, there seems to be a
simple formula describing the median m value of N. Here m is defined as such a value of N that the
same number of sample random walks stopped in smaller than m, as well in larger than m steps.
Since for a51 the RW with memory passes into the usual RW, N should diverge to infinity for
FIG. 4. The plot of dependence of the median m~a! versus u51/(12 a ) is shown. The solid line presents the least-square
fit to the points obtained from the Monte Carlo simulations, represented by circles, under the assumption that fit is made
by the exponential function.
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp
J. Math. Phys., Vol. 40, No. 6, June 1999
R. Rudnicki and M. Wolf
3083
a→1, thus we guessed that m is a function of 1/~12a!. Hence, in Fig. 4 the plot of the median m
versus u51/(12 a ) is shown. This figure suggests that m~a! grows exponentially with u—the
dashed line presents the exponential fit to the actual values obtained by the least-square method:
m ~ a ! ;exp
S D
1
.
12 a
~28!
Summarizing, the Monte Carlo simulations suggest that there seems to be strict, quantitative
rules governing the behavior of some characteristics of the RW with memory for the function
c a (x).
ACKNOWLEDGMENTS
This research was supported by the State Committee for Scientific Research ~Poland! Grant
No. 2 P03A 042 09 ~RR! and No. 2 P302 057 07 ~MW!.
F. Spitzer, Principles of Random Walk ~Springer-Verlag, New York, 1976!.
J. W. Haus and K. W. Kehr, Phys. Rep. 150, 263 ~1987!.
3
S. Havlin and D. Ben-Avraham, Adv. Phys. 36, 695 ~1987!.
4
H. E. Stanley, K. Kang, S. Redner, and R. L. Blumberg, Phys. Rev. Lett. 51, 1223 ~1983!.
5
Two talks by P. Diaconis at the 1987 Midwest Probability Conference, unpublished.
6
R. Pemantle, ‘‘Phase transition in reinforced random walk and RWRE on trees,’’ Ann. Prob. 16, 1229 ~1988!.
7
B. Davis, Probab. Th. Rel. Fields 84, 203 ~1990!.
8
T. Sellke, ‘‘Reinforced random walk on d-dimensional integer lattice,’’ Tech. Rept. No. #94-26, Dept. of Statistics,
Purdue Univ.
9
M. F. Barnsley, Fractals Everywhere ~Academic, New York, 1988!.
10
M. Cranston and Y. Le Jan, Math. Ann. 303, 87 ~1995!.
11
O. Raimond, Probab. Th. Rel. Fields 107, 177 ~1997!.
12
E. B. Dynkin, Markov Processes, Vol. I ~Springer-Verlag, New York, 1965!.
1
2
Downloaded 31 Aug 2004 to 137.30.117.37. Redistribution subject to AIP license or copyright, see http://jmp.aip.org/jmp/copyright.jsp