Swarm Intell
DOI 10.1007/s11721-013-0076-9
Ants find the shortest path: a mathematical proof
Jayadeva · Sameena Shah · Amit Bhaya · Ravi Kothari ·
Suresh Chandra
Received: 21 May 2011 / Accepted: 25 January 2013
© Springer Science+Business Media New York 2013
Abstract In the most basic application of Ant Colony Optimization (ACO), a set of artificial ants find the shortest path between a source and a destination. Ants deposit pheromone
on paths they take, preferring paths that have more pheromone on them. Since shorter paths
are traversed faster, more pheromone accumulates on them in a given time, attracting more
ants and leading to reinforcement of the pheromone trail on shorter paths. This is a positive
feedback process that can also cause trails to persist on longer paths, even when a shorter
path becomes available. To counteract this persistence on a longer path, ACO algorithms
employ remedial measures, such as using negative feedback in the form of uniform evaporation on all paths. Obtaining high performance in ACO algorithms typically requires fine
tuning several parameters that govern pheromone deposition and removal. This paper proposes a new ACO algorithm, called EigenAnt, for finding the shortest path between a source
and a destination, based on selective pheromone removal that occurs only on the path that is
actually chosen for each trip. We prove that the shortest path is the only stable equilibrium
for EigenAnt, which means that it is maintained for arbitrary initial pheromone concentraJayadeva () · S. Chandra
Department of Electrical Engineering, Indian Institute of Technology, Delhi, Hauz Khas,
New Delhi 110016, India
e-mail: [email protected]
S. Chandra
e-mail: [email protected]
S. Shah
Thomson Reuters Corp. R & D, 195, Broadway, New York 10007, USA
e-mail: [email protected]
A. Bhaya
Federal University of Rio de Janeiro, Department of Electrical Engineering, PEE/COPPE/UFRJ,
P.O. Box 68504, Rio de Janeiro, RJ 21945-970, Brazil
e-mail: [email protected]
R. Kothari
IBM India Research Lab., 4 Block C, Institutional Area, Vasant Kunj, New Delhi 110070, India
e-mail: [email protected]
Swarm Intell
tions on paths, and even when path lengths change with time. The EigenAnt algorithm uses
only two parameters and does not require them to be finely tuned. Simulations that illustrate
these properties are provided.
Keywords Ant Colony Optimization · Stability analysis · Pheromone · Collective
foraging · Stagnation · Self-organization · Selective removal of pheromone · Distributed
optimization · Optimization · Swarm intelligence
1 Introduction
Collective foraging by ants is one of the most remarkable instances of co-operative computation in the natural world. Ants deposit pheromone as they travel and can also sense
the concentration of pheromone that may already be present on a path. Amongst a set of
alternative paths that lead from the source to the destination, the path that happens to be
more heavily traveled on by ants, and therefore has a higher concentration of pheromone,
tends to be chosen by the ants that follow (Bonabeau et al. 1999; Deneubourg et al. 1990;
Dorigo 2007). The seemingly simple behavior of depositing and sensing pheromone can
lead to the emergence of a foraging trail on the shortest path between the source and the
destination.
The selection of the shortest path is remarkable considering that it is achieved without explicit communication about the merits of one path over another, since an ant that
has chosen one path has no idea of the length of any path. Moreover, the decision making is distributed, based only on local information and indirect communication, without any
centralized control. Self-organization in ants has motivated many distributed optimization
techniques. Ant algorithms are known to yield good results on many problems including the
traveling salesman problem (Abdelbar and Wunsch 2012; Dorigo and Gambardella 1997;
Dorigo et al. 1996; Meyer 2004; Stützle and Hoos 2000), vehicle routing (Reimann et al.
2002, 2003; Yildirim and Çatay 2012), scheduling (Merkle et al. 2002), and routing in
communications networks (Di Caro and Dorigo 1998a, 1998b; Ducatelle et al. 2010;
Ghazy et al. 2012; Mapisse et al. 2011) (see Dorigo and Stützle 2004 for a comprehensive
overview).
Throughout the paper, the term ant is used to mean artificial ant: an agent of ACO subject
to the assumptions that follow. Ants prefer to traverse paths that have a higher pheromone
concentration: in other words, the probability of choosing a path is, in some well defined
way, proportional to the pheromone concentration on it. Paths with more pheromone on
them are preferred by ants, leading to pheromone build-up on paths. Assuming constant
ant velocities, shorter paths are traversed in less time, causing pheromone to accumulate
more rapidly on them. Consequently, shorter paths are increasingly preferred, and eventually, almost all the ants choose the shortest path. We refer to this as a positive feedback that
reinforces shorter paths.
However, since ants have no knowledge of path lengths, positive feedback may also cause
stagnation in a situation where the shortest path changes over time. To be more explicit,
consider a situation in which a shorter path B is discovered a long time after the discovery of
the previously shortest path A. In such a case, a large amount of pheromone has already built
up on path A, and it may continue to be the preferred path (Shah et al. 2008). Quicker returns
on path B can revert some of the initial bias on the longer path A, but Shah et al. (2010)
showed that there is a theoretical limit to the initial bias on path A that can be overcome.
Above this limit, a longer path will continue to get chosen and reinforced, despite the fact
that a shorter one has been discovered subsequently. Thus, late appearance of a shorter path
Swarm Intell
will not always bring the system out of premature convergence to a longer one. We refer to
this stagnation as the lack of plasticity.
A commonly used strategy to aid in plasticity is to limit the maximum amount of
pheromone deposited on any path (Stützle and Hoos 2000). This improves the performance
of the foraging model, because the bias on any sub-optimal path does not increase above
a specified threshold, thus facilitating bias reversion. However, determining the optimal
threshold is difficult (Yuan et al. 2012) as its relation to the problem at hand, to the number
and quality of possible solutions, and to the exploration probability is not yet well understood. It is possible for several sub-optimal paths to end up with the maximum allowed
pheromone concentration, preventing convergence to the optimal path. It is therefore clear
that positive reinforcement alone may not be enough to counter the initial bias on paths, and
that there needs to be some form of negative reinforcement to aid plasticity. In this context,
negative reinforcement or negative feedback refers to a mechanism that reduces the amount
of pheromone on a path.
The most commonly adopted strategy for negative reinforcement is pheromone evaporation (Dorigo et al. 1996). It consists of decreasing the amount of pheromone on all
paths at a fixed rate. In the presence of evaporation, sustaining a pheromone trail requires continued reinforcement by way of more deposition. Evaporation is known to significantly improve the performance of ant algorithms (Dorigo and Stützle 2004). If a
shorter path is discovered at some instant, evaporation can help in overcoming the preexisting bias for an earlier established path. Once again, there is a limit to the initial
bias on sub-optimal paths that can be overcome in this manner. Evaporation does not
ensure or completely explain the selection of the optimal solution (Shah et al. 2010;
Van Vorhis Key and Baker 1982). Nevertheless, the introduction of evaporative pheromone
removal represents an important milestone in the literature, and is almost universally employed in applications (see, for example, Bandieramonte et al. 2010; Ding et al. 2003;
Dorigo and Gambardella 1997; Parpinelli et al. 2002; Zecchin et al. 2005, in addition to
those already cited above).
Negative feedback has been reported in biological studies, and arises in the form of
selective removal of pheromone in certain species (Robinson et al. 2008), deposition of
a lower amount of pheromone when a large quantity already exists (Jaffe 1980), or the
mechanical rubbing off of pheromone while walking (Dorigo and Gambardella 1997;
Jackson et al. 2006). Evaporation is also a kind of negative feedback. However, we propose
a negative feedback mechanism that reduces pheromone only on the path that is actually
chosen by each ant, and thus differs from evaporation which is a phenomenon affecting all
paths equally.
In this paper, we propose an ant colony optimization algorithm, called EigenAnt, for
finding the shortest path between a source and a destination. Analysis of EigenAnt, which
essentially deals with the dynamics of the pheromone concentrations, gives our main result.
We show that, irrespective of initial biases on paths, EigenAnt will always cause pheromone
to concentrate on the shortest path. If path lengths change, EigenAnt will cause pheromone
concentrations to change in response. In other words, EigenAnt is not only plastic enough
to shift to a subsequently discovered shorter path, but is also stable in the presence of perturbations. We show that power law variants of EigenAnt also possess the same property, but
require less pheromone to maintain a trail on the shortest path.
The remainder of this paper is organized as follows. In Sect. 2, we propose the EigenAnt
algorithm. In Sect. 3, we analyze its stability properties. Section 4 contains simulation results. Section 5 contains concluding remarks.
Swarm Intell
Fig. 1 The shortest path problem setup for the EigenAnt algorithm, showing the source and destination
nodes, the n paths connecting them, as well as the parameters and variables associated with each path. These,
at ant trip t for the ith path, are the path length Li , the pheromone concentration Cit , the probability of
choosing the ith path pit , the pheromone removal coefficient α and the deposition coefficient β. In addition,
a weight di = f (Li ) is associated with the ith path, where f is a monotonically decreasing function of Li
2 The EigenAnt algorithm
2.1 Motivation and main assumptions
We first consider the case of a graph with two nodes, a source and a destination node, which
are connected by n edges. Each of these edges is termed a path between the source and the
destination nodes, and has an associated length. In this section, we examine the application
of EigenAnt to finding the shortest path in such a graph; in other words, how EigenAnt can
select the shortest amongst n edges. In the sequel, we provide simulation results on larger
graphs, where the graph contains additional nodes, and where paths between the source and
the destination may consist of several edges that pass through intermediate nodes.
We denote the pheromone concentrations on all paths by the vector C. The ith component
of C, denoted by Ci , represents the amount of pheromone on the ith path. The number of
paths
n is denoted by n, and the total pheromone concentration is denoted by V , where V =
i=1 Ci . The length of the ith path is denoted by Li . For each relevant variable, whenever
necessary, we may use a superscript enumerating the trip at which the variable is being
considered: for example, Cit denotes the pheromone concentration on the ith path for the t th
ant trip.
Ants randomly choose a path: the probability of choosing the ith path on the t th trip is
Ct
denoted by pit , and is given by pit = V it . Observe that the probability pit also has the interpretation of normalized pheromone concentration and we will use this alternative terminology
whenever appropriate. In the EigenAnt algorithm, the pheromone on the chosen path, say
the ith, is updated after the t th trip as follows:
ΔCit = −αCit + βdi pit ,
(1)
where
:=
−
and di = f (Li ), f is a monotonically decreasing function of the
length Li . The n × n diagonal matrix which has the positive numbers di as its diagonal
entries is denoted as D. Figure 1 illustrates the notation used in the EigenAnt algorithm. In
summary:
ΔCit
Cit+1
Cit
1. Each path i = 1, 2, . . . , n is an edge, with length Li . The pheromone concentration on
the ith path at trip t is denoted by Cit .
Swarm Intell
2. Each ant chooses a path probabilistically: at trip t , the probability of choosing the ith
path is given by
pit = Cit /V t ,
where
Vt =
(2)
Cit .
(3)
i
3. Let the chosen path be the j th one. Each ant removes −αCjt pheromone and deposits
pheromone equal to βf (Lj )pjt , where f (·) is any monotonically decreasing function of
its argument.
The expected change in the pheromone concentration on the ith path at the t th epoch is
given by
(4)
ΔCit = pit −αCit + βdi pit ,
or, equivalently, using (2),
ΔCit
Cit
Cit
t
=
t −αCi + βdi
t ,
j Cj
j Cj
(5)
which can be written compactly as
1
1
ΔCt = t diag Ct −αCt + t βDCt ,
V
V
(6)
where ΔCt is a vector in Rn and diag(Ct ) is defined as the diagonal matrix having entries
C1t , . . . , Cnt on its principal diagonal.
A power law relating the probability of choosing a path and the pheromone concentration on it has also been proposed in the literature (Deneubourg et al. 1990; Sumpter and
Beekman 2003). Following this idea, a generalization of (2) is to assume that the probability
of choosing the ith path in the t th trip is given by pit =
(C t )k
i t k
j (Cj )
. Since a large difference
between the deposition and removal rates is not reasonable, the negative feedback on the
ith path is assumed to be of the form α(Cit )k . This leads to the following equation for the
expected change in pheromone concentration on the t th trip:
k
(C t )k
(C t )k
(7)
ΔCit = i t k −α Cit + βdi i t k ,
j (Cj )
j (Cj )
where k ≥ 1 is an integer. Clearly (5) is a special case of (7), for k = 1.
An important observation that emerges from the above discussion is that the initial distribution of pheromone concentrations should be such that Ci0 = 0, i = 1, . . . , n, because a
zero initial pheromone concentration, say Cj0 = 0, means that path j has zero probability of
being chosen (however, see Sect. 4.3).
The EigenAnt algorithm is summarized by the pseudocode given in Algorithm 1, in
which it is assumed that the exponent k in (7) is chosen as 1. The ith component Ji of
the path counter vector J maintains a count of the total number of ant trips on path i. We
assume that simulations are carried out for a total of Jmax trips. The pseudocode initializes
Ci0 = n1 , i = 1, . . . , n. In the sequel, we show that EigenAnt finds the shortest path regardless
of initial pheromone concentrations.
Remark The MATLAB code for EigenAnt is available from the authors on request.
Swarm Intell
Algorithm 1 EigenAnt pseudocode
1: Initialize trip counter vector J = (Ji ) = 0.
2: Initialize total ant trip counter t = 1. Choose Jmax ∈ N, the total number of ant trips.
3: Initialize path i with pheromone concentration: Cit = (1/n), i = 1, . . . , n.
4: Initialize probability of choosing the ith path: pit = (1/n), i = 1, . . . , n.
5: Initialize the weights di as 1/Li , where Li is the length of the ith path.
6: Choose the pheromone removal parameter, α and the pheromone deposition parameter, β.
7: while i Ji ≤ Jmax do
8:
Choose a path randomly in accordance with the distribution of probabilities pjt , j =
9:
10:
11:
1, . . . , n.
if the ith path is chosen then
Update the trip counter of the ith path: Ji ← Ji + 1.
Update pheromone only on path i: Cit+1 ← (1 − α)Cit + βdi pit .
13:
No changes in pheromone on paths j = i: Cjt+1 = Cjt .
end if
14:
Update path choice probabilities: pit+1 ← n
12:
Cit+1
t+1 , i = 1, . . . , n.
j =1 Cj
15:
Update total ant trip counter: t ← t + 1.
16: end while
17: Return normalized pheromone concentrations matrix (pit versus trip t) and final trip counter
vector J.
3 Analysis of the EigenAnt algorithm
This section presents the analysis of the EigenAnt algorithm, which involves studying the
equilibria of (5) and (7), as well as their stability properties.
3.1 Analysis of the EigenAnt pheromone update rule
We assume that, at steady state, the expected change in pheromone concentrations is zero:
ΔCi = 0, for each path i = 1, 2, . . . , n. This may compactly be written as
ΔC = 0.
(8)
We will refer to pheromone concentration vectors that satisfy (8) as equilibrium points of
the system (5) or (7).
In order to simplify the proof, we assume that the path lengths can be arranged in strictly
increasing order:1
L 1 < L2 < L3 < · · · < Ln ,
which, in turn, implies that the entries di of the diagonal matrix D satisfy
d 1 > d 2 > d 3 > · · · > dn .
(9)
The main theorem of this paper can now be stated as follows.
1 This implies that all paths are of different lengths. This assumption can easily be relaxed, at the cost of
slightly increased technicalities, without changing the main idea of the proof.
Swarm Intell
Theorem 1 The vectors βα di ei , i = 1, . . . , n, are all equilibrium points of the system (6),
where ei denotes the ith canonical vector in Rn . The equilibrium point ( βα d1 )e1 of (6) is
stable, and all the other equilibrium points βα di ei , i = 1 are unstable.
The proof is divided into three parts:
1. For equilibrium point analysis, it is shown that the power law rule given in (7) for k ≥ 1
can be reduced to the case k = 1, which is (5). Thus only (5), or equivalently, (6) needs
to be analyzed.
2. The equilibrium points for (6) are calculated and shown to be the vectors ( βα di )ei , i =
1, . . . , n.
3. We show that the equilibrium point ( βα d1 )e1 is stable. We do so by showing that perturbations from this equilibrium state cause the system to revert to it. Furthermore, all
other equilibrium points βα di ei , i = 1 are unstable, in the sense that there always exist
perturbations that cause divergence from them.
Proof [Part 1: Reduction of k ≥ 1 to k = 1] Let the concentrations at equilibrium be denoted
by Ci∗ , i = 1, . . . , n. Then, for either (5) or (7), the equilibrium points are found from (8).
Specifically, for (7), the equilibrium points are given by
k
(C ∗ )k
α Ci∗ = βdi i ∗ k .
j (Cj )
(10)
Notice that if (Ci∗ )k is replaced by C̄i∗ in (10), then it becomes exactly the equilibrium
point equation for (6) or (5). Thus, from this point onwards, the analysis is carried out for
the case k = 1.
[Part 2: Calculation of the equilibria of (6)] For this part of the proof, we write V = j Cj
as 1T C, in order to emphasize dependence on the vector C, where 1 ∈ Rn is the vector with
all n components equal to 1. Then, for k = 1, the equilibrium point equation (8) corresponding to (6) can be written in vector form as:
αC =
β
DC,
1T C
DC =
α T 1 C C.
β
which, in turn, can be written as
(11)
If we define the scalar λ := βα (1T C), then (11) becomes
DC = λC,
(12)
which looks like an eigenvalue-eigenvector equation, with λ being the eigenvalue and C the
eigenvector, although it should not be forgotten that λ is a function of C.
Observe that, since D is a diagonal matrix, its eigenvectors are the canonical vectors ei ,
corresponding to the eigenvalues di . This implies that all solutions of (12), and hence of (11)
are of the form C = μi ei . That is, equilibria are of the form Ci = μi , Cj = 0, ∀j = i, for
some index i ∈ {1, 2, . . . , n}. Indeed, observing that 1T μi ei = μi , and substituting into (11)
yields
α
(13)
Dμi ei = (μi )μi ei .
β
Swarm Intell
Dividing both sides by μi = 0 and simplifying, we see that
μi =
β
di .
α
(14)
This underlines the fact that (11) is not an eigenvalue-eigenvector equation,2 but a related
nonlinear equation, which has exactly n solutions βα di ei , i = 1, . . . , n.
From part 1 of the proof, at equilibrium, for any general value of k, (Ci∗ )k can be replaced
by Ci∗ . Thus, the one-to-one correspondence between the equilibria of (5) and (7) is as
follows:
1k
β
β
di ei of (5) corresponds to equilibrium
di ei of (7).
(15)
Equilibrium
α
α
This indicates that the equilibrium is maintained for k > 1 with less pheromone than is
required in the case k = 1.
This completes the second part of the proof, namely the calculation of the equilibrium
points. Note that the proof has also shown that all possible equilibrium points correspond to
bounded pheromone concentrations.
[Part 3: Stability of the equilibrium corresponding to the shortest path and instability of all
other equilibria]
In this final part of the proof, we will carry out a perturbation analysis around an equilibrium point μi ei . The analysis will show that there always exist perturbations around an
equilibrium point μi ei , i = 1, that will cause the system to move away from such an equilibrium point. On the other hand, perturbations around μ1 e1 will cause the system to move
in such a way as to return to it, nullifying the perturbation. The starting point of the analysis
is (6), which, after multiplying throughout by the scalar V 2 , can be rewritten in vector form
as follows.
2
(16)
V ΔC = diag C(−αV C + βDC).
Note that, since V 2 is always positive, the sign of the scalar product of the vector V 2 ΔC
with any other vector v is always the same as that of the scalar product of ΔC with v.
Thus, in the remainder of the proof, we will use (the right hand side of) Eq. (16) in the
perturbation analysis of the vector ΔC.
The perturbation analysis is carried out around the equilibrium point C = μi ei : C =
(0, 0, . . . , 0, μi , 0, . . . , 0)T . The pheromone concentration vector after perturbation is denoted
C + = μi ei + = (1 , 2 , . . . , i−1 , μi + i , i+1 , . . . , n )T .
(17)
Since the pheromone on any path must be non-negative, we have
j ≥ 0, ∀j = i;
i ≥ −μi ,
1T > −μi .
(18)
After perturbing the equilibrium as in (17), we have
V = 1T (C + ) =
β
di + 1T .
α
(19)
2 This is because the eigenvalue-eigenvector equation Ax = λx, for a fixed eigenvalue λ has infinitely many
solutions. In fact, if x is an eigenvector, then y = γ x, for all γ ∈ R is also a solution. However, in the present
case, there are exactly n solutions, corresponding to n unique values of γ .
Swarm Intell
Substituting (17) and (19) in (16) and dropping higher order (O( 3 )) terms, in order to
evaluate the component of V 2 ΔC in the direction of canonical vector ej leads to
j = i,
j2 β(dj − di ),
2
T
(20)
V ΔC ej =
2
T
−(μi + i ) (1 ), j = i.
We first analyze the equilibria μi ei , for the case i = 1. From (9), (dj − di ) > 0 for j < i,
and (dj − di ) < 0 for j > i. Thus, from (20), V 2 ΔCT ej > 0, for j < i (respectively,
V 2 ΔCT ej < 0, for j > i), so that perturbations in the j th coordinate direction, for j < i,
increase, whereas perturbations in directions corresponding to j > i tend to decrease. This
implies that, for all equilibria μi ei , i = 1, there exist directions (all ej , j < i) in which
perturbations cause movement away from the equilibrium: in other words, these equilibria
are unstable.
On the other hand, for the equilibrium μ1 e1 , (dj − d1 ) < 0 for all j , so that all perturbations in the directions ej , j = i decrease. For perturbations in the direction e1 , we observe
that if 1T > 0, then V 2 ΔC < 0; conversely, if 1T < 0, then V 2 ΔC > 0. This means
that the equilibrium μ1 e1 is stable. Since 1T is the perturbation in the total pheromone
on all paths, this also means that the system moves in a direction so that the total amount
of pheromone is conserved. Consequently, the discussion on stability of equilibrium points
leads us to conclude that pheromone remains concentrated on the shortest path, and that the
total pheromone remains conserved.
Remark 1 As the proof indicates, the scaled eigenvector ( βα d1 )e1 associated with the dominant eigenvalue (the largest diagonal element of βα D) is the only asymptotically stable equilibrium point of (6). This motivated the choice of the name EigenAnt for the algorithm. Our
proof parallels the stability proof for Oja’s rule (Hertz et al. 1991), for a system that finds the
first principal component of a given data set. The proof also makes it clear that the shortest
path always corresponds to the only locally asymptotically stable equilibrium. Thus, if the
shortest path changes at any time, convergence to the new shortest path should be expected
to occur, since path choice is probabilistic.
Remark 2 The proof also shows that the total pheromone concentration is bounded at steady
1
state, and is equal to μ1 = ( βα d1 ) k . At steady state, the trail is sustained by ants choosing
it, causing both deposition and removal to occur on the chosen path in accordance with the
proposed dynamics. The dependence on k indicates that power law variants of EigenAnt
require less pheromone to maintain a trail, which is a possible advantage for a power law
dependence of path selection probabilities on pheromone concentrations. Note that, in the
notation of the theorem, the normalized pheromone concentration vector can be expected,
in the presence of perturbations caused by the probabilistic path choices, to converge to the
vector e1 .
3.2 The difference between selective removal and evaporation
In this section, we briefly discuss the difference between evaporative removal of pheromone
of the kind employed in Ant System (AS) (Dorigo 2007; Dorigo et al. 1996), and selective
removal of pheromone (or selective deposition of negative pheromone). The latter is the
kind of negative feedback employed in EigenAnt. The difference between the two is that in
the case of evaporative removal, pheromone gets removed even when no pheromone might
be added, whereas in EigenAnt, removal takes place only on paths that are selected by ants;
Swarm Intell
ants also reinforce these trails by depositing pheromone on them. In mathematical terms, let
si be a binary switching variable defined as:
1 if path i is chosen,
si =
(21)
0 otherwise.
Then, the equation for pheromone update using a selective removal model such as EigenAnt
is of the form
Ci
ΔCi = si −αCi + βf (Li )
,
(22)
V
whereas the Ant System update equation, specialized to the case of a single source and
destination node, which assumes evaporative removal (Dorigo 2007; Dorigo et al. 1996) is
ΔCi = −ρCi + si Q/Li ,
for all i,
(23)
where Q is a suitably chosen constant, often set to Q = 1. Also, in the AS model, the path
choice probability for the ith path is given by
C μ (1/Li )ν
,
pi = i μ
ν
j Cj (1/Lj )
(24)
where μ ≥ 0 and ν ≥ 1 are exponents to be suitably chosen and, indeed, an adequate choice
is crucial for the good performance of AS, even though this is not always a simple task
(Yuan et al. 2012).
The main differences to be noted are as follows. The first is that the binary switching
variable si affects both removal and deposition in the case of a selective removal model
like EigenAnt, whereas it affects only deposition in the case of the AS model employing
evaporative removal. The second is that calculation of the path choice probability for the ith
path, pi , in the AS model (24) involves knowledge of all the path lengths Lj , j = 1, 2, . . . , n,
whereas the corresponding calculation for EigenAnt (2) does not.
We performed a set of simulations to compare AS with EigenAnt. We consider a simple
scenario in which there are three paths from the source to the destination, having lengths
L1 = 1, L2 = 2, and L3 = 4, respectively. On each trip, an ant probabilistically chooses one
of the three paths, based on the normalized pheromone concentration on each of them. The
initial pheromone concentration on path 1 is C10 = 1 while it is increased from 1 to 10 on the
other two paths in steps of 1. For each of the 100 initial pheromone concentration vectors
obtained in this way, 1000 ant trips were simulated and the resulting path counter for path
1 divided by 1000 in order to compute the approximate probability of convergence to path
1. In all simulations, f (Li ) = 1/Li . Figures 2 and 3 show these approximate probabilities
for EigenAnt and for AS, respectively. Simulations were performed using an arbitrary (i.e.,
untuned) choice of the parameters. This comparison is, however, merely illustrative of the
possibilities and, without further experiments, can only be interpreted to say that proper
tuning of parameters seems to be more critical for AS than it is for EigenAnt.
The next experiment consisted of fixing the initial concentration at C = (1, 10, 10)T and
plotting the approximate probability of convergence to path 1, despite the heavy initial bias.
In the case of EigenAnt, this is done for all values of α ∈ [0.1, 1] and β ∈ [0.1, 1], while
for AS, it is done for all values of ρ ∈ [0.1, 1] and Q ∈ [0.1, 1]. Plots of approximate convergence probabilities for EigenAnt and for AS are shown in Figs. 4 and 5, respectively.
AS uses four parameters: ρ, Q, μ, and ν. In order to display three dimensional plots of the
approximate probabilities, AS parameters μ and ν have been fixed at 0.5 and 2, respectively.
The plots show that EigenAnt is less sensitive to the choice of parameters.
Swarm Intell
Fig. 2 Variation in the approximate probability of convergence to the shortest path (path 1), where L1 = 1,
L2 = 2 and L3 = 4, with respect to increasing pheromone concentrations on longer paths. The initial
pheromone concentration on path 1 is C10 = 1 while it is increased from 1 to 10 on the other two paths
in steps of 1, using EigenAnt’s selective removal of pheromone, with α = 0.8, and β = 1. The approximate
probability of convergence to path 1 is always 1, irrespective of increasing initial pheromone concentrations
C20 and C30 on the longer paths
Fig. 3 Variation in the
approximate probability of
convergence to the shortest path
(path 1), where L1 = 1, L2 = 2
and L3 = 4, with respect to
increasing pheromone
concentrations on longer paths.
The initial pheromone
concentration on path 1 is C10 = 1
while it is increased from 1 to 10
on the other two paths in steps
of 1. Pheromone update is carried
out using (23) with μ = 1, ν = 1,
ρ = 0.2, Q = 1. The approximate
probability of convergence to
path 1 decreases with increasing
bias on the longer paths. Note
that the parameters μ, ν, ρ, Q
have not been optimized
4 Numerical experiments
In this section, we provide simulation results for EigenAnt under different conditions. In all
simulations, f (Li ) = 1/Li . First, we test whether EigenAnt finds the shortest path, when all
paths other than the shortest one have significantly larger initial pheromone concentrations.
All the simulations were performed on a laptop with 3 GB RAM and an Intel Core 2 Duo
CPU with a 2.66 GHz system clock.
Swarm Intell
Fig. 4 Approximate probability
of convergence to the shortest
path (path 1), using EigenAnt,
with respect to variation in
parameter values. The path
lengths are L1 = 1, L2 = 2, and
L3 = 4. The initial pheromone
concentration vector C is
(1, 10, 10)T for all simulations.
The approximate probability of
convergence to path 1 is
always 1, regardless of the
choices of α and β, showing
robustness to the choice of these
parameters
Fig. 5 Approximate probability
of convergence to the shortest
path (path 1) with respect to
variation in parameter values.
The path lengths are L1 = 1,
L2 = 2, and L3 = 4. The initial
pheromone concentration
vector C is (1, 10, 10)T for all
simulations. Pheromone update is
carried out using AS (23), for
which parameters μ and ν have
been fixed at 0.5 and 2,
respectively. The approximate
probability of convergence to
path 1 decreases rapidly for large
values of Q, but it is relatively
robust to changes in ρ, for Q
below 0.5
In the second set of experiments, we moved the destination during the experiment after
EigenAnt has converged to the shortest path. In other words, we permute the path lengths,
and a different path now becomes the shortest one. We test whether EigenAnt finds the
shortest path after this change in path lengths.
Thirdly, we examine the case when all paths are not known a priori, but are discovered
according to some exploration probability. We investigate how EigenAnt adapts in response
to these discoveries.
4.1 Convergence in the presence of a large initial bias on sub-optimal paths
In each experiment, there were ten possible paths between the source and the destination.
The length of each path is given by its index, for example, path 7 is 7 units long: L7 = 7.
The diagonal elements, di , of D were chosen as di = f (Li ) = L1i . The initial pheromone
distribution was chosen as Ci0 = i, intentionally biased towards long paths, with path 10
Swarm Intell
Fig. 6 The plots show (a) the
evolution of normalized
pheromone concentration and
(b) the number of ant trips on
each of a set of ten paths whose
lengths range from 1 to 10, with
path length Li = i. The initial
pheromone distribution was
chosen as Ci0 = i, intentionally
biased towards long paths, with
path 10 having 10 times more
pheromone than path 1. Despite
this, the pheromone rapidly
concentrates on the shortest path.
The parameter values used are
α = 0.5, and β = 1. The plots
correspond to a snapshot taken
after a total of 80 ant trips
initially having 10 times more pheromone than path 1. Large initial pheromone bias on suboptimal paths causes many ant algorithms to fail to converge to the optimal path (Chen et al.
2009). Figures 6 and 7 show the evolution of normalized pheromone concentration on a
set of ten paths. Despite the large initial bias, pheromone gets rapidly concentrated on the
shortest path.
4.2 Convergence results with changing path lengths
In this set of experiments, the initial path length vector was L = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
and was changed, at t = 80, to L = [10, 9, 8, 7, 6, 5, 4, 3, 2, 1], so that path 1, which was
initially the shortest path, with L1 = 1, became the longest one at t = 80. At this time, the
erstwhile longest path, path 10, became the shortest one, with length L10 = 1. Figures 8
and 9 present simulation results for the case when f (Li ) = 1/Li , showing that the nor-
Swarm Intell
Fig. 7 The plots show (a) the
evolution of normalized
pheromone concentration and
(b) the number of ant trips on
each of the ten paths in the
example referred to in Fig. 6. The
plots correspond to a snapshot
taken after 500 ant trips
malized pheromone distribution corrects itself, demonstrating the plasticity of the proposed
EigenAnt dynamics.
4.2.1 Experimental results on larger graphs
Practical applications of ant colony optimization involve finding the shortest path on a general graph, where nodes other than the source and destination node are present, and paths
from the source to the destination are composed of multiple edges spanning intermediate
nodes. Application of EigenAnt to such examples is on the lines of the procedure adopted
by well known ACO algorithms, using ‘forward’ and ‘backward’ ants (i.e., ants going from
source to destination and those going from destination to source). At each node, the relative
pheromone concentration of outgoing edges, where the directionality is defined by the type
of ant (forward or backward), is used to determine the probability of choosing any one of
Swarm Intell
Fig. 8 The plots show (a) the
normalized pheromone
concentration and (b) the number
of ant trips made on each of a set
of ten paths of lengths ranging
from 1 to 10, with path length
Li = i, corresponding to a
snapshot taken after a total of 120
ant trips. The initial pheromone
distribution was chosen as a
vector in R10 , with each entry
being 1/10. The initial path
length vector was
L = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
and was changed, at t = 80, to
L = [10, 9, 8, 7, 6, 5, 4, 3, 2, 1].
As a result, path 1, which was
initially the shortest path with
L1 = 1 became the longest one at
t = 80. The parameter values
used are α = 0.5, β = 1, and
k = 1. The path choice
probabilities are in the process of
converging to e1 when the
change in the length vector
occurs
them, while the deposition and removal of pheromone is as prescribed earlier. Figure 10
shows an example with 11 nodes, in which all paths between the source and the destination
have four segments. At node 4, for example, a forward ant chooses a path to one of nodes 5,
6, and 7, since outgoing edges from node 4 lead to one of these nodes.
Figure 11 shows a plot of the lengths of trips made by ants, against the total number of
ant trips. The initial pheromone concentration on each edge was chosen to be equal to its
length. Each path traversal is marked by a dot. In this example, paths between the source
and the destination range in length from 13 to 27. The figure shows that as ant trips progress,
the shortest path is chosen increasingly often. The shortest path, which has a length of 13,
appears as a dense black line in Fig. 11(a), while longer paths appear as isolated dots. The
histogram, Fig. 11(b), shows that most trips are made on the shortest path.
We also ran EigenAnt on a larger example, with 10 stages of 10 nodes each between
the source and the destination. Each node in a stage is connected to every node in the next
Swarm Intell
Fig. 9 The plots show (a) the
normalized pheromone
concentration and (b) the number
of ant trips made on each of a set
of ten paths of lengths ranging
from 1 to 10, with path length
Li = i, for the example referred
to in Fig. 8, corresponding to a
snapshot taken after 400 ant trips.
The path choice probabilities are
in the process of converging
to e1 , when the change in the
length vector occurs, and
subsequent convergence is to e10
Fig. 10 A three stage multi-hop
network with three nodes in each
stage. Node 1 is the source node
and 11 is the destination node.
The lengths of each edge are
indicated. The shortest path has a
length of 13, and is shown by the
heavier lines connecting nodes
1–4–7–9–11. Observe that the
optimal path from source to
destination requires the choice of
locally nonoptimal edges at
nodes 1 and 4
Swarm Intell
Fig. 11 (a) Evolution of lengths
of paths traversed by ants against
the total number of ant trips.
Each traversed path is marked by
a dot against its length. Initial
pheromone bias causes trails to
be formed on longer paths at first.
Over time, EigenAnt updates
cause ants to choose the shortest
paths more frequently.
Eventually, pheromone gets
concentrated only on the shortest
path. Parameter values are
α = 0.1, β = 2, and
f (Li ) = 1/Li . (b) Histogram of
traversed path lengths, in the case
of the network of Fig. 10
stage. The number of possible routes is 1010 . Edge lengths were assigned random values,
and the initial pheromone on each edge was chosen to be equal to its length. Figure 12 shows
the evolution of traversed path lengths against the number of ant trips. In this example, path
lengths range between 13 and 72. Each trip is marked by a dot against its length. The shortest
path with length 13 appears as a dense line, indicating that most trips are made on this path
as time progresses.
4.3 Convergence when new paths are discovered according to an exploration probability
Thus far, we have assumed that in EigenAnt all available paths are known or, equivalently,
that all paths have a non-zero initial pheromone concentration. We now examine the case
when all paths are not known in advance, or, equivalently, that some paths initially have
zero pheromone concentration. Suppose that an exploration probability q is specified. In
this case, EigenAnt can be thought of as a two stage process. The first stage consists of a
Swarm Intell
Fig. 12 Lengths of traversed
paths against the number of ant
trips. The total number of
possible routes between source
and destination is 1010 . Each
traversed path is marked by a dot
against its length. Initial
pheromone bias causes trails to
be formed on longer paths at first.
Over time, EigenAnt updates
cause ants to choose the shortest
path more frequently. The plot
shows only the first 4000 ant trips
Fig. 13 This plot shows how the
probability of choosing paths
varies when newer paths are
discovered. The simulation shows
that trails switch to newer paths
only if they are shorter. Please
refer to the text for further details
decision between exploration and exploitation of prior knowledge: in mathematical terms, a
pheromone-free path is chosen with probability q, while a path with non-zero concentration
is chosen with probability (1 − q). In the former case, the pheromone concentration of the
chosen path is set to a prespecified (small) positive value, and the current number of known
paths incremented by one, after which, in the second stage, the steps of Algorithm 1 are
followed. In the latter case, in which a known path is chosen, the first and second stages
collapse into one, which consists of merely applying Algorithm 1.
Figure 13 shows the results obtained for a simulation run in which the exploration probability was chosen as q = 0.1. Initially, only paths 1, 2, and 3 are known. The initial
pheromone concentrations on these paths are 0.8, 0.7, and 0.1, respectively. The lengths
of the 10 paths are 10, 9, 8, 6, 7, 5, 4, 2, 1, and 3. The shortest path, path 9, which has a
length of 1, is discovered after about 160 ant trips. The simulation shows that trails switch
to newly discovered paths that are shorter, but otherwise remain on the previously found
shorter paths.
Swarm Intell
This behavior may be understood in the light of the analysis presented in Sect. 3. As a
new path gets discovered, the D matrix gets updated, and a new diagonal element is added
to it. If the maximal eigenvector of the updated D matrix is larger than the previous one, the
previous eigenvector becomes unstable. This may be understood from (20). Of course, if the
maximal eigenvector is the same as the earlier one, indicating that the new path is not shorter
than the currently best known one, then the system remains in its previous equilibrium, and
the amount of pheromone on the new path does not grow.
5 Conclusion
In this paper we have proposed a new ant colony optimization algorithm called EigenAnt.
The principal novel feature of EigenAnt is the use of negative feedback in the form of “selective removal of pheromones”: in other words, removal only from the probabilistically
selected path. This is in contrast with evaporation, normally used in existing ACOs, which
affects all paths, regardless of whether they are chosen or not.
There are two problems that lead to deterioration in the performance of a conventional
ACO algorithm—stagnation and premature stabilization at a sub-optimal solution. For the
problem of selecting the shortest among a set of paths directly connecting the source and
the destination nodes, we provided a theoretical stability analysis of the proposed EigenAnt
algorithm, as well as simulation results that confirm that the EigenAnt algorithm prevents
premature stabilization at a sub-optimal solution, even if initial pheromone concentrations
on sub-optimal paths are much larger than on the shortest path. Furthermore, we have experimentally shown that EigenAnt does not suffer from stagnation when new paths are discovered, when path lengths change over time, or when paths that were previously optimal
ones suddenly become sub-optimal. Even in dynamically changing scenarios, pheromone
concentrations in EigenAnt quickly change to reflect the new situation. These properties
suggest that the proposed EigenAnt algorithm might be successful also when applied to
more challenging combinatorial optimization problems. We are currently working in this
direction.
Acknowledgements The authors would like to thank reviewers 2 and 3 for constructive criticism, as well
as reviewer 1, the Associate Editors and the Editor for detailed comments and suggestions. The work of the
first author (J) was partially supported by a grant from the DST (Indo–Brazil International Collaboration).
The work of the last author (AB) was partially supported by grants from FAPERJ (CNE) and CNPq (BPP,
Brazil–India International Collaboration).
References
Abdelbar, A. M., & Wunsch, D. C. (2012). Improving the performance of MAX-MIN ant system on the
TSP using stubborn ants. In Proceedings of the fourteenth international conference on genetic and
evolutionary computation conference companion, GECCO Companion’12 (pp. 1395–1396). New York:
ACM.
Bandieramonte, M., Di Stefano, A., & Morana, G. (2010). Grid jobs scheduling: the alienated ant algorithm
solution. Multiagent and Grid Systems, 6(3), 225–243.
Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). Swarm intelligence: from natural to artificial systems.
New York: Oxford University Press.
Chen, L., Sun, H. Y., & Wang, S. (2009). First order deceptive problem of ACO and its performance analysis.
Journal of Networks, 4(10), 993–1000.
Deneubourg, J. L., Aron, S., Goss, S., & Pasteels, J. M. (1990). The self-organizing exploratory pattern of the
Argentine ant. Journal of Insect Behaviour, 3, 159–168.
Swarm Intell
Di Caro, G., & Dorigo, M. (1998a). Ant colonies for adaptive routing in packet-switched communications
networks. In A. E. Eiben, T. Bäck, M. Schoenauer, & H. P. Schwefel (Eds.), Lecture notes in computer
science: Vol. 1498. Parallel problem solving from nature—PPSN V: 5th international conference (pp.
673–682). Berlin: Springer.
Di Caro, G., & Dorigo, M. (1998b). AntNet: distributed stigmergetic control for communications networks.
Journal of Artificial Intelligence Research, 9, 317–365.
Ding, Y. Y., He, Y., & Jiang, J. P. (2003). Multi-robot cooperation method based on the ant algorithm. In
Proceedings of the swarm intelligence symposium, 2003. SIS’03 (pp. 14–18). New York: IEEE Press.
Dorigo, M. (2007). Ant colony optimization. Scholarpedia, 2(3), 1461. http://www.scholarpedia.org/
article/Ant_Colony_Optimization.
Dorigo, M., & Gambardella, L. M. (1997). Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1), 53–66.
Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: optimization by a colony of cooperating agents.
IEEE Transactions Systems, Man, Cybernetics-Part B, 26(1), 29–41.
Dorigo, M., & Stützle, T. (2004). Ant colony optimization. Cambridge: MIT Press.
Ducatelle, F., Di Caro, G. A., & Gambardella, L. M. (2010). Principles and applications of swarm intelligence
for adaptive routing in telecommunications networks. Swarm Intelligence, 4(3), 173–198.
Ghazy, A. M., El-Licy, F., & Hefny, H. A. (2012). Threshold based AntNet algorithm for dynamic traffic
routing of road networks. Egyptian Informatics Journal, 13(2), 111–121.
Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computation. Redwood
City: Addison-Wesley.
Jackson, D. E., Martin, S. J., Holcombe, M., & Ratnieks, F. L. W. (2006). Longevity and detection of persistent foraging trails in Pharaoh’s ants, Monomorium pharaonis (L.). Animal Behaviour, 71, 351–359.
Jaffe, K. (1980). Theoretical analysis of the communication system for chemical mass recruitment in ants.
Journal of Theoretical Biology, 84, 589–609.
Mapisse, J., Cardoso, P., & Monteiro, J. (2011). Ant colony optimization routing mechanisms with bandwidth
sensing. In EUROCON—international conference on computer as a tool (pp. 39–42). Lisbon: IEEE
Press.
Merkle, D., Middendorf, M., & Schmeck, H. (2002). Ant colony optimization for resource-constrained project
scheduling. IEEE Transactions on Evolutionary Computation, 6(4), 333–346.
Meyer, B. (2004). Convergence control in ACO. In Lecture notes in computer science: Vol. 3103. Genetic
and evolutionary computation (GECCO) (pp. 1–12), Seattle, WA. Berlin: Springer.
Parpinelli, R. S., Lopes, H. S., & Freitas, A. A. (2002). Data mining with an ant colony optimization algorithm. IEEE Transactions on Evolutionary Computation, 6(4), 321–332.
Reimann, M., Doerner, K., & Hartl, R. F. (2003). Analyzing a unified ant system for the VRP and some of its
variants. In Lecture notes in computer science: Vol. 2611. Proceedings of EvoWorkshops: applications
of evolutionary computing (pp. 300–310). Berlin: Springer.
Reimann, M., Stummer, M., & Doerner, K. (2002). A savings based ant system for the vehicle routing problem. In Proceedings of the genetic and evolutionary computation conference, GECCO’02 (pp. 1317–
1326). San Francisco: Morgan Kaufmann Publishers.
Robinson, E. J. H., Ratnieks, F. L. W., & Holcombe, M. (2008). An agent based model to investigate the roles
of attractive and repellent pheromones in ant decision making during foraging. Journal of Theoretical
Biology, 255, 250–258.
Shah, S., Kothari, R., & Jayadeva Chandra, S. (2008). Mathematical modeling and convergence analysis of
trail formation. In Proceedings of the 23rd national conference on artificial intelligence, advancement
of artificial intelligence (AAAI)’08 (Vol. 1, pp. 170–175). Chicago: AAAI Press.
Shah, S., Kothari, R., Jayadeva, & Chandra, S. (2010). Trail formation in ants: a generalized Polya urn process. Swarm Intelligence, 4(2), 145–171.
Stützle, T., & Hoos, H. H. (2000). MAX-MIN ant system. Future Generation Computer Systems, 16(8),
889–914.
Sumpter, D. J. T., & Beekman, M. (2003). From non-linearity to optimality: pheromone trail foraging by ants.
Animal Behaviour, 66, 273–280.
Van Vorhis Key, S. E., & Baker, T. C. (1982). Trail-following responses of the Argentine ant Iridomyrmex
humilis (Mayr), to a synthetic trail pheromone component and analogs. Journal of Chemical Ecology,
8(1), 3–14.
Yildirim, U. M., & Çatay, B. (2012). A time-based pheromone approach for the ant system. Optimization
Letters, 6(6), 1081–1099.
Yuan, Z., Montes de Oca, M. A., Birattari, M., & Stützle, T. (2012). Continuous optimization algorithms for
tuning real and integer parameters of swarm intelligence algorithms. Swarm Intelligence, 6(1), 49–75.
Zecchin, A. C., Simpson, A. R., Maier, H. R., & Nixon, J. B. (2005). Parametric study for an ant algorithm
applied to water distribution system optimization. IEEE Transactions on Evolutionary Computation,
9(2), 175–191.