Investigation of antnet routing algorithm by
employing multiple ant colonies for packet
switched networks to overcome the stagnation
problem
Firat Tekiner (Phd Student)
Z. Ghassemlooy
Optical Communications Research Group, The
University of Northumbria, Newcastle upon Tyne
S. Alkhayatt
Faculty of ACES, Sheffield Hallam University
LCS 2004
Contents








Background Information
Reinforcement Learning
Behaviour of ants in real life
Antnet routing algorithm
Antnet with Multiple Ant Colonies
Antnet with Evaporation
Simulation Environment and Results
Concluding Remarks
2
Routing Problem
“In internetworking, the process of moving a packet of
data from source to destination.”
A routing algorithm is necessary to find the optimal path
(or the shortest path) from source to destination.
Problems:




Existing algorithms are mostly Table-Based (high cost)
Congestion and contention (requires traffic distribution)
Requires human intelligence
The routing algorithms that are in use are all static
algorithms
3
Reinforcement Learning - Routing

Q-Learning



Ant (software agent) based Routing Algorithms








Q-routing (Boyan et al, 94) (Tekiner et al. 1, 04)
Dual reinforcement Q-routing (Kumar et al., 97 & 01)
ABC routing (Schoonderwoerd et al., 96)
Regular and Uniform ant routing (Subramanian et al., 97)
Antnet (Dorigo et al., 98)
Antnet++ (Dorigo et al., 02)
Improved Antnet (Boyan et al., 02)
Modified Antnet (Tekiner et al. 2, 04)
Antnet with evaporation (Tekiner et al. 3, 04)
Agent Distance Vector Routing (ADVR)
& 02)
(Amin et al., 01
4
Comparison of Algorithms





Antnet uses probabilistic routing tables whereas in
well known Link State and Distance Vector algorithms
routing table entries are deterministic
Antnet uses less resources on the nodes
Antnet is dynamic and self organising whereas
Distance Vector and Link State algorithms require
human supervision
Q-Routing does not guarantee on finding the shortest
path always. Moreover, they can only find a single
path, they cannot explore multiple paths
In antnet stagnation is the main problem (routing
table freezes due to selecting same paths continously)
5
Ants In Nature -
“unsophisticated and simple”










Builds and protects their nests
Sorts brood and food items
Explore particular areas for food, and preferentially
exploits the richest available food source
Cooperates in carrying large items
Migrates as colonies
Leaves pheromones on their way back
Stores information in the nature (uses world as a
memory)
Make decision in a stochastic way
Always finds the shortest paths to their nests or food
source
Are blind, can not foresee future, and has very limited
memory
6
Ants –
How do they find their way ?
Notion of Stimergy: Indirect communication via nature.
Ants don’t know where to go initially, and choose paths randomly
i.
Ants taking the “shorter path” will reach the destinations before the
those taking a long route. The path is marked with pheromone.
ii.
There after the number of ants using the shorter path will keep
increasing, since more pheromone is laid on the path.
(i)
(ii)
(iii)
7
Ants in Antnet

Software Agents (Ants) communicates with each
other by using:



Two types of software agents (ants):



Probabilistic routing tables
An array which represents statistical local traffic
experienced by every node.
Forward Ants (collects information)
Backward Ants (updates prob. table entries)
Two types of queues:


Low priority queue (data packets and forward ants)
High priority queue (backward ants and forward ants)
8
Antnet Algorithm Overview
At regular intervals every node creates a forward ant to
randomly selected destinations.
Forward ants uses probabilistic routing tables together with
queue status at every intermediate node to choose its
output port from unvisited list of nodes.
Time elapsed and node identifier is pushed to ants stack.
If cycle is detected , cycle is deleted from ants memory.
When a forward ant reaches to its destination:





It transforms itself to a backwad ant,

Visits the list of the nodes in its stack in a reverse order,

Updates corresponding entries in the routing tables and array on its
way back to source by using its values stored on its stack.
Ants reinforces the solution by the reinforcement parameter
which is calculated by using trip times that it has
experienced.
9

Antnet - Example
Backward
Forward - Ants Stack
0
1
2
3
4
Node Id
0
1
3
Time
t0
t1
t2
Lets assume that:
Pfd = 7
14and
andPP
==77
ndnd
0
Dest. 1
Node 2
4
Next Node
0
1
4
0.33 0.33 0.33
0.33 0.33 0.33
0.33 0.33 0.33
0.33 0.48
0.26
0.33 0.26
0.33
Routing Table for 3
0
Dest. 2
Node 3
4
Next Node
0
1
2
4
3
0.33 0.33 0.33
0.33 0.33 0.33
0.33 0.48
0.26
0.33 0.26
0.33
0.33 0.33 0.33
Routing Table for 1
Next Node
0
Dest. 2
Node 3
4
1
0.50
0.50
0.50
0.50
0.57
3
0.50
0.50
0.50
0.50
0.43
Routing Table for 0
10
Antnet With Multiple Ant
Colonies
(i)
(ii)
(iii)
(iv)
11
Antnet with Evaporation




Evaporation is a real life scenario where pheromone laid by
real ants evaporates in time due to natural circumstances.
Link usage statistics are used to evaporate (e(x)) the
phernome laid by the ants. It is the proportion of number
of forward ants destined to the node x over the total ants
received by the current node in the given time window.
Frequency of evaporation is defined by the programmer.
Amount of probability to be evaporated is subtracted from
the associated link. Then this amount is equally distributed
over the other links.


P(i)  P(i)  E ( x), i  x

ant _ send ( x) 
 xP( x)
E ( x)  1  N
E ( x)


P
(
i
)

P
(
i
)

,i  x
ant _ send (i )

 i  0

N 1
12
Simulation Parameters

Parallel Virtual Machine (PVM) together with C Language
is used to simulate the algorithm

Every network node is assigned to a different process

Poisson traffic distribution

2500 packets created per node

Average of 15 simulation runs is used for accuracy

No packet loss due to node/link failures


Ant creation rate is set to 5sec per node and
Evaporation rate is set to 0.5 sec per link.
29 Node Random Network is used.
13
Results Table
Multi Antnet
Evaporation
Improved
Antnet
Average Delay
(sec.)
2.121
2.033
2.118
Throughput
(kbits/sec)
5698.966
5265.960
5277.300
0.455
0.521
0.520
Agent Overhead
(%)


Antnet with multiple ant colonies performed
the best in terms of throughput.
Antnet with evaporation performed best in
terms of average packet delay.
14
Results - Throughput
Throughput vs. Simulation Time
Evap
Multi
95
12
0
14
0
16
0
18
5
20
5
22
5
24
5
26
5
28
5
70
50
30
9000
8000
7000
6000
5000
4000
3000
2000
1000
0
10
Throughput(kbytes/sec)
Antnet
Time(sec.)
15
Concluding Remarks







Stagnation is a major problem but solution exists.
Multiple Ant Colonies applied to the antnet routing
algorithm for the packet switched networks.
Multiple Ant Colonies has increased the throughput of the
network whereas there is no improvement observed on the
average delay experienced per packet.
No interaction among the different colonies has been
considered. This can be taken into account in the future.
By evaporating the links probabilities in a predefined rate,
average delay experienced per packet is reduced by 7%.
No mathematical formula only constant variables are used
in antnet.
There is a need for a second meta-heruistic to optimise
antnet’s parameters
16
Thanks!
Any Questions?
17