Mobile Robotics COM596
Laboratory Session: week 4
Neural Networks for Khepera Robot Control using Matlab
Exercise 1: Matlab code for obstacle avoidance of Khepera
%This program is running OK avoiding
%obstacles on left/right or in front of
%Date of update 4/7/2009
ref=kopen([0,9600,100]);
for i=1:100
v=kProximity(ref);
u=kAmbient(ref);
d5=v(5);
d6=v(6);
lvL=u(7);
lvR=u(8);
lv=(lvL+lvR)/2;
%direction is set by
if(lv<1/15)
kMoveTo(ref,5,-1) %move right
%position in pulse 1 pulse = 1/12 =0.083mm
end
x=rand(1);
if(x==0)
kSetSpeed(ref,5,-1) %move right
else if(lv>1/15)
kSetSpeed(ref,-1,5) %move left
else
kSetSpeed(ref,4,-1) %move right
end
end
end
kMoveTo(ref,0,0);
kclose(ref);
Above code will be useful for developing the Braitenberg vehicle in Exercise 2.
Example 2: Braitenberg Vehicles: Moving around while avoiding obstacles is a key
issue in autonomous agent research. Braitenberg has proposed a simple architecture
for such tasks. Figure 4.1 shows a control architecture inspired by Braitenberg vehicle
based on the idea that a sensor with high proximity activation accelerates the motor on
the sensors’ side. The activation of the left and right motors M l and M r is expressed
8
8
i 1
i 1
by M l   Pi wil  w0l and M r   Pi wir  w0r respectively, where Pi denotes the
activation of proximity sensor i , wil and wir denote weights that connect the proximity
sensors with left and right motors respectively. w0l and w0r represent the idle activation
of the left and right motors respectively that are responsible for forward motion of the
robot in absence of obstacle.
Ml
Mr
Figure 4.1: Control architecture for Braitenberg vehicle.
The problem here is to determine the weights wil , wir , w0l , w0r , i  1,2, ,8 such that the
robot is moving around while avoiding obstacles. Since no training patterns are
available for training, thus the reinforcement/random search learning is an ideal
candidate for this learning problem.
In the learning of Braitenberg vehicle, the first important thing is to find an initial
controller that exhibits a meaningful behaviour. In order to exploit the morphology of
the robot, the left and right part of the controller weights are constrained to equal
wil = wir and w0l = w0r .
All weights wi , i  1,2, ,8 are initialised with very small negative random values,
i.e., wi   0.5,0 and the idle activation weights w0 are set to very small positive
values, i.e. w0  0,0.1 .
In order to learn a good Braitenberg vehicle, the reward function has to incorporate
motor speeds and distances to obstacles. The use of speed and distance in reward
function may result in useless activity such as spinning of the robot with high speed
far from the obstacles. Therefore, a penalty term vt  Vl  Vr has been used in the
fitness function to avoid spinning. Hence, the fitness measure for the learning strategy
at time t is defined as
Vl  Vr  ,


f t  Vt 1  vt 1  IPˆt

(4.1)
IPˆt  max IPi , i  1,2, ,8 . Vl and Vr are the left and right
2
wheel speed of the robot respectively. The term 1  vt ensures the two wheels to
rotate in the same direction making it to move forward. The fitness evaluation is
carried out over a period of time, e.g. t max  100 times. The final reward value over
t max will thus be
Where Vt 

t max



F  Vt 1  vt 1  IPˆt
t 1

(4.2)