Locomotion Gait Optimization for a Quadruped Robot
Miguel Oliveira , Cristina Santos, Manuel Ferreira1, Lino Costa 2
1
Introduction
Legged robot gait generation is a challenging task that involves the
control of a large number of degrees of freedom (DOF’s) within a
mechanical structure that varies during locomotion. A large number
of motion parameters have to be considered in order to obtain a stable, natural and efficient locomotion.
Legged robot locomotion applies nonlinear dynamical equations
of high order with a multidimensional and irregular set of parameters.
Therefore, hand-tuning is very hard to achieve and may not ensure
the best results. Biological inspired Evolutionary Computation (EC)
appears as the natural choice for gait optimization of legged robots.
Genetic Algorithms (GAs) is the approach most used.
In this study, we focus on the quadruped gait optimization. Usually
optimization systems [1] are applied to improve the locomotion performance in biped, quadruped and hexapod robots. Robocup is one
of the motivation engines for these works on the AIBO quadruped
robot. However, in these works [1, 7], the robot configuration was
the required to achieve higher velocities: the robot knees were completely folded and use a trot gait locomotion.
In this contribution, we take inspiration from the vertebrate biological motor systems [6], specifically on the neural control systems involved in generating quadruped locomotion. Motor patterns
are generated by networks of Central Pattern Generators (CPGs),
pools of neurons that produce coordinated patterns of motor activity while being initiated and modulated by simple command signals.
Based on previous work [8], we apply (oscillator-based) differential equations to model a limb-CPG, that generates trajectories for
hip robot joints. These CPGs are modelled as coupled oscillators and
solved using numeric integration. They have been previously applied
in drumming [4].
The presented work takes part of a larger project which aims at developing a closed loop control architecture based on dynamical systems for the autonomous generation of robust, adaptive goal-directed
quadruped locomotion in unknown, rough terrain. One objective of
the project is to generate different types of gaits and be able to
achieve transition among these gaits. Therefore, in this work we propose an approach to optimize a specific quadruped gait, a slow crawl
gait, which has to respect a large duty factor β . In fact, it is a slow
statically stable gait since three legs are kept in ground contact, suitable for postural control. The goal was not to achieve the highest
possible velocity in any gait, but rather to achieve the highest velocity for a slow crawl gait, which has to respect a given duty factor and
certain phase relationships.
1
2
Industrial
Electronics
Department,
School
of
Engineering,University of Minho, Portugal, email: [email protected],
[email protected], [email protected]
Production Systems Department, School of Engineering, University of
Minho, Portugal, email: [email protected]
In order to achieve the desired crawl gait, it is necessary to appropriately tune the CPG parameters. The proposed CPG is based on
Hopf oscillators, and allows to explicitly and smoothly modulate the
generated trajectories according to changes in the CPG parameters
such as amplitude, offset and frequency. We take advantage of this
particularity and combine the proposed CPG approach with an optimization strategy, the GA [5], that searches for the best set of CPG
parameters that result in the desired gait. Speed, vibration and stability are the evaluated criterions used to explore the parameter space
of the network of CPGs to identify the best crawl pattern.In previous
works [2], we noticed that solving this problem requires a considerable computational effort. Notably because several constraints are
imposed in this optimization problem. Thus, alternative techniques
for handling constraints can make the search more efficient.
In this work, several experiments on a simulated Aibo robot, are
performed in order to compare the performance of two distinct constraint handling techniques embedded onto the GA. The experimental results, demonstrate that our approach allows low vibration with
a high velocity and wide stability margin for a quadruped slow crawl
gait. Better results were obtained by the tournament selection in
terms of the evaluation criterion and the computational times. The
application of the chosen combination of technologies to quadruped
walking is new, as is the comparison of the conflict resolution mechanism. Furthermore, despite intensive research, it is our belief that
there are still many open questions in gait optimization, and further research should be done to improve the performance of EC approaches in this field.
2 Locomotion Generation
In this work we address the crawl gait generation [8]. This is a slow
symmetric gait (all the legs have the same duty factor).The rhythmic
movements of each hip joint of a limb, i=(FL,FR,HL,HR, are generated by a Hopf oscillator, given by
ẋi
=
żi
=
³
´
α µ − ri2 (xi − Oi ) − ω zi ,
³
´
α µ − ri2 zi + ω (xi − Oi ) ,
(1)
(2)
q
where ri = (xi − Oi )2 + z2i , amplitude of the oscillations is given
√
by A = µ , ω specifies the oscillations frequency (in rad s−1 ) and
relaxation to the limit cycle is given by 2 α1 µ .
This oscillator contains an Hopf bifurcation from a stable fixed
point at (xi , zi ) = (Oi , 0) (when µ < 0) to a structurally stable, harmonic limit cycle, for µ > 0.
The frequency (ω ) allows an independent control of speed of the
ascending and descending phases of the rhythmic signal, meaning an
independent control of the stance ωst and the swing durations ωsw .
We have four CPGs, one for each Hip joint. These four CPGs are
coupled in order to achieve the limb coordination required in a walking gait pattern.
3
Optimization System
We use a stochastic algorithm framework to search the optimal combination of the CPG parameters. A scheme of the optimization system is depicted in fig 1. Each chromosome consists in these 7 CPG
Locomotion System
Initial
Population
parameters
Robot
Servos
CPGs
vibration
velocity
wsm
no
Genetic
Algorithm
Fitness
Evaluation
Stopping
Criteria?
yes
Best
chromosome
Figure 1. Optimization Locomotion System
free parameters:amplitude of the fore and hind limbs ( µFL , µHL );
fore and hind limbs knee stance angle (KFL , KHL ); fore and hind
limbs offset (OFL , OHL ) and swing frequency (ωsw ).
Genetic Algorithms (GA) are population based algorithms that use
techniques inspired by evolutionary biology such as inheritance, mutation, selection, and crossover. In order to recombine and mutate
chromosomes, the Simulated Binary Crossover (SBX) and Polynomial Mutation were considered, respectively.
The performance of each chromosome and thus the resultant walk
gait, is evaluated in terms of robot forward velocity (v), robot body
vibration ( fa ) and stability (WSM),as followed:
f = Wa ∗
fa
fa,max
+Wv ∗
WSM
1
∗ v +WWSM ∗ e− WSMmax ,
v min
Table 1. Parameter bounds
OHL ωsw (rad/s) KFL ( ◦ ) KHL ( ◦ )
OFL
µHL
3600
400
3600 1600
12
127
127
Lower 0.0001 -1600 0.0001 -400
1
-30
5
Upper
1. when two feasible solutions are compared, the one with better objective function value is chosen;
2. when one feasible and one infeasible solutions are compared, the
feasible solution is chosen;
3. when two infeasible solutions are compared the one with smaller
constraint violation is chosen.
Therefore, the fitness function, in which infeasible solutions are compared based on only their constraint violation, can be expressed by:
(
f (x)
if g j (x) ≤ 0, ∀ j
F(x) =
(5)
fmax + ∑8j=1 |max(0, g j (x))| otherwise
where f(x) is the objective function given by Eq. 3, fmax is the
objective function value of the worst feasible solution in the population and g j (x) are the 8 inequality constraints to be satisfied. This
approach can be only applicable to population-based search methods
such as GAs. An important advantage of this approach is that it is
not required to compute the objective function value for infeasible
solutions.
4 Experiments
(3)
where Wa ,Wv and WW SM are the vibration, velocity and WSM
weights, respectively.
The search range of the CPG network parameters directly depend
on the Aibo Ers-7 robot, and were set before hand as shown in Table 1.
µFL
where li , ui are the lower and upper limit of i component, respectively. The problem has several simple boundary constraints (see Table 1) as well as inequality constraints (for OFL , OHL , KFL and KHL ).
In order to handle the inequality constraints, two approaches are
implemented: a repairing method and the tournament based constraint method [3]. In the approach based on a repairing mechanism,
any infeasible solution is repaired exploring the relations among
variables expressed by the inequality constraints, i.e., the values
OFL , OHL , KFL and KHL are repaired in order to satisfy the constraints.
The second approach is based on the tournament based constraint
method proposed by Deb [3] that is based on a penalty function
which does not require any penalty parameter. For comparison purposes, tournament selection is exploited to make sure that:
In order to handle the simple boundary constraints, each new generated chromosome is projected component by component in order
to satisfy boundary constraints as follows:


li if xi < li
xi = xi if li ≤ xi ≤ ui
(4)


ui if xi > ui
In this section, we describe the experiment done in a simulated ers-7
AIBO robot using Webots [9]. Webots is a software for the physic
simulation of robots based on ODE, an open source physics engine
for simulating 3D rigid body dynamics.
At each sensorial cycle (30 ms), sensory information is acquired.
Each chromosome is evaluated during 12 seconds. We apply the Euler method with 1ms fixed integration step, to integrate the system
of equations. At the end of each chromosome evaluation the robot is
set to its initial position and rotation, such that initial conditions are
equal for the evaluation of all chromosomes of all populations.
In our implementation, we depict results when a population was
established with 50 chromosomes and a preset number of 50 generations was set. Each experiment is run 30 times because this is a
stochastic-based method.The generated gaits have a fixed duty factor
β = 0.75.
We aim to compare the performance of the two constraint handling techniques: the repairing mechanism and the tournament based
constraint method, embedded in the genetic algorithm to solve the
optimization problem of the CPG parameters.
5 Results
Table 2 contains the Best, Mean and standard deviation (SD) values
of the solutions found (in terms of fitness function and time) over the
Table 2. Performance of GA algorithm in the optimization system
Fitness
Constraint handling Technique Best
SD
Best
Mean
SD
0.2088 0.2817 0.0743 0.0704 0.9721 0.4367
Repairing
0.2727 0.2973 0.0212 3.5350 4.0843 2.0975
30 runs, for the two constraint handling technique used. The best individual with the tournament mechanism has a fitness value of 0.2088
and the best individual with the repairing mechanism has a fitness
value of 0.2727, that was achieved at generation 50. When comparing stochastic algorithms, the most important is the metric in terms of
average values, since it reports the central tendency of the results over
the runs. By analyzing Table 2, we can see that the tournament selection method is the least time consuming algorithm and with the best
mean fitness value. We also remark that the tournament method had
0.9721 hours as average time, while the repairing method had 4.0843
hours as average time. This difference is justified for not being necessary to compute the objective function value for infeasible solutions.
Figures. 2(a) and 2(b) show the best (solid line) and mean (dashed
line) fitness function evolution of the 30 runs for the tournament and
repairing mechanism, respectively. Minimum and maximum values
over the 30 runs are represented by the error bars. Table 3 lists the
25
Fitness
20
15
10
5
0
5
10
15
20
25
30
Constraint handling Technique Vibration Velocitym s− 1 W SM(mm) Fitness
Tournament
0.0425
0.0804
23.288 0.2088
Repairing
0.0233
0.0500
31.253 0.2727
Time (hours)
Mean
Tournament
0
Table 3. Optimization Results
35
40
45
50
Generations
(a) Tournament mechanism.
ments were obtained over the previous functioning but non-optimal
quadrupedal gait. The walking speed was improved by around
(40.58%), the wide stability by 74.44% and vibration of 46.46%
compared to the hand-tuned gait.
6
Conclusions and Future work
In this article, we have addressed the locomotion optimization of a
quadruped robot that walks with a slow walking gait.
A locomotion controller based on dynamical systems to model
CPGs, generates quadruped locomotion. These CPG parameters are
tuned by an optimization system. This optimization system combines
CPGs and a genetic algorithm. Moreover, two constraint handling
techniques were embedded in the genetic algorithm to solve the constrained optimization problem.
The aim was to compare the performance of the two constraint
handling techniques according to the evaluation criterion and the
computational times. The best results were obtained by the tournament selection. However, both techniques were able to obtain a set of
parameters adequate for the implementation of a quadruped walking
gait with low vibration, high stability margin and a high velocity.
Currently, we are using other optimization methods such as evolutionary strategies , as well as other fitness functions. We will address other locomotion related problems, such as: the generation and
switch among different gaits according to the sensorial information
and the control of locomotion direction.
7 Aknowledgments
Work supported by the Portuguese Science Foundation (grant
PTDC/EEA-CRO/100655/2008).
0.46
0.44
REFERENCES
0.42
Fitness
0.4
0.38
0.36
0.34
0.32
0.3
0.28
0.26
0
5
10
15
20
25
30
35
40
45
50
Generations
(b) Repairing mechanism.
Figure 2. Fitness evolution during 50 generations.
achieved values of vibration, velocity and W SM for the best solution
found using the two constraints handling techniques. The velocity is
0.0804 (m s− 1) and 0.05 (m s− 1) for the tournament selection and
the repairing technique, respectively.
We have further used a hand-tuned gait as a seed to evolve
the quadrupedal gait. After 50 of evaluations significant improve-
[1] S. Chernova and M. Veloso. An evolutionary approach to gait learning
for four-legged robots. In In Proceedings of IROS’04, September 2004.
[2] H. M. M. F. Cristina Santos, Miguel Oliveira and L. Costa. Locomotion
gait optimization for a quadruped robot. 3rd International Workshop on
Evolutionary and Reinforcement Learning for Autonomous Robot Systems(ERLARS 2010), 2010.
[3] K. Deb. An efficient constraint handling method for genetic algorithms.
[4] S. Degallier, C. Santos, L. Righetti, and A. Ijspeert. Movement generation using dynamical systems: a humanoid robot performing a drumming
task. In IEEE-RAS International Conference on Humanoid Robots, 2006.
[5] D. Goldberg. Genetic Algorithms in Search, Optimization, and Machine
Learning. Addison-Wesley, 1989.
[6] S. Grillner. Neurobiological bases of rhythmic motor acts in vertebrates.
Science, 228(4696):143–149, 1985.
[7] M. Hebbel, W. Nistico, and D. Fisseler. Learning in a high dimensional
space: Fast omnidirectional quadrupedal locomotion. In RoboCup 2006:
Robot Soccer World Cup X.
[8] V. Matos, C. P. Santos, and C. M. A. Pinto. A brainstem-like modulation
approach for gait transition in a quadruped robot. In IROS, pages 2665–
2670, 2009.
[9] O. Michel. Webots: Professional mobile robot simulation. Journal of
Advanced Robotics Systems, 1(1):39–42, 2004.