An Adaptive Fitness Function for Evolutionary Algorithms
Using Heuristics and Prediction
Ping Tang1 and Gordon K.Lee2
1
Faculty of Computer
Guangdong University of Technology
Guangzhou 510090
China
Email: [email protected]
2
Dept. of Electrical & Comp. Engr
5500 Campanile Drive
San Diego State University
San Diego, California 92182-1326
Email:[email protected]
ABSTRACT
A genetic algorithm usually performs a search over a complex and multimodal space and is an
important component in several applications such as evolutionary learning and optimization. The
search is dependent on several parameters including the fitness function, parent selection process,
mutation rate and crossover rate. The fitness function is an important component in the
evolutionary process since this performance metric is used to select the best individuals in a
population that will then evolve through the mutation, crossover and reproduction process in
successive generations. In this paper, a fitness function is developed that employs heuristic
information based upon past history, current information and future knowledge; in particular,
prediction and expectation are integrated into the fitness function. Simulation results show an
improvement over classical fitness techniques.
Keywords: genetic algorithms, heuristics, fitness function
1. INTRODUCTION
Many applications of evolutionary algorithms focus on classes of parametric optimization
problems. In these approaches, genetic searching consists of a finite number of iterations of three
steps at each generation: selection of the parent chromosomes for the next generation (usually an
elitist percentage of the population), recombination using crossover and mutation operations [1],
and a fitness function that describes the goodness of individual members of each generation.
Proper selection of the fitness function can speed up the evolutionary process in a genetic
algorithm [2].
There are a number of approaches to constructing the fitness function, based upon the
required objectives; some examples include [3-4] summing up all the weighted individual
objective functions or multiplying the individual objectives (as in the classical example of the
Khepera robot). Pedrycz [5] employs fuzzy sets and weak encoding while Bhat uses adaptive
fuzzy weights to improve fitness performance [6].
These techniques can be beneficial for many classical evolutionary algorithms in
designing a fitness function. However, the fitness function should not only draw upon stationary
history; Sankar [7], for example, provides an approach for measuring the fitness of an individual
by considering its own fitness as well as that of its ancestors. Further, he proposes that crossover
and mutation probabilities may also be made adaptive to improve performance. Tuning these
probabilities is also used in [10]. Lee and others [8] discuss stability properties of genetic
algorithms and summarize the effects of selection of several evolutionary algorithm parameters
on algorithm convergence.
In this paper, an adaptive fitness function with heuristic information that is based upon
past prediction, current information and future knowledge is presented. Section 2 provides a brief
overview of fitness function structures while Section 3 discusses a heuristic fitness function using
historical data, current information and future knowledge. Section 3 also presents the main
contribution of the paper, a heuristic fitness function that employs prediction to adaptively tune
the heuristics of the fitness function.. Simulations tests are performed and the results are
presented in Section 4. A comparison is made with classical fitness functions and the results
illustrate the feasibility of the proposed heuristic fitness function approach.
2. THE FITNESS FUNCTION STRUCTURE
Consider a classical fitness function of the form:
(1)
f j(n) = ∑ wiui (n)
i
where j denotes the particular individual (chromosome) being evaluated, n is the current
generation number, wi are the weights associated with the importance of each objective function,
and ui(n) represents a particular objective function (e.g., the inverse of a tracking error, power
consumption or total time to complete a task). The summation is over all objective functions. In
general, the larger the value of fj(n), the better is the individual’s fitness. We omit the subscript j
in subsequent discussions for compactness sake.
Note that the fitness function described by (1) is static in that the value of the present
fitness is dependent only upon present information. In order to improve the fitness value, a
heuristic fitness function that is dynamically varied is developed here.
Sankar [7] considers modifying (1) as:
f(n) = β0 ⋅ g(n) + ∑ βi ⋅ a i
(2)
where g(n) is the standard fitness of an individual based upon current information such as (1), ai is
the fitness function of its n ancestors, e.g., ai = f(n-i) and βi are weighting factors, β0>βj for
j=1…M. Here, consider a heuristic fitness function that is not only based upon past and current
information but also on future knowledge. In [9] we employ the fitness function form as:
f(n)= g(n) + h(n) + q(n)
(3)
where h(n) is the fitness function component based upon some historical information related to a
predicted target (such as in (2)) and q(n) is a heuristic function based upon expected future
knowledge. Note here we differentiate between prediction, based upon deterministic historical
data and future heuristics, based upon probabilistic information. Note that we use just one
objective function in this study; the algorithm can be extended to multiple objective functions in a
straight-forward manner.
We have shown that by selecting a forward predictor and heuristic function based upon
expected future knowledge:
h(n) = −δ{g(n) − f̂(n | Fn −1)}
M
f̂(n | Fn -1) = θ(f(n - 1),..., f(n - M)) = ∑ w f(n − k)
k =1 k
(4)
q(n) = λ/f̂ c (n)
where f̂(n | Fn -1 ) is the predictor, δ and λ are weights, and f̂ c (n) is based upon expectation of the
fitness functions. Figure 1 shows the improvement using (4) with a predictor, h(n) and a heuristic
future expectation term, q(n) in the fitness function. One of the drawbacks in [9] however is that
q(n) is computed using a constant desired best final fitness value. This leads to a conservative
estimate as is seen in the transient response of Figure 1. This issue is addressed in this paper and
the algorithm is enhanced by constructing an adaptive expectation.
3. ADDING HEURISTIC INFORMATION ON
EXPECTED FUTURE KNOWLEDGE
The approach discussed in Section 2 considers current and past information in developing
a predictor but only uses the expected best final fitness value in developing the heuristic fitness
function. In effect, the components of Equation (3) are passively adaptive [10] in that, while the
second part predicts the fitness values based upon past and present values, it does not assess the
effect of the selection of GA parameters and current fitness function on future learning. Thus the
heuristic function should also include information from the current state forward to future state
that is unknown. The problem becomes one of finding the heuristic knowledge and its impact on
future fitness.
Figure 1: Comparison of the best and average fitness function values using
simply prediction based on past information (h(n) ,denoted by m2g3) and with
heuristic information from past knowledge (q(n), denoted by hg3)
Following the procedure of [9] and in order to be consistent with (3), the error between
the actual fitness and the desired or best fitness is chosen as the performance metric. The term
g(n) represents the contribution of the present information to present performance, as before. The
term h(n) represents the penalty (4) as below. But now consider the term q(n) which represents
how past and present information affect expected future performance. To see this, define the
heuristic fitness function term q(n) at generation n as:
(5)
q(n) = max{E[f̂(n + N)]}
where E[.] is the expected value and f̂(n + N) is the predicted estimate of the fitness function N
generations ahead. Rather than maximizing the expectation of the fitness function in the future
(5), consider minimizing the prediction error that will occur in the future, where the error is the
difference between f̂(n + N | Fn -1 ) and f*(n+N); the second term is the best fitness function value at
generation n+N, which we do not have. That is:
e(f̂(n + N)) ≡ f * (n + N) − f̂(n + N | Fn - 1)
where we only have information Fn-1 = {f(1), …f(n-1)} at generation n. Notice that this error can
be estimated using standard prediction. We use linear forward prediction here. That is, let
PM(n+N) denote the minimum mean-squared prediction error:
PM (n + N) ≡ E[| e(f̂(n + N)) |2 ]
Then:
M
e(f̂(n + N)) = ∑ a f̂(n − k)
k = -N k
k=0
 1
ak = 
−
w
k
=
1,..., M

k
(6)
where f̂(n + 1),...f̂(n + N) are forward predicted best fitness estimates and f(n), f(n-1),…,f(0) are
the actual values found from previous generations. Following [12], let:
R w0 = r
(7)
T
PM (n + N) = r(0) − r w 0
(8)
where R is the correlation matrix of the fitness function, r is the cross-correlation vector and w0
is the column vector containing the weights wk.
We may combine (7) and (8) into a single matrix as:
0000
r(0) r T   1   PM (n + N )


=

R  − w 0  

 r
(9)
From (9), we find:
r(1)
 r(0)
 r(-1)
r(0)

 ...
...

r(-M + 1) r(-M + 2)
... r(M - 1)   1  PM (n + i )
 

... r(M − 2)  a1  
0

=
...
...   ...   ... 
  

...
r(0)  a M  
0

(10)
Finally, from (9) and (10) using the Yule-Walker Equations and assuming a white-noise
process[12], the values of w1, w2, w3,…, wM can be deduced.
Then, to compute (5) consider formulating the term:
f̂ c (n) = min{pf (n + N) ⋅ E[e(f̂(n + N)] }
(11)
where pf(n+N) be the probability that the best fitness function occurs at generation n+N; that is,
p f (n + N) = P(f * (n + N)) where P(.) denotes probability. Note that it is difficult to compute pf
(n+i). However, (11) may be replaced by:
(12)
f̂ c (n) ≅ min{(pe (n) + pe (n)) ⋅ E[e(f̂(n + N)]}
where p e (n) is the probability that the best fitness function value f*(n) occurs, given the
hypothesis that the algorithm is converging, i.e., that H(n) is true (see below), while p e (n) is the
probability that the best fitness function value f*(n), given hypothesis H (n) is true. The
hypothesis H(n) is defined as:
H(n): f*(n-1) – f(n-1) > f*(n) – f(n)
and H(n) is the complement hypothesis. Now, we do not know the probabilities p e (n) or p e (n) .
But we can estimate the probability that H(n) occurs, given f*(n); further, we assume we have the
probability P(f*(n)). Hence, p e (n) can be approximated by Bayes’ rule as:
pe (n) ≅ P(H(n) | f * (n)) * P(f * (n))}/{P(H(n) | f * (n)) * P(f * (n)) + P(H(n) | f * (n)) * P(f * (n) )} (13)
where f * (n) is the complement of f * (n) occurring.
Finally, if we want to employ the philosophy of Eqn.(3), that is, the larger the fitness
function value is, the smaller should be the value f̂ c (n) , then q(n) is given as follows:
q(n) = λ/f̂c (n)
(14)
where λ is the attenuation factor and λ<<1 because q(n) is merely an offset to the fitness function.
Thus the heuristic fitness function for each chromosome is as follows:
(15)
f j(n) = g j (n) + h j (n) + q j (n) = g j(n) − δ{f j (n) - f̂(n | Fn - 1)} + λ/f̂ c, j(n)
4. SIMULATION RESULTS
To apply the modified evolutionary algorithm, we employ the Khepera robot [4] and use a
maze scenario to test our proposed heuristic fitness function and adaptive control parameters. The
classical Khepera fitness function is:
f j (n) = g j (n) = V(1 − ∆v )(1 − i)
Firstly when adding a forward linear prediction on past prediction, the fitness function is:
f j (n) = g j (n) + h j (n) ⇒ g j (n) − δ(g j (n) − f̂ j (n | Fn −1 )) = V(1 − ∆v )(1 − i) − δ[V(1 − ∆v) (1 − i) − f̂(n | Fn −1 )]
where δ = 0.001, M = 2, w0 = 1, w1 = -0.975, w2 = 0.95 [3]; so the modified fitness
function with h(n) is is:
f j (n) = g j (n) + h j (n) ⇒ V(1 − ∆v )(1 − i) − 0.001* [V(1 − ∆v )(1 − i) − 0.975 * f j (n − 1) + 0.95 * f j (n − 2)]
If we just add q(n) based upon desired constant final bets fitness, the fitness function becomes:
f j (n) = g j (n) + h j (n) + q j (n) ⇒ V(1 − ∆v (1 − i) − 0.001* [V(1 − ∆v )(1 − i) − 0.975 * f j (n − 1) + 0.95 * f j (n − 2)] +
λ/[p e (n + N) * (C − f j (n))]
where λ=0.0015*[C- f(n) ] and C=1 in the Khepera robot [4]. Figure 1 shows the results
Now consider using q(n) based upon an expected future value, using (14) and (12).
Figure 2 shows the improvement of this fitness function. Notice that the transient response is
much better in that the fitness function values reach the best steady-state value much quicker than
does the conservative heuristics approach. Another advantage is the adaptive tuning is that one
does not have to estimate the best desired final fitness value as in [9].
One could combine h(n) and q(n) into a single predictor function if desired, one which
incorporates the effect of past values on the present fitness as well as predict the effects on future
fitness; however the approach taken here is to separate the fitness into three components as in (3).
One may also employ other predictors in formulating q(n), which is an area of future research.
5. CONCLUSIONS
This paper presents a modification of a heuristic fitness function that includes the
effects of past, present and expected knowledge on the value of the present fitness
function. By employing linear prediction, results indicate that the fitness trajectories
follow a faster response towards the desired best value. Future work includes developing
methods to reduce oscillations around steady-state, when compared to the conservation
heuristic results. This would enhance the fitness function trajectory, an important
component of evolutionary algorithms.
fitness 0.45
m1phhg3 best
0.4
hg3best
hg3 averager
0.35
m1phhg3 average
0.3
0.25
0.2
0.15
0.1
0.05
0
0
50
100
150
200
250
300
generations
Figure 2: Comparison of the best and average fitness function values using
heuristics (q(n) in Eqn.. (4), denoted by hg3) and with adaptive prediction (q(n),
in Eqn. (14), denoted by m1phhg3)
6. REFERENCES
[1] D. Floreano and J. Urzelai, “Evolutionary Robots with On-Line Self-Organization and
Behavioral Fitness”, Journal of Neural Networks, Vol. 13, Nos. 4-5, pp. 431-434, May 2000.
[2]D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, AddisonWesley, New Jersey, 1989.
[3]E. Uchibe, M. Yanase, and M. Asada, “Behavior Generation for a Mobile Robot Based on an
Adaptive Fitness Function”, Journal of Robotics and Autonomous Systems, Vol. 40, Nos. 2-3, pp.
69-77, Aug. 2002.
[4]S. Nolfi and D. Floreano, Evolutionary Robotics, The Biology, Intelligence, and Technology of
Self-Organizing Machines, MIT Press/Branford Books, Cambridge, 2000.
[5] W. Pedrycz, Computational Intelligence: An Introduction, CRC Press, 1997.
[6] S. Bhat and G. Lee, “A Fuzzy Inference Function for Evolutionary Learning Systems”, Proc.
of the World Automation Congress, ISSCI, Seville, 2004.
[7] S.K. Pal and P.P. Wang, Genetic Algorithms for Pattern Recognition, CRC Press, 1996.
[8] H.K. Lee and G. Lee, “Convergence Properties of Genetic Algorithms”, Proc. of the ISCA
Int’l Conference on Computers and Their Applications, Seattle, 2004.
[9] P. Tang, S. Bhat, J. Kothari and G. Lee, “A Modified Evolutionary Algorithm Using a
Heuristic Evaluation Function with Mutation Rate and Crossover Rate Tuning”, Proc. Of the
ISCA Int’l Conference on Computers and Their Applications, New Orleans, 2005.
[10] J. Kothari, E. Grant and G. Lee,”Tuning an Adaptive Neural Network Fuzzy Inference
Controller Using Evolutionary Learning”, Proc. Of the ISCA Int’l Conference on Computer
Applications in Industry and Engineering, Orlando, 2004.
[11] P. Tang, S. Bhat, J. Kothari and G. Lee, “A Heuristic Fitness Function for Genetic Algorithms
Using Prediction and Adaptive Parameter Tuning”, SDSU Technical Report 2005-06, 2005.
[12] S. Haykin, Adaptive Filtering Theory, Prentice Hall, New Jersey, 1987.