Applied Soft Computing 28 (2015) 138–149
Contents lists available at ScienceDirect
Applied Soft Computing
journal homepage: www.elsevier.com/locate/asoc
A new particle swarm optimization algorithm with adaptive inertia
weight based on Bayesian techniques
Limin Zhang a,b,c,∗ , Yinggan Tang a , Changchun Hua a , Xinping Guan a,b
a
b
c
Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China
Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China
Department of Mathematics and Computer Science, Hengshui University, Hengshui 053000, China
a r t i c l e
i n f o
Article history:
Received 1 April 2014
Received in revised form 15 October 2014
Accepted 20 November 2014
Available online 9 December 2014
Keywords:
Particle swarm optimization
Monte Carlo
Gaussian distribution
Bayesian techniques
a b s t r a c t
Particle swarm optimization is a stochastic population-based algorithm based on social interaction of bird
flocking or fish schooling. In this paper, a new adaptive inertia weight adjusting approach is proposed
based on Bayesian techniques in PSO, which is used to set up a sound tradeoff between the exploration and
exploitation characteristics. It applies the Bayesian techniques to enhance the PSO’s searching ability in
the exploitation of past particle positions and uses the cauchy mutation for exploring the better solution.
A suite of benchmark functions are employed to test the performance of the proposed method. The
results demonstrate that the new method exhibits higher accuracy and faster convergence rate than
other inertia weight adjusting methods in multimodal and unimodal functions. Furthermore, to show
the generalization ability of BPSO method, it is compared with other types of improved PSO algorithms,
which also performs well.
© 2014 Elsevier B.V. All rights reserved.
1. Introduction
Particle swarm optimization (PSO) was firstly introduced by
Kennedy and Eberhart in 1995 [1]. It belongs to evolutionary algorithm (EA), however differs from other evolutionary algorithms,
which is inspired by the emergent motion of a flock of birds
searching for food. PSO performs well in finding good solutions for
optimization problems [2], and it has become another powerful tool
besides other evolutionary algorithms such as genetic algorithms
(GA) [3]. PSO is initialized with a population of particles randomly
positioned in an n-dimensional search space. Every particle in the
population has two vectors, i.e., velocity vector and position vector. The PSO algorithm is recursive, which motivates social search
behavior among particles in the search space, where every particle
represents one point. In comparison with other EAs such as GAs,
the PSO has better search performance with faster and more stable
convergence rates.
Maintaining the balance between global and local search in the
course of all runs is critical to the success of an optimization algorithm [4]. All of the evolutionary algorithms use various methods
to achieve this goal. To bring about a balance between the two
∗ Corresponding author at: Institute of Electrical Engineering, Yanshan University,
Qinhuangdao 066004, China. Tel.: +86 18230357919.
E-mail address: limin [email protected] (L. Zhang).
http://dx.doi.org/10.1016/j.asoc.2014.11.018
1568-4946/© 2014 Elsevier B.V. All rights reserved.
searches, Shi and Eberhart proposed a PSO based on inertia weight
in which the velocity of each particle is updated according to a fixed
equation [5]. A higher value of the inertia weight implies larger
incremental changes in velocity, which means the particles have
more chances to explore new search areas. However, smaller inertia weight means less variation in velocity and slower updating for
particle in local search areas.
In this paper, the inertia weight strategies are categorized into
three classes. The first class is simple that the value of the inertia
weight is constant during the search or is selected randomly. In [6],
the impact of the inertia weight is analyzed on the performance
of the PSO. In [7], Eberhart and Shi use random value of inertia
weight to enable the PSO to track the optima in a dynamic environment. In the second class, the inertia weight changes with time
or iteration number. We name the strategy as time-varying inertia
weight strategy. In [8,9], a linear decreasing inertia weight strategy
is introduced, which performs well in improving the fine-tuning
characteristic of the PSO. Lei et al. use the Sugeno function as inertia
weight declined curve in [10]. Many other similar linear approaches
and nonlinear methods are applied in inertia weight strategies such
as in [11–13]. The last class is some methods that inertia weight is
revised using a feedback parameter. In [4], a fuzzy system is proposed to dynamically adapt the inertia weight. In [14], the inertia
weight is determined by the ratio of the global best fitness and
the average of particles’ local best fitness in each iteration. In [15],
A new strategy is presented that the inertia weight is dynamically
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
adjusted according to average absolute value of velocity, which follows a given nonlinear ideal velocity by feedback control, which can
avoid the velocity closed to zero at the early stage. In [16], dynamic
acceleration parameters (DAP) method is proposed, which owns
a mechanism to self-tune the acceleration parameters by utilising
the averaged velocity information of the particles. In [17], a new
adaptive inertia weight strategy is proposed based on the success
rate of the particles. In this strategy the success rate of the particles
is used as a feedback parameter to realize the state of the particles in the search space and hence to adjust the value of inertia
weight.
A series of other studies based on mutation strategy has
been done on the analysis and development of the PSO since
it was introduced in 1995. These approaches aim to improve
the PSO’s convergence velocity. In [18], a mutation operator is
used that changes a particle dimension value using a random
number drawn from a Gaussian distribution (GPSO). A particle
is selected for mutation using a mutation rate that is linearly
decreased during a run. In [19], a mutation strategy is proposed
that a particle position is changed using a random number drawn
from a Gaussian distribution. A mutation operator in [20] is similar to that of Ref. [18], but a Cauchy probability distribution
is used instead (CPSO). The Cauchy distribution curve is similar
to the Gaussian distribution curve, except it has more probability in its tails and thus making it more likely to return larger
values. In [21], the HPSO with a wavelet mutation (HWPSO) is
proposed, in which the mutation incorporates with a wavelet function.
Hybrid PSOs (HPSOs) have been proposed to enhance the performance of the PSO, in which different mutation strategies are
used. In [22], premature convergence is avoided by adding a mutation strategy, i.e., a perturbation to a randomly selected particle’s
velocity vector. A Cauchy mutation (HPSO) [23] is proposed, which
is used in best particle mutation, so that the best particle could
lead the rest of the particles to better positions. The algorithm in
[24] called IPSO + ACJ is tested on a suite of well known benchmark
multimodel functions and the results. The main idea of our new
jump strategy is that pbest and gbest are selected as mutated particles when they have not been improved in a predefined number
of iterations. In [25], Ant colony optimization (ACO) and PSO work
separately at each iteration and produce their solutions. In [26],
The new mutation strategy makes it easier for particles in hybrid
MRPSO (HMRPSO) to find the global optimum and seek a balance
between the exploration of new regions and the exploitation of the
old regions.
Fuzzy approaches for PSO is a hot topic in these years. In [27],
a fuzzy system is used to dynamically adjust the inertia weight
and learning factors of PSO in each topology. In [28], a dynamic
parameter adaptation is proposed to improve the convergence and
diversity of the swarm in PSO using fuzzy logic. Valdezis et al.
[29] introduced an improved FPSO + FGA hybrid method, which
combines the advantages of PSO and GA. See Ref. [30] for a comprehensive review on the application of fuzzy logic in PSO, ACO and
Gravitational Search Algorithm (GSA).
There are some works that have used the Bayesian technique in
smart computing. In [31], the Dynamic Bayesian Network (DBN) is
used in PSO algorithm for particle motion. In swarm optimization,
each particle tries to track the trajectory toward the place of better
fitness. Martens et al. [32] explained scientifically the application
of the Bayesian network in ACO. In our paper, Bayesian techniques
is introduced into the PSO algorithm with a view to enhance its
adaptive search ability. We call this algorithm as PSO with Bayesian
techniques (BPSO). Different from other inertia weight strategies
in the references, BPSO algorithm adjusts the inertia weight ω
based on the past particle places automatically. This new development gives particles more opportunity to explore the solution
139
space than a standard PSO. Furthermore, the new algorithm accelerates search velocity for the particles in valuable search-space
regions.
This paper is organized as follows. In Section 2, generic PSO theory is reviewed and the change of particle position ε is analyzed.
Section 3 introduces the advantage of PSO with Bayesian Techniques. Section 4 shows the experimental settings for the benchmarks, simulation results and parameters analysis in the BPSO.
Finally, Section 5 presents conclusions resulting from the study.
2. Particle swarm optimization
2.1. Generic PSO theory
PSO is also a population-based stochastic optimization algorithm and starts with an initial population of randomly generated
solutions called particles. Each particle in PSO has a position and a
velocity. PSO remembers both the best position found by all particles and the best positions found by each particle in the search
process. For a search problem in an n-dimensional space, a potential solution is represented by a particle that adjusts its position and
velocity according to Eqs. (1) and (2):
vi,d (t + 1) = ωvi,d (t) + c1 r1 (Ppid − xi,d (t)) + c2 r2 (Pgd − xi,d (t))
(1)
xi,d (t + 1) = xi,d (t) + vi,d (t + 1)
(2)
where c1 and c2 are two learning factors which control the influence of the social and cognitive components and ri = randi , (i = 1,
2) are numbers independently generated within the range of [0,1].
vi,d (t) is the velocity of individual i on dimension d. xi,d (t) is current
particle position on dimension d. Ppid (pbest) is the best local position of individual i on dimension d, and Pgd (gbest) represents the
best particle position among all the particles in the population on
dimension d. ω is the inertia weight, which ensures the convergence
of the PSO algorithm.
According to Eq. (2), changes in the position of particles depend
exclusively upon the position item of the PSO. Therefore, we only
use the position item to investigate the search ability of particles during iterations. The implicit form of the position equation
presented in Eq. (2) is used for a multi-particle PSO working in a
multi-dimensional search space. Since the data of each dimension
in the PSO are independent, the analysis below will be restricted to
a single dimension. We simplify the Eqs. (1) and (2) as follow:
V (t + 1) = ωV (t) + c1 r1 (Pp − X(t)) + c2 r2 (Pg − X(t))
(3)
X(t + 1) = X(t) + V (t + 1)
(4)
The following formulas can be obtained from Eq. (4):
V (t + 1) = X(t + 1) − X(t),
V (t) = X(t) − X(t − 1)
(5)
By substituting Eq. (5) into Eq. (3), the following nonhomogeneous recurrence relation is obtained:
X(t + 1) = (1 + ω − c1 r1 − c2 r2 )X(t) − ωX(t − 1)
+ (c1 r1 Pp − c2 r2 X(t)Pg )
(6)
X(t + 1) = (1 + ω)X(t) − ωX(t − 1) + (c1 r1 Pp + c2 r2 Pg )
− (c1 r1 + c2 r2 )X(t)
(7)
Let ε = (c1 r1 Pp + c2 r2 Pg ) − (c1 r1 + c2 r2 )X(t), then
X(t + 1) = (1 + ω)X(t) − ωX(t − 1) + ε
(8)
140
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
positions in s iteration, ω is used to accelerate up the movement
of particles towards the optimum point or slow down the particles
so that they converge to the optimum.
Write Eq. (8) as follow
X(t + 1) = W1 X(t) + W2 X(t − 1) + ε
(9)
]T ,
Let W1 = (1 + ω), W2 = − ω, W = [W1 , W2
t = [X(t), X(t − 1)]T ,
T
= [t , t+1 , · · · , t+s−1 ] , s is the constant. Y = W, H = [X(t + 1),
X(t + 2), · · · , X(t + s)]T , Eq. (9) is written as
H = Y + ε = W + ε
-1 -7E-10 -7E-40
0
7E-40 7E-10
1
Fig. 1. Probability distribution histogram of ε using the Monte Carlo method.
The first and second items on the right side of Eq. (8) both memorize the past values of position which have different weight. They
also can be seen as a whole item to be called the memory item of
position. Since the value of the third item on the right side of Eq. (8)
is obtained from the previous experience of particles, we call it the
learning item of position. According to ε, (c1 r1 + c2 r2 )X(t) in ε is the
memory part and (c1 r1 Pp + c2 r2 Pg ) is the learning part. It is obvious
that ε represents the change of particle position in every iteration.
2.2. Analysis of the change of particle position ε
ε represents the gap between memory part and learning part.
If the particle position converges to one position, ε is a Gaussian
distribution. We can use Monte Carlo method to illustrate that.
Monte Carlo is the art of approximating an expectation by the
sample mean of a function of simulated random variables. It is
apparently first used by Ulam and von Neumann as a Los Alamos
code word for the stochastic simulations applied to building better atomic bombs. Monte Carlo methods are also online simulation
methods that learn from experience based on randomly generated
simulations. Given a random set of experiences, with the guarantee from the weak law of large numbers, the simulation results will
eventually converge when each state is encountered for an infinite
number of trials. It is evidenced in part by the voluminous literature of successful applications. In our paper, we use Monte Carlo
method to analyze the properties of ε, respectively, using a large
body of experimental data. Take the test function f6 as the example,
the PSO is initialized as follow:
In this paper, we propose a Bayesian PSO method that considers
a probability density function over the weight space, and the optimal inertia weight vector is calculated by maximizing the posterior
probability density function of the weight, where ε is an independent stochastic variable satisfying Gaussian distribution with zero
mean value and variance 2 , i.e. (p(ε) ∼ N(ε|0, 2 ))
Let D = {H, W}, ˇ = 1/ 2 , Then, the Bayesian method can be used
to estimate the weight vector W, in which the probability density
function over the weight vectors space is mainly considered.
In general, the priori information on the weight vector W is difficult to be found, so we choose a fairly broad distribution as a priori.
The expression of this distribution is Gaussian distribution, which
is written as
3. PSO with Bayesian techniques
3.1. PSO with Bayesian techniques
The inertia weight adjusting method proposed in our paper
is inspired by the idea of self-adaption. Based on past particle
1
exp(−˛EW )
ZW (˛)
P(W ) =
(11)
1
1 2
Wi
||W ||2 =
2
2
M
EW =
(12)
i=1
where ˛ controls the distribution of weight vector. ZW (˛) is a normalization factor. M is the total number of weights.
If W is subject to Gaussian prior, the normalization factor ZW (˛)
can be expressed by
ZW (˛) =
2 M/2
(13)
˛
The likelihood function can be written as
P(D|W ) =
1
exp(−ˇED )
ZD (ˇ)
(14)
where ˇ controls the distribution of Y, ZD (ˇ) is a normalization
factor. ED is an error function. P(D|W) can also be written as
P(D|W ) = P(H|(W, )) =
1
=
exp
ZD (ˇ)
(1) The maximum number of iterations is 500.
(2) The swarm size is 50.
(3) c1 and c2 are set as 1.49445.
In the experiment, We obtain the probability distribution of ε
shown in Fig. 1. Under the condition of pbest = gbest, the value of
gbest in the 50 particles is not changed after about 200 iterations.
The mean value of ε is equal to zero and the variance is 2 , i.e., ε is
an independent stochastic variable satisfying Gaussian distribution
with zero mean value and variance 2 (p(ε) ∼ N(ε|0, 2 )).
(10)
ZD (ˇ) =
e(˛ED ) =
s
P(X(t + i)|(W, ))
i=1
ˇ
2
−
{X(t + i) − i W }
2
s
(15)
i=1
2 2s
ˇ
(16)
using formula
P(W |D) =
P(D|W )P(W )
P(D)
(17)
By substituting Eqs. (11) and (14) into Eq. (17), we obtain the
posterior distribution as follows
P(W |D) =
1
1
exp(−ˇED − ˛EW ) =
exp(Q (W ))
ZQ
ZQ
(18)
where
Q (W ) = ˇED + ˛EW
(19)
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
ZQ (˛, ˇ) =
exp(−ˇED − ˛EW )dW
(20)
To obtain the posterior distribution of W, we apply Gaussian
approximation to W. This approximation is obtained through the
second-order Taylor expansion of Q(W) around its most probable
weights value WMP . Taylor expansion of Q(W) is written as
Q (W ) = Q (WMP ) +
1
(W − WMP )T A(W − WMP )
2
(21)
where A is the Hessian matrix of the total (regularized) error function. The remainder terms of Q(W) have been ignored.
141
3.3. W estimation
By maximizing the posterior distribution Eq. (28), we maximize
its logarithm equivalently, which results in the following optimization problem:
Q (W ) = ˇED + ˛EW =
ˇ
˛
T
(W − H) (W − H) + W T W
2
2
(29)
By fixing the hyperparameters ˛ and ˇ, we maximize Q(W) with
respect to W, and have
3.2. Parameters selection of ˛, ˇ
According to the content we have discussed, parameters ˛, ˇ are
unknown. Take parameters ˛, ˇ as integral variables. The posterior
distribution of network weights is given by
∂(W − H)T
∂(W − H)T
(W − H) +
(W −H)
∂W
∂W
+ ˛W =
P(W |D) =
∂Q (W )
ˇ
=
2
∂W
P(W, ˛, ˇ|D)d˛dˇ
ˇ
{2T (W − H)} + ˛W = ˇT W − ˇT H
2
+ ˛W = ˇT W − ˇT H + ˛W
(30)
=
P(W |˛, ˇ, D)P(˛, ˇ|D)d˛ dˇ
(22)
taking ∂Q (W ) = 0, then W can be written as
∂W
Since the posterior probability distribution P(W, ˛, ˇ|D) for the
hyper-parameters is around their most probable values ˛MP and
ˇMP [33]. Replace ˛, ˇ with ˛MP , ˇMP . Eq. (22) can be written as
P(W |D) ≈ P(W |˛MP , ˇMP , D)
3.4. BPSO algorithm and its computational cost
(24)
by combining Eq. (21) with (24), P(D|˛, ˇ) can be written as
1
ZD (ˇ)ZW (˛)
exp(−Q (W ))dW =
ZM (˛, ˇ)
ZD (ˇ)ZW (˛)
(25)
considering Eq. (25), the log of the evidence is then given as
follow
MP
MP
ln P(D|˛, ˇ) = −ˇED
− ˛EW
−
,
2EW
N
N
ln(ˇ) + ln(2)
2
2
ˇMP =
n−
2ED
(26)
(27)
where
=
2
i
i=1
i + ˛
,
(i |i = 1, 2)
The main process of BPSO algorithm is in the Algorithm 1. In
the algorithm, inertia weight ω can change according to the past
particle position. In every s iterations, ω is different in Eq. (3). However, the Bayesian approach suffers from local optimal problem. By
introducing a mutation operation into BPSO, the ability to escape
from local optimum is enhanced and the global search ability of
the BPSO algorithm is strengthened. In [18], we see that Gaussian mutation tends to have a fine-tuning ability, however, it only
produces short jump. To produce longer jumps, the cauchy mutation is selected as the local mutation operator, which is written
as
1
L
ln(det A) + ln(˛)
2
2
The optimal hyper-parameters values ˛MP , ˇMP are obtained by
differentiating Eq. (26) [34]. The optimal values are as follow
˛MP =
(31)
(23)
P(D|˛, ˇ)P(˛, ˇ)
P(D)
+
ˇT H
using the W, we can compute the inertia weight ω.
using the formula as follow
P(D|˛, ˇ) =
−1
P(˛, ˇ|D)d˛dˇ
= P(W |˛MP , ˇMP , D)
P(˛, ˇ|D) =
W = (ˇT + ˛)
(28)
i are the eigenvalues of the Hessian matrix of the un-regularized
error.
Pp,i = Pp,i + ˛ · cauchy(1)
(32)
Pp,i represents the best place of the ith particle in the iteration.
cauchy(1) returns a big value, which is drawn from a cauchy
distribution. Parameter ˛ controls the range of cauchy random
number.
The computation cost of the original PSO algorithm can be easily obtained, which is represent as O(maxgen * sizepop). sizepop is
the size of the whole population, and maxgen is the maximum
number of iterations. In BPSO algorithm, the total computational cost incurred by the computation of the W is denoted as
O(maxgen * sizepop * s). s is regrouping period, which is less than
100. In our experiment, maxgen * sizepop is far greater than s, so the
total computational cost of BPSO is the same as the original PSO
algorithm.
142
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
4. Simulations
Algorithm 1 The details of BPSO.
Initialize:
maxgen: max iterations; i: the number of iterations; sizepop: size of the
whole population; j: the number of particle; s: regrouping period; Pp,i :
the best position of one particle in one iteration; Pg : the best position of
all particles in one iteration; Initialize each particle’s position xi,d and
constant c1 = c2 = 1.49445; regrouping period s = 30; cout(j): the best
position of all particles in one iteration; q: the threshold;
Iterate:
1:
for i = 1 : maxgen do
for j = 1 : sizepop do
2:
Update each particle using local version PSO (1);
3:
if mod(i,s)=0 and i ≥ 2then
4:
5:
compute the weight W using (31);
6:
end if
end for
7:
for j = 1 : sizepop do
8:
if f(xj,d ) < f(Pp,j )then
9:
f(Pp,j ) = f(xj,d )
10:
else
11:
cout(j) = cout(j) + 1
12:
end if
13:
14:
if cout(j) > qthen
mutate Pp,j using (32)
15:
16:
cout(j) = 0
end if
17:
if f(xj,d ) < f(Pg )then
18:
f(Pg ) = f(xj,d )
19:
end if
20:
21:
end for
end for
22:
Two comparisons have been conducted in this section. One is
among unimodal functions and the other is among multimodal
functions.
4.1. Experiment setup
To test the performance of the proposed algorithm, we do experiments on 10 benchmark functions. These test functions, which
are shown in Table 1, can be classified into two groups. The first
five functions f1 –f5 are unimodal functions. For unimodal functions,
the convergence rate of an algorithm is more interesting than the
final result of optimization. Therefore, we not only give the final
achievement but also show the convergence rate of each algorithm.
The next five functions f6 –f10 are multimodal functions with many
local optima. For multimodal functions, we only give the final result
since it reflects an algorithm’s ability of escaping from poor local
optima. The functions are used to test the global search ability of
the algorithm in avoiding premature convergence.
In this paper, all empirical experiments related to the PSO and
its improvements are carried out with a same population size. Furthermore, in order to ensure that the initial values of particles in
each algorithm are same, we use the MATLAB command rand(state,
sum(i30)). The initial value of i of each run for all algorithms is the
same. Parameter s is the interval of the adjacent two inertia weight
change in all iterations. The parameters of the BPSO are used as
follow.
Table 1
Test function.
Test function
n 2
f1 =
i=1
xi n 2 f2 = exp 0.5
x −1
i=1 i
n
2
([xi + 0.5])
f3 =
i=1
n
n
f4 =
|xi | +
|xi |
i=1
i=1
n
2
f5 =
ixi
i=1
n
xi sin(
|xi |)
f6 = n ∗ 418.98291 −
i=1
n 2
(xi − 10 cos(2xi ) + 10)
f7 =
i=1
n
1
f8 = −20 exp −0.2
xi2
n
1 n
i=1
cos(2xi ) + 20 + e
− exp n )
2i=1
n−1
2
2
2
2
(xi − 1) ∗ [1 + sin (xi+1 )] + (xn − 1) [1 + sin (xn )]
f9 = 0.1 sin (3xi ) +
i=1
√
n
n
1
2
f10 =
4000
i=1
xi −
(a)
i=1
cos(xi / i) + 1
Domain
Global optimum
Name
[−100,100]
0
Sphere
[−1,1]
0
Hartmann
[−100,100]
0
Step
[−100,100]
0
Schwefel’s p 2.22
[−100,100]
0
Sum Squares
[−500,500]
0
Schwefel
[−5.12,5.12]
0
Rastrigin
[−32,32]
0
Ackley
[−500,500]
0
Levy–Montalvo
[−600,600]
0
Griewank
(b)
Fig. 2. Comparisons between different inertia weight strategies for f1 –f2 .
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
143
Fig. 3. Comparisons between different inertia weight strategies for f3 –f4 .
From all strategies introduced in the previous sections, four
strategies have been adopted for comparisons: decreasing inertia
weight (Linear Decreasing) [5,6], random inertia weight (Random)
[7],the Sugeno method (Sugeno) [10] and AIWPSO [17].
To further validate the effectiveness of the proposed algorithm
in this paper, we compare BPSO with six existing PSO variants,
including the PSO [5], PSO with Gaussian mutation (GPSO) [19],
PSO with Gaussian mutation cauchy (CPSO)[19], hybrid particle
swarm (HPSO) [23], improved PSO with a jump strategy(IPSO + AJS)
[24], PSO with an MRS strategy (HMRPSO) [26] under the same
maximum function evaluations (FEs).
4.2. Comparison results on unimodal functions
Four inertia weight strategies are applied to the five test function f1 –f5 on dimensions 10 and 50. The results are listed in
Tables 2 and 3, where “best” indicate the best function values found in the generation, “Mean” and “Std Dev” stand for
mean value and standard deviation. Figs. 2–4 are comparison
among BPSO and other strategies on dimension 10. The sphere
function f1 is unimodal test function, which is often used. For
this function, the performance of the BPSO is much more better
than the decreasing inertia weight, random inertia weight, the
Sugeno method and AIWPSO. Similar results are also obtained
on f4 and f5 , which are shown in Tables 2 and 3. The differences between the inertia weight adjusting methods on f1 , f4 ,
and f5 suggest that the BPSO is better at a fine-gained search
than both the other algorithms. f3 is called step function, which is
characterized by plateaus and discontinuity. From Tables 2 and 3,
using decreasing inertia weight, random inertia weight and the
Sugeno method, optimal function value can be found. However,
in Fig. 3(a), we can see that the convergence rate of the BPSO is
faster than the other two PSO inertia weight adjusting methods. The
performance of the BPSO is much more better than the other four
methods. The main differences attribute to the change of the inertia
weight ω, which affords the BPSO a greater chance to search in the
local area. In other words, the feature of ω directly affects the particle’s movement, so that the particle preserve the fast converging
feature in its searching area.
The above mentioned PSO-based optimization algorithms (PSO,
GPSO, CPSO, IPSO + AJS, HPSO, HMRPSO) are applied to the test
problems. Table 4 shows the comparisons of six other PSO variants for function f1 –f5 . From Table 4 and Figs. 5– 7, BPSO performs
better than the other three improved variants. Inf2 and f3 , globle
optimum both can be found using BPSO and HMRPSO algorithm.
But, in Figs. 5(b) and 6(a), we can see that the convergence rate of
the BPSO is faster than HMRPSO.
4.3. Comparison results on multimodal functions
On multimodal functions, the global optimum is more difficult to
locate. Therefore, in the comparison, we study the accuracy, speed,
and reliability of the PSO variants. Comparisons of solution accuracy
on multimodal functions are given in Tables 2, 3 and 5.
The Schwefel function f6 is a multimodal function with very deep
sinusoidal indentations. It is one of the most difficult problems for
optimization algorithms. It can be seen from the comparisons in
Tables 2, 3 and 5 that, although none of the seven algorithms considered in this paper is efficient in solving this function, BPSO method
is more efficient than the other four inertia weight adjusting methods and six PSO variant algorithms in producing the near optimal
value.
50
10
0
10
Mean of average fitness
(1) The number of runs is 500; the acceleration coefficients c1 = c2
=1.49445.
(2) The population size is 50.
(3) The parameter s of BPSO increase from 1 to 50, the step length
is 2.
−50
10
−100
10
−150
10
0
10
−200
10
−250
10
BPSO
AIWPSO
Decreasing
Random
Sugeno
200 205 210
−300
10
0
100
200
300
400
500
Iteration
Fig. 4. Comparisons between different inertia weight strategies for f5 on 10
dimensions.
144
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
Table 2
Comparisons between different inertia weight adjusting PSO methods (n = 10).
Function
n
f1
10
f2
10
f3
10
f4
10
f5
10
f6
10
f7
10
f8
10
f9
10
f10
10
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
BPSO
AIW
Decrease
Random
Sugeno
3.37E−271
1.60E−226
0.00E+00
0
0
0
0
0
0
1.93E−132
1.54E−122
1.47E−122
9.11E−277
9.43E−245
2.45E−241
0.0018
0.0020
0.0013
0
0
0
1.88E−16
8.88E−16
3.88E−14
4.28E−08
1.42E−06
1.61E−06
0
0
0
4.51E−01
7.61
7.64
7.56E−03
4.67E−03
0.004
2
2.5
0.71
16.28
3.21E+01
22.31
13.5
13.3
0.36
3.70E+03
3.73E+03
53.78
26.12
53.73
38.71
19.3
19.8
0.38
2.94
4.35
2.02
0.082
0.196
0.148
1.62E−09
5.55E−07
9.44E−07
6.20E−14
2.73E−10
3.86E−10
0
0
0
0.093
3.02
4.14
2.88E−07
2.67E−05
3.73E−05
3.73E+03
3.78E+03
75.16
11.94
18.24
6.47
19.96
20
0.0178
1.01E−05
3.70E−03
0.0063
0.062
9.68E−02
0.0317
1.39E−10
2.37E−02
4.11E−02
7.15E−04
6.48E−04
9.522E−05
0
0
0
0.012
3.13E−02
0.027
7.49E−09
4.84E−05
6.84E−05
3.69E+03
3.71E+03
34.04
42.8
68.12
21.95
1.65
7.95
10.58
2.07E−04
2.41
3.58
0.029
0.105
0.066
3.01E−09
1.87E−08
1.43E−08
7.39E−10
3.49E−09
3.89E−09
0
0
0
0.45
0.862
0.585
2.01E−05
6.39E−05
6.19E−05
3.73E+03
3.80E+03
63.65
12.93
25.54
18.49
2.58
14.2
10.06
2.63E−05
3.77E−03
0.0063
0.064
8.45E−02
0.029
The generalized Rastrigin’s function f7 can easily trap an optimization algorithm in a local optimum on its way to the global
optimum. By adjusting step length s, particles in the BPSO are able
to search more solution space. In Tables 2, 3 and 5, it can be clearly
seen that the mean values of the BPSO are better than those of the
other variants algorithms.
The Ackley function f8 has several local minima. Here, the
mean values and the standard deviations of the BPSO are also
better than those of the inertia weight adjusting methods and
six PSO variant algorithms. Therefore, it is evident that the BPSO
can provide more stable and high accuracy results on this function.
Table 3
Comparisons between different inertia weight adjusting PSO methods (n = 50).
Function
n
f1
50
f2
50
f3
50
f4
50
f5
50
f6
50
f7
50
f8
50
f9
50
f10
50
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
BPSO
AIW
Decrease
Random
Sugeno
2.46E−248
1.73E−218
0.00E+00
0
0
0
0
0
0
2.42E−123
6.36E−116
1.22E−120
6.75E−216
1.35E−216
2.33E−212
119
343
283
0
0
0
1.88E−16
8.88E−16
3.88E−14
4.33
4.61
0.27
0
0
0
1.14E+03
1.59E+03
4.00E+02
6.58E−01
9.01E−01
0.34
1149
1443
168
327
373
14
3.21E+04
5.38E+04
2.48E+04
1.95E+04
1.96E+04
71
2116.25
2421.16
269.13
20.6
19.6
1.5
283
343
67.72
1.21
1.32
0.09
5.86E+01
1.43E+02
6.62E+01
7.15E−02
8.83E−01
1.15
263
391
165
383
417
46
3.33E+03
4.16E+03
1.16E+03
1.92E+04
1.91E+04
251
859.94
1195.80
318
17.1
19.3
0.06
142
162
78.18
0.85
0.95
0.1
1.37
1.06E+02
1.81E+02
3.50E−01
4.64E−01
0.16
21
498
675
365
441
94
1.01E+03
8.38E+03
9218
1.95E+04
1.97E+04
190
1.07E+03
1387.65
425.37
17.2
18.8
1.77
138
181
86.05
1.09
1.45
0.08
3.55E+02
3.95E+02
3.95E+01
5.73E−01
1.15
0.21
162
185
32
343
373
42
3.83E+03
4.13E+03
414
1.90E+04
1.91E+04
75
833.94
1028.68
241.35
19.5
20
0.44
309
401
81.29
1.07
1.09
0.01
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
145
Table 4
Comparisons between different PSO methods for unimodal test function.
Function
n
f1
10
30
50
f2
10
30
50
f3
10
30
50
f4
10
30
50
f5
10
30
50
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
BPSO
PSO
GPSO
CPSO
HPSO
IPSO + ACJ
HMRPSO
3.37E−271
1.60E−226
2.60E−224
1.33E−263
2.53E−165
148E−166
2.46E−248
1.73E−218
0.00E+00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1.93E−132
1.54E−122
1.47E−122
4.24E−130
9.60E−104
1.662E−103
2.42E−123
6.36E−116
1.22E−120
9.11E−277
9.43E−245
2.45E−241
3.42E−264
9.15E−228
1.68E−228
6.75E−216
1.35E−216
2.33E−212
7.05E−10
1.04E−06
1.51E−06
1.17E+01
1.06E+01
1.96E+00
1.50E+02
1.56E+02
28.1
9.70E−09
1.17E−07
1.35E−07
5.64E−02
0.078
0.040
0.13
0.54
0.41
0
0
0
15
36.5
18.44
213
352.5
162.48
0.083
4.35E−01
0.335
2.46E+02
2.52E+02
8.618
418.19
447
33.77
4.59E−09
1.55E−06
2.51E−06
2.31E+02
4.12E+02
310.75
3.00E+03
4.36E+03
1426.56
0.01
3.40E−02
1.41E−02
23.22
2.45E+01
1.39E+00
131.43
182
35.8
0.023
6.24E−02
0.035
5.44E−07
9.61E−07
1.12E−06
7.92E−06
2.23E−06
1.28E−06
0
0
0
27
31
3.78
240
326.5
107.46
0.284
1.40E+01
18.55
2.07E+02
2.77E+02
73.67
1E+11
2.33E+16
4.6E+16
0.0878
4.83E−01
0.268
5.65E+02
6.44E+02
70.2
4299.33
5.47E+03
1205.36
8.03E−06
9.06E−05
1.22E−04
1.09E−03
3.35E−03
3.87E−03
2.09E−03
7.73E−03
4.06E−03
4.47E−08
5.7E−05
6.33E−05
1.63E−10
1.11E−07
1.61E−07
1.63E−8
1E−07
1.21E−07
0
0
0
0
0
0
0
0
0
0.0071
3.67E−02
0.043
0.0622
8.67E−02
0.0352
0.13
0.23
0.19
6.14E−09
1.84E−04
0.00025
6.54E−05
7.37E−03
0.0114
1.73E−03
2.00E−02
0.031
1.21E−13
4.99E−09
1.10E−08
1.05E+01
1.26E+01
3.42E+00
1.14E+02
1.35E+02
1.64E+01
8.55E−14
1.27E−10
2.04E−10
2.32E−02
0.098
0.14
0.59
1.13
0.57
0
0
0
50
56
7.93
218
328
119.25
0.184
8.29E−01
0.405
1.50E+02
2.37E+02
78.91
384.63
447
63.99
2.24E−07
3.19E−04
0.00046
4.81E+02
6.50E+02
147.08
2.57E+03
4.88E+03
1681.89
9.55E−08
2.85E−06
5.88E−06
9.20E+00
2.36E+01
1.31E+01
1.51E+02
2.41E+02
8.36E+01
9.11E−12
2.08E−10
2.36E−10
4.26E−02
0.054
0.043
0.17
1.61
1.71
0
0
0
38
91.25
80.78
38
250
101.28
0.157
6.06E+01
52.5
1.54E+02
2.05E+02
44.99
288.19
418
148.02
5.78E−07
2.53E−04
0.00047
4.13E+02
7.46E+02
406.54
5.45E+03
6.84E+03
987.69
1.57E−130
3.18E−126
7.10E−126
1.52E−126
5.84E−121
4.43E−121
1.53E−125
8.81E−121
6.43E−121
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1.82E−83
1.94E−80
3.82E−80
9.16E−83
1.75E−81
2.83E−81
9.45E−83
1.72E−81
3.63E−81
3.77E−175
2.69E−165
0
3.65E−170
8.93E−161
0
2.12E−166
5.32E−158
0
Fig. 5. Comparisons between different PSO methods for f1 –f2 .
146
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
Fig. 6. Comparisons between different PSO methods for f3 –f4 .
The Levy-Montalvo function f9 has approximately 15n local
optima. In Table 5, when the number of dimensions is increased
from 10 to 50, the PSO has been trapped in local optima. Meanwhile,
the BPSO performs much better principally because the changed
inertia weight ω affords the BPSO a greater opportunity to escape
from poor local optima.
f10 is the Griewank function. Because f10 is a difficult multimodal function that has an exponentially increasing number
Table 5
Comparisons between different PSO methods for multimodal test function.
Function
n
f6
10
30
50
f7
10
30
50
f8
10
30
50
f9
10
30
50
f10
10
30
50
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
est
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
Best
Mean
Dev
BPSO
PSO
GPSO
CPSO
HPSO
IPSO+ACJ
HMRPSO
0.0018
0.0020
0.0013
0.011
0.132
0.0108
119
343
283
0
0
0
0
0
0
0
0
0
1.88E−16
8.88E−16
3.88E−14
8.88E−14
7.88E−13
1.21E−12
1.88E−16
8.88E−16
3.88E−14
4.28E−08
1.42E−06
1.61E−06
1.49E−06
3.52E−06
3.06E−06
4.33
4.61
0.27
0
0
0
0
0
0
0
0
0
1.81E+03
1919.25
242.73
8.87E+03
8932.18
89.12
1.43E+04
1.53E+04
826.41
5.96
10.14
3.17
5.53E+01
72.92
20.12
124.52
147.03
29.58
1.13E−05
2.01
0.93
4.62
6.22
1.41
7.88
8.74
0.61
0.12
5.27E+01
76.13
1.08E+03
1.78E+03
642.13
1.47E+03
3.18E+03
1284.62
0.099
5.05E+00
9.54
1.48
1.55
0.054
3.11
3.65
0.64
1.25E+03
2.03E+03
534.71
7.56E+03
8051.24
577.66
1.51E+04
1.62E+04
851.98
2.09
11.27
8.66
1.46E+01
54.22
37.63
90.62
173.61
81.32
0.51
1.33
0.49
5.86
6.25E+00
0.51
7.23
8.15E+00
0.63
0.11
9.18E+02
203.1
1.34E+03
2.95E+03
2500.06
4.82E+03
5.24E+03
422
0.031
0.26
0.87
1.11
1.18
0.072
1.72
2.51
0.75
0.022
502.55
599.31
2.24E−03
0.27
0.46
1.14E−03
2.05E−03
1.14E−03
4.88E−06
5.07E−05
5.16E−05
7.96E−06
0.00019
0.00029
8.33E−05
0.0049
0.007
9.68E−04
7.60E−03
0.011
0.0011
8.75E−04
0.00027
0.009
1.32E−02
0.01
0.0031
6.97E−03
0.0071
3.67E−04
1.31E−02
0.018
7.45E−03
1.11E−02
0.04
4.01E−04
8.06E−04
0.00091
2.50E−03
5.90E−03
0.0069
4.65E−03
7.09E−03
0.01
1.58E+03
2075.33
361.05
6.81E+03
7706.27
1049.47
1.58E+04
1.62E+04
356.62
6.96
14.12
6.02
4.40E+01
60.31
15.83
148.04
213.82
73.58
7.10E−05
1.12
1.06
3.34
4.41
0.41
6.69
8.44
1.17
0.36
0.41
0.31
6.18E+02
8.48E+02
293.36
3.09E+03
8.23E+02
644.95
0.097
2.21E+01
43.078
1.132
1.138
0.12
2.33
2.99
0.79
1.54E+03
1810.02
259.83
7.70E+03
7.43E+03
231.53
1.18E+04
1.34E+04
1412.65
9.94
17.51
7.88
7.08E+01
113.61
38.11
223.65
26.2
38.11
2.97E−05
7.94E−01
1.11
4.11
5.68
1.62
7.94
8.54
0.47
1.64E−02
6.64E−02
9.64E−02
1.34E+03
1.64E+03
457.81
1.32E+03
1.51E+03
478.22
0.038
1.77E+01
34.62
1.33
1.81
0.75
3.19
4.53
1.51
1.35E+03
1674.16
329.65
8.13E+03
8102.19
231.06
1.28E+04
1.48E+04
1358.96
0
0
0
0
0
0
0
0
0
8.88E−16
4.44E−15
2.9E−15
7.99E−15
8.89E−13
7.81E−13
7.99E−15
4.67E−13
5.33E−13
0.0078
1.52E−02
0.011
2.33
2.42
0.11
4.23
4.59
0.26
0.048
8.96E−02
0.11
0
0
0
0
0
0
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
147
These results confirm that BPSO is also competitive on multimodal
functions.
50
10
0
Mean of average fitness
10
−50
4.4. Analysis of parameters s and ω in the BPSO
−100
The performance of BPSO is superior due to the adaptive nature
of this algorithm. Fig. 8 shows the process of the inertia weight ω
changing, in which BPSO is applied to the function f1 . We see that
the changing method of ω in our paper is different from the other
four inertia weight strategies and most value of ω is in [0.4, 0.9]. This
is mainly because ω is mainly affected by the particle positions in
s step length and adjusted automatically.
In the BPSO, s is a parameter that influences the convergence
rate. A smaller s means that the inertia weight ω has more opportunities to be changed but less past particle position informations
are used. Larger s makes ω having less chance to be changed and
more past particle position informations are referred. Thus, it is
important to identify the appropriate value for s in the BPSO. We
endeavor to get a better parameter in BPSO algorithm. In our paper,
an efficient method is used that s varies from 2 to 50 in linearincreasing method. Experiment setup is same as above. We carry
out experiments to get a better approximation to best s for the unimodal functions f2 and the multimodal functions f10 on dimensions
10 (see Figs. 9 and 10).
10
10
−150
10
BPSO
0
10
PSO
GPSO
CPSO
HPSO
IPSO+ACJ
HMRPSO
−200
10
−250
10
195 200 205
−300
10
0
100
200
300
400
500
Iteration
Fig. 7. Comparisons between different PSO methods for f5 on 10 dimensions.
of local minima as the dimension of the problem increases,
it is difficult for PSO algorithms to find the optimal solution
steadily. On the prospect of search speed, BPSO remains one
of the fastest algorithms in terms of FEs and execution time.
0.9
0.8
0.7
Inertia weight
0.6
0.5
0.4
0.3
0.2
0.1
0
−0.1
0
10
20
30
40
50
The number of w change
Fig. 8. Inertia weight adaptation in BPSO applied to f1 when s = 50.
0
0
10
s=2
s=4
s=6
s=8
s=10
−5
10
Mean of average fitness
Mean of average fitness
10
−10
10
−15
10
−20
10
(a)
0
s=50
s=40
s=30
s=20
s=10
−5
10
−10
10
−15
10
−20
5
10
15
20
Iteration
25
30
35
40
10
0
5
10
(b)
Fig. 9. Comparison different s in the BPSO for f2 on 10 dimension.
15
20
Iteration
25
30
35
148
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
5
5
10
10
s=2
s=4
s=6
s=8
s=10
Mean of average fitness
10
10
−5
10
−10
10
−5
10
−10
10
−15
−15
10
10
−20
10
s=50
s=40
s=30
s=20
s=10
0
Mean of average fitness
0
0
(a)
−20
10
20
30
40
50
Iteration
10
(b)
0
10
20
30
40
50
Iteration
Fig. 10. Comparison different s in the BPSO for f10 on 10 dimension.
In Fig. 9 and 10, the convergence rate with different values of
s are shown for the test function. When s = 2, 4, 6, 8, 10, results
on each test function become better with s increasing. However,
when s = 10, 20, 30, 40, 50, results on each test function become
worse with s increasing. From the figure, we can see that the value
of s influences the convergence velocity, but not very significant.
5. Conclusion
In this paper, we have proposed a new inertia weight adjusting strategy based on the Bayesian techniques. In this strategy,
the Bayesian techniques are used to adjust the inertia weight on
the basis of the past particle positions. Our objective is to use the
Bayesian techniques and the cauchy mutation to seek a tradoff
between the exploration of new positions and the exploitation of
the past particle’s position. In the simulation, BPSO method has
faster convergence rate than other methods in the five unimodal
test functions and higher accuracy than other methods in the other
five multimodal test functions. The results demonstrate that the
BPSO method is useful to solve optimization problems. The application of BPSO method in steel industry is our further research in
our future work.
Acknowledgments
This paper is partially supported by the Science Fund for
Hundred Excellent Innovation Talents Support Program of Hebei
Province, Doctoral Fund of Ministry of Education of China
(20121333110008), Hebei Province Applied Basis Research Project
(13961806D), Hebei Province Development of Social Science
Research Project (201401315) and the National Natural Science
Foundation of China (61273260, 61290322, 61273222, 61322303).
References
[1] J. Kennedy, R. Eberhart, Particle Swarm Optimization, in: IEEE International
Conference on Neural Networks, Perth, Australia, 1995, pp. 1942–1948.
[2] K.E. Parsopoulos, V.P. Plagianakos, G.D. Magoulas, M.N. Vrahatis, Objective
function “stretching” to alleviate convergence to local minima, Nonlinear Anal.
TMA 47 (5) (2001) 3419–3424.
[3] R. Eberhart, Y. Shi, Comparison between genetic algorithms and particle swarm
optimization, in: The 7th Annual Conference on Evolutionary Programming,
San Diego, USA, 1998, pp. 611–616.
[4] Y.H. Shi, R.C. Eberhart, Fuzzy adaptive particle swarm optimization, in: Proc.
of the IEEE Congress on Evolutionary Computation, Seoul Korea, vol. 1, 2001,
pp. 101–106.
[5] Y.H. Shi, R.C. Eberhart, A modified particle swarm optimizer, in: Proc. of the
IEEE Congress on Evolutionary Computation, USA, 1998, pp. 69–73.
[6] Y.H. Shi, R.C. Eberhart, Parameter selection in particle swarm optimization,
in: EP ’98: Proceedings of the 7th International Conference on Evolutionary
Programming VII, Springer-Verlag, London, UK, 1998, pp. 591–600.
[7] R.C. Eberhart, Y.H. Shi, Tracking and optimizing dynamic systems with particle
swarms, in: Proc. of the IEEE Congress on Evolutionary Computation, Korea,
2001, pp. 94–100.
[8] R.C. Eberhart, Y.H. Shi, Comparing inertia weights and constriction factors in
particle swarm optimization, in: Proc. of the IEEE Congress on Evolutionary
Computation, USA, vol. 1, 2000, pp. 84–88.
[9] Y.H. Shi, R.C. Eberhart, Empirical study of particle swarm optimization, in:
Proc. of the IEEE Congress on Evolutionary Computation, USA, vol. 3, 1999,
pp. 1945–1950.
[10] K. Lei, Y. Qiu, Y. He, A new adaptive well-chosen inertia weight strategy to
automatically harmonize global and local search ability in particle swarm optimization, in: ISSCAA, 2006.
[11] S. Fan, Y. Chiu, A decreasing inertia weight particle swarm optimizer, Eng.
Optim. 39 (2) (2007) 203–228.
[12] Y. Zheng, L. Ma, L. Zhang, J. Qian, Empirical study of particle swarm optimizer
with an increasing inertia weight, in: Proc. of the IEEE Congress on Evolutionary
Computation, vol. 1, 2003, pp. 221–226.
[13] Y. Zheng, L. Ma, L. Zhang, J. Qian, On the convergence analysis and parameter
selection in particle swarm optimization, in: Proceedings of the Second International Conference on Machine Learning and Cybernetics, vol. 3, 2003, pp.
1802–1807.
[14] M.S. Arumugam, M.V.C. Rao, On the improved performances of the particle
swarm optimization algorithms with adaptive parameters, crossover operators
and root mean square (RMS) variants for computing optimal control of a class
of hybrid systems, Appl. Soft Comput. 8 (1) (2008) 324–336.
[15] G. Xu, An adaptive parameter tuning of particle swarm optimization algorithm,
Appl. Math. Computat. 219 (9) (2013) 4560–4569.
[16] M.K. Lim, C.H. Chan, D. Monekosso, P. Remagnino, Refined particle swarm
intelligence method for abrupt motion tracking, Inform. Sci. 283 (2014)
267–287.
[17] A. Nickabadi, M.M. Ebadzadeh, R. Safabakhsh, A novel particle swarm optimization algorithm with adaptive inertia weight, Appl. Soft Comput. 11 (2011)
3658–3670.
[18] N. Higashi, H. Iba, Particle swarm optimization with gaussian mutation, in:
Proceedings of the IEEE Swarm Intelligence Symphosium 2003, IEEE Press,
2003, pp. 72–79.
[19] P.S. Andrews, An investigation into mutation operators for particle swarm optimization, in: Proc of the IEEE Congress on Evolutionary Computation, 2006, pp.
1044–1051.
[20] A. Stacey, M. Jancic, I. Grundy, Particle swarm optimization with mutation, in: Proc. of the IEEE Congress on Evolutionary Computation, 2003,
pp. 1425–1430.
[21] S.H. Ling, H.H.C. Iu, K.Y. Chan, H.K. Lam, B.C.W. Yeung, F.H. Leung, Hybrid particle
swarm optimization with wavelet mutation and its industrial applications, IEEE
Trans. Syst. Man Cybern. B Cybern. 38 (3) (2008) 743–763.
[22] A. Ratnaweera, S.K. Halgamuge, H.C. Watson, Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients, IEEE Trans.
Evol. Comput. 8 (3) (2004) 240–255.
[23] H. Wang, Y. Liu, A hybrid particle swarm algorithm with Cauchy mutation, in:
Proc IEEE Conf. Swarm Intell. Symp., 2007, pp. 356–360.
[24] H. Wang, Y. Liu, Z. Wu, H. Sun, S.Y. Zeng, L.S. Kang, An improved particle swarm
optimization with adaptive jumps, in: Proc. of the IEEE Congress on Evolutionary Computation, 2008, pp. 392–397.
[25] M.S. Kiran, M. Gunduz, O.K. Baykan, A novel hybrid algorithm based on particle swarm and ant colony optimization for finding the global minimum, Appl.
Math. Computat. 219 (4) (2012) 1515–1521.
L. Zhang et al. / Applied Soft Computing 28 (2015) 138–149
[26] H. Gao, W.B. Xu, A new particle swarm algorithm and its globally convergent modifications, IEEE Trans. Syst. Man Cybern. B, Cybern. 41 (5) (2011)
1334–1351.
[27] J.C. Vzquez, F. Valdez, Fuzzy logic for dynamic adaptation in PSO with multiple
topologies, IFSA/NAFIPS (2013) 1197–1202.
[28] P. Melin, F. Olivas, O. Castillo, F. Valdez, Optimal design of fuzzy classification
systems using PSO with dynamic parameter adaptation through fuzzy logic,
Expert Syst. Appl. 40 (8) (2013) 3196–3206.
[29] F. Valdez, P. Melin, O. Castillo, Modular Neural Networks architecture optimization with a new nature inspired method using a fuzzy combination of
particle swarm optimization and genetic algorithms, Inform. Sci. 270 (2014)
143–153.
149
[30] F. Valdez, P. Melin, O. Castillo, A survey on nature-inspired optimization algorithms with fuzzy logic for dynamic parameter adaptation, Expert Syst. Appl. 1
(14) (2014) 6459–6466.
[31] C.K. Monsun, K.D. Seppi, Bayesian optimization models for particle swarms, in:
Proceedings of the 2005 Conference on Genetic and Evolutionary Computation,
2005.
[32] D. Martens, B. Baesens, T. Fawcett, Editorial survey: swarm intelligence for data
mining, Mach. Learn. 82 (1) (2011) 1–42.
[33] M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press,
1995.
[34] H. Jaeger, H. Haas, Harnessing nonlinearity: predicting chaotic systems and
saving energy in wireless communication, Science 304 (5667) (2004) 78–80.