MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
179
Application of Genetic Algorithm in Solving
Linear Equation Systems
1
Al Dahoud Ali , 2 Ibrahiem M. M. El Emary, and 3Mona M. Abd El-Kareem
Abstract-- There are several algorithms for
solving linear system of equations. Iteration
algorithms are recommended for the large
linear systems with sparse matrix. But in the
case of general n x m matrices, the classic
iterative algorithms are not applicable with a
few exceptions. So, In this paper, we have used
genetic algorithms to find the solution of a
system of linear equations since it is difficult to
describe the solution set of a linear system
with infinitely many solutions. To avoid the
disadvantages of solving large system of linear
equations such as rounding errors, inverting
large matrixes, the genetic algorithms are
introduced. The obtained results indicate that
a GA is effective and represents an efficient
approach to solve the systems of linear
equations. When we use G.A. in solving linear
system, we see that the solution is almost equal
to the analytical one. Nevertheless on using the
G.A. to solve linear system we obtain the result
very swiftly.
Index Term-- Crossover, Fitness Function,
Genetic Algorithms Mutation, Optimization
Problem, Simple Genetic Algorithm
I.
INTRODUCTION
T
he problem of solving systems of interval
linear equations with using GA based
approaches is analyzed and studied in this paper.
First, this problem is viewed in terms of an
optimization task. A cost function with interval
variables is defined. Next, for a given system of
equations, instead of the exact algebraic solution
its approximation is determined by minimizing
the cost function. This is done by use of GA.
1
Faculty of Science and IT, Al Zaytoonah University,
Amman, Jordan
2
Faculty of Engineering, King abdul-Aziz University, Jeddah,
King Saudi Arabia
3
Faculty of Science, Suez Canal University, Ismailia, Egypt.
[email protected], [email protected] ,
[email protected]
A number of numerical evaluations are provided
in order to verify the proposed techniques. The
results are compared, discussed and some final
conclusions are drawn.
The major purpose of this paper is to apply
Genetic Algorithm as a soft computing technique
in solving the linear equation system and
investigating the major returned benefits as a
result of using GA. First of all, GAs are
computer programs which create an environment
where populations of data can compete and only
the fittest survive, sort of evolution on a
computer. GAs is excellent for all tasks requiring
optimization and highly effective in any situation
where many inputs (variables) interact to
produce a large number of possible outputs
(solutions). Some example situations are: ♦ Optimization: such as data fitting,
clustering, trend spotting, path finding, and
ordering.
♦ Management:
distribution,
scheduling,
project management, courier routing,
♦ container packing, task assignment, and
timetables.
♦ Financial: portfolio balancing, budgeting,
forecasting, investment analysis, and
payment scheduling.
♦ Engineering: structural design (e.g. beam
sizes), electrical design (e.g. circuit boards),
mechanical design (e.g. optimize weight,
size & cost), process control, network design
(e.g. computer networks).
Genetic algorithms utilize the concept of natural
selection and genetics. They work on a very
simple principle, which will be presented shortly.
First, we need to define encoding of each
optimization parameter into one string over some
finite alphabet consisting of GA’s building units
called genes. Usually, binary coding is used, so
we have genes that take values of 0 and 1.
Structure consisting of genes that make up all the
parameters is called genotype. Merging all these
MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
genes, the string called chromo-some is get.
Such chromosome represents one point in the
search space, that is, encoded parameter vector.
By selecting N random points from search space
and converting them to chromosomes, initial
population of individuals is get. This population
will now traverse through search space by
utilizing three operators [2, 11]: selection,
crossover, mutation. Each of the individuals in
population undergoes fitness evaluation, that is,
evaluation of the objective function. The
resulting number (fitness) is the quantitative
measure of solution “goodness” and it represents
the only connection between GA and the
problem solved. Individual’s fitness directly
affects the probability of its survival
A. Selection
Now, from the current population, the algorithm
selects individuals that will proceed (survive) to
the next generation. Temporary population is
made from the current population by random
selection of N individuals; probability of
selecting each individual is biased by its fitness –
fitter individuals have greater chance to be
selected. Each individual can be selected more
than once. There are different selection
strategies, but the most popular in the literature
is roulette wheel selection. Here, individuals are
selected based on a probability of selection given
in Equation 1 where f (parent i) denotes the
fitness of the ith parent [1].
Pselection = F(parent i) / ∑ F(parent i)
(1)
This selection method requires the objective
function to be strictly positive. If this is hard to
achieve, there are different selection schemes
(like tournament selection [1,2]), which do not
impose such constraint.
B. Crossover
From temporary population, individuals are
selected, with the probability, which will serve
as parents for crossover. Crossover means
exchanging genetic material between individuals
by copying parts of the chromosome from one
parent to another to produce children. This
process would ideally combine “good” portions
of each chromosome, producing a child with
superior fitness than its parents, but we do not
know in advance which portions are “good”. The
180
simplest kind is single-point crossover, where a
random
location
(locus)
in
parents’
chromosomes is selected, and then exchanges the
portion of chromosome preceding the selected
point from parent 1 to child 1 and from parent 2
to child 2. Portion of chromosome following the
selected point is copied from parent 1 to child 2
and from parent 2 to child 1. After crossover,
children are put back into the temporary
population in place of their parents. Pcross is set to
an arbitrary value and typically high probability
values within range [0.6, 0.8] have been found to
work best in most situations [1].
C. Mutation
Mutation is a mechanism for extending the
search on the new areas of search space. It
consists of random altering of bit values inside
chromosome with given probability. Random
number P is generated for each allele in each
chromosome and if it satisfies relation P < Pmut
is selected allele is altered [1]. Pmut is the
probability of mutation and it is typically set to
low values within range [0.01, 0.1]. While crossover works towards convergence of the
algorithm to one solution, mutation works
against this goal, enabling the algorithm to look
for better solution in yet unexplored areas.
Finally, the current population with temporary
population and repeat the process until the
termination condition (such as closeness to the
solution is replaced, maximum number of
generations, maximum number of simulations,
and so on) is met. The algorithm described is
called simple genetic algorithm (SGA). The
Schema Theorem, also known as the
Fundamental Theorem of Genetic Algorithms
[1], explains why this relatively simple
optimization concept really works. There are
many improvements to SGA by altering the
coding scheme (Gray-coded chromosomes, realcoded chromosomes [1]), selection function
(tournament selection, rank selection [1,2] ), or
introducing new concepts (elitism, fitness
scaling, steady-state GA, diploidy and
dominance, inversion, niche and speciation [1,2]
for usage of GA in different applications.
II. RELATED WORKS
V.Kelner, O.Lèonard [6] used GA for lubrication
pump stacking design. Sizing a pump stacking
used in an aircraft lubrication system is a
challenging task. The combination of several
pumps, in parallel and in a single casing, must
MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
deliver specified oil flow rates, on a variable
number of circuits and under given flight
conditions. The optimal assembly has to
minimize overall dimensions, weight and cost.
This optimization problem involves a large space
search, continuous and discrete variables and
multi objectives. Genetic algorithm is well suited
to solve that kind of problems.
G.Renner, A.Ekàrt [8] used genetic algorithms in
computer aided design. Design is a complex
engineering activity, in which computers are
more and more involved. The design task can
often be seen as an optimization problem in
which the parameters or the structure describing
the best quality design algorithms and several
ways in which they can solve difficult design
problems. They discussed several advanced
genetic algorithms that have proved to be
efficient in solving difficult design problems.
B.M.Kariuki et al [5] used genetic algorithms for
solving crystal structures from powder
diffraction data. That work used GA to tackle
crystal structure solution from powder diffraction
data in the case of a previously unknowns
structure-ortho-thymotic acid. In that structure
solution calculation, the structural fragment was
subjected to combine translation and rotation
within the unit cell, together with variation of
selected intermolecular degree of freedom under
the control of a genetic algorithm, in which a
population of trail structures is allowed to evolve
subject to well defined procedures for mating,
mutation, and natural selection. Importantly, the
genetic algorithm approach adopts the ‘directspace’ philosophy for structure solution, and
implicitly avoids the problematic step of
extracting the intensities of individual reflections
from the powder diffraction data. The structure
solution was found efficiently in the genetic
algorithm calculation, and was then used as the
initial structural model in Riveted refinement
calculations.
R.K.Sahoo et al [9] used genetic algorithms for
multi-component aromatic extraction. That work
applied (GA) which leads to globally optimal
binary interaction parameters from multicomponent liquid-liquid equilibrium data, has
been recently demonstrated for some ternary,
Quaternary and quinary systems. The binary
interaction parameters are related to each other
through the
closure equations. In that work, the binary
interaction parameters based on non-random two
liquid (NRTL) activity coefficient model have
been estimated using GA, without and with
closure equations for 65 multi-component
181
aromatic extraction systems: 53 ternary, 9
quaternary and 3 quinary systems. Parameters
that satisfy the closure equations exhibit better
root mean square deviations than those that do
not satisfy the closure equations. Root mean
square deviation value without implementation
of closure equations is 0–80% better than
literature as compared to 0–90% better with
implementation of closure equations. Aromatics,
such as benzene, toluene and xylene are
considered essential in the chemical industry
because they are the source of many organic
chemicals. These aromatics are presented in
naphtha. High purity aromatics are difficult to be
separated using ordinary distillation operation,
since they form several binary azeotropes with
non-aromatics. Extraction is, therefore, a better
choice to separate the aromatics from naphtha, as
they are preferentially soluble in a variety of
solvents. To predict the separation, it is
necessary to know the liquid–liquid equilibrium
(LLE) data for a particular system. Various
activity coefficient models, such as the universal
quasi chemical (UNIQUAC) and the non-random
two liquid (NRTL) can be used to predict the
LLE. Each of these models requires proper
binary interaction parameters that can represent
LLE for highly non-ideal liquid mixtures usually
encountered in aromatic extraction. These
parameters are usually estimated from the known
experimental LLE data via optimization of a
suitable objective function.
P.E.Amiolemhen et al [1] used genetic
algorithms for determination of the optimal
machining parameters in the conversion of a
cylindrical bar stock into a continuous finished
profile. That work proposed an optimization
technique based on genetic algorithms for the
determination of the cutting parameters in multipass machining operations by simultaneously
considering multi-pass roughing and single-pass
finishing operations. The optimum machining
parameters are determined by minimizing the
unit production cost of converting a cylindrical
bar stock into a continuous finished profile
involving seven machining operations; with each
operation subject to many practical constraints.
The cutting model developed for each machining
operation is a non-linear, constrained problem.
Experimental results show that the proposed
technique is both effective and efficient.
III.GENETIC ALGORITHMS
As mentioned in the introduction section, we see
that a genetic algorithm is a search technique
MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
used in finding true or approximate solutions to
optimization and search problems. Genetic
algorithms are categorized as global search
heuristics. Genetic algorithms form a particular
class of evolutionary algorithms that use
techniques inspired by evolutionary biology such
as inheritance, mutation, selection, and crossover
(also called recombination).Genetic algorithms
are implemented as a computer simulation in
which a population of abstract representations
(called chromosomes or the genotype or the
genome) of candidate solutions (called
individuals, creatures, or phenotypes) to an
optimization problem evolves toward better
solutions. Traditionally, solutions are represented
in binary strings of 0s and 1s, but other
encodings are also possible. The evolution
usually starts from a population of randomly
generated individuals. Genetic algorithms form
one of the best ways to solve a problem for
which a little is known. They are very general
algorithms that work well in any search space. A
genetic algorithm is able to create a high quality
solution.
A. Background
Genetic algorithm (GِA) is a heuristic solution
search or an optimization technique, originally
motivated by the Darwinian principle of
evolution through (genetic) selection. A GA uses
a highly abstract version of evolutionary
processes to evolve solutions to some given
problems. Each GA operates on a population of
artificial chromosomes. These are strings in a
finite alphabet (usually binary). Each
chromosome represents a solution to a problem
and has fitness, a real number, which is a
measure of the quality of the solution; to the
particular problem. Starting with a randomly
generated population of chromosomes, a GA
carries out a process of fitness-based selection
and recombination to produce a successor
population, the next generation. During
recombination, selecting parent chromosomes
and their genetic material is recombined to
produce child chromosomes. These then pass
into the successor population. As this process is
iterated, a sequence of successive generations
evolves and the average fitness of the
chromosomes tends to increase until some
stopping criterion is reached. In this way, a GA
“evolves” a best solution to the given problem,
[7]. GAs was first proposed by John Holland’s,
as a means to find good solutions to problems
that were otherwise computationally intractable.
182
Holland’s schema theorem [4], and the related
building block hypothesis, provided a theoretical
and conceptual basis for the design of efficient
GAs. It also proved straightforward to
implement GAs due to their highly modular
nature. As a consequence, the field grew quickly
and the technique was successfully applied to a
wide range of practical problems in science,
engineering and industry. GA theory is an active
and growing area, with a range of approaches
being used to describe and explain phenomena
not anticipated by earlier theory. In tandem with
this, more sophisticated approaches for directing
the evolution of a GA population are aimed at
improving performance on classes of problem
known to be difficult for GAs, [4]. The
development and success of GAs contributed
greatly to a wider interest in computational
approaches based on natural phenomena. It is
now a major stand of the wider field of
computational intelligence, which encompasses
techniques such as neural networks, and artificial
immunology. Genetic algorithms are search
methods that can be used for both solving
problems and modeling evolutionary systems.
Since it is heuristic (it estimates a solution), GAs
differs from other heuristic methods in several
ways. The most important difference is that it
works on a population of possible solutions,
while other heuristic methods use a single
solution in their iterations. Another important
difference is that GAs is not a deterministic but a
probabilistic one.
B. Biological Background
All living organisms consist of cells. In each cell
there is the same set of chromosomes.
Chromosomes are strings of DNA and serve as a
model for the whole organism. The genes
determine a chromosome’s characteristic. Each
gene has several forms or alternatives, which are
called alleles, producing differences in the set of
characteristics associated with that gene. The set
of chromosome is called the genotype. Which
define a phenotype (the individual) with certain
fitness. During reproduction first occurs
recombination (or crossover). Genes from
parents form in some way the whole new
chromosome. The new created offspring can then
be mutated. Mutation means that the elements of
DNA are bit wise changed. The fitness of an
organism is measured by success of the organism
in its life. According to Darwinian Theory the
highly fit individuals are given opportunities to
“reproduce” whereas the least fit members of the
MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
population are less likely to get selected for
reproduction, and so “die out”, [2, 10].
IV. SYSTEMS OF LINEAR EQUATIONS
A system of equations is a collection of two or
more equations with the same set of unknowns.
In solving a system of equations, we try to find
values for each of the unknowns that will satisfy
every equation in the system. A system of linear
equations (or linear system) is a collection of
linear equations involving the same set of
variables
A. General form
A general system of m linear equations with n
unknowns can be written as
Here
x1 , x 2 ,..., x n are
the
a11 , a12 ,..., a mn , are the coefficients
b , b ,..., bm are the constant
of the system, and 1 2
183
linear combination of the given left-hand vectors,
then any solution is unique. In any event, the
span has a basis of linearly independent vectors
that do guarantee exactly one expression; and the
number of vectors in that basis (its dimension)
cannot be larger than m or n, but it can be
smaller. This is important because if we have m
independent vectors a solution is guaranteed
regardless of the right-hand side, and otherwise
not guaranteed.
C. Matrix equation
The vector equation is equivalent to a matrix
equation of the form Ax=b where A is an m×n
matrix, x is a column vector with n entries, and b
is a column vector with m entries.
The number of vectors in a basis for the span is
now expressed as the rank of the matrix.
unknowns
D. Solution set
terms.
A solution of a linear system is an assignment of
values to the variables x1, x2, , ..., xn such that
each of the equations is satisfied. The set of all
possible solutions is called the solution set.
A linear system may behave in any one of three
possible ways:
1. The system has infinitely many
solutions.
2. The system has a single unique solution.
3. The system has no solutions.
B. Vector equation
One extremely helpful view is that each
unknown is a weight for a column vector in a
linear combination.
E. System of linear equations in two unknowns
First, a linear equation in two unknowns x and y
is an equation of the form
This allows all the language and theory of vector
spaces (or more generally, modules) to be
brought to bear. For example, the collection of
all possible linear combinations of the vectors on
the left-hand side is called their span, and the
equations have a solution just when the righthand vector is within that span. If every vector
within that span has exactly one expression as a
ax+by=c
(2)
Where a, b, and c are numbers, and where a and
b are not both zero.
Second, a system of linear equations is just a
collection of these beasts. To solve a system of
linear equations means to find a solution (or
MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
solutions) (x, y) that simultaneously satisfies all
of the equations in the system.
V. THE PROPOSED METHODOLOGY OF
SOLUTION
A. Coding Scheme
The coding scheme of this problem is
accomplished using a relatively standard coding
scheme. In this scheme, the parameters of the
search problem are represented as bit strings.
The equations are presented collectively as:
5GW1 - 4 GW2 -3 = 0
5GW1 - 8 GW2 -2 = 0
184
Generation
Number
GW1
GW2
1
7.6410000000E-01
5.726000000E-01
2
6.1635767147E-01
1.0835431448E-01
3
4.7684443427E-01
6.111848630E-01
4
1.7105363546E-01
3.4225986114E-01
5
1.5466544591E-01
6.8268863965E-01
6
9.8512245365E-01
2.483543144E-01
7
9.8512245365E-01
2.483543144E-01
(3)
The unknown's parameters of the above equation
are GW1and GW2.
The exact solution of this system is
B. Fitness function
The fitness function to be minimized by the
genetic algorithms is
GW1
GW2
0.8
0.25
g = max [abs (fi)] for i =1,2,3,…,N
(4)
Where g is the fitness function to be minimized,
max [abs (fi)] is the maximum absolute value of
individual equations in the system f(x) = 0, and
N is the number of equations in the system [3].
VI. SIMULATED OUTPUT RESULTS
In this section, we solve the linear system of the
equation (3) using the following parameters:String length of each parameter is = 16
The number of generation = 7
From this output result, we notice that we reach
to the optimum solution when the results are
equal in the final generations, so we stopped it
and take the final solution to compare it within
the exact solutions. Also, from the results of the
table1which indicates G.A. in solving linear
system, we see that the solution is almost equal
to the analytical one. Nevertheless on using the
G.A. to solve linear system, we obtain the result
very swiftly.
VII. CONCLUSION
TABLE 1 SIMULATED OUTPUT
RESULTS
In this paper, we have shown how genetic
algorithms can be successfully applied to solve
linear equations systems. The result of using GA
compared to the exact solutions obtained by
some different numerical methods turned to be
confirming. As a future work we plan to apply
genetic algorithms to training and designing
artificial intelligence systems such as artificial
neural networks, prediction of three dimensional
MASAUM Journal of Basic and Applied Science, Vol.1, No.2 September 2009
protein structure. Also, the same technique can
be used and applied to more complex non-linear
systems.
REFERENCES
[1] Amiolemhen, P. E. , and A. O. A. Ibhadode , “
Application Of Genetic Algorithms-Determination
Of The Optimal Machining Parameters In The
Conversion Of A Cylindrical Bar Stock Into A
Continuous Finished Profile ” , International
Journal of Machine tools & Manufacture 44 , pp
1403 – 1412 , 2004.
[2]“GeneticAlgorithms”.http://www.tjhsst.edu/~ai/
AI2001/GA.htm
[3] Ilgin, M. A. , “ Joint Optimization Of Spare
Parts Inventory And Maintenance Policies Using
Hybrid
Genetic
Algorithms
“2006.
http://www.fbe.deu.edu.tr/tezler/2006/YL_t2021.p
df
[4] Jähne, B. , “ Digital Image Processing ” ,
Springer – Verlag Berlin, Heidelberg , 2002.
[5] Kariuki, B. M. , H. S. González , R. L. Johnston
, and K. D. M. Harris, “ The Application Of A
Genetic Algorithm For Solving Crystal Structures
From Powder Diffraction Data ” , Chemical
Physics Letters 280 , pp. 189 – 195 , 1997.
[6] Kelner, V. , and O. Léonard, “ Application Of
Genetic Algorithms To
Lubrication Pump
Stacking Design ” , Journal of Computational and
applied Mathematics 168, pp. 255 – 265 , 2004.
[7] McCall, J., “Genetic Algorithms for Modeling
and Optimization”, Journal of Computational and
Applied Mathematics 184, pp. 205 – 222,
2005.
[8] Renner, G. , and A. Ekárt , “ Genetic
Algorithms In Computer Aided Design ” ,
Computer Aided Design 35 , pp. 709 – 726 , 2003.
[9] Sahoo, R. K. , T. Banerjee , S. A. Ahmed , and
A. Khanna , “Improved Binary Parameters
Using GA For Multi-Component Aromatic
Extraction : NRTL Model Without And With
Closure Equations ” , Fluid phase Equilibria 239
, pp. 107 – 119 , 2006.
[10] Sandikci, B. , “ Genetic Algorithms ” .
www.ie.bilkent.edu.tr/~Lors/ie572/barhaneddin.
pdfhttp://
[11]http://www.phd.etfbl.net/files/Works_PDF/Ti
marac%20Igor.pdf
185