Optimization Techniques
Population-Based
Algorithms
José M PEÑA ([email protected])
Antonio LATORRE ([email protected])
(Universidad Politécnica de Madrid)
1
Optimization Techniques (03 PBA)
Contents
1. 
2. 
Initial concepts: Heuristics, taxonomy
Population-based optimization techniques:
– 
– 
– 
– 
– 
– 
3. 
4. 
2
Genetic Algorithms
Estimation of Distribution Algorithms
Differential Evolution
Genetic Programming
Ant Colony Optimization
Particle Swarm Optimization
Behavior, parameters and configuration
Coevolution: Cooperation & competition
Optimization Techniques (03 PBA)
Heuristics vs Meta-Heuristics
l 
l 
l 
Heuristic derives from the greek verb heuriskein (ενρισκειν) which
means “to find”
Faster than mathematical optimization (branch & bound, simplex,
etc)
“Meta” means “beyond, in an upper lever”
– 
– 
– 
– 
– 
3
Meta-heuristics are strategies that “guide” the search process
The goal is to explore the search space in order to find (near-)optimal
solutions
Meta-heuristics are not problem-specific
The basic concepts of meta-heuristics permit an abstract level
description
They may incorporate mechanisms to avoid getting trapped in
confined areas of the search space
Optimization Techniques (03 PBA)
Classification
l 
Heuristic
– 
– 
l 
Constructive algorithms (greedy)
Local search algorithms (hill-climbing…)
Meta-heuristic
– 
Trajectory methods: Describe a trajectory in the search space
during the search process
l 
l 
l 
l 
– 
Population-based: Perform search processes which describe
the evolution of a set of points in the search space
l 
4
Variable Neighbourhood Search
Iterated Local Search
Simulated Annealing
Tabu Search
Best example: Evolutionary computation (Genetic Algorithms)
Optimization Techniques (03 PBA)
Classification
5
© Wikipedia
Optimization Techniques (03 PBA)
Population-Based
Optimization
General characteristics:
l  Trial-and-error search (not brute-force)
l  Graduated solution quality
l  Stochastic search of solution landscape
l  Population of candidate solutions
6
Optimization Techniques (03 PBA)
Abstract Problem Description
Production of the next population
Solution description
(format of the solution)
?
?
?
?
?
?
Instantiation
Solution evaluation
(fitness function)
Evaluation
f(x)
7
PROBLEM
DESCRIPTION
Candidate solutions
(population)
s1
s2
s3
f(s1)
f(s2)
f(s3)
s4
s5
s6
s7
s8
s9
s10
f(s4)
f(s5)
f(s6)
f(s7)
f(s8)
f(s9)
f(s10)
s11
f(s11)
Fitness assignment
Recombination
Generate
new solutions
POPULATION-BASED
OPTIMIZATION TECHNIQUE
Optimization Techniques (03 PBA)
Population-Based Techniques
l 
Different techniques
based on different
criteria for:
– 
– 
– 
8
Recombination
(action)
Generation of new
solution (procedure)
Production of the next
population (action)
Production of the next population
Recombination
Generate
new solutions
Optimization Techniques (03 PBA)
Population-Based Techniques
l 
These techniques are
independent of the complexity
and nature of the objective
function (black box
optimization)
Solution evaluation
(fitness function)
f(x)
9
l 
But it depends on how the
solution is described:
– 
– 
– 
– 
Continuous domain.
Discrete domain.
Permutations.
Sequence of actions.
Solution description
(format of the solution)
?
?
?
?
?
?
Optimization Techniques (03 PBA)
Solution Description
l 
Continuous description:
– 
– 
– 
l 
10
Solution is based on a fixed
list of …
continuous values (real
numbers) which are …
bounded between two limit
values.
l 
Discrete description:
– 
– 
– 
Solution is based on a fixed
list of …
discrete values of a …
set of possible values.
l 
Example: Equipment
configuration (possible
modules, empty spaces).
l 
Particular case: Binary
description: e.g., selection
problem.
Example: Firing a projectile
(angles, velocity, spin, …)
Optimization Techniques (03 PBA)
Solution Description
l 
Permutations description:
– 
– 
– 
l 
11
l 
Solution is based on a
ordered list of …
a list of given values …
(with no repetition).
Example: Scheduling
(sequence of tasks), or
routing (sequence of path
points).
Sequence of actions:
– 
– 
– 
l 
Solution is based on a fixed
or dynamic list of …
possible actions and …
conditions.
Example: Program or
planning (operations in
robotics).
Optimization Techniques (03 PBA)
Derived Descriptions
l 
There are derived descriptions, such as:
– 
– 
– 
– 
12
Dynamic versions of continuous or discrete
descriptions.
Mixed descriptions.
Structured solutions: Matrices or trees.
…
Optimization Techniques (03 PBA)
Fitness Landscapes
Fitness:
Function that evaluates individual capabilities.
Graphical representation
N+1 dimensions
fitness
Solution evaluation
(fitness function)
f(x)
13
© L.J. Pit, ‘96
Optimization Techniques (03 PBA)
Differences between PBA and
Conventional Search Algorithms
l 
l 
l 
l 
l 
14
PBA works on a coding of the parameters set, not the
parameter themselves
PBA searches from a population of points, not a single
point
PBA uses only a payoff function, and no domain
knowledge
PBA uses probabilistic transition rules, not
deterministic ones
PBA can provide a number of potential solutions to a
given problem. The final choice is left to the user.
Optimization Techniques (03 PBA)
Population-Based Algorithms
GENETIC ALGORITHMS
15
Optimization Techniques (03 PBA)
Overview
l 
l 
l 
l 
l 
16
A class of heuristic optimization algorithms
Inspired by the biological evolution process
Uses concepts of “Natural Selection” and “Genetic
Inheritance” (Darwin 1859)
Originally developed by John Holland (1975)
Widely-used in business, science and engineering
Optimization Techniques (03 PBA)
Natural Evolution I
l 
l 
l 
l 
17
Population(s) of individuals
Individuals have a fitness
Various operators: Mutation, Recombination
Selection towards higher fitness (“survival of
the fittest”)
Optimization Techniques (03 PBA)
Natural Evolution II
Darwin: The Origin of Species
– 
– 
Search of optimal forms (fittest in the
environment): Natural Selection
Based on:
Phenotypic variability: small, random,
apparently undirected deviations of offspring
from parents
l  Fitness: capability of an individual to survive
and reproduce in an environment
l  Adaptation: state of being suitable for an
environment
l 
18
Optimization Techniques (03 PBA)
Darwinian Evolution I
l 
l 
19
Diversification of species into ecological
niches
Example: Galapagos finches
Optimization Techniques (03 PBA)
Darwinian Evolution (1)
• Darwinian
DiversificationEvolution
of species into
II ecological niches
• Example: Galapagos finches
20
Optimization Techniques (03 PBA)
Darwinian
III
DarwinianEvolution
Evolution (2)
l 
l 
• A common metaphor: Adaptive landscape [Wright 19
A common
metaphor: Adaptive
landscape
Better model: due to
interactions,
landscape changes
dynamically over
timeExample: Two traits of an
Australian grashopper
21
Abstract view of an adaptive landscape
• Better model: Due to interactions, landscape
Optimization Techniques (03 PBA)
Neo-Darwinism I
Neo-Darwinism (1)
l  •Molecular-genetic
processofofevolution
evolution
Molecular-genetic processes
Building plan
DNA
l  •Building
planofoflife:
life:
DNA
“Alphabet”: Nucleotide
bases
–  –“Alphabet”:
Nucleotide
bases
– Purines:
A(denine),
l  Purines:
A(denine), G(uanine)
G(uanine)
– Pyrimidines:
T(hymine), C(ytosine)
l  Pyrimidines: T(hymine),
C(ytosine)
22
Optimization Techniques (03 PBA)
Neo-Darwinism II
Neo-Darwinism (2)
Neo-Darwinism
(2)
– Watson-Crick complementarity: A – T, C – C
l 
l 
23
Watson-Crick complementarity of bases
– Watson-Crick
complementarity:
A spiral
– T, C – C
–
Structure:
Double-helix,
(1953): A-T, G-C
– Structure: Double-helix, spiral
Structure: double-helix, spiral
Optimization Techniques (03 PBA)
Neo-Darwinism III
Neo-Darwinism (3)
• Genotype versus phenotype:
l  Genotype vs. Phenotype:
– Central dogma of molecular genetics
–  Central dogma of molecular genetics
– Information flow:
–  Information flow
Genotype
DNA
24
Transcription
Phenotype
RNA
Translation
Proteine
Phenotype
– NOT: Genotype
(Lamarckism), typically not true ... but exceptions
Optimization Techniques (03 PBA)
Neo-Darwinism IV
l 
Hierarchy of genetic information:
– 
– 
– 
– 
– 
– 
– 
25
Nucleotide base: single symbol
Codon: unit of translation
Gene: unit encoding a proteine
Scripton: unit of transcription
Replikon: unit of reproduction
Chromosome: Few replicons (meiotic unit)
Genome: Set of chromosomes (mitotic unit)
Optimization Techniques (03 PBA)
Neo-Darwinism I
26
Chromosome is divided in parts: genes
Genes code for proteines (properties)
Possible values of the genes: allele
Position of the gene in the chromosome: locus
Optimization Techniques (03 PBA)
Neo-Darwinism II
•  The entire combination of genes: genotype
•  A genotype is expressed as a phenotype
•  During reproduction “errors” occur
•  Due to these “errors” genetic variation exists
•  Most important “errors” are:
•  Recombination (cross-over)
•  Mutation
27
Optimization Techniques (03 PBA)
Genetic Algorithms (GAs)
John Holland (~1973)
– 
– 
GAs use the principle of natural selection to solve complex
optimization problems.
An alternative of brute-force search on a complex solution
space.
l 
l 
– 
– 
28
Instead of complete exploration
Stochastically-driven search
Mathematical expression of success in life: fitness
“Preservation of favourable variations and rejection of
unfavourable variations.”
Optimization Techniques (03 PBA)
Key Aspects of EAs I
Concepts:
– 
– 
– 
– 
Individual: A possible solution of the problem.
Gene: A solution component (a.k.a. an attribute) [eye color]
Allele: A gene value [eye color=green]
Population: group of potential solutions
Asumption: there exists a relationship between genetic information
and individual fitness.
+
10011101011011
-19 4 108 46
29
0.433 -33.345 0.0013
A
x
B
3
Optimization Techniques (03 PBA)
Key Aspects of EAs II
How new individuals are created?
– 
Crossover (recombination)
Mixing existing genetic material, assortment
l  Inheritage of useful characteristics
l  2 (or more) parents à 1 (or more) offspring
l 
– 
Mutation:
Rare (very rare in biological perspective)
l  Randomness
l  1 parent à 1 offspring
l 
30
They simulate the evolution process
The new individuals created are called offspring
Optimization Techniques (03 PBA)
Key Aspects of EAs III
l 
l 
l 
l 
31
Quality measure of individuals: Fitness
function, objective function
Survival-of-the-fittest principle
Mating-of-the-fittest principle
Algorithm cycles through iterations
(generations)
Optimization Techniques (03 PBA)
Main Components of GAs
1. 
2. 
3. 
4. 
5. 
6. 
7. 
32
Representation of individuals: Coding
Evaluation method for individuals: Fitness
Initialization procedure for 1st generation
Definition of variation operators (crossover,
mutation)
Parent (mating) selection mechanism
Survivor (environmental) selection mechanism
Technical parameters: mutation and crossover
rates, population size, etc.
Optimization Techniques (03 PBA)
Evolutionary Algorithms
Selection
Population
33
Recombination
& Mutation
Population
Optimization Techniques (03 PBA)
Advantages of GAs I
1. 
Widely applicable, also in cases where no
good solution techniques are available
1. 
2. 
3. 
4. 
2. 
34
Multimodalities, discontinuities, constraints
Noisy objective functions
Multiple criteria decision making problems
Implicitly defined problems (simulation models)
No presumptions with regards to the problem
space
Optimization Techniques (03 PBA)
Advantages of GAs II
3. 
4. 
5. 
6. 
35
Low development cost, i.e., costs to adapt to
new problem spaces
The solutions of GAs have straightforward
interpretations
Can run interactively, always deliver solutions
Self-adaptation of parameters
Optimization Techniques (03 PBA)
Disadvantages of GAs
l 
l 
l 
No guarantee for finding optimal solutions
within a finite amount of time. This is true for all
global optimization methods.
No complete theoretical basis (yet), but much
progress is being made
Parameter tuning is sometimes based on trial
and error
– 
36
We will see alternatives for this
Optimization Techniques (03 PBA)
Natural Evolution and
Operators
Discrete Crossover (1-point)
#9
00010100010001
Variants:
l 
l 
l 
37
10011101011011
00010100011011
2, 3, n-points crossover
Uniform crossover
Other...
Optimization Techniques (03 PBA)
Natural Evolution and
Operators
Continuous Crossover (BLX-α)
0.3 0.7 1.4 2.5 0.1 0.8 2.0 1.3 0.2
1.2 0.4 0.2 1.5 1.2 0.2 0.4 1.2 0.3
0.4 0.6 1.5 1.1 0.2 0.3 1.1 1.3 0.2
Variants:
l 
l 
l 
38
Gaussian
Uniform crossover
Other...
-αd=-0.75
-0.25
α=1.25
+αd=0.75
0.2
0.8
d=0.6
1.25
Optimization Techniques (03 PBA)
Natural Evolution and
Operators
Discrete Mutation (Flip)
#5
00010100011011
00011100011011
Variants:
l 
l 
l 
39
Mutation-by-swap
Biased mutation
Cataclysmic mutation
Optimization Techniques (03 PBA)
Natural Evolution and
Operators
Continuous Mutation (Gaussian)
#7
0.4 0.6 1.5 1.1 0.2 0.3 1.1 1.3 0.2
0.4 0.6 1.5 1.1 0.2 0.3 0.9 1.3 0.2
Variants:
l 
l 
l 
40
µ=1.1
[original value]
σ=α(M-m)
Uniform
Biased mutation
Cataclysmic mutation
1.1
[domain]
Optimization Techniques (03 PBA)
Natural Evolution and
Operators
Selection:
Evaluation of individuals (fitness)
2.  Parent selection (for each crossover operation).
Selective pressure: relationship between capacities (fitness) of
the individual and its possibilities to generate new individuals
(participate in the reproduction).
1. 
Alternatives:
– 
– 
– 
41
Tournament selection
Rank-based: Continuous function
Roulette wheel
Optimization Techniques (03 PBA)
Roulette Wheel
42
f(x)
%
6.82
1.11
8.48
2.57
3.08
31%
5%
38%
12%
14%
22.05
100%
Optimization Techniques (03 PBA)
Tournament Selection
f(x)
6.82
1.11
8.48
2.57
3.08
Tournament (p,k):
Take k individuals (randomly)
if((rand() ≤ p)
Select best individual
else if((rand() ≤ p)
Select second individual
…so on…
1.11
8.48
43
Selected (p probability), otherwise
Optimization Techniques (03 PBA)
Canonical Genetic Algorithm
P0
Initial population
end condition?
Fitness-function
selection
P’i
Selected population
crossover/mutation
Pi+1
44
Next population
Includes the
chances of
participation in
reproduction
Optimization Techniques (03 PBA)
GA Variants (Replacement)
Elitism:
– 
– 
The best individuals(s) in Pi stay in population Pi.
Provides monotonic fitness increment
Steady-state:
– 
– 
45
Only one individual is created per generation.
New one replaces worst individual.
Optimization Techniques (03 PBA)
Replacement Variants
Selection
Crossover/mutation
Pi
Current population
Offi
Offspring population
P’i
l 
– 
– 
l 
Next population
Elitism:
The best(s) individuals in Pi stay in population Pi.
Provides monotonic fitness increment
Steady-state:
– 
– 
Only one individual is created per generation.
New one replaces worst individual
Canonical
46
Steady-sate
Elitism
Offspring size
N
1
>, = or < N
Current à Next
0
N-1
<< N
Replacement
All
Worst
Worst
Optimization Techniques (03 PBA)
Parameters and Functions
l 
l 
l 
l 
l 
47
Chromosome representation
Creation of initial population
Fitness function
Genetic operators
Parameters (population size,
probabilities of the genetic
operators, etc.)
Typical configuration for
small problems:
Population size:
50 – 100
Children per
generation:
= population
size
Crossovers:
> 85%
Mutations:
< 5%
Generations:
20 – 20,000
Optimization Techniques (03 PBA)
Example (The MAXONE
problem)
•  Suppose we want to maximize the number of ones
in a string of m binary digits
Is it a trivial problem?
•  It may seem so because we know the answer in
advance
•  However, we can think of it as maximizing the number
of correct answers, each encoded by 1, to m yes/no
difficult questions`
48
Optimization Techniques (03 PBA)
MAXONE problem
l 
l 
l 
49
An individual is encoded (naturally) as a string
of m binary digits
The fitness f of a candidate solution to the
MAXONE problem is the number of ones in its
genetic code
We start with a population of n random strings.
Suppose that m = 10 and n = 6
Optimization Techniques (03 PBA)
MAXONE problem
(initialization)
•  We toss a fair coin 60 times and get the
following initial population:
50
s1 = 1111010101
f (s1) = 7
s2 = 0111000101
f (s2) = 5
s3 = 1110110101
f (s3) = 7
s4 = 0100010011
f (s4) = 4
s5 = 1110111101
f (s5) = 8
s6 = 0100110000
f (s6) = 3
Optimization Techniques (03 PBA)
MAXONE problem (selection)
•  Suppose that, after performing selection, we
get the following population:
51
s1` = 1111010101
(s1)
s2` = 1110110101
(s3)
s3` = 1110111101
(s5)
s4` = 0111000101
(s2)
s5` = 0100010011
(s4)
s6` = 1110111101
(s5)
Optimization Techniques (03 PBA)
MAXONE problem (crossover)
•  Next we mate strings for crossover. For each
couple we decide according to crossover
probability (for instance 0.6) whether to actually
perform crossover or not
52
•  Suppose that we decide to actually perform
crossover only for couples (s1`, s2`) and (s5`, s6`).
For each couple, we randomly extract a
crossover point, for instance 2 for the first and 5
for the second
Optimization Techniques (03 PBA)
MAXONE problem (crossover)
Before crossover:
s1` = 1111010101
s5` = 0100010011
s2` = 1110110101
s6` = 1110111101
After crossover:
53
s1`` = 1110110101
s5`` = 0100011101
s2`` = 1111010101
s6`` = 1110110011
Optimization Techniques (03 PBA)
MAXONE problem (mutation)
•  The final step is to apply random mutation: for each bit that
we are to copy to the new population we allow a small
probability of error (for instance 0.1)
•  Before applying mutation:
s1`` = 1110110101 s4`` = 0111000101
s2`` = 1111010101 s5`` = 0100011101
s3`` = 1110111101 s6`` = 1110110011
54
Optimization Techniques (03 PBA)
MAXONE problem (mutation)
After applying mutation:
s1``` = 1110100101 f (s1``` ) = 6
s2``` = 1111110100 f (s2``` ) = 7
s3``` = 1110101111 f (s3``` ) = 8
s4``` = 0111000101 f (s4``` ) = 5
s5``` = 0100011101 f (s5``` ) = 5
s6``` = 1110110001 f (s6``` ) = 6
55
Optimization Techniques (03 PBA)
MAXONE problem
•  In one generation, the total population
fitness changed from 34 to 37, thus improved
by ~9%.
•  At this point, we go through the same
process all over again, until a stopping
criterion is met
56
Optimization Techniques (03 PBA)
Programming with GAlib
Provides:
– 
– 
57
Different algorithms
Different genomes
Optimization Techniques (03 PBA)
GA Example
Programming a problem
requieres:
– 
– 
– 
58
To define de fitness
function
To define de genome
format
To setup the algorithm
parameters (most of them
have default values).
float Objective(GAGenome&)
{ /*objective function here*/}
main(){
// Create a genome
GA1DBinaryStringGenome
genome(size, Objective);
// Create the GA
GASimpleGA ga(genome);
// Evolve the GA
ga.evolve();
// Print out the results
cout <<ga.statistics()<< endl;
}
Optimization Techniques (03 PBA)
Extra Features
Defining genetic operators (see GAGenome)
–  g.crossover(…)
g.mutator(...)
Bounding gene values (see GAAlleleSet)
–  GARealAlleleSet
a(0,1)
GAAlleleSet<int> a(5,v)
Evolution loop control (see GAAlgorithm)
–  ga.done()
–  ga.step()
59
ga.population()
ga.generation()
Optimization Techniques (03 PBA)
Population-Based Algorithms
ESTIMATION OF
DISTRIBUTION ALGORITHMS
60
Optimization Techniques (03 PBA)
Recombination Operators
l 
In Genetic Algorithms the generation of new
solutions (offspring) is based on recombination
operators:
– 
– 
– 
l 
61
Selection
Crossover
Mutation
The appropriate configuration of these
operators represent extra parameters to be
defined.
Optimization Techniques (03 PBA)
Estimation of Distribution
Algorithms
l 
Estimation of Distribution Algorithms (EDAs)
propose the use a probabilistic model:
– 
– 
– 
62
To recombine existing population to produce the
new offspring.
To model of (currently) promising regions of the
search space.
To convert the optimization problem into a search
over probability distributions.
Optimization Techniques (03 PBA)
GAs vs. EDAs
P0
P0
Initial population
Initial population
end condition?
selection
63
Selected
population
Selected
population
P’i
P’i
end condition?
selection
crossover/mutation
model/sampling
Pi+1
Pi+1
Next population
Next population
Optimization Techniques (03 PBA)
EDA Model and Sampling
Selected
population
P’i
Best solutions
from the
current
population
64
Offspring
population
Modeling
Sampling
A probabilistic
model that
describe the
best solutions
Oi+1
New
individuals
derived from
sampling the
model
Optimization Techniques (03 PBA)
EDA Model and Sampling
l 
There are two types of EDAs:
– 
– 
l 
l 
65
EDAs for Discrete Representations (e.g., Binary-coded) and
EDAs for Real-Code Representations (Continuous Domains).
EDAs for discrete optimization will evolve probability
distribution functions, such as bayesian networks.
EDAs for continuous domain problems will evolve
probability density functions, such as gaussian
networks.
Optimization Techniques (03 PBA)
Univariate Marginal
Distribution Algorithm (UMDA)
l 
Simplest discrete bayesian network:
– 
Assumes genes are mutually independent.
( P(x=V1), P(x=V2), P(x=V3) … P(x=Vn) ) =
( p 1, p 2, p 3 … p n )
General expresion for a
discrete set of n values
In the case of binary encoding, it can be sumarized as the
probability, for a given gene position to have value x=1
66
Optimization Techniques (03 PBA)
MAXONE problem
(initialization)
•  Let’s consider the same example:
67
s1 = 1111010101
f (s1) = 7
s2 = 0111000101
f (s2) = 5
s3 = 1110110101
f (s3) = 7
s4 = 0100010011
f (s4) = 4
s5 = 1110111101
f (s5) = 8
s6 = 0100110000
f (s6) = 3
Optimization Techniques (03 PBA)
MAXONE problem (selection)
•  Select the top 4 individuals:
68
s1 = 1111010101
f (s1) = 7
s2 = 0111000101
f (s2) = 5
s3 = 1110110101
f (s3) = 7
s4 = 0100010011
f (s4) = 4
s5 = 1110111101
f (s5) = 8
s6 = 0100110000
f (s6) = 3
Optimization Techniques (03 PBA)
MAXONE problem (model)
•  Select the top 4 individuals:
s1 = 1111010101
f (s1) = 7
s2 = 0111000101
f (s2) = 5
s3 = 1110110101
f (s3) = 7
s5 = 1110111101
f (s5) = 8
p(x=1)=(0.75, 1.0, 1.0, 0.5, 0.5, 0.75, 0.25, 1.0, 0.0, 1.0)
69
Optimization Techniques (03 PBA)
MAXONE problem (sampling)
•  Sample new individual according to the model:
p(x=1)=(0.75, 1.0, 1.0, 0.5, 0.5, 0.75, 0.25, 1.0, 0.0, 1.0)
70
s’1 = 1110010101
f (s’1) = 6
s’2 = 0111111101
f (s’2) = 8
s’3 = 1111111101
f (s’3) = 9
s’4 = 0111010101
f (s’4) = 6
s’5 = 1110101101
f (s’5) = 7
s’6 = 1111100101
f (s’6) = 7
Optimization Techniques (03 PBA)
MAXONE problem
•  In one generation, the total population
fitness changed from 34 to 43, thus improved
by ~26%.
•  At this point, we go through the same
process all over again, until a stopping
criterion is met.
71
Optimization Techniques (03 PBA)
Continuous UMDA (UMDAC)
l 
If the coding represents continuous domain,
instead of using probability values, it uses
density functions based on a given distribution.
– 
– 
72
Usually, normal distributions.
Distribution parameters are calculated using the
maximum likelihood estimators.
Optimization Techniques (03 PBA)
UMDAC
l 
Simplest discrete gaussian network:
– 
Assumes genes are mutually independent.
N(µ,σ)
General expresion for a
discrete set of n values
73
Optimization Techniques (03 PBA)
MAXSUM Problem
l 
Similar to MAXONE but on contninuous
domains.
s1 s2 s3 s4 s5 s6 g1 0,23 0,69 0,23 0,27 0,30 0,32 g2 0,12 0,91 0,54 0,94 0,26 0,61 g3 0,78 0,12 0,81 0,07 0,73 0,26 g4 0,56 0,82 0,98 0,56 0,51 0,03 f(s) 1,69 2,54 2,55 1,85 1,80 1,22 g4 0,56 0,82 0,98 0,56 0,51 0,03 f(s) 1,69 2,54 2,55 1,85 1,80 1,22 Generation 0
74
s1 s2 s3 s4 s5 s6 g1 0,23 0,69 0,23 0,27 0,30 0,32 g2 0,12 0,91 0,54 0,94 0,26 0,61 g3 0,78 0,12 0,81 0,07 0,73 0,26 After selection
Best value
Optimization Techniques (03 PBA)
MAXSUM Problem
µ
σ
0,40 0,26 0,80 0,23 0,33 0,41 0,79 0,21 µ
σ
0,36 0,20 Modeling
s1 s2 s3 s4 s5 s6 g1 0,59 0,22 0,46 0,36 0,18 0,27 g2 0,82 0,96 0,81 0,31 1,00 0,96 g3 0,00 0,50 0,00 0,08 0,00 1,00 75
s1 s2 s3 s4 s5 s6 g2 0,82 0,96 0,81 0,31 1,00 0,96 g3 0,00 0,50 0,00 0,08 0,00 1,00 0,50 0,50 0,87 0,12 Modeling
g4 0,85 1,00 0,62 0,33 0,86 0,76 f(s) 2,26 2,67 1,89 1,07 2,04 2,99 Generation 1
g1 0,59 0,22 0,46 0,36 0,18 0,27 0,91 0,08 g4 0,85 1,00 0,62 0,33 0,86 0,76 f(s) 2,26 2,67 1,89 1,07 2,04 2,99 s1 s2 s3 s4 s5 s6 g1 0,19
0,43
0,58
0,24
0,32
0,56
g2 0,87
0,95
0,92
0,90
0,94
0,86
g3 0,62
0,53
1,00
0,89
0,71
0,87
Generation 2
…
…
g4 0,70
0,99
1,00
0,75
0,81
0,84
f(s) 2,38
2,90
3,50
2,77
2,78
3,14
Optimization Techniques (03 PBA)
EDA Variants
l 
Population-based Incremental Learning:
– 
l 
Mutual Information Maximization for Input Clustering (MIMIC)
– 
l 
Bayesian network model where nodes have at most k parents.
Estimation of Bayesian Networks Algorithm (EBNA)
– 
76
Uses a tree-structured graphical model
Bayesian Optimization Algorithm (BOA)
– 
l 
Lineal dependencies between variables
Combining Optimizers with Mutual Information Trees (COMIT)
– 
l 
Probability is updated accordint to an α value.
Multivariant bayesian network.
Optimization Techniques (03 PBA)
Population-Based Algorithms
DIFFERENTIAL EVOLUTION
77
Optimization Techniques (03 PBA)
Differential Evolution
l 
Optimization heuristic with similarities to
Evolutionary Algorithms. It has
– 
– 
– 
l 
l 
78
population,
variation operators, and
selection
It is based on adding the weighted difference
between two solutions to a third.
It provides continuous domain optimization.
Optimization Techniques (03 PBA)
Differential Evolution Process
Selection
Target vector xi,g
Donor vectors xr0,g, xr1,g, xr2,g
Mutation
Mutant vector vi,g
= xr0,g + F(xr1,g – xr2,g)
79
Replacement
Replacement
xi,g+1 ui,g if(f(ui,g)>f(xi,g))
xi,g otherwise
Recombination
Trial vector ui,g
ui,j,g vj,i,g if(rand[0,1]<Cr)
xj,i,g otherwise
Optimization Techniques (03 PBA)
Example
Selection
xr0,g
Target vector xi,g
Donor vectors xr0,g, xr1,g, xr2,g
xi,g
xr2,g
80
xr1,g
Optimization Techniques (03 PBA)
Example
xr0,g
xr0,g + F(xr1,g – xr2,g)
xi,g
Mutation
Mutant vector vi,g
= xr0,g + F(xr1,g – xr2,g)
xr1,g – xr2,g
xr2,g
81
xr1,g
vi,g
Optimization Techniques (03 PBA)
Example
ui,g
xi,g
Replacement
vi,g
Replacement
xi,g+1 ui,g if(f(ui,g)>f(xi,g))
xi,g otherwise
Recombination
Trial vector ui,g
ui,j,g vj,i,g if(rand[0,1]<Cr)
xj,i,g otherwise
82
Optimization Techniques (03 PBA)
MAXSUM Problem
s1 s2 s3 s4 s5 s6 g1 0,23 0,69 0,23 0,27 0,30 0,32 g2 0,12 0,91 0,54 0,94 0,26 0,61 g3 0,78 0,12 0,81 0,07 0,73 0,26 g4 0,56 0,82 0,98 0,56 0,51 0,03 f(s) 1,69 2,54 2,55 1,85 1,80 1,22 g4 0,56 0,82 0,98 0,56 0,51 0,03 f(s) 1,69 2,54 2,55 1,85 1,80 1,22 Generation 0
xi s2 xr2
xr0 xr1 s6 g1 0,23 0,69 0,23 0,27 0,30 0,32 g2 0,12 0,91 0,54 0,94 0,26 0,61 g3 0,78 0,12 0,81 0,07 0,73 0,26 Selection
83
F=0.5
xr1 – xr2
0,07 -0,28 -0,08 -0,47 F(xr1 – xr2)
0,04 -0,14 -0,04 -0,23 xr0 + F(xr1 – xr2)
xi
0,31 0,80 0,03 0,33 0,23 0,12 0,78 0,56 0,23 0.80 0.78 0,33 2,14
max
Optimization Techniques (03 PBA)
DE Variants
l 
Differential mutation
– 
– 
– 
One difference vector: F·(xr1- xr2)
Two difference vector: F·(xr1- xr2)+F·(xr3- xr4)
Mutation scale factor F
Crucial role: balance exploration and exploitation
l  Randomization: different distributions of F
DE/rand/1: vi,g= xr0,g + F(xr1,g- xr2,g)
DE/best/1: vi,g= xbest,g + F(xr1,g- xr2,g)
DE/current-to-best/1: vi,g= xi,g + F(xbest,g- xi,g) + F(xr1,g- xr2,g)
DE/rand/2: vi,g= xr0,g + F(xr1,g- xr2,g) + F(xr3,g- xr4,g)
DE/best/2: vi,g= xbest,g + F(xr1,g- xr2,g) + F(xr3,g- xr4,g)
l 
84
Optimization Techniques (03 PBA)
Population-Based Algorithms
GENERAL CHARACTERISTICS
85
Optimization Techniques (03 PBA)
Characteristics
l 
Pros:
– 
– 
– 
– 
– 
86
More general purpose
than traditional
optimization
algorithms
Ability to solve
“difficult” problems
Solution availability
Robustness
Inherent parallelism
l 
Cons:
– 
– 
– 
– 
Fitness function and
search techniques
often not obvious
Premature
convergence
Computationally
intensive
Difficult parameter
optimization
Optimization Techniques (03 PBA)
Aspects to consider:
Population Size
l 
Population aspects are important for the
performance of the algorithm:
– 
– 
– 
87
Population size
Population dynamics (including and removing
individuals)
End condition (based on population characteristics)
Optimization Techniques (03 PBA)
Population Size
l 
Population size has large impact
– 
– 
– 
88
Size too small then algorithm may converge too
quickly
Size too large may waste computational resources
Population size (and distribution) must ensure a
good provision of building blocks
Optimization Techniques (03 PBA)
Micropopulations
l 
There are problems in which population size is
restricted, due to:
– 
– 
89
Memory (or time) restrictions
Computational cost of the fitness function
l 
Theoretical work has shown that populations sizes
between n and 2n (n is binary string size) are
appropriate.
l 
If population size is <25 à micro-population GAs
Optimization Techniques (03 PBA)
Micropopulations
l 
Small populations have high risk to:
– 
– 
l 
Minimum size:
– 
– 
90
Under-cover the solution space
Premature convergence
At least one instance of
every allele (value) at
each locus (position).
Higher order alphabets
require bigger
populations.
Binary
alphabet
100111
011000
103
Quaterly 0 2 0
alphabet
211
332
Optimization Techniques (03 PBA)
Micropopulations
l 
Alternative: Systematic selection of the initial
population:
– 
– 
– 
Distance based population generation
Coverage conditions
Error detection codes
Not valid individual
91
Iterative process for
the generation of
initial population
(based on
distances)
Optimization Techniques (03 PBA)
Exploration and Exploitation
l 
l 
l 
92
Adaptation in natural systems
is framed by a tension
between exploration and
exploitation
Exploitation: Looking for the
best solution close to a good
one.
Exploration: Looking for
individuals of even higher
Exploration
fitness far away form the
current best ones.
Explotation
Exploration
Exploitation
Mutation
Crossover
High
Low
Low
High
Highly-dependant on the fitness & coding
Optimization Techniques (03 PBA)
Evolutionay Pressure
l 
It can be considered as a pressure system:
– 
– 
– 
– 
Definition: Relationship between fitness value and
individual’s chance to produce offspring.
Strong selective pressure “supports” premature
convergence of algorithm search
Weak selective pressure can make the search
ineffective
Pressure depends on:
Fitness evaluation,
l  Selection policy and
l  Reproduction chances
l 
93
Optimization Techniques (03 PBA)
Ending Condition
l 
Evolutionary algorithms can finish when:
– 
– 
– 
94
A number of generations are met.
Best population fitness is not improved since
several generations. (Convergence)
Population characteristics are almost the same.
(Population Convergence)
Optimization Techniques (03 PBA)
Parameter Tuning vs Control
l 
Parameter Tuning: A priori optimization of fixed
strategy parameters
l 
Parameter Control: On-the-fly optimization of
dynamic strategy parameters
Optimization Techniques (03 PBA)
Parameter Tuning methods
l 
l 
l 
l 
Start with stock parameter values
Manually adjust based on user intuition
Monte Carlo sampling of parameter values on
a few (short) runs
Meta-tuning algorithm (e.g., meta-EA)
Optimization Techniques (03 PBA)
Parameter Tuning drawbacks
l 
l 
l 
l 
l 
Exhaustive search for optimal values of
parameters, even assuming independency, is
infeasible
Parameter dependencies
Extremely time consuming
Optimal values are very problem specific
Different values may be optimal at different
evolutionary stages
Optimization Techniques (03 PBA)
Parameter Control methods
l 
Deterministic
– 
Example: replace pi with pi(t)
l 
l 
Adaptive
– 
l 
akin to cooling schedule in Simulated Annealing
Example: Rechenberg’s 1/5 success rule
Self-adaptive
– 
Example: Mutation-step size control in ES
Optimization Techniques (03 PBA)
Population-Based Algorithms
CO-EVOLUTION
99
Optimization Techniques (03 PBA)
Co-Evolution
l 
Co-Evolutionary algorithms (CEAs) evolve individuals
to search the best strategy according to a fitness
function that requires the comparison with other
individuals.
– 
– 
– 
– 
10
0
Based on Game Theory scenarios
The fitness of an individual depends on the relationship
between that individual and other individuals
In fact, individuals are not evaluated, their relationships do
Other perspectives is to consider fitness landscapes as
dynamic functions.
Optimization Techniques (03 PBA)
Co-Evolution
l 
l 
l 
l 
Do these algorithms ‘optimize’?
Fitness is subjective
Two types of Co-Evolutionary Algorithms:
Competitive CEA:
l  Cooperative CEA:
– 
– 
10
1
Individuals are rewarded
at the expense of those
with which they interact.
Continued minor
adaptations in some
individuals will force
competitive adaptations in
others.
– 
– 
Individuals are rewarded
as they work well
together.
Parallel adaptive
evolutionary forces help
keep the algorithm driving
along a (possibly infinite)
gradient
Optimization Techniques (03 PBA)
Competitive CEAs
l 
General Model:
– 
Predator-prey relationship:
l 
10
2
Application to the
optimization of a system
(population of the system
solutions) vs. different test
cases (population of
problems)
Predator
Population
Fitness: The
number of
individuals it
can actually
capture
Prey
Population
Fitness: The
number of
individuals
from which it
escapes
Optimization Techniques (03 PBA)
Cooperative CEAs
l 
Different models:
– 
Problem decomposition: Subpopulation
l 
10
3
Each population contains
individuals representing a
component of a larger
solution, and evolution of
these populations occurs
almost independently,
interacting only to obtain Fitness: Combination
of solutions. The result
fitness.
contributes to the
fitness of each
element.
Subpopulation
Optimization Techniques (03 PBA)
Pathologies of Co-Evolution
l 
Loss of gradient:
– 
one population comes to severely dominate the
others, thus creating an impossible situation in
which the other participants do not have enough
information from which to learn
l 
10
4
e.g., a small child, new to the game of chess, attempting to
learn to play by playing a grand master at her best
Optimization Techniques (03 PBA)
Pathologies of Co-Evolution
l 
Cyclic behavior:
– 
l 
Mediocre stability:
– 
10
5
where intransitivities in the reward system can allow
one population to adapt slightly to gain an
advantage over the others, then the others follow
suit, only for the original population to change again,
eventually, back to the original strategy
limiting behaviors are obtained, but do so at
particularly suboptimal points in the space, from
some external perspective.
Optimization Techniques (03 PBA)
Pathologies of Co-Evolution
l 
Focussing problems:
– 
l 
Red Queen effect:
– 
– 
10
6
brittle solutions because the coevolutionary search
has driven players to over-specialize on their
opponent’s weaknesses.
populations seem to be changing, but the internal
subjective measure shows no progress is occurring
“It takes all the running you can do, to keep in the
same place”. Through the Looking Glass (L. Carroll)
Optimization Techniques (03 PBA)
Co-Evolution Scenario:
Iterated Prisoner’s Dilemma
l 
Two-players game:
– 
– 
10
7
Defect
– 
Cooperate
– 
Pris #1
Pris
#2
– 
Each player has two choices:
cooperate or defect.
T>R and P>S (Defection
always pay more)
R>P (Mutual cooperation beats
mutual defection)
R>(S+T)/2 (Alternating doesn’t
pay)
Multiple iterations: undefined
end.
#1
Cooperate
#1
R=4
#2
#1
#2
#2
Defect
#1
#2
T=5
R=4
S=0
S=0
P=1
T=5
Payoff matrix
P=1