‘Hang On a Minute’: Investigations on the Effects of
Delayed Objective Functions in Multiobjective
Optimization
Richard Allmendinger
1 University
2 University
1
and Joshua Knowles2
College London
of Manchester
March 20, 2013
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
1
Closed-loop optimization
Applications include: shape design optimization, experimental quantum
control, drug discovery, instrument optimization, taste optimization, ...
[image from PK Wong (2008), PNAS 105(13) ]
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
2
Delayed objective functions
Batch evaluation
Assumption: experiments are done in batches
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
3
Delayed objective functions
Batch evaluation
Assumption: experiments are done in batches
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
4
Delayed objective functions
Focus of research
Multiobjective optimization problems where at least one of the objective
functions requires a relatively longer time to be evaluated than the
cheapest/quickest of the objective functions → at any given time, fitness
estimates of some solutions may only be partial
∆ti - Evaluation delay of objective i relative to the quickest objective
f1
∆t1 = 0
EA ~x
- Objectives are
evaluated in batches
- No possibility of
speeding-up evaluations
f2
∆t2 = t ′′ − t ′
t t′
Richard Allmendinger and Joshua Knowles
t ′′
‘Hang On a Minute’: Delayed Objectives
time
March 20, 2013
5
Delayed objective functions
Focus of research
Multiobjective optimization problems where at least one of the objective
functions requires a relatively longer time to be evaluated than the
cheapest/quickest of the objective functions → at any given time, fitness
estimates of some solutions may only be partial
f1 (~x ), f2 (~x )
Wait until all objective
values are available
before updating the
population
f1
EA ~x
f2
t t′
Richard Allmendinger and Joshua Knowles
t ′′
‘Hang On a Minute’: Delayed Objectives
time
March 20, 2013
6
Delayed objective functions
Focus of research
Multiobjective optimization problems where at least one of the objective
functions requires a relatively longer time to be evaluated than the
cheapest/quickest of the objective functions → at any given time, fitness
estimates of some solutions may only be partial
f1 (~x )
Return available
objective value and
assign pseudovalue to
the other objectives
f1
EA ~x
f2
t t′
Richard Allmendinger and Joshua Knowles
t ′′
‘Hang On a Minute’: Delayed Objectives
time
March 20, 2013
7
Related work
Finding minimal sets of objective functions without conflicting with
the full set (Brockhoff and Zitzler, 2009)
Asynchronous evaluation in optimization in the context of grid
computing (Scriven et al., 2008; Lewis et al., 2009)
Age-layered populations to allow solutions from previous generations
to take part in reproduction (Hornby, 2006)
Estimating objective values using surrogate modeling techniques
including fitness inheritance (Smith et al., 1995; Runarsson, 2004)
Ephemeral resource constraints (Allmendinger and Knowles, 2011,
2011a, 2012; Allmendinger, 2012): Temporary limitations in the
capacity to evaluate certain otherwise feasible solutions during the
optimization process.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
8
Population update strategies
Waiting strategy: Wait until all evaluations have been completed →
standard EAs and population update rules can be applied
Non-waiting strategy: Solutions with complete and partial information
on objective values co-exist in a population growing without bound
e.g. f1 needs 1 time step to be evaluated, and f2 has an evaluation delay
of ∆t2 = 2
P0
P3
O0
Wait for ∆t2
time steps
P6
O3
Wait for ∆t2
time steps
O6
...
Pi - (Ranked) population at time step i
Oi - Offspring population at time step i
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
9
Population update strategies
Waiting strategy: Wait until all evaluations have been completed →
standard EAs and population update rules can be applied
Non-waiting strategy: Solutions with complete and partial information
on objective values co-exist in a population growing without bound
P0
O0
P1
O1
P2
O2
...
Pi - (Ranked) population at time step i
Oi - Offspring population at time step i
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
10
Selecting solutions for evaluation on the delayed objective
function fm
Sweep selection: Select always the most recently generated solutions
Priority-based selection: Select solutions based on a score indicating a
solution’s potential to change the ranking of all (completely
evaluated) solutions in P
e.g. expensive objective function needs 3 time steps to be evaluated
Ot
Ot+1
Ot+2
Ot+3
Ot+4
...
Oi - Offspring population at time step i
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
11
Selecting solutions for evaluation on the delayed objective
function fm
Sweep selection: Select always the most recently generated solutions
Priority-based selection: Select solutions based on a score indicating a
solution’s potential to change the ranking of all (completely
evaluated) solutions in P
e.g. expensive objective function needs 3 time steps to be evaluated
Ot
Ot+1
Ot+2
Ot+3
Ot+4
...
Oi - Offspring population at time step i
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
12
Strategies for dealing with delayed objectives
Assignment of pseudovalues to the delayed objective fm
1
Random pseudovalue assignment: Uniform variate within the
observed objective range(s) um value of objective fm of all solutions
in P that have actually been evaluated on objective fm
2
Noise-based pseudovalue assignment: Add noise to a value drawn
from an existing solution value of the delayed objective
3
Fitness inheritance-based pseudovalue assignment: Use simple 1-NN
scheme in decision space n all objectives
Pseudovalues are reassigned at each generation
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
13
Strategies for dealing with delayed objectives
Ranking of solutions
1
Performance ranking: Sort all solutions in P according to their
non-dominated sorting ranks only
2
Performance + age ranking: Sort P based on the age of solutions
where more recently generated solutions are favoured in
environmental selection. Parental selection is then done on
non-dominated sorting ranks of solutions.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
14
Experimental Setup
EA parameter settings
Ranking-based EMOA with a non-fixed population size
For environmental selection the setting was µ = λ = 50
Solutions are evaluated in a batch of size ki = µ, i = 1, ..., m
Binary Tournament selection, simulated binary crossover (pc = 0.9),
and polynomial mutation (pm = 1/l)
Test problems
WFG1-WFG9 of the Walking Fish Group toolkit (Huband et al., 2006)
Problems consisted of l = 6 continuous decision variables and m = 2
or 3 objectives
Evaluation delay ∆ti measured in time steps (here generations)
20 independent algorithmic runs were performed for each experiment
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
15
Results - Standard EA / Waiting
WFG3
5
Without delayed objectives
With delayed objectives
True Front
4
3
f2
2
1
0
-1
-2
0
0.5
1
1.5
f1
2
2.5
3
Figure : Estimated true Pareto Front and median attainment surface obtained on
WFG3 with m = 2 objectives with objective f2 having an evaluation delay of
∆t2 = 3 time steps. The EMOA employed a waiting strategy.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
16
Results
WFG1 - Sweep Selection
WFG1 - Priority-based Selection
No waiting, fitness inheritence-based p. a.
No waiting, random p. a.
No waiting, noise-based p. a.
Waiting
0.25
Average hypervolume
Average hypervolume
0.25
0.2
0.15
0.1
0.05
0.2
0.15
0.1
0.05
0
2
4
6
Delay ∆tm
8
10
0
2
4
6
Delay ∆tm
8
10
Figure : Average hypervolume on WFG1 with m = 3 objectives and one objective
function, f3 , delayed by ∆t3 .
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
17
Results - Sweep versus Priority-Based Selection
WFG2 - Sweep Selection
0.9
0.8
0.8
Average hypervolume
0.85
Average hypervolume
WFG2 - Priority-based Selection
0.9
No waiting, random p. a.
No waiting, noise-based p. a.
No waiting, fitness inheritence-based p. a.
Waiting
0.75
0.7
0.65
0.6
0.55
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.5
0
0
2
4
6
Delay ∆tm
8
10
0
2
4
6
Delay ∆tm
8
10
Figure : Average hypervolume on WFG2 with m = 3 objectives and one objective
function, f3 , delayed by ∆t3 .
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
18
Average hypervolume
Results - Sweep on 2- and 3-Objective Problems
0.9
WFG2
0.8
No waiting, fitness inh.-based p. a.
No waiting, random p. a.
Waiting
0.7
0.6
m = 3 objectives
0.5
0.4
m = 2 objectives
0.3
0
2
4
6
Delay ∆tm
8
10
Figure : Average hypervolume on WFG2 with m = 2 and 3 objectives using 1
delayed objective function, f2 , with ∆t2 time steps. Sweep Selection.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
19
Results - Sweep Selection (2 Delayed Objectives)
WFG2
0.9
No waiting, fitness inh.-based p. a., 1 delayed obj. fun.
No waiting, fitness inh.-based p. a., 2 delayed obj. fun.
No waiting, random p. a., 1 delayed obj. fun.
No waiting, random p. a., 2 delayed obj. fun.
Waiting
Average hypervolume
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0
2
4
6
8
10
Delay ∆tm
Figure : Average hypervolume on WFG2 with m = 3 objectives using 1 and 2
delayed objective functions, f2 and f3 , with ∆t2 = ∆t3 time steps. The EMOAs
employed Sweep Selection.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
20
Conclusion
Delayed objective functions degrade performance of a standard EA
For short delays, waiting performs relatively well
For longer delays:
employ a fitness inheritance-based pseudovalue assignment,
rank solutions based on performance only
evaluate most recently generated solutions on delayed objectives
Observations hold on WFG2–9, for 2 or 3 objectives.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
21
Future Work
Improve pseudovalue assignment and selection of solutions for
evaluation on delayed objectives
Develop strategies for switching between waiting and not waiting
during the optimization (Allmendinger and Knowles, 2011)
Consider many-objective problems where several objectives are subject
to delays of different durations
Establish a framework for describing algorithms that can cope with
delayed objective functions
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
22
Questions ?
Acknowledgments:
Doug Kell and many others (closed-loop applications work)
Ian Stott and Jane Shaw (Unilever)
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
23
References
R. Allmendinger. Tuning Evolutionary Search for Closed-Loop Optimization. PhD
thesis, School of Computer Science, The University of Manchester, 2012.
R. Allmendinger and J. Knowles. On-line purchasing strategies for an evolutionary
algorithm performing resource-constrained optimization. In Proceedings of PPSN XI,
pages 161-170, 2011.
R. Allmendinger and J. Knowles. Policy learning in resource-constrained
optimization. In Proceedings of GECCO, pages 1971-1978, 2011.
R. Allmendinger and J. Knowles. On handling ephemeral resource constraints in
evolutionary search. Evolutionary Computation, 2013. Posted Online November 19,
2012. (doi:10.1162/EVCO a 00097).
D. Brockhoff and E. Zitzler. Objective reduction in evolutionary multiobjective
optimization: theory and applications. Evolutionary Computation, 17(2):135-166,
2009.
G. Hornby. ALPS: the age-layered population structure for reducing the problem of
premature convergence. In Proceedings of GECCO, pages 815-822, 2006.
A. Lewis, S. Mostaghim, and I. Scriven. Asynchronous multi-objective optimisation in
unreliable distributed environments. Biologically-Inspired Optimisation Methods,
pages 51-78, 2009.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
24
References
T. P. Runarsson. Constrained evolutionary optimization by approximate ranking and
surrogate models. In Proceedings of PPSN VIII, pages 401-410, 2004.
I. Scriven, D. Ireland, A. Lewis, S. Mostaghim, and J. Branke. Asynchronous multiple
objective particle swarm optimisation in unreliable distributed environments. In
IEEE Congress on Evolutionary Computation, pages 2481-2486, 2008.
R. Smith, B. Dike, and S. Stegmann. Fitness inheritance in genetic algorithms. In
Proceedings of the ACM symposium on Applied computing, pages 345-350, 1995.
S. Huband, P. Hingston, L. Barone, and L. While. A review of multiobjective test
problems and a scalable test problem toolkit. IEEE Transactions on Evolutionary
Computation, 10(5):477-506, 2006.
Richard Allmendinger and Joshua Knowles
‘Hang On a Minute’: Delayed Objectives
March 20, 2013
25