Multiple Objective Optimization
Oswaldo Aguirre
Agenda
•  Introduction
•  Multiple objective optimization
•  Pareto Dominance
•  Mathematical methods
•  Heuristic methods
•  Numerical Examples
•  Questions
Introduction
•  Optimization:
▫  Refers to choosing the best element or alternative
from some set of available alternatives
▫  Minimize or maximize an objective function.
Introduction
•  Linear Programming
Min 3x1+4y
s.t.
x + 2y ≤ 14
3x +y ≥ 0
x-y≤2
Introduction
•  Some Optimization Problems
•  Development of a New Alloy:
▫  Strong and light
•  Develop an Engine:
▫  Power and economy
•  Investments
▫  Low risk and high profit
Multiple Objective Optimization
•  Optimize several objectives simultaneously
•  Mathematical model
Min , Max (f1(x), f2(x), .., fn(x))
x∈D
s.t. Constrains
Where:
x decision variable
n ≥ 2 number of objectives
D feasible region of solutions
Single VS Multiple Objective
Optimization
  Single optimization
Unique optimal solution
  Multiple Objective optimization
Pareto set of solutions
Min
f2(x)
constraint
improve
improve
Minf1(x)
Figure 1: Single objective
improve
Minf1(x)
Figure 2: Multiple objectives
7
Pareto Dominance
  Non Dominated Solutions
Min
f2(x)
f2(x)
1
.90
100
2
.80
130
3
.95
120
2
3
120
110
1
improve
solution f1(x)
130
improve .80
.90 .95
Minf
1(x)
8
Optimality of Solutions
Proximity & Diversity
Min
Min
f2(x)
f2(x)
True Pareto
front
True Pareto
front
L1
improve
L2
Minf
1(x)
Figure 3: Proximity
improve
improve
L3
improve
Min f1
(x)
Figure 4: Diversity
Methodology
•  Mathematical Methods
•  Heuristic Methods
Mathematical Methods
•  Goal Programming
Min d1+d2 +d3
Min 3x+4y -d1
=35
Max 5x -4y +d2 =100
x + 2y ≤ 14
3x +y ≥ 0
x-y≤2
3x+4y -d1= 35
5x- 4y +d2 = 100
x+y +d3 =20
Max x + y +d3 =20
Mathematical Methods
•  Weighted Sum Method
Min .60d1+.20d2 +.20d3
Min 3x+4y +d1
Max 5x -4y +d2
Max x + y +d3
60%
20%
20%
Mathematical Methods
•  Lexicographic Method
Min 3x+4y
Max 5x -4y
Max x + y
Min 3x+4y
Max
x +-4y
5x
y
3x+4y ≥ 35
5x- 4y ≤ 35
x+y ≥ 35
3x+4y=35
5x -4y =95
Heuristic Methods
•  Ant Colony Optimization
▫  Mimics how ants search for food.
  ants deposit pheromone on the ground in order to
mark some paths
  Find optimal path that should be followed by other
members of the colony.
Heuristic Methods
•  Ant Colony Optimization
▫  The first ant finds the food source (F ), then returns
to the nest (N), leaving behind a trail pheromone
(b)
▫  Ants indiscriminately follow several possible ways
▫  Ants take the shortest path, the one with highest
concentration of pheromones
Heuristic Methods
•  Particle swarm optimization
▫  It is mainly motivated by the social behavior of
organisms such as bird flocking and fish
schooling.
▫  The swarm optimization method is a population
method, that instead of fighting one against the
other, its concept is about mutual cooperation
Xi, Vi Xi, Vi Xi, Vi Xi, Vi Xi, Vi Xi, Vi •  Who’s the Global
Best now??
Xi, Vi Xi+1,Vi+1
•  What’s going to
happen in the
next iteration??
Xi+1,Vi+1
Global
Best
Xi, Vi Xi+1,Vi+1
Xi+1,Vi+1
Xi, Vi Xi, Vi Heuristic Methods
•  Genetic Algorithms
▫  Genetic algorithm (GA) introduces the principle of evolution
▫  The idea is to simulate the process in natural systems.
▫  The main principle of evolution used in GA
is “survival of the fittest”.
▫  The good solution survive, while bad ones die.
Heuristic Methods
•  Genetic Algorithms
Population
Selection
Parents
Reproduction
Replacement
offspring
Numerical Examples
• Redundancy allocation problem
• Data Survivability
• Reliability network design problem
Reliability Allocation Problem
•  The redundancy allocation problem is a practical
problem of determining the appropriate number of
redundant components that minimize the cost of the
system reliability under different resource
constraints
Subsystem
number of
components
PROBLEM DESCRIPTION (CONT’D...)
Reliability Allocation Problem
• Objective:
▫  How many redundancy parts
▫  Which supply
•  To optimize
▫  Total Cost
▫  Max reliability
Reliability Allocation Problem
Reliability Allocation Problem
PROBLEM
DESCRIPTION
Data
Survivability
Problem
  Survivability
Copy # 1
Copy #3
Copy # 2
  Data security
P1
P2
P3
27
Data Survivability Problem
•  Storage devices
PROBLEM
DESCRIPTION (CONT’D...)
• Probability of destruction
• Probability of Theft
• Cost
•  Encoding
Copy 1
Part 1
1
Part 2
3
Copy 3
Copy 2
Part 3
1
Part 1
Part 2
Part 3
Part 1
Part 2
Part 3
2
2
1
3
3
1
28
PROBLEM
DESCRIPTION
(CONT’D...)
Data Survivability Problem
• Objective:
▫  How many parts
▫  How many copies
▫  Which storage devise
•  To optimize
▫  Total Cost
▫  Prob. of destruction
▫  Prob. of theft
Data Survivability Problem
Network Reliability Design Problem
• Reliability
• Cost
• Weight
Network Reliability Design Problem
• 
Network configuration
x=(x12,..,x1n,x23,..,x2n,..,xij,..,xnn-1)
xij=0 The link is broken
xij=1 The link is established
x12
x13
x14
x23
x24
x34
0
0
1
1
0
1
2
1
4
3
31
31
PROBLEM DESCRIPTION (CONT’D...)
Network Reliability Design Problem
• Objective:
▫  Which links to enable
•  To optimize
▫  Min Total Cost
▫  Max reliability
▫  Min Weight
Reliability network design problem
Reliability network design problem
Post Pareto Optimality
•  K-means
•  Hierarchical clustering
•  Self organizing trees