Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Numerical Algorithms for
Optimization and Control of PDE’s Systems
Application to Fluid Dynamics
Surrogate models
Conclusion
R. Duvigneau
INRIA Sophia Antipolis-Méditerranée, OPALE Project-Team
KIT German - French Summer School, September 2010
1/79
Introduction
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Some examples and illustrations ...
Global optimization of the wing shape of a business aircraft
Local optimization of the shape of an airfoil
Identification of optimal parameters for flow control
KIT German - French Summer School, September 2010
2/79
Wing design
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Problem description (Piaggio-Aero P180)
Parameterization
Automated grid
generation
Drag minimization
Gradient evaluation
Lift constraint
Surrogate models
Global variables : span, root/tip
chord ratio, sweep, twist and
incidence angles
Conclusion
Local variables : airfoil section (10
variables)
Cruise regime M∞ = 0.83
Euler solver (Finite-Volume)
Mesh 56,512 nodes 311,820 cells
KIT German - French Summer School, September 2010
3/79
Wing design
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
0.014
Parameterization
Gradient evaluation
Surrogate models
Conclusion
0.012
drag coefficient
Automated grid
generation
0.01
0.008
0.006
0.004
0
20
40
60
80
number of iterations
→ Convergence in about 80 iterations ∼ 960 simulations
KIT German - French Summer School, September 2010
4/79
Wing design
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Comparison of initial (red) and final (blue) wing shapes
KIT German - French Summer School, September 2010
5/79
Wing design
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Comparison of initial (top) and final (bottom) pressure fields
KIT German - French Summer School, September 2010
6/79
Wing design
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Comparison of initial (top) and final (bottom) planforms
KIT German - French Summer School, September 2010
7/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Airfoil modification
Problem description
Navier-Stokes, k − ω turbulence modeling
Optimization
algorithms
Reynolds number Re = 19 106 , Mach number 0.68
Parameterization
Drag reduction
Automated grid
generation
Gradient evaluation
0.4
Surrogate models
NUWTUN
Experiment
1
0.2
0.5
0
-Cp
Conclusion
-0.2
-1
-0.4
-0.5
0
-0.5
0
0.5
1
1.5
-1.5
0
0.2
0.4
0.6
0.8
1
x/c
Computational mesh: 353 × 97 nodes
Comparison with experiments
Xcr
∆Yh
Xbr
Xbl
Definition of shape modification: four parameters
KIT German - French Summer School, September 2010
8/79
Airfoil modification
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Annotation = Number of CFD evaluations
48
52
56
60
64
Optimization
algorithms
0.86
68
72
76
80
Parameterization
96
0.82
14
0
14
4
14
8
15
2
15
6
16
0
16
4
16
8
13
6
0.8
12
4
12
8
13
2
Conclusion
0.84
10
0
10
4
10
8
11
2
11
6
12
0
Surrogate models
Objective function
Gradient evaluation
84
88
92
Automated grid
generation
0.78
0
5
10
15
Number of iterations
20
25
30
→ Convergence in about 30 iterations ∼ 150 simulations
KIT German - French Summer School, September 2010
9/79
Airfoil modification
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Comparison of initial (left) and final (right) pressure fields
0.05
RAE5243
Optimized
0
-0.05
0
0.2
0.4
0.6
0.8
1
Comparison of initial (black) and final (red) airfoil shapes
KIT German - French Summer School, September 2010
10/79
Flow control
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Problem description
Optimization
algorithms
Navier-Stokes modeling (laminar flow)
Parameterization
Reynolds number Re = 200
Automated grid
generation
Oscillatory rotating cylinder
Gradient evaluation
Objective : reduce the time-averaged drag
Surrogate models
Two control parameters : amplitude and frequency
Conclusion
Mesh: 70,000 nodes 330,000 cell
control system
KIT German - French Summer School, September 2010
11/79
Flow control
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Comparison of vorticity fields without (top) and with control (bottom)
KIT German - French Summer School, September 2010
12/79
Flow control
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1
Optimization
algorithms
Parameterization
0.95
Gradient evaluation
Surrogate models
Conclusion
cost function
Automated grid
generation
0.9
0.85
0.8
0
2
4
6
8
10
12
optimization iterations
→ Convergence in about 5 iterations ∼ 35 simulations
KIT German - French Summer School, September 2010
13/79
Flow control
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Surrogate model of drag coefficient
KIT German - French Summer School, September 2010
14/79
Introduction
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
How to carry out such an optimization process ?
Surrogate models
Conclusion
What are the tools involved ?
How are they organized ?
KIT German - French Summer School, September 2010
15/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
16/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
17/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
18/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
19/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
20/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
21/79
Design loop
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
22/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
3 Automated grid generation
Surrogate models
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
23/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
3 Automated grid generation
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
24/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Definition of an optimization problem
Minimize
Subject to
f (x)
gi (x) = 0
hj (x) > 0
x ∈ Rn
i = 1, · · · , l
j = 1, · · · , m
cost function
equality constraints
inequality constraints
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
25/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Definition of an optimization problem
Minimize
Subject to
f (x)
gi (x) = 0
hj (x) > 0
x ∈ Rn
i = 1, · · · , l
j = 1, · · · , m
cost function
equality constraints
inequality constraints
Automated grid
generation
Gradient evaluation
Role of the optimizer
Surrogate models
Provide a candidate design vector x ∈ Rn
Conclusion
KIT German - French Summer School, September 2010
25/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Definition of an optimization problem
Minimize
Subject to
f (x)
gi (x) = 0
hj (x) > 0
x ∈ Rn
i = 1, · · · , l
j = 1, · · · , m
cost function
equality constraints
inequality constraints
Automated grid
generation
Gradient evaluation
Role of the optimizer
Surrogate models
Provide a candidate design vector x ∈ Rn
Conclusion
Some approaches
Descent methods : suitable for smooth convex functions
steepest descent, conjugate gradient, quasi-Newton, Newton methods
Pattern search methods : suitable for noisy convex functions
Nelder-Mead simplex, Torczon’s multidirectional search algorithms
Evolutionary methods : suitable for multimodal functions
genetic algorithms, evolution strategies, particle swarm optimization, ant
colony methods, simulated annealing
KIT German - French Summer School, September 2010
25/79
Steepest descent
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Algorithm
For each iteration k:
Estimation of gradient ∇f (xk )
Define search direction dk = −∇f (xk )
Line search : find the step length ρ such as :
f (xk + ρdk ) < f (xk ) + α∇f (xk ) · ρdk (Armijo rule)
f (xk + ρdk ) > f (xk ) + β∇f (xk ) · ρdk (Goldstein rule)
Conclusion
KIT German - French Summer School, September 2010
26/79
Steepest descent
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Summary
Properties of steepest descent method:
Gradient evaluation
Proof of convergence to a local optimum
Surrogate models
Linear convergence rate :
Conclusion
limk→∞
kxk+1 − x ? k
=a >0
kxk − x ? k
In practice:
Unable to take into account function curvature
Oscillatory optimization path
KIT German - French Summer School, September 2010
27/79
Quasi-Newton methods
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Algorithm
Parameterization
Improvement of the search direction:
Automated grid
generation
Definition according to the function curvature:
Gradient evaluation
e k dk = −∇f (xk ),
H
Surrogate models
Conclusion
e approximation of the Hessian matrix (second-order derivative)
with H
Iterative construction of the Hessian matrix (BFGS formula) :
e 0 = Id
H
e k+1 = H
ek −
H
1
e k sk
skT H
e k sk s T H
eT +
H
k
k
1
yk ykT ,
ykT sk
with sk = xk+1 − xk and yk = ∇f (xk+1 ) − ∇f (xk )
KIT German - French Summer School, September 2010
28/79
Quasi-Newton methods
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Summary
Properties of quasi-Newton methods:
Parameterization
Automated grid
generation
Take into account the curvature of the past iteration:
e k+1 (xk+1 − xk ) = ∇f (xk+1 ) − ∇f (xk )
H
Gradient evaluation
Surrogate models
Conclusion
e k positive definite (if line search efficient enough)
H
Proof of convergence to local optimum
Super-linear convergence rate :
limk→∞
kxk+1 − x ? k
=0
kxk − x ? k
In practice:
Very efficient algorithm for convex problems (close to Newton method)
Gradient should be available
KIT German - French Summer School, September 2010
29/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
(λ, µ)−Evolution Strategy
Algorithm
For each iteration k, according to x̄k and σ̄k :
Generate a set of λ perturbation lengths σi = σ̄k e τ
N(0,1)
Generate a sample of λ individuals xi = x̄k + σi N(0, Id) (mutation)
where N(0, Id) multivariate normal distribution with 0 mean and Id
covariance matrix
Evaluate the performance of λ individuals
Surrogate models
Choose the best µ parents among λ individuals (selection)
Conclusion
Update distribution properties (crossover and self-adaption) :
x̄k+1 =
µ
1X i
x
µ i=1
σ̄k+1 =
µ
1X i
σ
µ i=1
KIT German - French Summer School, September 2010
30/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
(λ, µ)−Evolution Strategy
Summary
Properties of evolution strategies:
Proof of convergence to the global optimum in a statistical sense
e.g. ∀ > 0
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
limk→∞ P(|f (xk ) − f (x ? )| 6 ) = 1
linear convergence rate
In practice:
Able to avoid local optima
Curse of dimensionality
Low local convergence rate due to isotropic search
KIT German - French Summer School, September 2010
31/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Covariance Matrix Adaption (CMA-ES)
Algorithm
Improvement by using an anisotropic distribution:
Optimization
algorithms
Use of a covariance matrix Ck to generate the population:
Parameterization
xi = x̄k + σ̄k N(0, Ck )
Automated grid
generation
= x̄k + σ̄k Bk Dk N(0, Id)
Gradient evaluation
Surrogate models
Conclusion
1/2
with Bk matrix of eigenvectors of Ck
and Dk diagonal matrix of eigenvalues
Iterative construction of the covariance matrix :
C0 = Id
µ
c
1 X i i
ω (y )(y i )T
Ck+1 = (1 − c)Ck + pk pkT + c(1 − )
m
m i=1
| {z }
| {z }
previous estimate
|
{z
}
1D update
covariance of parents
with :
pk evolution path (moves performed during last iterations)
y i = (x i − x̄k )/σk
KIT German - French Summer School, September 2010
32/79
Covariance Matrix Adaption (CMA-ES)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Summary
Properties:
Ck is an approximation of the inverse of the Hessian matrix
Gradient evaluation
Surrogate models
Conclusion
x̄k + σ̄k N(0, Id)
x̄k + σ̄k N(0, D 2 )
x̄k + σ̄k N(0, BD 2 B T )
Several invariance properties:
Invariance under order-preserving transformations of the function
Invariance under scaling of the search space
Invariance under rotation of the search space
In practice:
Outperforms most evolutionary methods
Only a few parameters defined by the user
KIT German - French Summer School, September 2010
33/79
Analytical example
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Rosenbrock function
Non-convex function ”Banana valley”
Dimension n = 16
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
34/79
Analytical example
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
steepest-descent
KIT German - French Summer School, September 2010
quasi-Newton
35/79
Analytical example
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
ES
KIT German - French Summer School, September 2010
CMA-ES
36/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Analytical example
Camel back function
R. Duvigneau
Dimension n = 2
Optimization
algorithms
Six local optima
Parameterization
Two global optima
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
37/79
Analytical example
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Quasi-Newton
KIT German - French Summer School, September 2010
Optimization path
38/79
Analytical example
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
ES
KIT German - French Summer School, September 2010
Optimization path
39/79
Analytical example
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
CMA-ES
KIT German - French Summer School, September 2010
Optimization path
40/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
3 Automated grid generation
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
41/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Role of the parameterization tool
Parameterization
Automated grid
generation
Describe the system with a few variables
Gradient evaluation
Transform a given design vector to a new engineering system
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
42/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Role of the parameterization tool
Parameterization
Automated grid
generation
Describe the system with a few variables
Gradient evaluation
Transform a given design vector to a new engineering system
Surrogate models
Conclusion
A particular case: shape parameterization
Discrete shape : set of moving nodes
Parametric surface : Bézier, B-Splines, NURBS with control points
Composite approach : set of baseline components with parameters
Free-form deformation : deform a lattice that embeds the system
KIT German - French Summer School, September 2010
42/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Discrete shape representation
Principle
R. Duvigneau
Shape described by a surface grid
Optimization
algorithms
Each node can be moved independently
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
43/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Discrete shape representation
Principle
R. Duvigneau
Shape described by a surface grid
Optimization
algorithms
Each node can be moved independently
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Advantages
No strong assumption concerning the search space
No need for CAD software (CAD-free)
KIT German - French Summer School, September 2010
43/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Discrete shape representation
Principle
R. Duvigneau
Shape described by a surface grid
Optimization
algorithms
Each node can be moved independently
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Advantages
No strong assumption concerning the search space
No need for CAD software (CAD-free)
Drawbacks
Smoothing required at each optimization step
Very large number of design variables (∼ 10,000)
Optimized shape = surface grid
KIT German - French Summer School, September 2010
43/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Parametric shape representation
Principle
Shape described by a parametric surface: x(ξ) =
Pn
i=0
Ni (ξ)Xi
Choice of basis functions Ni :
Bézier : global influence, C ∞
B-Spline : compact support, C k−2 (k order)
NURBS : conic surfaces possible
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
44/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Parametric shape representation
Principle
Shape described by a parametric surface: x(ξ) =
Pn
i=0
Ni (ξ)Xi
Choice of basis functions Ni :
Bézier : global influence, C ∞
B-Spline : compact support, C k−2 (k order)
NURBS : conic surfaces possible
Gradient evaluation
Surrogate models
Conclusion
Advantages
Low number of design variables (∼ 10)
Optimal shape is parametric and smooth
Hierarchical representation
KIT German - French Summer School, September 2010
44/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Parametric shape representation
Principle
Shape described by a parametric surface: x(ξ) =
Pn
i=0
Ni (ξ)Xi
Choice of basis functions Ni :
Bézier : global influence, C ∞
B-Spline : compact support, C k−2 (k order)
NURBS : conic surfaces possible
Gradient evaluation
Surrogate models
Conclusion
Advantages
Low number of design variables (∼ 10)
Optimal shape is parametric and smooth
Hierarchical representation
Drawbacks
Mild assumption concerning the design space
KIT German - French Summer School, September 2010
44/79
Composite shape representation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Principle
Use a set of baseline components (edges, circles, etc) with parameters
(lengths, angles, positions)
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
45/79
Composite shape representation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Principle
Use a set of baseline components (edges, circles, etc) with parameters
(lengths, angles, positions)
Gradient evaluation
Surrogate models
Conclusion
Advantages
Optimal shape obtained from baseline components (manufacturing
constraint)
Low number of design variables
Variables chosen according to engineering experience
KIT German - French Summer School, September 2010
45/79
Composite shape representation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Principle
Use a set of baseline components (edges, circles, etc) with parameters
(lengths, angles, positions)
Gradient evaluation
Surrogate models
Conclusion
Advantages
Optimal shape obtained from baseline components (manufacturing
constraint)
Low number of design variables
Variables chosen according to engineering experience
Drawbacks
Strong assumption concerning the design space
KIT German - French Summer School, September 2010
45/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Free-Form Deformation (FFD)
Principle
Represent deformation instead of the shape
Embed the the system in a lattice, described as parametric volume
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
46/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Free-Form Deformation (FFD)
Principle
Represent deformation instead of the shape
Embed the the system in a lattice, described as parametric volume
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Advantages
Low number of design variables, whatever the system complexity
KIT German - French Summer School, September 2010
46/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Free-Form Deformation (FFD)
Principle
Represent deformation instead of the shape
Embed the the system in a lattice, described as parametric volume
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Advantages
Low number of design variables, whatever the system complexity
Drawbacks
Not easy to handle
No representation of the optimal shape
KIT German - French Summer School, September 2010
46/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
3 Automated grid generation
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
47/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Role of the grid generation tool
Optimization
algorithms
Parameterization
Automated grid
generation
Generate a grid in accordance with the parameterized geometry
Requirement: suitable for computations
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
48/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Role of the grid generation tool
Optimization
algorithms
Parameterization
Automated grid
generation
Generate a grid in accordance with the parameterized geometry
Requirement: suitable for computations
Gradient evaluation
Surrogate models
Conclusion
Difficulties
Automated task
Need for robustness (large variations of geometry)
Need for accuracy (mesh quality maintained)
KIT German - French Summer School, September 2010
48/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Role of the grid generation tool
Optimization
algorithms
Parameterization
Automated grid
generation
Generate a grid in accordance with the parameterized geometry
Requirement: suitable for computations
Gradient evaluation
Surrogate models
Conclusion
Difficulties
Automated task
Need for robustness (large variations of geometry)
Need for accuracy (mesh quality maintained)
Possible Approaches
Generate completely a new grid
Deform an existing reference grid
KIT German - French Summer School, September 2010
48/79
Complete mesh generation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Principle
Store all the steps of the mesh construction (usually in a script) with some
parameters as input (geometry)
Run the grid generation software automatically for each new geometry
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
49/79
Complete mesh generation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Principle
Store all the steps of the mesh construction (usually in a script) with some
parameters as input (geometry)
Run the grid generation software automatically for each new geometry
Surrogate models
Conclusion
Advantages
Very large geometry changes are possible
KIT German - French Summer School, September 2010
49/79
Complete mesh generation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Principle
Store all the steps of the mesh construction (usually in a script) with some
parameters as input (geometry)
Run the grid generation software automatically for each new geometry
Surrogate models
Conclusion
Advantages
Very large geometry changes are possible
Drawbacks
Regularity of the mesh w.r.t. geometry change ?
treatment of boundary layers tedious
KIT German - French Summer School, September 2010
49/79
Illustration
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
parameterization
KIT German - French Summer School, September 2010
initial mesh
50/79
Illustration
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
51/79
Reference mesh deformation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Principle
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Construct a reference mesh (initial geometry)
Move the nodes according to geometry change (same topology)
Need to solve an extra problem for the displacement field (e.g. structural
analogy)
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
52/79
Reference mesh deformation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Principle
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Construct a reference mesh (initial geometry)
Move the nodes according to geometry change (same topology)
Need to solve an extra problem for the displacement field (e.g. structural
analogy)
Surrogate models
Conclusion
Advantages
Control of the grid quality
Smoothness of the deformation
KIT German - French Summer School, September 2010
52/79
Reference mesh deformation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Principle
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Construct a reference mesh (initial geometry)
Move the nodes according to geometry change (same topology)
Need to solve an extra problem for the displacement field (e.g. structural
analogy)
Surrogate models
Conclusion
Advantages
Control of the grid quality
Smoothness of the deformation
Drawbacks
Moderate geometry change
KIT German - French Summer School, September 2010
52/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Example: discrete mechanical analogy
Method
Consider all edges as lineal springs and all angles as torsional springs
Solve a linear system representing the mechanical equilibrium:
„ «
0
(Klin + Ktors |Id) q =
q
with:
K stiffness matrices
q displacement
q imposed boundary displacement
C ijk
k
kik
C ijk
i
k
i
kij
kjk
j
C ijk
j
KIT German - French Summer School, September 2010
53/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Example: discrete mechanical analogy
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Initial grid
Deformed grid
KIT German - French Summer School, September 2010
54/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Example: use of Free-Form Deformation
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Initial grid
Deformed grid
KIT German - French Summer School, September 2010
55/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
3 Automated grid generation
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
56/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Role of sensitivity analysis
Compute the gradient of the cost function (e.g. drag) with respect to
design variables
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
57/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Role of sensitivity analysis
Compute the gradient of the cost function (e.g. drag) with respect to
design variables
Surrogate models
Conclusion
Possible Approaches
Finite-difference approximation
Complex estimation
Adjoint approach
KIT German - French Summer School, September 2010
57/79
Finite-difference approximation of the gradient
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Method
Optimization
algorithms
Perform n + 1 or 2n simulations for perturbed design variable values x ± ei
Parameterization
Compute approximated gradient:
Automated grid
generation
Gradient evaluation
∇f (x)|i '
f (x + ei ) − f (x)
∇f (x)|i '
f (x + ei ) − f (x − ei )
2
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
58/79
Finite-difference approximation of the gradient
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Method
Optimization
algorithms
Perform n + 1 or 2n simulations for perturbed design variable values x ± ei
Parameterization
Compute approximated gradient:
Automated grid
generation
Gradient evaluation
∇f (x)|i '
f (x + ei ) − f (x)
∇f (x)|i '
f (x + ei ) − f (x − ei )
2
Surrogate models
Conclusion
Drawbacks
Choice of perturbation coefficient Low accuracy → noisy cost function
Very expensive for large n
KIT German - French Summer School, September 2010
58/79
Finite-difference approximation of the gradient
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Method
Optimization
algorithms
Perform n + 1 or 2n simulations for perturbed design variable values x ± ei
Parameterization
Compute approximated gradient:
Automated grid
generation
Gradient evaluation
∇f (x)|i '
f (x + ei ) − f (x)
∇f (x)|i '
f (x + ei ) − f (x − ei )
2
Surrogate models
Conclusion
Drawbacks
Choice of perturbation coefficient Low accuracy → noisy cost function
Very expensive for large n
Advantages
Non-intrusive approach
KIT German - French Summer School, September 2010
58/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Complex estimation
Method
Move all real variables x of the code to complex variables: z = x + iy
The cost function (output) is also complex: f = u + iv
∂u
∂x
∂v
∂y
'
v (x+i(y +h))−v (x+iy )
h
Parameterization
The Cauchy-Riemann condition :
Automated grid
generation
But values are real : y = 0 v (x) = 0 and u = f
Gradient evaluation
Then, the following approximation can be used:
Surrogate models
Conclusion
=
∇f (x) '
Im[f (x + ih)]
h
Propagate imaginary part of input variable to estimate gradient
KIT German - French Summer School, September 2010
59/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Complex estimation
Method
Move all real variables x of the code to complex variables: z = x + iy
The cost function (output) is also complex: f = u + iv
∂u
∂x
∂v
∂y
'
v (x+i(y +h))−v (x+iy )
h
Parameterization
The Cauchy-Riemann condition :
Automated grid
generation
But values are real : y = 0 v (x) = 0 and u = f
Gradient evaluation
=
Then, the following approximation can be used:
Surrogate models
∇f (x) '
Conclusion
Im[f (x + ih)]
h
Propagate imaginary part of input variable to estimate gradient
Advantages
Accurate gradient estimation (no difference)
KIT German - French Summer School, September 2010
59/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Complex estimation
Method
Move all real variables x of the code to complex variables: z = x + iy
The cost function (output) is also complex: f = u + iv
∂u
∂x
∂v
∂y
'
v (x+i(y +h))−v (x+iy )
h
Parameterization
The Cauchy-Riemann condition :
Automated grid
generation
But values are real : y = 0 v (x) = 0 and u = f
Gradient evaluation
=
Then, the following approximation can be used:
Surrogate models
∇f (x) '
Conclusion
Im[f (x + ih)]
h
Propagate imaginary part of input variable to estimate gradient
Advantages
Accurate gradient estimation (no difference)
Drawbacks
Move CFD code to complex !
n simulations required
KIT German - French Summer School, September 2010
59/79
Adjoint approach
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Method
Let consider the cost function as a constrained functional:
Parameterization
Automated grid
generation
Gradient evaluation
f : x 7→ f (x) = F (x, W ) ∈ R
with W = W (x) ∈
RN
is the solution of the state equation:
Ψ(x, W ) = 0
Surrogate models
Conclusion
Apply chain rule:
df
∂F
∂F dW
=
+
dxi
∂xi
∂W dxi
∂Ψ dW
∂Ψ
+
=0
∂xi
∂W dxi
by combining equations:
df
∂F
∂F
=
−
dxi
∂xi
∂W
„
∂Ψ
∂W
«−1
KIT German - French Summer School, September 2010
∂Ψ
∂xi
60/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Adjoint approach
Method
Finally, gradient can be computed by solving the (linear) system first:
Optimization
algorithms
„
Parameterization
Automated grid
generation
∂Ψ
∂W
«T
„
Π=
∂F
∂W
«T
where Π are the adjoint variables, and then by computing:
Gradient evaluation
Surrogate models
Conclusion
„
df
dx
«T
„
=
∂F
∂x
«T
„
−
∂Ψ
∂x
KIT German - French Summer School, September 2010
«T
Π
61/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Adjoint approach
Method
Finally, gradient can be computed by solving the (linear) system first:
Optimization
algorithms
„
Parameterization
Automated grid
generation
∂Ψ
∂W
«T
„
Π=
∂F
∂W
«T
where Π are the adjoint variables, and then by computing:
Gradient evaluation
„
Surrogate models
Conclusion
df
dx
«T
„
=
∂F
∂x
«T
„
−
∂Ψ
∂x
«T
Π
Advantages
Adjoint system do not depend on x
Cost rather independent from the size of x
KIT German - French Summer School, September 2010
61/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Adjoint approach
Method
Finally, gradient can be computed by solving the (linear) system first:
Optimization
algorithms
„
Parameterization
Automated grid
generation
∂Ψ
∂W
«T
„
Π=
∂F
∂W
«T
where Π are the adjoint variables, and then by computing:
Gradient evaluation
„
Surrogate models
Conclusion
df
dx
«T
„
=
∂F
∂x
«T
„
−
∂Ψ
∂x
«T
Π
Advantages
Adjoint system do not depend on x
Cost rather independent from the size of x
Drawbacks
The adjoint system has to be constructed (intrusive approach)
Inversion of adjoint system difficult (badly conditionned)
KIT German - French Summer School, September 2010
61/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
3 Automated grid generation
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
62/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Some reasons to use surrogate models
Cost function evaluations are very expensive
Some results are just discarded (in particular with ES)
Weak use of past results (iterative process without history)
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
63/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Some reasons to use surrogate models
Cost function evaluations are very expensive
Some results are just discarded (in particular with ES)
Weak use of past results (iterative process without history)
Gradient evaluation
Surrogate models
Principles
Conclusion
Store all cost function values into a database
Use the database to build a surrogate model
Use the surrogate model as cheap and approximate evaluation
KIT German - French Summer School, September 2010
63/79
Presentation
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Some reasons to use surrogate models
Cost function evaluations are very expensive
Some results are just discarded (in particular with ES)
Weak use of past results (iterative process without history)
Gradient evaluation
Surrogate models
Principles
Conclusion
Store all cost function values into a database
Use the database to build a surrogate model
Use the surrogate model as cheap and approximate evaluation
Some possible surrogate models
Radial basis functions (RBFs)
Artificial Neural Networks (ANNs)
Gaussian processes (kriging)
KIT German - French Summer School, September 2010
63/79
Radial Basis Functions (RBF)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Evaluation
Approximation of the function J (x), x ∈ <n of the form:
Automated grid
generation
Gradient evaluation
f (x) =
Surrogate models
Nc
X
ωj φj (x)
(1)
j=1
Conclusion
where φj are radial functions:
2
φj (x) = Φ(kx − xj k)
(xj )j=1,...,Nc
s
(ωj )j=1,...,Nc
Φ(r ) = e
− r2
s
(2)
points stored in the database
attenuation factor
weights adjusted to fit the data
KIT German - French Summer School, September 2010
64/79
Radial Basis Functions (RBF)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Training
(ωj )j=1,...,Nc are determined from interpolation conditions:
f (xi ) =
Nc
X
ωj φj (xi )
i = 1, . . . , Nc
(3)
j=1
Conclusion
(ωj )j=1,...,Nc is the solution of the linear system:
9 8
9
0
18
φ1 (x1 )
...
φNc (x1 )
ω > >
J (x1 ) >
>
>
>
< 1>
= >
<
=
B φ1 (x2 )
C
...
φNc (x2 ) C
ω2
J (x2 )
B
=
@ ...
...
... A >
.
.
.
.
.
.
>
>
>
>
>
>
:
; >
:
;
φ1 (xNc ) . . . φNc (xNc )
ωNc
J (xNc )
(4)
s set by the user or optimized by internal algorithm
KIT German - French Summer School, September 2010
65/79
Gaussian processes (Kriging)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Principle
The vector of known function values FN is assumed to be one realization of
a Gaussian process:
“
”
exp − 12 FN> CN−1 FN
p(FN ) = p
(2π)N det(CN )
Conclusion
with a given covariance matrix Cmn = c(xm , xn ).
It can be shown that (conditional probabilities):
"
#
(fN+1 − fˆN+1 )2
p(fN+1 |FN ) ∝ exp −
2σf2
N+1
where:
fˆN+1 = k > CN−1 FN ,
σf2N+1 = κ − k > CN−1 k
KIT German - French Summer School, September 2010
66/79
Illustration for 1D problem
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
2.0
Parameterization
Automated grid
generation
1.5
Training data
Exact
Evaluated
Lower bound
Upper bound
Gradient evaluation
Surrogate models
1.0
f
Conclusion
0.5
0.0
−0.5
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
x
KIT German - French Summer School, September 2010
67/79
Gaussian Processes (Kriging)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Choice of the covariance function
Distance dependent correlation, scaling, offset
"
Gradient evaluation
c(x, y ) = θ1 exp −
Surrogate models
d
1 X (xi − yi )2
2 i=1
ri2
#
+ θ2 ,
Conclusion
Parameters to be optimized Θ = (θ1 , θ2 , r1 , r2 , . . . , rd )
Choice of the parameters (training phase)
Choose Θ to maximize the likelihood of the known function values
Internal optimization
KIT German - French Summer School, September 2010
68/79
Application to Inexact Pre-Evaluation (IPE)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Principle
For evolutionary optimizers several evaluations are just not used
Use surrogate models as pre-screening criterion for CFD evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
69/79
Application to Inexact Pre-Evaluation (IPE)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Principle
For evolutionary optimizers several evaluations are just not used
Use surrogate models as pre-screening criterion for CFD evaluation
Surrogate models
Conclusion
Expected benefits
Avoid useless evaluations for evolutionary optimizers
Reduce drastically the cost of evolutionary optmizers
Low coupling between optimizer and metamodel
KIT German - French Summer School, September 2010
69/79
Inexact Pre-Evaluation (IPE) approach
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
1
2
Gradient evaluation
Surrogate models
Conclusion
3
Set n = 0
If n ≤ Ne compute cost function f (xkn ), k = 1, . . . , K using the exact
model, else using metamodel f˜(xkn ), k = 1, . . . , K .
If n > Ne , then select a subset of points S n for exact evaluation.
4
Update the optimizer parameters (mean, standard deviation) using only the
exactly evaluated cost functions
5
Store exactly evaluated function values into a database
6
If n < Nmax , then n = n + 1 and go to step (iii), else STOP.
KIT German - French Summer School, September 2010
70/79
Illustration for business jet
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
CFD evaluations for the 10% best points or points with predicted
improvement
500 iterations
RBF model with local database (40 pts)
Number of exact evaluations
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
71/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
Illustration for business jet
Cost function
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Final shapes
KIT German - French Summer School, September 2010
72/79
Application to Efficient Global Optimization (EGO)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Main ideas
Use a metamodel to drive the search
Use a Gaussian Process model to take into account the probability of
obtaining a better design
Conclusion
KIT German - French Summer School, September 2010
73/79
Application to Efficient Global Optimization (EGO)
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Main ideas
Use a metamodel to drive the search
Use a Gaussian Process model to take into account the probability of
obtaining a better design
Conclusion
Expected benefits
Global optimization (proof of convergence)
Deterministic approach
Measure of the probability of improvement
KIT German - French Summer School, September 2010
73/79
EGO algorithm
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Basics
Build iteratively a database and corresponding Gaussian process
Enrichement chosen in order to:
Minimize the current Gaussian process model
Explore where the probability of improvement is high
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
74/79
EGO algorithm
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Basics
Optimization
algorithms
Build iteratively a database and corresponding Gaussian process
Enrichement chosen in order to:
Parameterization
Automated grid
generation
Minimize the current Gaussian process model
Explore where the probability of improvement is high
Gradient evaluation
Surrogate models
Conclusion
Algorithm
1
Build an a priori database (Latin Hypercube Sampling)
2
Construct a global Gaussian process
Find the points xi? that minimize / maximize a merit function :
3
Statistical lower bound
Probability of improvement
Expected improvement
4
Evaluate the p points (xi? )i=1,...,p and add them in the database
5
Return to step 2 until convergence
KIT German - French Summer School, September 2010
74/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Illustration for business jet
EGO algorithm with three merit functions
150 iterations
Initial database size : 60 points
Cost function
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
75/79
Comparison IPE - EGO
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Cost function
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
76/79
Use of surrogate models
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Main ideas
Use all evaluations already performed
Use expensive simulation only if required
Gradient evaluation
Surrogate models
Conclusion
KIT German - French Summer School, September 2010
77/79
Use of surrogate models
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Main ideas
Use all evaluations already performed
Use expensive simulation only if required
Gradient evaluation
Surrogate models
Advantages
Conclusion
Drastic reduction of CPU cost
Interesting for post-processing / interactive study
KIT German - French Summer School, September 2010
77/79
Use of surrogate models
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Main ideas
Use all evaluations already performed
Use expensive simulation only if required
Gradient evaluation
Surrogate models
Advantages
Conclusion
Drastic reduction of CPU cost
Interesting for post-processing / interactive study
Drawbacks
More sophisticated approach
Restricted to problems of low dimension
KIT German - French Summer School, September 2010
77/79
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
1 Optimization algorithms
Optimization
algorithms
Parameterization
2 Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
3 Automated grid generation
Conclusion
4 Gradient evaluation
5 Surrogate models
6 Conclusion
KIT German - French Summer School, September 2010
78/79
Conclusion
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Conclusion
Optimization and control in CFD:
A multi-disciplinary field:
Applied mathematics
Numerical methods
Physical phenomena
Geometry
Computer sciences
Efficiency of methods are strongly problem dependent
Efficiency depends strongly of the global coherency of the loop
KIT German - French Summer School, September 2010
79/79
Conclusion
Numerical
Algorithms for
Optimization and
Control of PDE’s
Systems
R. Duvigneau
Optimization
algorithms
Parameterization
Automated grid
generation
Gradient evaluation
Surrogate models
Several topics have not been discussed:
Multi-objective optimization: How to minimise several criteria ?
Multi-disciplinary optimization: How to couple several disciplines ?
Constrained optimization: How to take into account constraints ?
Conclusion
Robust optimization: How to take into account uncertainties ?
Hierarchical optimization: How to develop multi-level strategies ?
Distributed optimization: How to use parallel computing ?
Automatic differentiation: How to use AD softwares ?
...
KIT German - French Summer School, September 2010
80/79