Numerical optimization methods
Gintarė Viščiūtė
Department of Applied Mathematics
Classification
 Analytical methods
▫ classical methods
▫ mathematical programming
 liner programing
 nonliner programing
▫ game theory
 Heuristic methods
Optimization problem
Generic minimization problem
f ( x*)  min f ( x)
xX
where
•X is the search space
• f(x) objective function
• the value f(x*) is the minimum
,
,
Local vs. global minimum
 Local minimum f ( x*)  f ( x)
x  x*  
 Global minimum f ( x*)  f ( x) x  X
Local minimum
Global minimum
x X
Optimality conditions
• Necessary conditions for optimality
f ( x* )  0
• Sufficient conditions for optimality
1. Gradient of the function is zero
2. Hessian matrix - positive definite
Optimization methods without
limitations
•
•
•
•
•
Gradient method
Newton method
Reliable field methods
Variable metric methods
Conjugate direction methods
Nonlinear programming
h1 ( x1 ,..., xn )  0,
n
f ( x)   c j x j  min, then
j 1
h2 ( x1 ,..., xn )  0,
hl ( x1 ,..., xn )  0,
g1 ( x1 ,..., xn )  0,
g m ( x1 ,..., xn )  0.
•
•
•
•
Fines and barrier methods
Methods of allowable directions
Gradient projection and reduction methods
Lagrange function method
Linear programming
a11 x1  a12 x2 
a1n xn  b1 ,
a21 x1  a22 x2 
a2 n xn  b2 ,
ar1 x1  ar 2 x2 
arn xn  br ,
n
f ( x)   c j x j  min, then
j 1
ar 11 x1  ar 12 x2 
am1 x1  am 2 x2 
x1  0, x2  0,
•
•
•
•
Geometric interpretation
Simplex method
Ellipsoid method
Interior-point methods
ar 1n xn  br 1 ,
amn xn  bm ,
, xs  0.
Mathematical programming
applications
•
•
•
•
•
•
•
•
Production Planning
Agriculture
Distribution of resources
Transport Planning
Schedules
Economic, Finance
Human behavior modeling
Chemical Technology
Conclusion
• There is no universal optimization method for
resolving any task effectively.
• Initial value selection has a significant influence for
the minimum value of the found function.
• Optimization techniques and its software is being
actively developed branch of science.
• Recently, the genetic algorithms are especially
popular in optimization problems.
Thank you for your attention
• Define local and global minimum.
• Necessary and sufficient conditions for a minimum.
• Differences between linear and nonlinear
programming.
• Formulate the optimization problem of your own
practice.
• Philosophical task. Who is better for optimization: a
computer or a human?