Optimization of thermal processes
Lecture 3
Maciej Marek
Czestochowa University of Technology
Institute of Thermal Machinery
Optimization of thermal processes
2007/2008
Overview of the lecture
• Multivariable optimization with no constraints
• Multivariable objective function
− Extreme points
− Necessary and sufficient conditions for extreme points
• Differential calculus methods
• Multivariable optimization with equality constraints
− Solution by direct substitution
− Lagrange multipliers method
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization problem
Find
 x1 
x 
X   2
 
 
 xn 
which minimizes f ( X)
What about constraints?
•The constraints are not significant in some of the problems
•It is instructive to study unconstrained problems first
•There are powerful methods for constrained optimization problems that use
unconstrained minimization techniques
Optimization of thermal processes
2007/2008
Multivariable objective function
x2
10
f ( x1 , x2 )
f ( X)
5
0
x1
x2
-5
minimum
Surface plot
-10
-10
x1
-5
0
5
10
Contour plot
f ( X)  40
Objective function surfaces
f ( X)  100
f ( X)  C
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization
(differential calculus methods)
Necessary condition
f
f
f
*
*
X

X

...

X*   0





x1
x2
xn
X*
Stationary point
Just as in the case of single-variable function, this condition
is not
sufficient:
Saddle point
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization
(differential calculus methods)
To formulate sufficient condition we have to introduce matrix H:
The Hess matrix
The Hessian
 2 f

 x1x1
 2 f

H   x2 x1


 2 f

 xn x1
2 f
x1x2
2 f
x2x2
2 f
xn x2
2 f 

x1xn 
2 f 

x2xn 


2
 f 

xn xn 
Sufficient condition for minimum at the extreme point X*:
If the Hessian is positive definite, then X* is minimum.
What does it mean?
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization
(differential calculus methods)
The matrix H is positive definite when:
Q  hT Hh
  f

 x1x1
 2 f

H   x2 x1


 2 f

 xn x1
2
 f
x1x2
2
2 f
x2x2
2 f
xn x2
X X
*
 f 

x1xn 
2 f 

x2xn 


2
 f 

xn xn 
0
for every non-zero h
2
Optimization of thermal processes
2 f
H1 
x1x1
H2 
2 f
x1x1
2 f
x1x2
 f
x2 x1
 f
x2 x2
2
2 f
x1x1
2 f
x1x2
2 f
x1xn
2 f
H n  x2 x1
2 f
x2x2
2 f
x2xn
2 f
xn x1
2 f
xn x2
2 f
xn xn
2007/2008
2
...
Unconstrained multivariable optimization
(differential calculus methods)
• With the help of determinants H1, H2, ..., Hn we can formulate
the sufficient condition in a more convenient way:
− If all the values H1, H2, ..., Hn are positive, then the Hessian is
positive definite and the extreme point is minimum
j
− If the sign of Hj is (1) for j=1,2,...,n, then the Hessian is negative
definite and the extreme point is maximum
• For instance in the case of two variables (suppose all the
derivatives are evaluated at the extreme point X*):
2 f
2 f
H1 
 2 0
x1x1 x1
H2 
2 f
x1x1
2 f
x1x2
2 f
x2 x1
2 f
x2 x2
2 f 2 f
2 f 2 f



x1x1 x2 x2 x2 x1 x1x2
2
2 f 2 f  2 f 
 2

 0
x1 x22  x1x2 
Relative minimum at X*
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization
(differential calculus methods)
But if:
H1 
 f
 f
 2 0
x1x1 x1
2
2
H2 
2 f
x1x1
2 f
x1x2
2 f
x2 x1
2 f
x2 x2
2
2 f 2 f  2 f 
 2

 0
x1 x22  x1x2 
then there is relative maximum at X*
Note, that H2>0 in both of the cases. If, on the other hand:
2
2 f 2 f  2 f 
H2  2

 0
2
x1 x2  x1x2 
then there is a saddle point at X* (H is indefinite).
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization
(differential calculus methods)
EXAMPLE
Find the extreme points of the function:
f ( x1 , x2 )  x13  x23  2 x12  4 x22  6
f
 x1 (3x1  4)  0,
x1
f
 x2 (3 x2  8)  0
x2
(0, 0), (0, -8 / 3), (-4/3,0), (-4/3,-8/3)
2 f
 6 x1  4,
2
x1
2 f
 6 x2  8
2
x2
2 f
0
x1x2
Necessary condition
Stationary points
0 
 6 x1  4
H

0
6
x

8

2

Hessian
What is the nature of the extreme points?
Optimization of thermal processes
2007/2008
Unconstrained multivariable optimization
(differential calculus methods)
EXAMPLE contd
H1  6 x1  4
Point X
H2 
6 x1  4
0
0
6 x2  8
H1
H2
Nature of H
(0,0)
+4
+32
Positive definite
Relative minimum
(0,-8/3)
+4
-32
Indefinite
Saddle point
(-4/3,0)
-4
-32
Indefinite
Saddle point
(-4/3,-8/3)
-4
+32
Negative definite
Relative maximum
Optimization of thermal processes
Nature of X
2007/2008
Constrained multivariable optimization problem
(equality constraints)
 x1 
x 
Find X   2 
 
 
 xn 
subject to
which minimizes
g j ( X)  0,
f ( X)
j  1, 2,..., m
Equality constraints
Number of independent variables (degrees of freedom):
So there should be:
nm
mn
Otherwise, the problem is overdefined (no solution, in general)
Optimization of thermal processes
2007/2008
Constrained multivariable optimization problem
(equality constraints)
x2
10
g1 ( x1 , x2 )  0
5
Minimum with constraint g1
Minimum point with no
constraints
0
x1
g2 ( x1 , x2 )  0
-5
Minimum with constraint g2
-10
-10
-5
0
5
10
With both constraints there is no solution!
Optimization of thermal processes
2007/2008
Constrained multivariable optimization problem
(equality constraints)
EXAMPLE
Find the dimensions of a box of largest volume that can be inscribed
in a sphere of unit radius.
x3
( x1 , x2 , x3 )
x12  x22  x32  1
2x3
x2
2x1
2x2
x1
Optimization of thermal processes
V  f ( x1 , x2 , x3 )  8x1 x2 x3
2007/2008
Constrained multivariable optimization problem
(equality constraints)
EXAMPLE contd
f ( x1 , x2 , x3 )  8 x1 x2 x3
Objective function
x12  x22  x32  1
Constraint
We can transform this problem into unconstrained optimization problem.
x3  1  x12  x22
So, substituting in f:
f ( x1 , x2 )  8x1 x2 1  x12  x22
We need only to maximize f (use classical differential calculus method).
Homework: find the extreme point and make
sure it is maximum!
Optimization of thermal processes
2007/2008
Constrained multivariable optimization problem
(equality constraints)
This is the idea of
Method of Direct Substitution.
• Suppose we have n variables and m equality constraints
• Then there are n-m independent variables
• Choose a set of m variables and express them in terms of
independent variables
• The new objective function involves only n-m variables and is not
subjected to any constraints.
Voilà!
But it is not always that simple. For many nonlinear constraints it is
impossible to express any m variables in terms of the remaining ones.
Optimization of thermal processes
2007/2008
Constrained multivariable optimization problem
(equality constraints)
Method of Lagrange Multipliers is more general.
Here are the basic features for the problem with two variables and one
constraint.
Minimize
f ( x1 , x2 )
subject to
g ( x1 , x2 )  0
• Construct a function L (Lagrange function) as
L( x1 , x2 ,  )  f ( x1 , x2 )   g ( x1 , x2 )
Lagrange multiplier
• Necessary conditions for the extremum are:
L L L


0
x1 x2 
Optimization of thermal processes
2007/2008
Constrained multivariable optimization problem
(equality constraints)
EXAMPLE
Minimize
subject to
f ( x, y )  kx 1 y 2
k, a - constants
g ( x, y)  x 2  y 2  a 2  0
L( x, y,  )  kx 1 y 2   ( x 2  y 2  a 2 )
L
 kx 2 y 2  2 x  0
x
L
 2kx 1 y 3  2 y  0
y
L
 x2  y 2  a2  0

Optimization of thermal processes
2 
Lagrange function
k
2k

x3 y 2 xy 4
a
x 
3
*
2007/2008
x* 
a
y  2
3
*
1 *
y
2
Thank you for your attention
Optimization of thermal processes
2007/2008