Block 4 Nonlinear Systems
Lesson 15 – Nonlinear Models
(Classical Optimization)
Is It Not the Best
of All Possible
Worlds?
Charles Ebeling
University of Dayton
An engineer who
forgot to optimize
1
The Goal of this Lesson
"It is demonstrable," said he, "that things cannot be otherwise
than as they are; for as all things have been created for
some end, they must necessarily be created for the best end.”
Candide by Voltaire
Goal: To make this “best of all possible nonlinear
worlds” - a little better!
Right on.
2
The Optimization Problem
Max/Min f ( x1 , x2 ,..., xn )
subj to :
мЈ ь
п п
gi ( x1 , x2 ,..., xn ) н = э bi , i = 1, 2,..., m
пі п
о ю
where f, g1, …,gm are real-valued functions
3
The Single Variable Problem
open interval:
Max / Min f ( x)
where    x  
closed interval:
Max / Min f ( x) where    a  x  b  
4
The Difficulty
unbounded
global
max
local
max
f(x)
global
min
local
min
a
b
x
closed interval
5
Local Minimum
local min: x’ is a local minimum (maximum) if for an arbitrary
small neighborhood, N, about x’, f(x’)  () f(x) for all x in N.
f(x)
x’
N
x’
N
x
6
Global Minimum
global min: x* is a global minimum if
f(x*)  f(x) for all x such that a  x  b.
f(x)
x*
a
x
b
7
Global Maximum
global max: x* is a global maximum if
f(x*)  f(x) for all x such that a  x  b.
f(x)
x*
a
x
a
b
8
stationary
point
Inflection pt
+
stationary
point
-
f(x)
+
x
d f ( x)
dx
x
d 2 f ( x)
dx 2
x
concave
convex
9
Our very first nonlinear
optimization problem
y  f ( x)  2 x  x
2
2
4
dy
y' 
 4 x  4 x3  0
dx
4 x 1  x 2   0; x  0,  1
2
d y
2

f
''(
x
)

4

12
x
dx 2
f ''(0)  4  0 minimum
1
-3
-2
-1
0
-1 0
1
2
3
-2
-3
-4
-5
-6
-7
-8
-9
f ''(1)  8  0 maximum
d2y
2

f
''(
x
)

4

12
x
0
2
dx
12 x 2  4
f ''(1)  8  0 maximum
x 2  1/ 3; x  .57735
10
Global Minimum – Convex
Functions
If f(x) is a convex function if and only if
d 2 f ( x)
 0; for    x  
2
dx
df ( x*)
then if
 0, x * is the global minimum
dx
f(x)
x
11
Global Maximum – Concave
Functions
If f(x) is a concave function if and only if
d 2 f ( x)
 0; for    x  
2
dx
df ( x*)
then if
 0, x * is the global maximum
dx
f(x)
x
12
An Unbounded Function
f ( x)  100 ln( x)
100
100
f '( x) 
; f ''( x)   2  0 therefore concave
x
x
200
150
100
50
0
-50 0
1
2
3
4
5
-100
-150
-200
-250
13
The Single Variable Problem
on the Open Interval
necessary condition for global solution:
d f ( x)
f(x) is bounded and
0
dx
sufficient condition:
for all x:
d 2 f ( x)
 0 for a min (convex)
2
dx
d 2 f ( x)
 0 for a max (concave)
2
dx
14
A Bounded Example
f ( x)  100 ln( x)  2 x 2 ; x  0
100
f '( x) 
 4 x  0; 4 x 2  100; x 2  25; x  5
x
100
f ''( x)   2  4  0
concave function
x
150
100
50
0
-50 0
5
10
15
20
-100
-150
-200
-250
-300
15
Our very first word problem
A pipeline from the port in NYC to St. Louis, a distance of 1000
miles, is to be constructed by the Leak E. Oil Company with
automatic shutoff values installed every x miles in the event of a
leak. Environmentalists have estimated that such a pipeline is
likely to have two major leaks during its lifetime. The cost of a
valve is $500 and the cost of a cleanup in the event of a leak is
$2500 per pipeline mile of oil spilled. How far apart should the
valves be placed?
f(x) = 2 (2500) x + 500 (1000) / x
0  x  1000
16
Our very first word problem
(continued)
f(x) = 2 (2500) x + 500 (1000) / x
d f ( x)
500, 000
 f '( x)  5000 
0
2
dx
x
500, 000
2
x 
 100
500
x*  10 miles
d 2 f ( x)
2(500, 000)
 f ''( x) 
 0 for x  0
2
3
dx
x
therefore f(x) is convex and x* is a global minimum
17
The Single Variable Problem
on the Closed Interval
Max f ( x) where    a  x  b  
define a stationary point as any point x’ such that
find
f '( x ')  0
max  f (a), f ( x1' ), f ( x2' ),..., f ( xk' ), f (b)
x
This looks too
easy. There must
be more to it.
18
Our very next example
problem
1 4
f ( x)  x  3x 3  13x 2  24 x  20.1 ; 1  x  6
4
f '( x)  x3  9 x 2  26 x  24  0
I bet that can
be factored!
( x  2) ( x  3) ( x  4)  0
x  2,3, 4
19
Our very next example problem
(continued)
1 4
f ( x)  x  3x3  13x 2  24 x  20.1
4
f '( x)  x3  9 x 2  26 x  24  0
( x  2) ( x  3) ( x  4)  0
f ''( x)  3x  18 x  26
2
x
1
2
3
4
6
f(x)
6.25
4
4.25
4
20
x  2,3, 4
f”(x)
2
-1
2
local/global min
local max
local/global min
global max
20
Another example
For a particular government 12-year health
care program for the elderly, the number of
people in thousands receiving direct benefits
as a function of the number of years, t, after
the start of the program is given by
My health
benefits
will expire
soon!
t3
n   6t 2  32t 0  t  12
3
For what value of t does the maximum
number receive benefits?
21
The Answer
t3
n   6t 2  32t
3
dn 2
 t  12t  32  0
dt
 t  4  t  8  0; t  4,8
d 2n
 2t  12;
2
dt
d 2n
 4  0
2
dt t  4
d 2n
40
2
dt t 8
t = 0 (n = 0),
t= 4 (n = 53/3)
t = 8 (n = 42.67)
t = 12 (n = 96)
n
local max
local min
f
22
Multi -Variable Optimization
i.e. going from one to two
23
2-Variable Function with a
Maximum
z = f(x,y)
24
2-Variable Function with both
Maxima and Minima
z = f(x,y)
25
2-Variable Function with a Saddle
Point
z = f(x,y)
26
The General Problem
Max/Min f ( x1 , x2 ,..., xn )
   x1 , x2 ,..., xn  
necessary conditions:
 f ( x1 , x2 ,..., xn )
 0 for all j
 xj
sufficient conditions:
f(x1,x2,…,xn) is convex for a minimum
f(x1,x2,…,xn) is concave for a maximum
27
Recall Taylor’s Series
Approximation in 2-variables?
I sure do!
 x  x0 
f ( x, y )  f ( x0 , y0 )   f x ( x0 , y0 ) f y ( x0 , y0 )  

 y  y0 
 f xx ( x0 , y0 ) f xy ( x0 , y0 )   x  x0 
1
  x  x0 y  y0  


f
(
x
,
y
)
f
(
x
,
y
)
2
yy
0
0   y  y0 
 yx 0 0
 higher order terms
28
2-Variable Problem
sufficient conditions:
f xx ( x0 , y0 )  0
f xx ( x0 , y0 )
f xy ( x0 , y0 )
f yx ( x0 , y0 )
f yy ( x0 , y0 )
for a local min
f xx ( x0 , y0 )  0 for a local max
f xx ( x0 , y0 )  f yy ( x0 , y0 )   f xy ( x0 , y0 )   0
2
and
f xx ( x0 , y0 )  f yy ( x0 , y0 )   f xy ( x0 , y0 )   0
2
saddle
point
29
Why so?
0
 x  x0 
f ( x, y )  f ( x0 , y0 )   f x ( x0 , y0 ) f y ( x0 , y0 )  

y

y
0

 f xx ( x0 , y0 ) f xy ( x0 , y0 )   x  x0 
1
  x  x0 y  y0  


f
(
x
,
y
)
f
(
x
,
y
)
y

y
2
yy
0
0 
0
 yx 0 0
 higher order terms
f ( x, y)  f ( x0 , y0 )  0 ( x0 , y0 )is a local min; x t Hx  0
f ( x, y)  f ( x0 , y0 )  0 ( x0 , y0 )is a local max; x t Hx  0
30
A 2-variable example
Max f(x,y) = 100 – (x – 4)2 – 2 (y – 2)2
necessary conditions:
f
 2( x  4)  0
x
 x4
f
 4( y  2)  0
y
 y2
2 f
 2  0
x
sufficient conditions:
2 f
 4
y
2
 f  f   f 


 80
 x  y   x y 
2
concave function
;
2 f
0
 x y
2
2
31
Not Another Example? A Cubic no
less!
f(x,y) = 2x3 – 2x2 – 10x + y3 – 3y2 + 20
f
 6 x 2  4 x  10  0
x
2(3x – 5) (x + 1) = 0
x = 5/3, -1
f
 3y2  6 y  0
y
3y (y – 2) = 0
y = 0, 2
…and it has
four
solutions!
(x*,y*) = (5/3,0), (5/3, 2), (-1,0), (-1,2)
32
Not Another Example (continued)
f
 6 x 2  4 x  10
x
f
 3y2  6 y
y
2 f
 12 x  4
2
x
2 f
 6y  6
2
y
2 f
0
 x y
x
y
2 f
 x2
2 f
 y2
5/3
5/3
-1
-1
0
2
0
2
16
16
-16
-16
-6
6
-6
6
saddle pt
local min
local max
saddle pt
33
A Logistics Design Problem
A special container must be constructed to transport 40
cubic yards of material. The transportation cost is one
dollar per round trip. It costs $10 per square yard to construct
the sides, $30 per square yard to construct the bottom of the
container and $20 dollars to construct the ends.
It has no top and no salvage value. It must be rectangular in
shape and only one can be made. Find the dimensions which
will minimize the construction and transportation costs.
I need a box,
quick!
34
The Formulation
let x = the length, y = the width, and z = the height
then volume = xyz and transportation cost = $1 [40 / (xyz)]
cost of bottom = $30 xy
cost of sides = $10 xz
cost of ends = $20 yz
40
f ( x, y, z ) 
 30 xy  10 xz  20 yz
xyz
35
The necessary conditions
40
f ( x, y, z ) 
 30 xy  10 xz  20 yz
xyz
f ( x, y, z ) 40
 2  30 y  10 z  0
x
x yz
f ( x, y, z ) 40
 2  30 x  20 z  0
y
xy z
f ( x, y, z ) 40

 10 x  20 y  0
2
z
xyz
y  .5609776
x  1.1219551
z  1.6829321
36
Is the function convex?
 2 f ( x, y , z )
80  2 f ( x, y, z )
80  2 f ( x, y, z )
80
 3 ;
 3 ;

2
2
2
x
x yz
y
xy z
z
xyz 3
 2 f ( x, y , z )
40
 2 f ( x, y , z )
40
 2 2  30;
 2 2  10
xy
x y z
xz
x yz
 2 f ( x, y , z )
40
 2 2  20
yz
xy z
They show us how to do
that in MSC 521. I am
going to sign up today!
I see, all 9 2nd
partials must be
analyzed.
37
Adventures in Optimization
Presented by the Department of
Engineering Management &
Systems
38