L8 Optimal Design concepts pt D
•
•
•
•
•
•
•
Homework
Review
Equality Constrained MVO
LaGrange Function
Necessary Condition EC-MVO
Example
Summary
1
MV Optimization- UNCONSTRAINED
For x* to be a local minimum: f  f (x )  f (x*)  0
1 T
T
f  f ( x*)d  d H d
2
f T (x*)  0
1rst order
Necessary
Condition
dT H d  0
2nd order
Sufficient
Condition
i.e. H(x*) must be positive definite
2
MV Optimization- CONSTRAINED
For x* to be a local minimum:
MINIMIZE :
f (x )
Subject To :
h j (x )= 0
j = 1 p
(U )
(L )
i = 1 n
xi  xi  xi
3
LaGrange Function
If we let x* be the minimum f(x*) in the feasible region:
All x* satisfy the equality constraints (i.e. hj =0)
Let’s create the LaGrange Function by augmenting the
objective function with “0’s”
Using parameters, known as LaGrange multipliers, and the
equality constraints
Lx, ν   f ( x )  υ1h1 ( x )  υ2 h2 ( x )  ... υ p h p ( x )
or in summation notation
p
Lx, ν   f ( x )   υi hi ( x )
i 1
and using vectors
Lx, ν   f ( x )  νT h( x )
" υ"  upsilon  scalar
" ν"  nu  vector
4
Necessary Condition
Necessary condition for a stationary point
Given f(x), one equality constraint, and n=2
Lx, ν   0
{ f ( x )  υ1h1 ( x )}  0
L f
h ( x )

 υ1 1
0
x1 x1
x1
L f
h ( x )

 υ1 1
0
 x2  x2
 x2
L
 h1 ( x )  0
υ1
Lx, ν   0
f
h1 ( x )
 υ1
0
x1
x1
f
h1 ( x )
 υ1
0
 x2
 x2
h1 ( x )  0
5
2D Example
Minimize f ( x1 , x2 )  ( x1  1.5)2  ( x2  1.5)2
subject to : h(( x1 , x2 )  x1  x2  2  0
6
Example cont’d
L(x, υ)  f (x )  υh(x )
L(x, υ)  ( x1  1.5)2  ( x2  1.5)2  υ( x1  x2  2)
Lx, ν   0
f
h ( x )
υ
 2( x1  1.5)  υ  0
x1
x1
f
h ( x )
υ
 2( x2  1.5)  υ  0
x2
x2
h1 ( x )  x1  x2  2  0
system of 3 eqns and 3unkns
2( x1  1.5)  υ  0
2( x2  1.5)  υ  0
x1  x2  2  0
simultaneo us solution gives :
υ 1
x1*  1
x2*  1
7
Example cont’d
optimal solution
υ  1 x1*  1 x2*  1
optimal value
f (1,1)  (1 - 1.5)2  (1  1.5)2  0.5
2( x1  1.5)   1
f x *  
 

2( x2  1.5)  1
1
hx *   
1
8
Geometric meaning?
 1
f x *   
 1
1
hx *   
1
f x *  υhx *  0
f x *  υhx *
9
Stationary Points
Lx, ν  0
Points that satisfy the necessary
condition of the LaGrange Function are
stationary points
Also called “Karush-Kuhn-Tucker” or
KKT points
10
Lagrange Multiplier Method
1. Both f(x) and all hj(x) are differentiable
2. x* must be a regular point:
x* is feasible (i.e. satisfies all hj(x)
Gradient vectors of hj(x) are linearly
independent (not parallel, otherwise
no unique solution)
3. LaGrange multipliers can be +, - or 0.
Can multiply h(x) by -1, feasible region
is the same.
11
Summary
•
•
•
•
LaGrange Function L(x,u)
Necessary Conditions for EC-MVO
Example
Geometric meaning of multiplier
12