EE 572 OPTIMIZATION THEORY
HOMEWORK I
Due: 20 November 2008
1. Find all the stationary points for the functions and classify them as minimum,
maximum and inflection point:
(a) f ( x1 , x2 )  (2  x1  x2 )2  (1  x1  x2  x1x2 )2
(b) f ( x1 , x2 )  x13  3x1 x2  2x23
(c) f ( x1 , x2 , x3 )  2x12  x1 x2  x22  x2 x3  x32  6x1  7 x2  8x3  9
2. Determine whether the following function is convex or not. Compute its global
minimum if it is a convex function.
f ( x1 , x2 , x3 )  3x12  x1x2  4x22  x2  2x1x3  2x32
3. Find an equation of the form g ( x)  c0  c1 x  c2 x2  c3 x3 which best approximates the
1
x
function f ( x)  in the interval 0.5  x  2 . ( Hint: minimize the integral of the squarederror over the given interval with respect to the parameters c0 ,..., c3 .)
4. Let f ( x)  x be a norm on R n . Show that it is a convex function on R n .
5. Let f : R  R be a convex function. Show that its moving average defined by
x
F ( x) 
1
f (t ).dt , x  0
x 0
is convex. Assume that f is differentiable.
6. Let f : R n  R, A  R nm and b  R n . Define g : R m  R by
g  f ( Ax  b)
Show that g is convex if f is convex.
SOLUTION
1-) (a)
f
 2(2  x1  x2 )  2(1  x1  x2  x1 x2 )(1  x2 )  0
x1
f
 2(2  x1  x2 )  2(1  x1  x2  x1 x2 )(1  x1 )  0
x2

x1  1,
x2  1
Hessian matrix:
2 f
 2  2(1  x2 ) 2 ,
2
x1
2 f
 2  4 x1  4 x2  4 x1 x2 ,
x1x2
 2 2 
 H (1,1)  
 principal minors: 2, 0
 2 2 
2 f
 2  2(1  x1 ) 2
2
x2
 inflection point.
(b)
f
 3x12  3x2  0,
x1
Hessian matrix:
f
 6 x22  3x1  0  soln. (0,0), (0.7937,0.63)
x2
2 f
 6 x1 ,
x12
2 f
 3,
x1x2
2 f
 12 x2
x22
 4.7622 3 
 H (0.7937,0.63)  
principal minors: 4.7644, 27  mimimum point.
7.56 
 3
 0 3
H (0,0)  
 inflection point.
 principal minors: 0,  9
 3 0 
(c)
f
 4 x1  x2  6  0,
x1
f
 x1  2 x2  x3  7  0,
x2
f
 x2  2 x3  8
x3
 4 1 0   x1   6 
f  0  1 2 1   x2   7   x1  1.2, x2  1.2, x3  3.4
 0 1 2   x3   8 
4 1 0
Hessian matrix: H (x)  1 2 1  , principal minors: 4, 7, 10  positive definite.
 0 1 2 
 minimum point.
2-)
f ( x)   6 x1  x2  2 x3
 6 1 2 
H ( x)   1 8 0 
 2 0 4 
 f is convex
f ( x )  0 
 x1  8 x2  1 2 x1  4 x3 
principal minors: 6, 47, 182
Hx  b   0 1 0 
T
 positive definite
 x   0.0256 0.1282 0.0128
T
3-) Integral of the squared-error
2
1

E   [ g ( x)  f ( x)] dx   c0  c1 x  c2 x 2  c3 x3   dx
x
0.5
0.5 
2
2
2
The integral can be evaluated using the Symbolic Toolbox of Matlab:
E  3c1  3.75c2  5.25c3  2.7726c0  1.5  7.9688c0c3  5.25c0c2  7.9688c1c2  3.75c0c1
 12.7875c1c3  21.3281c2 c3  1.5c02  2.625c12  6.3938c22  18.2846c32
Equating the gradient of E with respect to the unknown vector c  c0 c1 c2 c3  , and
solving the resulting linear equations gives:
T
c1  3.9841, c2  5.5417, c3  3.2103, c4  0.6591
To verify the correctness of the result, f ( x) and g ( x) are plotted:
4-)
f  x1  (1   ) x2    x1  (1   ) x2   x1  (1   ) x2

from the triangle inequality
f  x1  (1   ) x2    x1  (1   ) x2   f  x1   (1   ) f  x2 
since 0    1.
 f ( x)  x is convex.
5-) It is sufficient to show that
d 2F
 0 x  0 .
dx 2
dF 1
  f ( x)  F ( x)  
dx x
d 2F
1
1  df 1

  2  f ( x)  F ( x)      f ( x)  F ( x)  
dx 2
x
x  dx x

1  df
dF 
  2

x  dx
dx 
Consider the Taylor series expansion of f :


1 (n)
f (0) x n
n
!
n 0
1
f ( n ) (0) x n
(
n

1)!
n 0
f ( x)  
 F ( x)  
Also,
df ( x) 
1

f ( n ) (0) x n 1 ,
dx
n 1 ( n  1)!

d 2 f ( x) 
1

f ( n ) (0) x n  2
2
dx
n  2 ( n  2)!
d 2F 1 
1
2   1
1
(n)
n 1

f
(0)
x

  f ( n ) (0) x n

2
2 
dx
x n 1 (n  1)!
x n 0  (n  1)! n! 



1 
1
n
(n)
n
f
(0)
x

2
f ( n ) (0) x n 


2 
x  n 1 (n  1)!
n  0 ( n  1)!

which simplifies to
d 2F 
1
1

f ( n ) (0) x n  2
2
dx
(
n

1)!
(
n

2)!
n 0
Since f is convex
d 2 f ( x) 
  n x n  2  0
dx 2
n2

6-)
where  n 
1
f ( n ) (0) n  2
(n  2)!
d 2 F  n

xn2  0
dx 2 n 0 (n  1)!
g  x1  (1   ) x2   f  A  x1  (1   ) x2   b   f  Ax1  (1   ) Ax2  b 
 f   Ax1  b   (1   )  Ax2  b  
 f  y1  (1   ) y2 
where y1  Ax1  b, y2  Ax2  b
f is convex    f  y1   (1   ) f  y2    g  x1   (1   ) g  x2 
since f  y1   g  x1  and
f  y2   g  x2  .
 g is convex.