Answers: Practice Problems - Optimization 1
Math Prefresher 2013
August 29, 2013
Problem 1
Determine the leading principal minors and definiteness of the following matrices.
!
2 −1
1.
LPMs are 2,1 so positive definite
−1
1
2.
3.
4.

−3
4
4
−5
2
4
4
8
!
LPMs are -3,-1 so indefinite
!
LPMs are -3,-2 so negative definite
−3
4
4
−6
!
LPMs are 2,0 so positive semidefinite

1
2
0

5.  2
0
4

5  LPMs are 1,0,-25 so indefinite
6
5

−1
1

6. 
1
0
−1
0

0


0  LPMs are -1,0,0 so negative semidefinite
−2

1
0
3
0

 0
7. 
 3

0
2
0
0
4
5
0

5 
 LPMs are 1,2,-10 so indefinite
0 

6
1
Problem 2
Given x4 + x2 − 6xy + 3y 2 , find the first order conditions (gradient and critical points) and the second order
conditions (hessian, definiteness of hessian, and whether maxima/minima/saddle).
1. Gradient: ∇f (x) =
4x3 + 2x − 6y
!
−6x + 6y
2. Critical Points: −6x + 6y = 0 → x =
! y which yields three critical points: (0,0); (1,1); (-1,-1).
12x2 + 2 −6
3. Hessian: H(x) =
−6
6
!
2 −6
4. H(0,0) =
; LPMs are 2 and -24; indefinite; saddle point
−6 6
!
14 −6
H(1,1) =
; LPMs are 14 and 48; positive definite; local min
−6 6
!
14 −6
H(-1,-1) =
; LPMs are 14 and 48; positive definite; local min
−6 6
Problem 3
Given xy 2 + x3 y − xy, find the first order conditions (gradient and critical points) and the second order
conditions (hessian, definiteness of hessian, and whether maxima/minima/saddle).
1. Gradient: ∇f (x) =
y 2 + 3x2 y − y
!
2xy + x3 − x
−1 2
2. Critical Points: six critical points: (0,0); (0,1); (1,0); (-1,0), ( √15 , 25 ); ( √
, ).
5 5
!
2
6xy
2y + 3x − 1
3. Hessian: H(x) =
2
2y + 3x − 1
2x
!
0 −1
4. H(0,0) =
; LPMs are 0 and -1; indefinite; saddle point
−1 0
!
0 1
H(0,1) =
; LPMs are 0 and -1; indefinite; saddle point
1 0
!
0 2
H(1,0) =
; LPMs are 0 and -4; indefinite; saddle point
2 2
!
0 2
H(-1,0) =
; LPMs are 0 and -4; indefinite; saddle point
2 −2
!
H( √15 , 25 ) =
−1 2
H( √
, )=
5 5
12
√
5 5
2
5
−12
√
5 5
−2
5
2
5
√2
5
2
5
√2
5
; LPMs are
12
√
5 5
and 45 ; positive definite; local minima
; LPMs are
−12
√
5 5
and 45 ; negative definite; local maxima
!
2
Problem 4
Given x2 + 6xy + y 2 − 3yz + 4z 2 − 10x − 5y − 21z, find the first order conditions (gradient and critical points)
and the second order conditions (hessian, definiteness of hessian, and whether maxima/minima/saddle).


2x + 6y − 10


1. Gradient: ∇f (x) =  6x + 2y − 3z − 5 
−3y + 8z − 21
2. Critical Points: one critical point: (2,1,3).


2 6
0


3. Hessian: H(x) =  6 2 −3 
0 −3 8


2 6
0


4. H(2,1,3) =  6 2 −3 ; LPMs are 2, -32, and -274; indefinite; saddle point
0
−3
8
Problem 5
We have data, x1 , x2 , ...xn , that are independently and identically exponentially distributed with parameter
λ (xi ∼iid Expo(λ)). We are going to find the maximum likelihood estimator1 of λ given x1 , x2 , ...xn . Given,
L(λ) =
n
Y
λe−λxi
i=1
1. Take the log of L(λ) and call this l(λ).
L(λ) = λn e−λ
Pn
i=1
l(λ) = n log(λ) − λ
xi
Pn
i=1
x
2. Find the gradient of l(λ).
∂l
∂λ
=
n
λ
−
Pn
i=1
xi
3. Set the gradient equal to 0 and solve the system of equations for λ. This is our maximum likelihood
estimator. Note that λ will be a function of x.
λ=
Pnn
i=1
x
4. Check that this is a maximum. Find the hessian of l(µ, σ 2 ), determine its concavity, and confirm that
this is a maximum.
H(λ) = − λn2 which is negative always, therefore concave and a maximum.
1 You
will learn more about what this is in Gov 2001.
3
Problem 6: R Practice
1. Load any of the data from this week. Then save this data as a matrix.
load("day3data.RData")
ans.1 <- as.matrix(example.matrix)
2. Create a vector that holds the mean of every variable (column) from your data.
ans.2 <- apply(ans.1, 2, FUN=mean)
3. Write a function to calculate the mean of the square of vector (First square the vector, then calculate
the mean of the new vector).
sqmean.function <- function(x){
y <- x^2
ans.3 <- sum(y)/length(y)
return(ans.3)
}
4