STAT 280: OPTIMIZATION
SPRING 2017
PROBLEM SET 2
1. (a) Show that for any A, B ∈ Rm×n ,
tr(AT B)2 ≤ tr(AT A) tr(B T B).
(b) Show that for any A ∈ Rn×n , the series
I +A+
Ak
A2
+ ··· +
+ ···
2
k!
is always convergent.
(c) Show that the set of invertible matrices
GL(n) := {A ∈ Rn×n : det(A) 6= 0}
is an open set in Rn×n .
(d) Show that the set of nilpotent matrices {A ∈ Rn×n : Ak = 0 for some k ∈ N ∪ {0}} is a
closed set in Rn×n .
(e) Show that the set of symmetric positive definite matrices
Sn++ := {A ∈ Sn : xT Ax > 0 for all 0 6= x ∈ Rn }
is an open set in Sn := {A ∈ Rn×n : AT = A} and that its closure is the set of symmetric
positive semidefinite matrices
Sn+ := {A ∈ Sn : xT Ax ≥ 0 for all x ∈ Rn }.
2. (a) Let f : Rn → R be continuous and S ⊆ Rn be a compact set. By emulating the proofs for
the case n = 1 or otherwise, show that (i) there exists M > 0 such that
|f (x)| ≤ M
for all x ∈ S;
and (ii) there exist xmin , xmax ∈ S such that
f (xmin ) ≤ f (x) ≤ f (xmax )
for all x ∈ S.
(b) Let f : Rn → R be in C 1 (Rn ). Suppose f satisfies
∂f
≤ K, i = 1, . . . , n,
(x)
∂xi
for all x ∈ Rn where K > 0 is a constant. Show that
√
|f (x) − f (y)| ≤ nKkx − yk2
for all x, y ∈ Rn (Hint: Apply the univariate mean-value theorem n times).
(c) Define f : R2 → R by

2
 1 − cos x px2 + y 2 if y 6= 0,
f (x) =
y

0
if y = 0.
Show that f is continuous but not differentiable at (0, 0). For which v ∈ R2 does the
directional derivative dv f (0, 0) exist?
Date: April 18, 2017 (Version 1.0); due: May 2, 2017.
1
2
STAT 280 ASSIGNMENT 2
3. Let A : R → Rn×n be the matrix-valued function

a11 (x) · · ·
 ..
A(x) =  .
an1 (x) · · ·

a1n (x)
.. 
. 
ann (x)
where the functions aij : R → R are all in C 1 (R) for all i, j = 1, . . . , n.
(a) Show that the function f : R → R defined by f (x) = tr A(x)3 is differentiable and that
f 0 (x) = 3 tr A(x)2 A0 (x)
where
a011 (x) · · ·

A0 (x) =  ...

a0n1 (x) · · ·

a01n (x)
..  .
. 
a0nn (x)
(b) Let n = 2. Suppose det A(x) > 0 for all x ∈ R and define B : R → R2×2 by B(x) = A(x)−1 .
Show that
X2
d
log det A(x) =
a0 (x)bji (x).
i,j=1 ij
dx
This is actually true for arbitrary n. For n = 1, it reduces to (log a(x))0 = a0 (x)/a(x).
4. (a) Find the total derivative of the function f : R4 → R2×2 defined by
2 T
x y
x y
f (x, y, z, w) =
+
= X 2 + X T.
z w
z w
Find a point x ∈ R4 where Df (x) is invertible and another where it is not.
(b) Consider the functions f, g : Rn×n → Rn×n defined by
f (X) = X + X 2 ,
g(X) = X 4 .
Show that the total derivatives Df (X) and Dg(X) are given by
[Df (X)](Y ) = Y + XY + Y X,
[Dg(X)](Y ) = X 3 Y + X 2 Y X + XY X 2 + Y X 3 ,
for all Y ∈ Rn×n . Are these invertible when X = 0, the zero matrix?
5. Find the gradients of the following functions.
(a) f : Rn → R defined by
f (x) = kxk22 .
(b) f : Rn → R defined by
f (x) = kxk2 .
Find also the set of points where f is not differentiable.
(c) f : Rn → R defined by
xT Ax
f (x) = T
x Bx
where A, B ∈ Sn++ .
(d) f : Rm×n → R defined by
f (X) = tr(AT X)
where A ∈ Rm×n .
(e) f : Rm×n → R defined by
f (X) = aT Xb
where a ∈ Rm and b ∈ Rn .
(f) f : Sn++ → R defined by
f (X) = tr(X −1 ).