Analysis & Optimization
Problem Set 5
June 25
1. (a) Here the Lagrangian is
L(p) = I(p) − λ
n
X
pi =
i=1
so that
n
X
pi log pi − λ ,
i=1


log p1 + 1 − λ


..
∇L(p) = 
.
.
log pn + 1 − λ
λ−1
We then have ∇L(p) = 0 if and only P
if pi = e
for all i = 1, . . . , n. In particular p1 = · · · = pn ,
so that since we have the constraint
pi = 1, we have
p1 = · · · = pn =
1
.
n
(b) Here the Lagrangian is
L(p) = S(p) − λ
n
X
i=1
so that
pi − µ
n
X
i=1
pi Ei = −
n
X
pi (log pi + λ + µEi ),
i=1


log p1 + 1 + λ + µE1


..
∇L(p) = − 

.
log pn + 1 + λ + µEn
It is then easy to see that ∇L(p) = 0 only if pi = e−1−λ e−µEi . We can then take Z = e1+λ and
c = −µ to obtain the desired form.
e )2 subject to the constraints x2 + y 2 = r2 and
2. We can interpret this as wanting to minimize d(x, x
ye − ae
x = b. Then the Lagrangian here is
e ) = (x − x
L(x, x
e)2 + (y − ye)2 − λ(x2 + y 2 ) − µ(e
y − ae
x),
so that


2(x − x
e) − 2λx
 2(y − ye) − 2λy 
.
e) = 
∇L(x, x
 2(e
x − x) + aµ 
2(e
y − y) − µ
e ) = 0. If λ = 0 then we have x = x
e is a point of intersection
Suppose L(x, x
e and y = ye, so that x = x
between the line and the circle. If λ 6= 0, then
(x, y) =
1
1
2(x − x
e), 2(y − ye) =
− aµ, µ = k(a, −1),
2λ
2λ
1
µ
where we have taken k = − 2λ
. Now
r2 = x2 + y 2 = k 2 (a2 + 1),
so that
k = ±√
So
x=±
√
r
.
1 + a2
ar
r
, −√
1 + a2
1 + a2
.
e are proportional. Let x
Since x − x
e − λx = y − ye − λy = 0, it is easy to see that x and x
e = `x, so that
−√
r`
a2 r`
−√
= b.
1 + a2
1 + a2
Then
b
`=− √
,
r 1 + a2
implying
e=
x
ab
b
−
,
.
1 + a2 1 + a2
If λ ≤ 0, we see that L is a sum of convex functions and is consequently itself convex. Furthermore if
λ < 0, then
e is precisely the point(s) of intersection. If the
In summary, if the line intersects the circle then x = x
line does not intersect the circle, then
ar
b
r
ab
e= −
x = ±√
,
.
,√
and
x
1 + a2 1 + a2
1 + a2
1 + a2
To discuss which points are actually minimum : notice the Lagrangian is convex as soon as λ ≤ 0.
This happens at intersection points (when the line and the circle are intersecting.)
e − x are pointing in the same direction.
Otherwise, this is equivalent to say that the vectors x and x
When the line and circle are not intersecting we see that the Lagrangian is going to be convex for the
e are on the same side of the origin point.
critical point where x and x
3. (a) Rewriting the boundary conditions in the form −x ≤ 0, −y ≤ 0, and x + y ≤ 3, we have the
Lagrangian
L(x, y; λ1 , λ2 , λ3 ) = x3 + y 3 − 3x − 3y + λ1 x + λ2 y − λ3 (x + y),
and we can apply the Kuhn-Tucker conditions to see that we must solve the system
0 = 3x2 − 3 + λ1 − λ3
0 = 3y 2 − 3 + λ2 − λ3
0 ≤ λ1
0 ≤ λ2
0 ≤ λ3 ,
with λ1 = 0 if x > 0, λ2 = 0 if y > 0, and λ3 = 0 if x + y < 3. We have a number of possible cases:
i. x = 0, y = 0, and x + y < 3. Clearly the only such point is (0, 0), with f (0, 0) = 0.
ii. x = 0, y > 0, and x + y = 3. Since x = 0 and x + y = 3, the only such point is (0, 3), with
f (0, 3) = 18.
iii. x = 0, y > 0, and x + y < 3. Here λ2 = λ3 = 0, and so 0 = 3y 2 − 3 and then y = 1. The
corresponding point is (0, 1), and f (0, 1) = −2.
2
iv. x > 0, y = 0, and x + y = 3. Since y = 0 and x + y = 3, the only such point is (3, 0), with
f (3, 0) = 18.
v. x > 0, y = 0, and x + y < 3. Here λ1 = λ3 = 0, and so 0 = 3x2 − 3 and then x = 1. The
corresponding point is (1, 0), and f (0, 1) = −2.
vi. x > 0, y > 0, and x + y = 3. Here λ1 = λ2 = 0, and so 3x2 = 3y 2 = 3 + λ3 . Then x = y;
since x + y = 3, we have x = y = 23 , with f ( 32 , 32 ) = − 94 .
vii. x > 0, y > 0, and x + y < 3. Here λ1 = λ2 = λ3 = 0, and so 3x2 = 3y 2 = 3, i.e. (x, y) =
(1, 1); we have f (1, 1) = −4.
Since D is a compact domain, f has a global maximum and minimum on D; by comparison of
the above values, we see that the maxima are (0, 3) and (3, 0), and the minimum is (1, 1). The
maximum value is 18 and the minimum value is −4.
(b) This question was terrible.
4. (a) The Lagrangian here is
L(x, y) = 3x2 − 4xy − λ(x2 + y 2 ),
so that
∇L(x, y) =
6x − 4y − 2λx
.
−4x − 2λy
So ∇L(x, y) = 0 if and only if (3 − λ)x = 2y and −2x = λy. Due to the second equation, if
y = 0 then x = 0 and so x2 + y 2 = 0; thus y 6= 0. Substituting the second equation into the first,
we find −(3−λ) λ2 y = 2y or equivalently (λ−4)(λ+1)y = 0. Since y 6= 0, we have λ = 4 or λ = −1.
If λ = 4, then both equations simplify to −x = 2y. Substituting into the constraint x2 + y 2 = 1,
we have 5y 2 = 1, so that
2
1
(x, y) = ± √ , ∓ √
.
5
5
At either of these points, q(x, y) = 4.
If instead λ = −1, then both equations simplify to 2x = y. As before substituting into the
constraint x2 + y 2 = 1, we have 5x2 = 1, so that
2
1
.
(x, y) = ± √ , ± √
5
5
At either of these points, q(x, y) = −1.
(b) As in the proof of the spectral theorem, we have from part (a) that1 the eigenvalues of the matrix
associated to the quadratic form q are 4 and −1, with associated eigenvectors √15 (2, −1) and
√1 (1, 2). Given the diagonalization
5
3
−2

√2
5
−2

=
0
− √1
5
!
√2
5
− √15

√1
5
√2
5
√1
5

4
0
√2
5

0
−1

,
it is easy to see that
q(x, y) = x
y
3
−2
2 2
2x
y
x
2y
4
1
−2
x
=4 √ −√
− √ +√
= (2x − y)2 − (x + 2y)2 .
0
y
5
5
5
5
5
5
1 Writing q(x, y) = xT Ax, we have L(x, y) = xT Ax − λ(x2 + y 2 ), so ∇L(x, y) = 2Ax − 2λx; that is, the values of the
Lagrange multipliers are precisely the eigenvalues of A and the critical points are the corresponding eigenvectors.
3
(c) In the above, the value of the Lagrange multiplier corresponding to the maximum was 4. Then
an approximation of the maximum taken by q on D is 4 + 0.01 · 4 = 4.04, as per the discussion of
Lagrange multipliers as shadow prices in §3.3 of the book.2
d
5. Here F (t, x, ẋ) = 21 ẋ2 + xet , and so the Euler-Lagrange equation is3 et = dt
ẋ = ẍ. Integrating once, we
t
t
find ẋ(t) = e +c1 for constant c1 . Integrating once more, we have x(t) = e +c1 t+c2 , where now both c1
and c2 are constants. Now 0 = x(0) = 1+c2 , so that c2 = −1. Also −1 = x(1) = e+c1 +c2 = e+c1 −1,
so that c1 = −e. We then have
x(t) = et − et − 1.
Since 21 ẋ2 is convex in ẋ and xet is convex in x, their sum (that is, F (t, x, ẋ)) is convex in both. So
x(t) as given above is a minimum.
2 If
q ∗ (k) denotes the maximum of q over the disk Dq = {(x, y) ∈ R2 : x2 + y 2 = k}, then we have
dq ∗
= λk ,
dk
so that here (since λ1 = 4) we can say q ∗ (k) ≈ q ∗ (1) + 4(k − 1). Taking k = 1.01, we have our result.
3 We have ∂F = et and ∂F = ẋ.
∂x
∂ ẋ
4