Optimization
13:49, November 3, 2015.
Single period models. Assume that we have a single step model with a real world probability
P. Given some initial wealth, we want to maximize the expected utility of our wealth at T = 1.
We let this utility be measured by a utility function u(v) with the following properties
u0 (v) > 0,
u00 (v) < 0.
Example 1. Assume that we have a binary model with a single stock and zero interest rate given
by the following table and assume that B0 = B1 = 1. Let the utility function be defined by
ω1
S0
S1
P
2
3
1/3
ω2 2
1 2/3
Table 1. Example 1.
u(v) = ln(v + 1), and assume that v > −1. Given an initial wealth V0 we want to find a portfolio
ϕ = (x, y) which maximizes
g(x, y) := EP (u(V1ϕ )), if V0ϕ = V0 .
We can write the condition V0ϕ = V0 as
x + 2y = V0 , or x = V0 − 2y.
Thus we choose y such that f (y) = g(V0 − 2y, y) is maximal. We have that
2
1
f (y) = ln(V0 − 2y + 3y + 1) + ln(V0 − 2y + y + 1)
3
3
1
= ln (α2 − y 2 )(α + y) ,
3
where α = V0 + 1. Solving for f 0 (y) = 0,
1
1
2
1
0=
−
,
3 V0 + 1 + y 3 V0 + 1 − y
which implies
V0 + 1
5V0 + 2
y=−
,
x=
.
3
3
This is a maximum for f since we have that f 00 < 0 for −α < y < α.
Returning to the general case, we have a single period model, with one bond, n scenarios;
ω1 , . . . , ωn , and m stocks; S 1 , . . . , S m . We are given a convex utility function u. For each initial
wealth V0 , we want to find a portfolio ϕ = (x, y) ∈ R1+m such that
EP (u(V1ϕ )) is maximal, where V0ϕ = V0 .
(1)
Since V0ϕ = V0 ,
x=
1
(V0 − y · S0 ) .
B0
Therefore we can write
V1ϕ = xB1 + y · S1
B1
=
(V0 − y · S0 ) + y · S1
B0
= (1 + r) V0 + y · ∆S̄1 .
Hence (1) can be rewritten
maximizey1 ,...,ym EP u((1 + r) V0 + y · ∆S̄1 .
1
2
Thus we must maximize
g(y) =
n
X
pi u (1 + r) V0 + y · ∆S̄1 (ωi )
.
i=1
If we have a maximum at some ŷ = (ŷ1 , . . . , ŷm ) then ∇g(ŷ) = 0, or
n
(2)
X
∂g
0=
(ŷ) =
pi u0 (V1ϕ (ŷ, ωi )) ∆S̄1j (ωi ),
∂yj
i=1
for j = 1, . . . , m.
Observe that if we have a solution of this, then we automatically can define a risk neutral probability
measure by setting
pi u0 (V ϕ (ŷ, ωi ))
qi = Pn
.
0
ϕ
k=1 pk u (V (ŷ, ωk ))
(3)
Pn
We obviously have i=1 qi = 1 and since u0 > 0, pi > 0, we also have qi > 0. Setting Q =
(q1 , . . . , qn ) (2) reads
n
X
qi ∆S̄1 (ωi ) = 0,
i=1
which means that Q is a risk neutral probability measure. Thus if we can find a portfolio which
maximizes g, then we have a risk neutral probability and the model is viable. Conversely, if the
model contains arbitrage opportunities, then the maximization problem (1) has no solution.
This can also be seen directly as follows. If ŷ is a solution to (1) and y is an implicit arbitrage
opportunity, set ỹ = ŷ + y. Then
V1ϕ (ỹ) = (1 + r) V0 + ỹ · ∆S̄1
= (1 + r) V0 + ŷ · ∆S̄1 + (1 + r)y · ∆S̄1
= V1ϕ (ŷ) + y · ∆S̄1 ≥ V1ϕ (ŷ),
with strict inequality for at least one ωi . Since the utility function u is increasing and pi > 0, this
means that
EP (V1ϕ (ỹ)) > EP (V1ϕ (ŷ)).
Hence the maximization problem (1) has no solution.
The converse it not necessarily true, there exist viable models and convex increasing utility
functions u and initial wealths V0 such that (1) has no solution.
Example 2. Consider the following model with three scenarios and two stocks.
S̄11 = 6
S̄12 = 13
S01 = 6
S02 = 10
S̄11 = 8
S̄12 = 9
S̄11 = 4
S̄12 = 8
B0 = 1
B1 = 9/8
3
Finding the discounted differences
0
∆S̄1 (ω1 ) =
,
3
∆S̄1 (ω2 ) =
2
.
−1
∆S̄1 (ω3 ) =
−2
.
−2
Hence a risk neutral probability measure satisfies
q1 + q2 + q3 = 1,
0q1 + 2q2 − 2q1 = 0,
3q1 − q2 − 2q3 = 0.
This model is viable and complete, since Q = (1/3, 1/3, 1/3) is the unique solution to this system.
Let the real world probability be P = (p1 , p2 , p3 ) and the utility function u(v) = −e−v . The
function to be maximized is
g(y1 , y2 ) = −p1 e−9(V0 +3y2 )/8 − p2 e−9(V0 +2y1 −y2 )/8 − p3 e−9(V 0−2y1 −2y2 )/8 .
If we have a solution (y1 , y2 ), this solution must solve
0 = 2p2 e−9(V0 +2y1 −y2 )/8 − 2p3 e−9(V0 −2y1 −2y2 )/8
0 = 3p1 e−9(V0 +3y2 ) − p2 e−9(V0 +2y1 −y2 )/8 − 2p3 e−9(V0 −2y1 −2y2 )/8 .
There seems to be no easy way of solving this nonlinear system of equations, except for the case
where P = (1/3, 1/3, 1/3), in which case a solution is (0, 0).
One can of course, use a numerical solver to find what is hopefully an approximation to the
solution. In the case where P = (1/4, 1/2, 1/4) the Matlab routine “fsolve” gave the answer
(4)
(y1 , y2 ) = (0.1716, −0.0684).
Repetition: Lagrange multipliers. If we want to find the maximum of a function f (y) subject to
the condition g(y) = c we can use Lagrange multipliers. For any such maximum ŷ, geometric
considerations demand that the gradient of f is parallel to the gradient of g at ŷ, i.e., ∇f (ŷ) =
λ∇g(ŷ) for some λ. We can solve this equation and the equation g(ŷ) = c to find candidates
for ŷ.
Example 3. Find the maximum of f (x, y, z) = ln(x) + ln(y) + 3 ln(z) for (x, y, z) on the part
of the sphere x2 + y 2 + z 2 = 5 where x > 0, y > 0 and z > 0.
We solve
∇f (x, y, z) = (1/x, 1/y, 3/z) = λ∇g(x, y, z) = λ(2x, 2y, 2z),
2
to find that x = 1/(2λ) = y 2√and z 2 = 3/(2λ). Inserting this in x2 + y 2 + z 2 = 5 we find that
λ = 1/2 and (x, y, z) = (1, 1, 3).
Finding optimal portfolios using the risk neutral probability measure. Sometimes it is
easier to use the risk neutral probability measure Q to find optimal portfolios. The strategy is as
follows: First solve the optimization problem; find a “derivative” D1 (ω1 ), . . . , D1 (ωn ) such that
EP (D1 ) is maximal, given that EQ (D̄1 ) = V0 . Then find a portfolio ϕ = (x, y) replicating D1 , i.e.,
V1ϕ = D1 . If the model is complete, this can always be done.
Using a Lagrange multiplier λ, the first order conditions are
∇D EP (u(D)) − λEQ (D̄) = 0,
subject to the condition
EQ (D̄) = V0 .
Here ∇D f (D) is the vector with components
∂f
for i = 1, . . . , n.
∂D(ωi )
4
We must find D = D(ωi ) for i = 1, . . . , n. Now
f (D) = EP (u(D)) − λEQ (D̄) =
n
X
pi u(D(ωi )) −
i=1
=
n
X
λ
qi D(ωi )
1+r
pi u(D(ωi )) −
i=1
λ qi
D(ωi ) .
1 + r pi
We have that
λ qi
∂f
= pi u0 (D(ωi )) −
.
∂D(ωi )
1 + r pi
Since pi > 0, the condition for a maximum for each scenario is
u0 (D(ωi )) =
λ
li ,
1+r
for i = 1, . . . , n,
where li is the state price density qi /pi . We have u00 < 0, hence u0 is decreasing, and it has an
inverse I. This means that
λ
li , for i = 1, . . . , n.
D(ωi ) = I
1+r
We can use this to determine λ since
V0 = EQ (D̄) =
n
n
1 X
1 X
λ
qi D(ωi ) =
qi I
li .
1 + r i=1
1 + r i=1
1+r
This is a single (probably nonlinear) equation for the unknown λ, so it is likely to be easier to
solve than the nonlinear system we get by trying to optimize directly.
Once we have the optimal “derivative” D, we can find a replicating portfolio ϕ.
Example 4. Consider Example 2 and assume that the real world probabilities are P = (1/4, 1/2, 1/4).
Since Q = (1/3, 1/3, 1/3) the state price densities are
l1 =
4
2
4
, l2 = , l3 = .
3
3
3
The utility function u(v) = −e−v , u0 (v) = e−v , and hence I(z) = − ln(z). Therefore
8λ 4
8λ 2
8λ 4
D(ω1 ) = − ln
, D(ω2 ) = − ln
, D(ω3 ) = − ln
.
9 3
9 3
9 3
Recall that ν = V0 (1 + r) = 98 V0 , the equation determining λ is
3
X
1
ν=
qi D(ωi ) =
3
i=1
−2 ln
3 !
16
32
16
1
λ − ln
λ
= − ln 4
λ3 .
27
27
3
27
This gives
λ=
27 −ν
√ e .
16 3 4
For the value of the derivative we get
√
32 27
3
−ν
√
D(ω1 ) = − ln
e
= ν + ln 4 − ln 2.
3
27 16 4
Similarly
D(ω2 ) = ν + ln
√
3
4, and D(ω3 ) = D(ω1 ).
5
Now we must find the replicating portfolio ϕ = (x, y1 , y2 ). The determining equations are x + y ·
S̄1 = D̄ for each ω,
8 √
3
x + 6y1 + 13y2 = V0 +
ln 4 − ln 2 ,
9
8 √
3
x + 8y1 + 9y2 = V0 + ln 4,
9
8 √
3
ln 4 − ln 2 .
x + 4y1 + 8y2 = V0 +
9
Set


1 6 13
A = 1 8 9  .
1 4 8
And let α and β solve

 
 √
− ln 2
ln 3 4√
1
Aα = 1 , and Aβ =  √
ln 3 4  .
3
1
ln 4 − ln 2
We get
 


1
−0.3851
α = 0 and β ≈  0.1925  .
0
−0.0770
The optimal portfolio is then given by

 

x
V0 − 0.3423
8
y1  = V0 α + β ≈  0.1711  .
9
y2
−0.0685
Observe that this is approximately the same portfolio as found by the Matlab routine, cf. (4).
Often, one uses the utility function u(v) = ln(v). This simplifies matters a bit. In this case
u0 (v) =
1
1
, so that i(z) = .
v
z
Therefore
D(ωi ) =
1+r
,
λli
and the equation determining λ becomes
V0 = EQ (D̄) =
n
1 X 1+r
1
qi
= .
1 + r i=1
λli
λ
Thus λ = 1/V0 , and
pi
.
qi
Hence the equations determining the optimal portfolio ϕ = (x, y) are
D(ωi ) = (1 + r)V0
xB0 +
m
X
j=1
yj S̄1j (ωi ) = V0
pi
,
qi
for i = 1, . . . , n.
Example 5. We use Example 2 and optimize the expected utility for the utility function u(v) =
ln(v). In this case
p1
3 p2
3
p3
3
= ,
= , and
= .
q1
4 q2
2
q3
4
To find the optimal portfolio we must solve the equation

 
 
1 6 13
x
1
3V
1 8 9  y1  = 0 2 ,
4
1 4 8
y2
1
6
which has solution
 


x
28/27
y1  = V0  10/27  .
y2
−4/27
Optimization in incomplete markets. The preceding discussion assumed that the market
was complete, i.e., that every derivative, in particular the optimal one, can be replicated. In an
incomplete market the optimal derivative may not be attainable.
Therefore we must seek the maximum among those derivatives which can be attained starting
with the initial wealth V0 . This is simply the set
DV0 = D EQ (D̄) = V0 , for all risk neutral Q .
However, since there is an infinite number of risk neutral probability measures Q, this is impractical
to use as a constraint.
We can overcome this difficulty by using a “basis” for the set of risk neutral probabilities. Let
M denote this set, i.e.,
(
)
n
X
X
M = Q = (q1 , . . . , qn ) 0 < qi < 1,
qi = 1 and
qi ∆S̄1 (ωi ) = 0 .
i
i=1
This is a bounded and convex planar set. Hence one can find K vectors Q1 , . . . , QK in the closure
of M such that for any Q ∈ M one has
Q=
K
X
K
X
ck Qk , and
k=1
ck = 1.
k=1
The coefficients ck may be positive or negative (or zero). By the continuity of the map Q 7→ EQ (D),
if EQ (D̄) = V0 for all Q ∈ M, then EQ (D̄) = V0 for all Q ∈ M (the closure of M). Hence if D is
attainable from a starting wealth V0 , then
EQk (D̄) = V0 , for k = 1, . . . , K.
This observation reduces an infinite number of constraints to K constraints. Thus the optimization
problem reads
maximizeD EP (u(D)),
subject to the conditions
(5)
EQk (D̄) = V0 , for k = 1, . . . , K.
Now we need K Lagrange multipliers λ1 , . . . , λK , and our task is:
K
X
(6)
maximize EP (u(D)) −
λk EQk (D̄) ,
k=1
such that the K conditions (5) are satisfied. To fix the notation let
Qk = (qk1 , qk2 , . . . , qkn ).
The function to be maximized is
n
X
pi
u (D(ωi )) −
i=1
K
X
k=1
!
λk qki
D .
(1 + r)pi
This gives that
(7)
D(ωi ) = I
K
X
λk lki
1+r
!
,
k=1
where I = (u0 )−1 and lki = qki /pi . We have K equations determining the multipliers λk , namely
!
n
n
K
X
1 X
1 X
λ` l`i
qki D (ωi ) =
qki I
, for k = 1, . . . , K.
V0 =
1 + r i=1
1 + r i=1
1+r
`=1
7
Again matters simplify slightly if u(v) = ln(v), in this case
#
"
n
X
qki
(8)
V0 =
pi PK
, for k = 1, . . . , K.
`=1 λ` q`i
i=1
Example 6. We shall show how this works in an example. Consider the following model with
one stock S and interest rate r = 1/9. We can assume that B0 = 1. We start by checking whether
ω S0
ω1 5
ω2 5
ω3 5
Table
S1
S̄1 P
20/3 6 1/3
40/9 4 1/3
30/3 3 1/3
2. Example 6
the model is viable or complete by looking for a risk neutral probability measure Q solving the
equations
q1 + q2 + q3 = 1,
q1 − q2 − 2q3 = 0.
This has solutions given by
Q=
1 1 1 3
+ λ, − λ, λ , λ ∈ (0, 1/3).
2 2 2 2
Thus the model is viable, but not complete.
To choose a “basis” for all risk neutral probability measures we can choose Q1 = Q(λ = 0) and
Q2 = Q(λ = 1/3), i.e.,
1 1
2
1
Q1 =
, ,0
Q2 =
, 0,
.
2 2
3
3
This gives
3
3
l11 = , l12 = , l13 = 0,
2
2
l21 = 2, l22 = 0, l23 = 1.
Assume now that the utility function is u(v) = ln(v). The

1
2

2
 2 λ1 + 3 λ2
X
λ` q`i = 12 λ1

1
`=1
3 λ2
denominators in (8) are
i = 1,
i = 2,
i = 3.
Hence the equations in (8) are
1
2
1
3
1
2 λ1
2
3
1
2 λ1
+ 23 λ2
1
3
+ 23 λ2
+
1
3
+
1
3
1
2
1
2 λ1
1
3
1
3 λ2
= V0 ,
= V0 ,
which can be rewritten
3
1
+
= 3V0 ,
3λ1 + 4λ2
λ1
1
4
+
= 3V0 .
3λ1 + 4λ2
λ2
This has a solution (I confess I used Matlab to get this...)
λ1 ≈
0.46482
0.53519
, λ2 ≈
.
V0
V0
8
Since we use u = ln the optimal derivative D in (7) is
D(ωi ) = (1 + r) P2
1
k=1
λk lki
Inserting λk and lki we find that


0.5657
D(ωi ) ≈ (1 + r)V0 1.4342


1.8685
i = 1,
i = 2,
i = 3.
We can check that this derivative is attainable, since D(ω3 ) = (3/2)D(ω2 ) − (1/2)D(ω1 ). The
equations for the optimal portfolio are
x + 6y = 0.5657V0 ,
x + 4y = 1.4342V0 ,
x + 3y = 1.8685V0 .
Of course, one of these equations is redundant. Solving two of them, we find that
x = 3.1713V0 , y = −0.4343V0 .
Multi-period models. For multi-period models we can use the same strategy to find optimal
trading strategies. If the model is viable and complete, there is a unique equivalent martingale
measure defined by the filtration given by the stock prices. Then the optimization problem reads
(9)
maximize [EP (u(D))] ,
subject to the condition
EQ (D̄) = V0 .
This is not so different from the single period case as the next example shows.
Example 7. Consider model 5.1 from the textbook, see Figure 1. If we set the interest rate to
0.1 this model has a uinque equivalent martingale measure given by
ω
ω1
ω2
ω3
ω4
ω5
ω6
ω7
ω8
ω9
.
Q 0.2309 0.0402 0.1054 0.0445 0.0082 0.0296 0.1109 0.4194 0.0108
Thus the model is viable and complete.
Assume that the real world probability measure is
ω1
ω2
ω3
ω4
ω5
ω6
ω7
ω8
ω9
ω
.
P 0.1635 0.1647 0.0269 0.1657 0.1634 0.0829 0.1366 0.0242 0.0720
We write pi = P(ωi ). Assume that the utility function is u(v) = ln(v), then the optimal derivative
is
pi
D(ωi ) = (1 + r)2 V0 .
qi
Now we must find the optimal trading strategy reproducing this portfolio. If and B0 = 1 then
ω1
ω2
ω3
ω4
ω5
ω6
ω7
ω8
ω9
ω
.
D/V0 0.8567 4.9586 0.3089 4.5059 24.1145 3.3876 1.4908 0.0699 8.0677
To find the optimal trading strategy at the node u, we must solve
x2 (u)B2 + y2 (u) · S2 (ωi ) = D(ωi ),
for i = 1, 2, 3. Setting y2 (u) = (y21 (u), y22 (u)), this reads
x2 (u)(1.1)2 + y21 (u)13 + y22 (u)29 = D(ω1 )
x2 (u)(1.1)2 + y21 (u)10 + y22 (u)23 = D(ω2 )
x2 (u)(1.1)2 + y21 (u)7 + y22 (u)22 = D(ω3 ),
which gives
(x2 (u), y2 (u)) = (19.7375, 2.1333, −1.7503),
V1Φ (u)
= x2 (u)(1.1) + y2 (u) · (10, 24) = 1.0371.
9
Figure 1. Model 5.1.
Similaly at the node m we solve
x2 (m)(1.1)2 + y21 (m)8 + y22 (m)23 = D(ω4 )
x2 (m)(1.1)2 + y21 (m)9 + y22 (m)20 = D(ω5 )
x2 (m)(1.1)2 + y21 (m)13 + y22 (m)18 = D(ω6 ),
which gives
(x2 (m), y2 (m)) = (259.2525, −10.1398, −9.9161),
V1Φ (m) = x2 (m)(1.1) + y2 (m) · (9, 19) = 5.5129,
and at the node d,
x2 (d)(1.1)2 + y21 (d)10 + y22 (d)25 = D(ω7 )
x2 (d)(1.1)2 + y21 (d)14 + y22 (d)17 = D(ω8 )
x2 (d)(1.1)2 + y21 (d)15 + y22 (d)20 = D(ω9 ),
which gives
(x2 (d), y2 (d)) = (−57.9619, 2.9860, 1.6706),
V1Φ (d)
= x2 (d)(1.1) + y2 (d) · (12, 17) = 0.4738.
Finally we can solve for the initial portfolio Φ1 = (x, y),
x(1.1) + y1 10 + y2 24 = V 1 (u),
x(1.1) + y1 9 + y2 19 = V 1 (m),
x(1.1) + y2 12 + y2 17 = V 1 (d),
10
with solution
(x1 , y1 ) = (29.9689, −2.0087, −0.4934).
To control this answer we can compute x1 + y1 · (10, 18) = 1.0008 ≈ 1, so this is probably correct.
Dynamic programming. The idea of dynamic programming is that the optimal trading strategy at
some t-node λ can be found if we know the optimal trading strategies at all the successor nodes of
λ, and the value of these optimal trading strategies for all values. Call this function Ut+1 (v, µ) for
all successor nodes µ. To find the optimal trading strategy at a t-node λ can solve the optimization
problem
maximize EP [Ut+1 (xt+1 Bt+1 + yt+1 St+1 )] ,
over all those xt+1 and yt+1 giving a value v at λ, i.e., v = xt+1 Bt + yt+1 St . Having done that,
we know the function Ut (v) at the node λ. Doing this for all t-nodes, we can repeat the process
for all (t − 1)-nodes, eventually finding U0 (v) for all values of v, along with the trading strategy
T
Φ(v) = {xt (v), yt (v)}t=0 .
To be more concrete, assume that xt Bt + yt St = v, then
v
+ yt+1 ∆S̄t .
xt+1 Bt+1 + yt+1 St+1 = Bt+1
Bt
The dynamic programming equation is then
v
+ yt+1 ∆S̄t
(10)
Ut (v) = max EP Ut+1 Bt+1
Ft ,
yt+1
Bt
where the maximum is taken over all Ft measurable yt+1 . Since Ut (v) will be Ft measurable, we
sometimes write Ut = Ut (v, λ) where λ is a t-node. At t = T , we have that UT (v, ω) = u(v) for
all scenarios ω.
If we let pλµ denote the conditional real world probability to reach a successor node µ given a
node λ, then we have
X
v
+ y∆S̄t+1 (µ)
,
pλµ Ut+1 Bt+1
(11)
Ut (v, λ) = max
y
Bt
µ∈succ(λ)
and yt+1 (λ) is the argument which maximizes the expression. The equation(s) for y is(are)
X
v
0
(12)
0=
pλµ Ut+1
Bt+1
+ y∆S̄t+1 (µ)
Bt+1 ∆S̄t+1 (µ).
Bt
µ∈succ(λ)
Example 8. Consider the following two-period model
ω ω1 ω2 ω3
S0 20 20 20
S1 40 40 10
S2 80 20 20
with one stock and binary branching:
ω4
20
.
10
5
For simplicity assume that the interest rate is zero, and that the utility function is u(v) = ln(v),
and that the real world probabilities are P(ωi ) = 1/4 for i = 1, 2, 3, 4.
At the node u = ω1 ↑ 1 the equation for U1 (v) is
U1 (v, u) = max
y
1
(ln(v + 40y) + ln(v − 20y)) .
2
Differentiating, we find that the maximum is for y = v/80 = y1 (u) and hence
9
U1 (v, u) = ln
v .
8
Similarly at the node ω3 ↑ 1 = d, we have
U1 (v, d) = max
y
1
(ln(v + 10y) + ln(v − 5y)) .
2
11
This gives y1 (d) = v/20 and
U1 (v, d) = ln
9
v .
8
At the root we have that U0 (v) is given by
1
9
U0 (v) = max (ln(v + 20y) + ln(v − 10y)) + ln( ).
y 2
8
This gives y1 = v/40 and
9
U0 (v) = ln(v) + 2 ln( ).
8