Macroeconomics Review Sheet: Laibson
Matthew Basilico
Spring, 2013
Course Outline:
1.
Discrete Time Methods
(a) Bellman Equation, Contraction Mapping Theorem, Blackwell's Sucient Conditions, Numerical Methods
i. Applications to growth, search,
2.
consumption, asset pricing
Continuous Time Methods
(a) Bellman Equation, Brownian Motion, Ito Proccess, Ito's Lemma
i. Application to search, consumption, price-setting,
investment, I.O., asset-pricing
Lecture 1
Sequence Problem
What is the sequence problem?
Find
v(x0 )
such that
v (x0 ) =
subject to
sup
∞
X
{xt+1 }∞
t=0 t=0
xt+1 ∈ Γ(x)
Variable denitions
• xt
is the state vector at date
• F (xt , xt+1 )
t
is the ow payo at date
function
F
t
is stationary
a.k.a. the ow utility function
• δt
δ t F (xt , xt+1 )
is the expontential discount function
1
with
x0
given
Discount Variable Denitions
What are the denitions of the dierent discount variables?
• δt
is the exponential discount function
• δ
is referrred to as the exponential discount factor
• ρ
is the discount rate
which is the rate of decline of the discount function, so
∗ ρ ≡ − ln δ = −
·
so
dδ t
dt
δt
e−ρ = δ
Bellman Equation
What is the Bellman Equation? What are its components?
Bellman equation expresses the value function as a combination of a ow payo and a discounted
continuation payo
v(x) =
sup
{F (x, x+1 ) + δv(x+1 )}
x+1 ∈Γ(x)
Components:
•
Flow payo is
•
Current value function is
•
Equation holds for all (feasible) values of
• v(·)
F (x, x+1 )
v(x).
Continuation value function is
v(x+1 )
x
is called the solution to the Bellman Equation
Any old function won't solve it
Policy Function
What is a policy function? What is an optimal policy?
A policy is a mapping from
x
to the action space (which is equivalent to the choice variables)
An optimal policy achieves payo
v(x)
for all feasible
x
Relationship between Bellman and Sequence Problem
A solution to the Bellman will also be a solution to the sequence problem, and vice versa.
1. A solution to the Sequence Problem is a solution to the Bellman
2
2. A solution to the Bellman Equation is also a solution to the sequence problem
Notation with Sequence Problem and Bellman example: Optimal Growth with CobbDouglas Technology
How do we write optimal growth using log utility and Cobb-Douglas technology?
How can this be
translated into sequence problem and Bellman Equation notation?
Optimal growth:
sup
{ct }∞
t=0
subject to the constraints
∞
X
δ t ln(ct )
t=0
c, k ≥ 0, k α = c + k+1 ,
and
k0
given
Optimal growth in Sequence Problem notation: [∞- horizon]
v(k0 ) =
such that
sup
∞
X
{kt+1 }∞
t=0
δ t ln(ktα − kt+1 )
t=0
kt+1 ∈ [0, k α ] ≡ Γ(k)
and
k0 given
Optimal growth in Bellman Equation notation: [2-period]
v(k) =
sup
{ln (k α − k+1 ) + δv(k+1 )}
∀k
k+1 ∈[0,kα ]
Methods for Solving the Bellman Equation
What are the 3 methods for solving the Bellman Equation?
1. Guess a solution
2. Iterate a functional operator analytically (This is really just for illustration)
3. Iterate a functional operator numerically (This is the way iterative methods are used in most
cases)
3
FOC and Envelope
What are the FOC and envelope conditions?
First order condtion (FOC):
Dierentiate with respect to choice variable
0=
∂F (x, x+1 )
+ δv 0 (x+1 )
∂x+1
Envelope Theorem:
Dierentiate with respect to state variable
∂F (x, x+1 )
∂x
v 0 (x) =
Optimal Stopping using G&C
How do we approach optimal stopping using Guess and Check?
1. Write the Bellman Equation
2. Propose Threshold Rule as Policy Function
3. Use Policy Function to Write Value Function [Bellman Guess]
4. Rene Value Function using continuity of
5. Find value of threshold
x∗
v(·)
so that Value Function solves Bellman (using indierence)
Expanded:
Agent draws an oer
x
from a uniform distribution with support in the unit interval
1. Write the Bellman Equation
v(x) = max {x, δEv(x+1 )}
2. Propose Threshold Rule as Policy Function
(a) Stationary threshold rule, with threshold
Accept
Reject
x∗ :
if
if
x ≥ x∗
x < x∗
3. Use Policy Function to Write Value Function [Bellman Guess]
(a) Stationary threshold rule implies that there exists some constant
v(x) =
4. Rene Value Function using continuity of
x
v
if
if
x ≥ x∗
x < x∗
v(·)
v = x∗
x ≥ x∗
x < x∗
(a) By continuity of the Value Function, it follows that
v(x) =
x
x∗
4
if
if
v
such that
x∗
5. Find value of threshold
so that Value Function solves Bellman
(a) Use indierence condition
v(x∗ ) = x∗ = δEv(x+1 )
ˆ
so
ˆ
x=x∗
∗
∗
x =δ
x=1
1
1
xf (x)dx = δ (x∗ )2 + δ
2
2
p
x∗ = δ −1 1 − 1 − δ 2
x f (x)dx + δ
x=x∗
x=0
Giving solution
Lecture 2
Bellman Operator
What is the Bellman operator? What are its properties as a functional operator?
Bellman operator
B,
operating on a function
(Bw) (x) ≡
sup
w, is dened
{F (x, x+1 ) + δw(x+1 )} ∀x
x+1 ∈Γ(x)
Note: The Bellman operator is a contraction mapping that can be used to iterate until convergence to
a xed point, a function that is a solution to the Bellman Equation.
Bw on the LHS frees the RHS continuation value function w from
sup Bw(x) 6= w(x), then keep getting a better (closer) function
Bw(x) = w(x).
Having
being the same function.
Hence if
to the xed point where
Notation: When you iterate, RHS value function lags (one period). When you are not iterating, it
doesn't lag.
Properties as a functional operator
•
Denition is expressed pointwise - for one value of
•
Operator
B
maps function
Hence operator
B
w
to new function
x
- but applies to all values of
x
Bw
is a functional operator since it maps functions
Properties at solution to Bellman Equation
•
If
•
Function
v
is a solution to the Bellman Equation, then
v
is a xed point of
B (B
maps
v
to
Bv = v
(that is,
(Bv)(x) = B(x) ∀x)
v)
Contraction Mapping
Why does
Bnw
converge as
n → ∞?
What is a contraction mapping? What is the contraction mapping
theorem?
The reason
Bnw
Denition : Let
converges as
(S, d)
n→∞
is that
B
is a contraction mapping.
B : S → S be a function mapping S onto itself. B
δ ∈ (0, 1), d (Bf, Bg) ≤ δd(f, g) for any two functions f and g .
be a metric space and
contraction mapping if for some
5
is a
•
Intuition:
B
is a contraction mapping if operating
closer together. (Bf and
•
Bg
B
on any two functions moves them strictly
are strictly closer together than
f
and
g)
A metric (distance function) is just a way of representing the distance between two functions
(e.g. maximum pointwise gap between two functions)
Contraction Mapping Theorem:
If
(S, d)
1.
B
is a complete metric space and
has exactly one xed point
2. for any
3.
B n v0
B: S→S
is a contraction mapping, then:
v∈S
v0 ∈ S , lim B n v0 = v
has an exponential convergence rate at least as great as
− ln δ
Blackwell's Theorem
What is Blackwell's Theorem? How is it proved?
Blackwell's Sucient Conditions -
these are sucienct (but not necessary) for an operator to
be a contraction mapping
Let
Let
X ⊆ Rl and let C(X) be a space of bounded functions f : X → R,
B : C(X) → C(X) be an operator satisfying:
1.
(Monotonicity) if
2.
(Discounting) there exists some
f, g ∈ C(X)
and
f (x) ≤ g(x)∀x ∈ X ,
δ ∈ (0, 1)
Then
B
is a contraction with modulus
a
is a constant, and
f +a
∀f ∈ C(X), a ≥ 0, x ∈ X
δ.
is the function generated by adding a constant to the function
Proof:
For any
f, g ∈ C(X)
we have
(Bf )(x) ≤ (Bg)(x)∀x ∈ X
such that
[B(f + a)] (x) ≤ (Bf )(x) + δa
[Note that
then
with the sup-metric.
f ≤ g + d(f, g)
Properties 1 & 2 of the conditions imply
Bf ≤ B (g + d(f, g)) ≤ Bg + δd(f, g)
Bg ≤ B (f + d(f, g)) ≤ Bf + δd(f, g)
Combining the rst and last terms:
Bf − Bg ≤ δd(f, g)
Bg − Bf ≤ δd(f, g)
|(Bf )(x) − (Bg)(x)| ≤ δd(f, g) ∀x
sup |(Bf )(x) − (Bg)(x)| ≤ δd(f, g)
x
d(Bf, Bg) ≤ δd(f, g)
6
f]
Checking Blackwell's Conditions
Ex 4.1: Check Blackwell conditions for a Bellman operator in a consumption problem
Consumption problem with stochastic asset returns, stocastic labor income, and a liquidity constraint
(Bf )(x) = sup
n
o
u(c) + δEf R̃+1 (x − c) + ỹ+1
∀x
c∈[0,x]
1. Monotonicity:
f (x) ≤ g(x)∀x. Suppose c∗f is the optimal policy when the continuation value function is f .
n
o
(Bf )(x) = sup u(c) + δEf R̃+1 (x − c) + ỹ+1
= u(c∗f ) + δEf R̃+1 (x − c∗f ) + ỹ+1
Assume
c∈[0,x]
n
o
= (Bg)(x)
≤ u(c∗f ) + δEg R̃+1 (x − c∗f ) + ỹ+1 ≤ sup u(c) + δEg R̃+1 (x − c) + ỹ+1
c∈[0,x]
[Note (in top line) elimination of sup by using optimal policy
c∗f .
In bottom line, can use any policy
to add in sup]
2. Discounting
Adding a constant
(∆)
to an optimization problem does not eect optimal choice, so:
[B(f + ∆)] (x) = sup
n
h io
u(c) + δE f R̃+1 (x − c) + ỹ+1 + ∆
c∈[0,x]
n
o
+ δ∆ = (Bf )(x) + δ∆
= sup u(c) + δEf R̃+1 (x − c) + ỹ+1
c∈[0,x]
Iteration Application: Search/Stopping
What is the general notation?
v0 (x) = 0
Taking lecture 1 example, can you iterate from an initial guess of
for 2 steps? How about proving convergence?
Notation: Iteration of the Bellman operator
B:
vn (x) = (B n v0 )(x) = B(B n−1 v0 )(x) = max x, δE(B n−1 v0 )(x)
Let
xn
xn
xn ≡ δE(B n−1 v0 )(x+1 ),
is sucient statistic for
vn (x) = (B n v0 )(x)
Bellman Operator for the Problem:
w
vn (x).
vn (x) = (B n v0 )(x)
which is the continuation payo for
is also the cuto threshold associated with
(Bw)(x) ≡ max {x, δEw(x+1 )}.
*Note there is no lag here in the
index since we are not iterating
Iterating Bellman Operator once on
v0 (x) = 0:
v1 (x) = (Bv0 )(x) = max {x, δEv0 (x+1 )} = max {x, 0} = x
Iterating again on
v1 (x) = x:
v2 (x) = (B 2 v0 )(x) = (Bv1 )(x) = max {x, δEv1 (x+1 )} = max {x, δEx+1 }
x2 = δEx+1 =
δ
2
7
v2 =
x2
x
x ≤ x2
x ≥ x2
if
if
Proof at limit:
We have:
(B n−1 v0 )(x) =
"ˆ
xn = δE(B
n−1
xn−1
x
x ≤ xn−1
x ≥ xn−1
if
if
ˆ
x=xn−1
x=0
We set
xn = xn−1
#
x=1
xn−1 f (x)dx +
v0 )(x) = δ
xf (x)dx =
x=xn−1
δ 2
xn−1 + 1
2
to conrm that this sequence converges to
p
lim xn = δ −1 1 − 1 − δ 2
n→∞
Lecture 3
Classical Consumption Model
1. Consumption; 2. Linearization of the Euler Equation; 3. Empirical tests without precautionary
savings eects
Classical Consumption
What is the classical consumption problem, in Sequence Problem and Bellman Equation notation?
1. Sequence Problem Representation
Find
v(x)
such that
v(x0 ) = sup E0
{ct }∞
0
(
subject to
Variables:
x
∞
X
δ t u(ct )
t=0
a static budget constraint for consumption:
a dynamic budget constraint for assets:
is vector of assets,
c
is consumption,
R
xt+1
C
ct ∈ Γ (x)
∈ ΓX xt , ct , R̃t+1 , ỹt+1 , . . .
is vector of nancial asset returns,
y
is vector of
labor income
Common Example:
x:
xt , ct , R̃t+1 , ỹt+1 , . . . ≡ R̃t+1 (xt − ct ) + ỹt+1 ; x0 = y0
The only asset is cash on hand, and consumption is constrained to lie between 0 and
ct ∈ ΓC (xt ) ≡ [0, xt ];
Assumptions:
ỹ
xt+1 ∈ ΓX
is exogenous and iid;
u
is concave;
8
limc↓0 u0 (c) = ∞
(so
c>0
as long as
)
x > 0)
2. Bellman Equation Representation
[It is more convenient to think about
c
as the choice variable. The state variable,
it is not directly chosen (rather a distribution for
v(xt ) =
xt+1
is chosen at time
x,
is stochastic, so
t)]
sup {u(ct ) + δEt v(xt+1 )} ∀x
ct ∈[0,xt ]
xt+1 = R̃t+1 (xt − ct ) + ỹt+1
x0 = y0
Euler Equation 1: Optimization
How is the Euler equation derived from the Bellman Equation of the consumption problem above using
optimization?
v(xt ) =
sup
n
o
u(ct ) + δEt v R̃t+1 (xt − ct ) + ỹt+1
∀x
ct ∈[0,xt ]
1. First Order Condition:
u0 (ct ) = δEt R̃t+1 v 0 (xt+1 )
u0 (ct ) ≥ δEt R̃t+1 v 0 (xt+1 )
h
F OCct : 0 = u0 (ct ) + δEt v 0 (xt+1 ) × (−R̃t+1 )
if
if
0 < ct < xt
ct = xt
i
(interior)
(boundary)
using chain rule
2. Envelope Theorem
v 0 (xt ) = u0 (ct )
h
h
Dierentiate with respect to state variable
∂ct
= u0 (ct )
xt : v 0 (x) = u0 (ct ) ∂x
t
i
i
xt+1 +ỹt+1
R̃t+1
Putting the FOC and Envelope Condition together, and moving the Envelope Condition forward one
Note:
xt+1 = R̃t+1 (xt − ct ) + ỹt+1 =⇒ ct R̃t+1 = R̃t+1 xt − xt+1 + ỹt+1 =⇒ ct = xt −
period, we get:
IEuler
Equation:
u0 (ct ) = δEt R̃t+1 u0 (ct+1 )
u0 (ct ) ≥ δEt R̃t+1 u0 (ct+1 )
if
if
0 < ct < xt
ct = xt
(interior)
(boundary)
Euler Equation 2: Perturbation
How is the Euler equation derived from the Bellman Equation of the consumption problem above using
perturbation?
[*Prefers that we know it this way*]
1. General Intuition
•
Cost of consuming one dollar less today?
9
Utility loss today
•
= u0 (ct )
Value of saving an extra dollar Discounted, expected utility gain of consuming
R̃t+1
more
dollars tomorrow:
Utility gain tomorrow
= δEt R̃t+1 u0 (ct+1 )
2. Perturbation
**Idea: at an optimum, there cannot be a perturbation that improves welfare of the agent.
If there is such a feasible pertubation, then we've proven that whatever we wrote down is not an
optimum.
Proof by contradiction: [Assume inequalities, and show they produce contradictions so cannot hold]
1. Suppose
[If
0
u (ct )
u0 (ct ) < δEt R̃t+1 u0 (ct+1 )
.
is less than the discounted value of a dollar saved, then what should I do? Save more,
consume less.]
(a) Then we can reduce
ct
by
ε
and raise
ct+1
by
R̃t+1 · ε
to generate a net utility gain :
h
i
0 < δEt R̃t+1 u0 (ct+1 ) − u0 (ct ) · ε
=⇒ this
perturbation raises welfare [Q? - dening welfare here]
[Note the expression is from simply moving
u0 (ct )
to RHS from above and multiplying by
ε]
(a) If this perturbation is possible that raises welfare, then can't be true that we started this
analysis at an optimum.
i. This perturbation is always possible along the equilibrium path.
ii. Hence, if this cannot be an optimum, we've shown that at an optimum, we must have:
u0 (ct ) ≥ δEt R̃t+1 u0 (ct+1 )
2. Suppose
u0 (ct ) > δEt R̃t+1 u0 (ct+1 )
(a) Then we can raise
i. [If
0
u (ct )
ct
by
ε
and reduce
ct+1
by
R̃t+1 · ε
to generate a net utility gain :
is greater than the value of a dollar saved, then what should I do? Save less,
consume more.]
h
i
ε · u0 (ct ) − δEt R̃t+1 u0 (ct+1 ) > 0
=⇒ this
perturbation raises welfare [Q? - dening welfare here]
(b) If this perturbation is possible that raises welfare, then can't be true that we started this
analysis at an optimum.
i. This perturbation is always possible along the equilibrium path, as long as
ct < x t
(liquidity constraint not binding).
ii. Hence, if this cannot be an optimum, we've shown that at an optimum, we must have:
u0 (ct ) ≤ δEt R̃t+1 u0 (ct+1 )
10
as long as
ct < xt
It follows that:
u0 (ct ) = δEt R̃t+1 u0 (xt+1 )
u0 (ct ) ≥ δEt R̃t+1 u0 (xt+1 )
if
if
0 < ct < xt
ct = xt
(interior)
(boundary)
Linearizing the Euler Equation
How do you linearize the Euler Equation?
Goal: Can we linearize this and make it operational without unrealistic assumptions?
Equation tells us what I should expect consumption growth
1. Assume
u
to do
is an isoelastic (i.e. CRRA) utility function
u(c) =
Note: limγ→1
2. Assume
∆ ln ct+1
Rt+1
cγ−1 −1
=ln c.
1−γ
c1−γ − 1
1−γ
Important special case.
is known at time
t
3. Rewrite the Euler Equation [using functional form]:
c−γ
= Et δRt+1 c−γ
t
t+1
4. [Algebra] Divide by
c−γ
t
γ
1 = Et δRt+1 c−γ
t+1 ct
5. [Algebra] Rearrange, using exp[ln]:
γ 1 = Et exp ln δRt+1 c−γ
t+1 ct
6. [Algebra] Distribute ln:
1 = Et exp [rt+1 − ρ + (−γ) ln ct+1 /ct ]
[Note: ln δ = −ρ; ln Rt+1 = rt+1 ]
7. [Algebra] Substitute notation for consumption growth:
1 = Et exp [rt+1 − ρ − γ∆ ln ct+1 ]
[Note: ln(ct+1 /ct ) = ln ct+1 − ln ct ≡ ∆ ln ct+1 ]
8. [Stats] Assume
∆ ln ct+1
is conditionally normally distributed. Apply expectation operator:
1 2
1 = exp Et rt+1 − ρ − γ∆ ln ct+1 + γ Vt ∆ ln ct+1
2
h
i
ã
Eã+ 21 V arã
Note: Ee = e
where ãis a random variable
2
Here we have: − γ∆ ln ct+1 ∼ N (−γ∆ ln ct+1 , γ Vt ∆ ln ct+1 )
11
9. [Algebra] Take ln:
1
0 = Et rt+1 − ρ − γ∆ ln ct+1 + γ 2 Vt ∆ ln ct+1
2
10. [Algebra] Divide by
γ,
rearrange:
(∗) ∆ ln ct+1 =
Terms of linearized Euler Equation
∆ ln ct+1 = consumption growth
Vt ∆ ln ct+1 =conditional variance
1
1
(Et rt+1 − ρ) + γVt ∆ ln ct+1
γ
2
(∗):
in consumption growth conditional on information known at time
t.
(i.e. conditional on being in year 49, what is the variance in consumption growth between 49 and 50)
It is also known as the precautionary savings term as seen in the regression analysis
∆ ln ct+1
Why is ∆ ln ct+1
consumption growth?
ct+1 − ct
ct+1
ct+1
ct+1 − ct
≈ ln 1 +
= ln 1 +
− 1 = ln
≡ ∆ ln ct+1
ct
ct
ct
ct
[the second step uses the log approximation,
∆x ≈ ln (1 + ∆x)]
Important Consumption Models
What are the two important consumption models?
In 1970s, began recognizing that what is interesting is change in consumption, as opposed to just levels
of consumption. Since then have stayed with this approach.
1.
Life Cycle Hypothesis (Modigliani & Brumberg, 1954; Friedman) = Eat the Pie
Problem
(a) Assumptions
i.
ii.
R̃t = R
δR = 1 [δ < 1; R > 1]
iii. Perfect capital markets, no moral hazard, so future labor income can be exchanged for
current capital
(b) Bellman Equation
v(x) = sup {u(c) + δEv(x+1 )} ∀x
c≤x
x+1 = R(x − c)
x0 = E
∞
X
t=0
12
R−t ỹt
[At date 0, sells all his labor income for a lump sum, discounted at rate
r.
This sum is his
only source of wealth now]
[Eat the Pie, except every piece he leaves grows by a factor
r]
(c) Euler Equation
i.
ii.
u0 (ct ) = δRu0 (ct+1 ) = 1 · u0 (ct+1 ) = u0 (ct+1 )
=⇒ consumption is constant in all periods
(d) Budget Constraint
∞
X
−t
R ct = E0
t=0
∞
X
R−t ỹt
t=0
[(Discounted sum of his consumption) must be equal to [the expectation at date 0 of (the
discounted sum of his labor income)]]
(e) Subsitute Euler Equation
∞
X
−t
R c0 = E0
t=0
∞
X
R−t ỹt
t=0
i. To give nal form: [Euler Equation + Budget Constraint]
c0 =
1
1−
R
E0
∞
X
R−t ỹt
h
Note:
P∞
t=0
R−t =
1
1
1− R
i
∀t
t=0
Intuition: Consumption is an annuity. Consumption is equal to interest rate scaling my total
net worth at date
1−
2.
1
R
0
is the annuity scaling term; approximately equal to the real net interest rate
Certainty Equivalence Model (Hall, 1978)
(a) Assumptions
i.
ii.
iii.
iv.
R̃t = R
δR = 1 [δ < 1; R > 1]
Can't sell claims to labor income
Quadratic utility : u(c) = αc − β2 c2
A. Note: This admits negative consumption, and does not imply that
B. [Hence it is possible to consume a negative amount]
(b) Bellman Equation
v(x) = sup {u(c) + δEv(x+1 )} ∀x
c
x+1 = R(x − c) + ỹ+1
x0 = y0
(c) Euler Equation
13
limc↓0 u0 (c) = ∞
ct = Et ct+1 = Et ct+n
u0 (c) = α − βc. u0 (ct ) = δEt R̃t+1 ; u0 (ct+1 ) = Et u0 (ct+1 ) [Since δR = 1]
=⇒ α − βct = Et [α − βct+1 ] = α − βEt ct+1 =⇒ −βct = −βEt ct+1
Et ct+1
So ct = Et ct+1 = Et ct+n implies consumption is a random walk :
i. Euler becomes:
ii.
=⇒
ct =
ct+1 = ct + εt+1
iii.
So ∆ct+1 can not be predicted by any information available at time t
A. If I know
ct ,
no other economic variable should have predictive power at
ct+1 ;
any
other information should be orthogonal (d) Budget Constraint
∞
X
R−t ct ≤
t=0
[Sum from t to
∞
∞
X
R−t ỹt
t=0
of discounted consumption is
≤the
discounted value of stochastic labor
income]
i. Budget Constraint at time
t:
∞
X
R
−s
ct+s = xt +
s=0
∞
X
R−s ỹt+s
s=1
[Discounted sum of remaining consumption = cash on hand (at date t) and the discounted sum of remaining labor income]
ii. Now applying expectation: [if it's true on every path, it's true in expectation ]
Et
∞
X
R−s ct+s = xt + Et
s=0
(e) Subsitute Euler Equation
∞
X
R−s ỹt+s
s=1
[ct = Et ct+s ]
∞
X
R−s ct = xt + Et
s=0
∞
X
R−s ỹt+s
s=1
h
i. To give nal form: [Euler Equation + Budget Constraint] Note:
!
∞
X
1
xt + Et
R−s ỹt+s
ct = 1 −
R
s=1
P∞
−t
=
t=0 R
1
1
1− R
i
∀t
Intuition: Just like in Modigliani, consumption is equal to interest rate scaling my total net
worth at date
t
i. (but get there just by assuming a quadratic utility function)
1 − R1 is the annuity scaling term (approximately equal
P∞
(xt + Et s=1 R−s ỹt+s ) is total net worth at date t
14
to the real net interest rate)
Empirical Tests of Linearized Euler Equation: Without Precautionary Savings Eects
Want to prove that consumption growth is unpredictable between
t
and
t+1
A. Testing Linearized Euler Equation: Without Precautionary Savings Eects
1. Start with: Linearized Euler Equation, in regression form:
∆ ln ct+1 =
[where
εt+1
1
1
(Et rt+1 − ρ) + γVt ∆ ln ct+1 + εt+1
γ
2
is orthogonal to any information known at time
t.]
2. Assume (counterfactually) precautionary savings is constant
(a) Euler equation reduces to:
∆ ln ct+1 = constant + γ1 Et rt+1 + εt+1
[Note: when we replace precautionary term with a constant, we are eectively ignoring its eect
(since it is no longer separately identied from the other constant term
ρ
γ )]
B. Estimating the Linearized Euler Equation: 2 Goals and Approaches
1. Goal 1: Estimate
(a)
1
γ
1
γ is EIS, the elasticity of intertemporal substitution [for this model, EIS is the inverse of
1
the CRRA; ∴ in other models EIS not
γ]
(b) Estimate using
∂∆ ln ct+1
∂Et rt+1 , with equation
2. Goal 2: Test the orthagonality restriction
∆ ln ct+1 = constant + γ1 Et rt+1 + εt+1
Ωt ⊥εt+1
(a) This means, test the restriction that information available at time
sumption growth in the following regression [new
∆ ln ct+1 = constant +
(a) For example, does the date
t
βXt
t
does not predict con-
term, for any variable]:
1
Et rt+1 + βXt + εt+1
γ
expectation of income growth,
Et ∆ ln Yt+1
predict date
t+1
consumption growth [for Shea, etc test below]?
i. [Model is trying to say that an expectation of rising incomei.e.
marketdoesn't translate into a rise in consumption.
∆ ln ct+1 = constant +
α
being hired on job
should be 0.]
1
Et rt+1 + αEt ∆ ln Yt+1 + εt+1
γ
C. Summary of Results:
1.
1
γ
∈ [0, 0.2]
(Hall, 1988)
(a) EIS is very smallpeople are fairly unresponsive to expected movements in the interest rate;
this doesn't seem to be a big driver of consumption growth
i. the parameter that is scaling the expected value of the interest rate is estimated to
be close to 0
15
(b) But estimate may be messed up once you add in liquidity contraints. Someone who is very
liquidity constrained
→
EIS of 0.
(c) So what exactly are we learning when we see American consumers are not responsive to
the interest rate? [Is it just about liquidity constraints, or does it mean that even in their
absence, something deep about responsiveness to interest rate]
2.
α ∈ [0.1, 0.8]
(a)
(b)
=⇒
t+1
=⇒
(Campell and Mankiw 1989, Shea 1995)
expected income growth (predicted at date
t)
predicts consupmtion growth at date
the assumptions (1) the Euler Equation is true, (2) the utility function is CRRA, (3)
the linearization is accurate, and (4)
Vt ∆ ln ct+1
is constant, are jointly rejected
(c) Theories on why does expected income predict consumption growth:
i. Leisure and consumption expenditure are substitutes
ii. Work-based expenses
iii. Households support lots of dependents in mid-life when income is highest
iv. Houesholds are liquidity-constrained and impatient
v. Some consumers use rules of thumb:
cit = Yit
vi. Welfare costs of smoothing are second-order
Lecture 4
Precautionary Savings and Liquidity Contraints
1. Precautionary Savings Motives; 2. Liquidity Constraints; 3. Application: Numerical solution of
problem with liquidity constraints; 4. Comparison to eat-the-pie problem
Precautionary Motives
Why do people save more in response to increased uncertainty?
1. Uncertainty and the Euler Equation
γ
1
(Et rt+1 − ρ) + Vt ∆ ln ct+1
γ
2
= Et [∆ ln ct+1 − Et ∆ ln ct+1 ]2
Et ∆ ln ct+1 =
where
Vt ∆ct+1
(a) Increase in economic uncertainty raises
Vt ∆ct+1 ,
raising
Et ∆ ln ct+1
i. Note: this is not a general result (i.e. doesn't apply to quadratic)
2. Key reason is the convexity of marginal utility
(a) An increase in uncertainty raises the expected value of marginal utility
i. Convexity of marginal utility curve raises marginal utility in expectation in the next
period
A. Euler equation (for CRRA):
u0 (ct ) 6= u0 (Et (ct+1 ))
16
but rather
u0 (ct ) = Et u0 (ct+1 )
B. (Since by Jensen's inequality
=⇒
Et (ct+1 )
ct
> 1)
(b) This increase the motive to save
i.
=⇒
referred to as the precautionary savings eect
3. Denition: Precautionary Savings
(a) Precautionary saving is the reduction in consumption due to the fact that future labor
income is uncertain instead of being xed at its mean value.
Liquidity Constraints
Buer Stock Models
What are the two key assumptions of the buer stock models? Qualitatively, what are some of their
predictions
Since the 1990s, consumption models have emphasized the role of liquidity constraints (Zeldes, Caroll,
Deaton).
1. Consumers face a borrowing limit: e.g.
ct ≤ x t
(a) This matters whether or not it actually binds in equilibrium (since desire to preserve buer
stock will aect marginal utilty as you get close to full-borrowing level)
2. Consumers are impatient:
•
ρ>r
Predictions
Consumers accumulate a small stock of assets to buer transitory income shocks
Consumption weakly tracks income at high frequencies (even predictable income)
Consumption strongly tracks income at low frequencies (even predictable income)
17
Eat the Pie Problem
A model in which the consumer can securitize her income stream. In this model, labor can be transformed into a bond.
1. Assumptions
(a) If consumers have exogenous idiosyncratic labor income risk, then there is no risk premium
and consumers can sell their labor income for
W0 = E0
∞
X
R−t yt
t=0
(b) Budget constraint
W+1 = R(W − c)
2. Bellman Equation
v(W ) = sup {u(c) + δEv(R(W − c))} ∀x
c∈[0,W ]
3. Solve Bellman: Guess Solution Form
(
v(W ) =
1−γ
ψ W1−γ
φ + ψ ln W
γ ∈ [0, ∞], γ 6= 1
γ=1
if
if
4. Conrm that solution works (problem set)
5. Derive optimal policy rule (problem set)
1
c = ψ− γ W
1
1
ψ − γ = 1 − (δR1−γ ) γ
Lecture 5
Non-stationary dynamic programming
18
)
Non-Stationary Dynamic Programming
What type of problems does non-stationary dynamic programming solve? What is dierent about the
approach generally?
Non-Stationary Problems:
•
So far we have assumed problem is stationary - that is, value function does not depend on time,
only on the state variable
•
Now we apply to nite horizon problems
E.g. Born at
t = 1,
terminate at
t = T = 40
backwards induction
We index the value function, since each value function is now date-specic
Two changes for nite horizon problems:
1.
• vt (x)
•
E.g.,
represents value function for period
vt (x) = Et
PT
s=t
t
δ s−t u(cs )
2. We don't use an arbitrary starting function for the Bellman operator
•
Instead, iterate the Bellman operator on
vT (x),
where
T
is the last period in the problem
In most cases, the last period is easy to calculate, (i.e. end of life with no bequest
vT (x) = u(x))
motive:
Backward Induction
How does backward induction proceed?
•
Since you begin in the nal period, the RHS equation is one period ahead of the LHS side
equation (opposite of previous notation with operator)
Each iteration takes on more step back in time
•
Concretely, we have the Bellman Equation
vt−1 (x) = sup {u(c) + δEt vt (R(x − c) + ỹ)} ∀x
c∈[0,x]
•
*To generate vt−1 (x), apply the Bellman Operator
vt−1 (x) = (Bvt )(x) = sup {u(c) + δEt vt (R(x − c) + ỹ)} ∀x
c∈[0,x]
(As noted above, generally start at termination date
•
Generally we have:
vT −n (x) = (B n vT )(x)
19
T
and work backwards)
Lecture 6
Hyperbolic Discounting
Context:
For
β < 1, Ut = u(ct ) + β δu(ct+1 ) + δ 2 u(ct+2 ) + δ 3 u(ct+3 ) + . . .
Hyperbolic Bellman Equations
Bellman Equation in set up in the following way. Dene three equations where:
V is the continuation value function
W is the current value function
C is the consumption function
1.
V (x) = U (C(x)) + δE[V (R(x − C(x)) + y)]
2.
W (x) = U (C(x)) + βδE[V (R(x − C(x)) + y)]
3.
C(x) = arg maxc U (c) + βδE[V (R(x − c) + y)]
Three further identities:
•
Identity linking V and W
βV (x) = W (x) − (1 − β)U (C(x))
•
Envelope Theorem
W 0 (x) = U 0 (C(x))
•
FOC
U 0 (C(x)) = RβδE [V 0 (R(x − C(x)) + y)]
Hyperbolic Euler Equation
How do we derive an Euler equation for the hyperbolic scenario?
What is the intuition behind the
equation?
1. We have the FOC
u0 (ct ) = RβδEt [V 0 (xt+1 )]
2. Use the Identity linking
V
and
W
to subsitute for
V0
dCt+1
= RδEt W 0 (xt+1 ) − (1 − β)u0 (ct+1 )
dxt+1
3. Use envelope theorem
20
(a) [substitute
u0 (xt+1 )
for
W 0 (ct+1 )]
dCt+1
0
0
= RδEt u (xt+1 ) − (1 − β)u (ct+1 )
dxt+1
4. Distribute
dCt+1
dCt+1
= REt βδ
+δ 1−
u0 (ct+1 )
dxt+1
dxt+1
Interpretation of Hyperbolic Euler Equation
•
•
First, note that
(∗)
(∗)
dCt+1
dxt+1 is the marginal propensity to consume (MPC).
We then see that the main dierence in this Euler equation is that it is a
discount factors
weighted average of


 

dCt+1 
dCt+1
 u0 (ct+1 )
+δ
1
−
βδ
= REt 


dx
dx
t+1
t+1
 | {z }
{z
}
|
short run
•
long run
More cash on hand will shift to to the long-run term (less hyperbolic discounting)
Sophisticated vs. Naive Agents
[From Section Notes]
•
Sophisticated Agents
The above analysis has assumed sophisticated agents
A sophisticated agent knows that in the future, he will continue to be quasi-hyperbolic
(that is, he knows that he will continue to choose future consumption in such a way that
he discounts by
βδ
between the current and next period, but by
δ
between future periods.)
This is reected in the specication of the consumption policy function (as above)
•
Naive Agents
Doesn't realize he will discount in a quasi-hyperbolic way in future periods
He wrongly believes that, starting from the next period, he will discount by
δ
between all
periods.
∗
(Another way of saying: he wrongly believes that he will be willing to commit to the
path that he currently sets)
He will decide today, believing that he will choose from tomorrow forward by the exponential discount function. (
e
superscript); however, when he reaches the next date, he again
reoptimizes:
21
1. Dene
(a) Continuation value function, as given by standard exponential discounting Bellman
equation:
e
Vt+1
(x) =
max
e
ct+1 ∈[0,x]
e
u cet+1 + δEt+1 Vt+2
R(x − cet+1 ) + ỹ
(b) Current value function, as given by the hyperbolic equation:
Wtn (x) = emax
ct ∈[0,x]
e
u (cn ) + δEt Vt+1
(R(x − cet ) + ỹ)
2. Since he is not able to commit to the exponential continuation function, realized consumption path will be given:
(a) Realized consumption path:
(b) Solution at each time
t
T
{cnτ (xτ )}τ =t
is characterized by
u0 (cet+1 ) = δREt+1 u0 cet+2
0
u0 (cnt ) = βδREt V e (xt+1 ) = βδREt u0 cet+1
Lecture 7
Asset Pricing
1. Equity Premium Puzzle; 2. Calibration of Risk Aversion; 3. Resolutions to the Equity Premium
Puzzle
Intuition
•
Classical economics says that you should take an arbitrarily small bet with any positive re-
as long as the payos of the bet are uncorrelated with your consumption path
turn...
(punchline)
•
Don't want to make a bet where you will give up resources in a low state of consumption (when
the marginal utility of consumption is higher) for resources in a high state of consumption (where
marginal utility of consumption is lower) [all because of curvature of the utility function]
•
Not variance-based risk, covariance based risk
Asset Pricing Equation
What is the asset pricing equation? How is it derived?
Asset Pricing Equation
General Form:
π ij = γσic − γσjc
22
where
π ij = ri − rj
(the gap in rate of returns between assets
iand j )
σic = Cov(σ i εi , ∆ ln c)
Risk-Free Asset:
Denition: we denote the risk free return:
When
i =equities
and
j =risk
Rtf .
Specically, we assume that at time
t − 1, Rtf
is known.
free asset, we have the simple relationship
π equity,f = γσequity,c
Intuition:
•
The equity premium
risk,
π equity,f ,
is equal to the amount of risk,
σequity,c ,
multiplied by the price of
γ.
Asset Notation and Statistics:
1. Allowing for many dierent types of assets:
1
2
i
I
Rt+1
, Rt+1
, . . . , Rt+1
, . . . , Rt+1
2. Assets have stochastic returns:
i
Rt+1
=e
εit+1 has unit variance
=⇒
2
i
rt+1
+σ i εit+1 − 21 [σ i ]
where
εit+1 has
unit variance
εit+1 is a normally distributed random variable:
x ∼ N (µ, σ 2 ) −→Ax + B ∼ N (Aµ + B, A2 σ 2 )
−→ E [exp (Ax + B)] = exp Aµ + B + 21 A2 σ 2
1
2
[Since x ∼ N (µ, σ )−→ E(exp(x)) = exp(E(x) + V ar(x));
2
1 2
= exp(µ + 2 σ )]
εit+1 ∼ N (0, 1)
so if standard normal:
i
i
i
σ i εit+1 + rt+1
− 12 [σ i ]2 ∼ N (σ i ∗ 0 + rt+1
− 12 [σ i ]2 , σ i2 ∗ 12 ) = N (rt+1
− 12 [σ i ]2 , σ i2 )
i
E [Rit ] = E exp(rt+1
+ σ i εit+1 − 21 [σ i ]2 ) which as we've seen is now exp(a random variable)
Hence:
so
we can use
i
i
= exp(mean + 12 V ar) = exp(rt+1
− 12 [σ i ]2 + 21 σ i2 ) = exp(rt+1
)
Finally we have that since for small
x: ln(1 + x) ≈ x
=⇒ 1 + x ≈ ex
i
∴ exp(rti ) ≈ 1 + rt+1
.
Derivation of Asset Pricing Equation:
π ij = γσic − γσjc
How is the Asset Pricing Equation derived?
Use Euler, Substitute, Take Expectation (Random Variable), Dierence Two Equations, Apply Covariance Formula
1. Start with the Euler Equation
i
u0 (ct ) = Et δRt+1
u0 (ct+1 )
2. Assume
u
is isoelastic (CRRA):
u(c) =
23
c1−γ − 1
1−γ
3. Substitute into the Euler equation
(a)
δ = exp(−ρ)
(b)
i
Rt+1
=e
(c)
0
2
i
rt+1
+σ i εit+1 − 12 [σ i ]
−γ
u (c) = c
where
εit+1 has
unit variance
(given CRRA)
i
c−γ = Et δRt+1
c−γ
t+1
−γ
c
1 i 2 −γ
i
i i
σ
ct+1
= Et exp −ρ + rt+1 + σ εt+1 −
2
"
1 = Et
1 i 2
i
σ
exp −ρ + rt+1
+ σ i εit+1 −
2
ct+1
ct
−γ #
4. Re-write in terms of exp(·), using exp(ln)
1 i 2
ct+1
i
1 = Et exp −ρ + rt+1
+ σ i εit+1 −
σ
exp −γ ln
2
ct
1 i 2
i
σ − γ∆ ln (ct+1 )
1 = Et exp −ρ + rt+1
+ σ i εit+1 −
2
(a) Rearrange, and note






1 i 2
i
i i


1 = Et 
εt+1 − γ∆ ln (ct+1 )
exp −ρ + rt+1 − 2 σ + σ
{z
}
{z
} |
|
non-stochastic
random variable
5. Take Expectation, applying the formula for the expectation of a Random Variable
Sx ∼ N (Sµ, S 2 σ 2 ) −→ E [exp (Sx)] = exp Sµ + 21 S 2 σ 2
i i
For the random variable σ εt+1 − γ∆ ln (ct+1 )
i i
i i
1
That is, E σ εt+1 − γ∆ ln (ct+1 ) = exp −γEt [∆ ln (ct+1 )] + V σ εt+1 − γ∆ ln (ct+1 )
2
(a) That is, use:
i.
ii.
1 i 2
1 i i
i
1 = exp −ρ + rt+1 −
σ − γEt [∆ ln (ct+1 )] + V σ εt+1 − γ∆ ln (ct+1 )
2
2
6. Take Ln
i
0 = −ρ + rt+1
−
1 i 2
1 σ − γEt [∆ ln (ct+1 )] + V σ i εit+1 − γ∆ ln (ct+1 )
2
2
7. Dierence Euler Equation for assets
j
i
0 = rt+1
− rt+1
−
i
and
j,
and Re-write:
h
ii
2 i 1 h i i
1 h i 2
σ − σj
+
V σ εt+1 − γ∆ ln (ct+1 ) − V σ j εjt+1 − γ∆ ln (ct+1 )
2
2
24
(a) Note that
−ρ, γEt [∆ ln (ct+1 )]
drop out of the equation
(b) Re-write with interest rate dierence on LHS:
j
i
=
rt+1
− rt+1
ii
h
2 i 1 h i i
1 h i 2
−
V σ εt+1 − γ∆ ln (ct+1 ) − V σ j εjt+1 − γ∆ ln (ct+1 )
σ − σj
2
2
8. Apply formula for
(a)
V (A + B)
[V (A + B) = V (A) + V (B) + 2Cov(A, B)]
2Cov(A, −αB) = −2αCov(A, B)
V σ i εit+1 − γ∆ ln (ct+1 ) = V σ i εit+1 +V (−γ∆ ln (ct+1 ))+2Cov σ i εit+1 , γ∆ ln (ct+1 )
i. Also note
Note, in our case:
= σi
2
+ γ 2 V ∆ ln (ct+1 ) − 2γσic
j
i
rt+1
−rt+1
=
i h 2
ii
2 h 2
1 h i 2
σ − σ j − σ i + γ 2 V ∆ ln (ct+1 ) − 2γσic + σ j + γ 2 V ∆ ln (ct+1 ) − 2γσjc
2
Just cancel terms:
rti −rtj =
1
2
2
2
2
2
(σ i )\
− (σ i ) + (σ j )\
− (σ j ) + γ 2 V ∆ ln (ct+1\
) − γ 2 V ∆ ln (ct+1 ) + 2γσic − 2γσjc
To get out asset pricing equation:
j
i
π ij = rt+1
− rt+1
= γσic − γσjc
Equity Premium Puzzle
What is the equity premium puzzle?
We can rearrange the asset pricing equation for the risk free asset
γ=
π equity,f
σequity,c
Empirical Date: In the US post-war period
• π equity,f ≈ .06
• σequity,c ≈ .0003
γ=
.06
= 200
.0003
25
equity,f
π
= γσequity,c
to isolate
γ:
Lecture 8
Summary Equations
•
Brownian Motion; Discrete to Continuous
∆x ≡ x(t + ∆t) − x(t) =
+h
−h
p
q =1−p
with prob
with prob
t
∆t (p
E [x(t) − x(0)] = n ((p − q)h) =
− q)h (*)
t
2
V [x(t) − x(0)] = n 4pqh = ∆t 4pqh2 (*)
•
Calibration
√
h = σ ∆t
h
√ i
p = 12 1 + α
σ ∆t , ⇒ (p − q) =
E [x(t) − x(0)] =
V [x(t) − x(0)] =
t
∆t (p
α
σ
√
∆t
t
∆t
− q)h =
√
α
σ
∆t
√ σ ∆t =
αt
|{z}
linear drift
σ2 t
|{z}
linear variance
•
Weiner
1.
√
∆z = ε ∆t
where
ε ∼ N (0, 1),
2. Non overlapping increments of
•
∆z
Vertical movements
√
∝ ∆t
are independent
Ito:
dx = a(x, t)dt + b(x, t)dz ; dx = a(x, t)dt +
| {z }
drift
1. Random Walk:
dx = αdt + σdz
2. Geometric Random Walk:
variance
•
∆z ∼ N (0, ∆t)]
[also stated
Ito's Lemma:
b(x, t)dz
| {z }
standard deviation
[with drift
α
and variance
dx = αxdt + σxdz
σ2 ]
[with proportional drift
α
and proportional
σ2 ]
z(t)
is Wiener,
x(t)
is Ito with
dx = a(x, t)dt + b(x, t)dz .
Let
V = V (x, t),
then
2
1∂ V
2
dV = ∂V
dt + ∂V
∂x dx + 2 ∂x2 b(x, t) dt
h ∂t
i
∂V
1 ∂2V
2
= ∂V
dt +
∂t + ∂x a(x, t) + 2 ∂x2 b(x, t)
•
∂V
∂x
b(x, t)dz
Value Function as Ito Process:
Value Function is Ito Process
(V ) of an Ito Process (x) or a Weiner Process (z)
(need to check this)
∂V
∂V
1 ∂2V
∂V
2
dV = â(x, t)dt + b̂(x, t)dz =
+
a(x, t) +
b(x, t) dt +
b(x, t) dz
∂t
∂x
2 ∂x2
|∂x {z }
{z
}
|
b̂
â
•
Proof of Ito's Lemma:
dV =
∂V
∂t
dt +
1 ∂2V
2 ∂t2
(dt)2 +
∂V
∂x
dx +
1 ∂2V
2 ∂x2
(dx)2 +
2
∂2V
∂x2
dxdt + h.o.t.`
(dx) = b(x, t)2 (dz)2 + h.o.t. = b(x, t)2 dt + h.o.t.
26
Motivation: Discrete to Continuous Time
Brownian Motion Approximation with discrete intervals
Have continuous time world, but use
length goes to zero, that is
discrete time steps
in
continuous time
∆t.
Want to see what happens as the step
∆t → 0.
Process with jumps at discrete intervals:
•
At every
∆t
intervals, a process
x(t)
either goes up or down:
+h
−h
∆x ≡ x(t + ∆t) − x(t) =
•
with prob
with prob
p
q =1−p
Expectation of this process:
E(∆x) = ph + q(−h) = (p − q)h
•
Variance of this process
2
V (∆x) = E [∆x − E∆x]
2
2
2
We have: V (∆x) = E [∆x − E∆x] = E (∆x)
− [E∆x] = 4pqh2
2
2
(Since)E (∆x)
= ph2 + q (−h) = h2
Denition of Variance of a random variable:
Analysis of
•
in this process
Time span of length
•
x(t) − x(0)
t
=⇒ x(t) − x(0)
implies
is
n
k
x(t) − x(0)
is a binomial random variable
General form for probability that
Probability that
t
∆t steps in
n=
x(t) − x(0)
is a certain combination of
h
and
−h:
x(t) − x(0) = (k)(h) + (n − k)(−h)
k n−k
p q
n
k
(where
=
n!
k!(n−k)! )
• Expectation and Variance:
E [x(t) − x(0)] = n ((p − q)h) =
∗ (n
t
∆t (p
times the individual step
− q)h (*)
∆x.
Since these are iid steps, the sum is
n
times each
element in the sum)
V [x(t) − x(0)] = n 4pqh2 =
∗
(Since all steps are iid:
t
2
∆t 4pqh (*)
when we look at sum of random variables, where random
variables are all independent, the variance of the sum is the sum of the variances)
We have binomial random variable. As we chop into smaller and smaller time steps, will converge to
normal density.
∆x given a time step ∆t,
t, during which time there
x(t) − x(0), is a binomial random
So far: we've taken the discrete time process, discussed the individual steps
and then we've aggregated across many time steps to an interval of length
are
n steps n =
t
∆t , and we've observed that this random variable,
27
t
t
2
∆t (p − q)h and variance ∆t 4pqh . Our next job is to move from discrete number
of steps to smaller and smaller steps.
variable with mean
Calibrate h and p to give us properites we want as we take ∆t → 0: [Linear drift and variance]
•
Let:
√
h = σ ∆t (*)
h
√ i
p = 12 1 + α
σ ∆t
(*)
These Follow:
h
√ i
q = 1 − p = 12 1 − α
σ ∆t
√
(p − q) = α
σ ∆t
•
∆t to h, p, q ) We are trying to get to continuous time
random walk with drift. Random walk has variance that increases linearly with t forcasting
Where did this come from? (linking
horizon. Would like this property to emerge in continuous time.
t
Consider the variance (of the aggregate time random variable): ∆t
4pqh2 .
∆t is going to zero; p and q are numbers that are around .5 so can think of them as constants
4 is constant, t is the thing we'd like everything to be linear with respect to
√
∗ So given all this, h must be proportional to ∆t [then put in scaling parameter σ ]
t
· But then, what must (p−q) be for drift to be linear in t E [x(t) − x(0)] = ∆t
(p − q)h ?
√
(p − q) must also be proportional to ∆t, which (subject to ane transformations)
pins down p, q and (p − q).
in the limit,
∗
•
No other way to calibrate this for it to make sense as a continuous time random walk
Expectation and Variance
E [x(t) − x(0)] =
t α√ √ t
∆t σ ∆t =
(p − q)h =
∆t
∆t σ
αt
|{z}
linear drift
V [x(t) − x(0)] =
t
t
4pqh2 =
4
∆t
∆t
α 2 α 2 1
1−
∆t σ 2 ∆t = tσ 2 1 −
∆t −→
4
σ
σ
σ2 t
|{z}
(as ∆t → 0)
linear variance
Intuition as
•
•
∆t → 0
As take
∆t
At a point
to
0,
t,we
mathematically converging to a continuous time random walk.
have a
variable with mean
•
αt
binomial random variable. It is converging to a normal random
and var
σ 2 t.
[These are the four graphs simulating the process, which appear closer and closer to Brownian
motion as
∆t
gets smaller]
28
•
Hence this Random Walk, a continuous time stochastic process, is the limit case of a family of
stochastic processes
Properties of Limit Case (Random Walk):
√
1. Vertical movements proportional to
2.
[x(t) − x(0)] →D N (αt, σ 2 t),
∆t (not ∆t)
Binomial →D N ormal
√
1
√1
period > nh =
∆t σ ∆t = σ ∆t → ∞
since
3. Length of curve during 1 time
(a) (Since length will be greater than the number of step sizes, time the length of each step
size)
(b) Hence length of curve is innity over any nite period of time
4.
∆x
∆t
=
√
±σ ∆t
∆t
=
±σ
∆t
→ ±∞
(a) Time derivative,
E
dx
dt , doesn't exist
(b) So we can't talk about expectation of the derivative.
However, we can talk about the
following derivative-like objects (5,6):
5.
E(∆x)
∆t
=
(p−q)h
∆t
( ασ
=
(a) So we write
√
√
∆t)(σ ∆t)
∆t
E (dx) = αdt
i. That is, in a period
6.
V (∆x)
∆t
=
2
2
4( 14 ) 1−( α
σ ) ∆t σ ∆t
∆t
(a) So we write
•
=α
∆t,
you expect this process to increase by
α
→ σ2
V (dx) = σ 2 dt
Finally we see what the process is everywhere continuous, nowhere dierentiable
When we let
∆t
converge to zero, the limiting process is called a continuous time random walk with
(instantaneous) drift
α
and (instantaneous) variance
σ2 .
We generated this continuous-time stochastic process by building it up as a limit case.
We could have also just dened the process directly as opposed to converging to it:
Note that ”x” above becomes ”z” for following :
Wiener Process
Weiner processes: Family of processes described before with zero drift, and unit variance per unit time
Denition
If a continuous time stoachastic process
sponding to a time interval
1.
√
∆z = ε ∆t
where
∆t,
z(t)
is a Wiener Process, then any change in
satises the following conditions:
ε ∼ N (0, 1)
29
z, ∆z ,
corre-
∆z ∼ N (0, ∆t)]
√
piece, and
∆t is
(a) [often also stated
(b)
ε
is the gausian
2. Non overlapping increments of
(a) A.K.A. If
∆z
t1 ≤ t2 ≤ t3 ≤ t4
the scaling piece
are independent
then
E [(z(t2 ) − z(t1 )) (z(t4 ) − z(t3 ))] = 0
A process tting these two properties is a Weiner process. It is the same process converged to in the
rst part of the lecture above.
Properties
•
A Wiener process is a continuous time random walk with zero drift and unit variance
• z(t)
has the Markov property: the current value of the process is a sucient statistic for the
distribution of future values
History doesn't matter. All you need to know to predict in
y
periods is where you are right
now.
• z(t) ∼ N (z(0), t)
so the
variance of
z(t) rises linearly with t
(Zero drift case)
Note on terminology :
Wiener processes are a subset of Brownian motion (processes), which are a subset of Ito Processes.
Ito Process
Now we generalize. Let drift (which we'll think of as
t and state x.
lim∆t→0 E∆x
∆t equal
a
x
with arguments
and
t, a(x, t))
depend on
time
[Let
to a function
a
with those two arguments]
Similarly, generalize variance.
From Wiener to Ito Process
Let
z(t)
be a Wiener Process. Let
x(t)
be another continuous time stocastic process such that
lim∆t→0 E∆x
∆t = a (x, t)
2
∆x
lim∆t→0 V∆t
= b (x, t)
i.e.
i.e.
E (dx) = a (x, t) dt
2
V (dx) = b (x, t) dt
Note that the i.e. equations are formed by simply bringing the
We summarize these properties by writing the expression for an
dt
"drift"
"variance"
to the RHS.
Ito Process :
dx = a(x, t)dt + b(x, t)dz
•
This is not technically an equation, but a way of saying
•
Drift and Standard Deviation terms:
dx = a(x, t)dt +
| {z }
drift
30
b(x, t)dz
| {z }
standard deviation
We think of
dz
as increments of our Brownian motion from Weiner process/limit case of
discrete time process.
What is telling us about variance is whatever term is multiplying
as
z
x.
Weiner processcause a change in
change in
•
dz .
The process changes
changes, and what you want to know is how does a little change in
x
with scaling factor
Deterministic and Stochastic terms:
z this
background
In this case, an instantaneous change in
z
causes a
b.
dx =
a(x, t)dt
| {z }
deterministic
+ b(x, t)dz
| {z }
stochastic
Ito Process: 2 Key Examples
•
Random Walk
dx = αdt + σdz
Random Walk with drift
•
α
and variance
σ2
Geometric Random Walk
dx = αxdt + σxdz
Geometric Random Walk with proportional drift
∗
∗
α
and proportional variance
x
dx
σ2
Drift and variance are now dependent on the state variable
dx
Can also think of as
x
= αdt + σdz ,
so percent change in
is a random walk
**Note on Geometric Random Walk: If it starts above zero, if will never become negative
(or even become zero): even with strongly negative drift, given its proportionality to
x
the
process will be driven asymptotically to zero at the lowest.
Laibson: Asset returns are well approximated by Geometric Random Walk
Ito's Lemma
Motivation is to work with functions that take Ito Process as an argument
Ito's Lemma
Basically just a taylor expansion
z(t) be
V = V (x, t), then
Theorem: Let
Let
a Wiener Process. Let
dV =
x(t)
be an Ito Process with
∂V
1 ∂2V
∂V
dt +
dx +
b(x, t)2 dt
∂t
∂x
2 ∂x2
31
dx = a(x, t)dt + b(x, t)dz .
=
∂V
∂V
∂V
1 ∂2V
2
b(x,
t)
dt +
+
a(x, t) +
b(x, t)dz
∂t
∂x
2 ∂x2
∂x
[the second line simply follows from expanding
dx = a(x, t)dt + b(x, t)dz ]
Motivation: Work with Value Functions (which will be Ito Processes) that take Ito
Processes as Arguments
•
We are going to want to work with functions (value functionsthe solutions of Bellman equations)
that are going to take as an argument, an Ito Process
•
Motivation is to know what these functions look like, that take
x
as an argument, where
x
is an
Ito Process
More specically, we'd like to know what the Ito Process looks like that characterizes the
value function
∗
Value function
·
•
V.
V
is
2nd
Ito Process, which takes an Ito Process (x) as an argument
Not only want to talk about Ito Process
x,
but also about Ito Process
V
Since we are usually looking for a solution to a question: i.e. Oil Well
Price of oil is given by
The Ito Process
x
dx = a(x, t)dt + b(x, t)dz
x,
describes the path of
the price of oil (not the
oil well)
∗
e.g. price of oil is characterized by geometric Brownian motion:
V (x, t)
∗
is value of Oil Well, which depends on the price of oil
Want to characterize the value function of the oil well,
x
dx = αxdt + σxdz
and time
V (x, t),
where
x
follows an Ito
Process
·
·
Ito Process that characterizes value function over time:
But we will see that the dependence of
â
and
b̂
on
V
dV = â(V, x, t)dt+b̂(V, x, t)dz
will drop
We want to write the stochastic process which describes the evolution of
V:
dV = â(x, t)dt + b̂(x, t)dz
∗
∗
which is called the total dierential of
ˆ terms
Note that the hat
(â, b̂)
V
are what we are looking for in this stochastic process,
which are dierent than the terms in the Ito Process
•
(a, b)
Ito's Lemma gives the solution to this problem in the following form:
dV =
∂V
∂V
1 ∂2V
∂V
1 ∂2V
∂V
∂V
2
2
dt+
dx+
b(x,
t)
dt
=
+
a(x,
t)
+
b(x,
t)
dt+
b(x, t)dz
2
2
∂t
∂x
2 ∂x
∂t
∂x
2 ∂x
∂x
∂V
∂V
1 ∂2V
∂V
2
dV =
+
a(x, t) +
b(x, t) dt +
b(x, t) dz
∂t
∂x
2 ∂x2
|∂x {z }
|
{z
}
b̂
â
32
where:
Proof of Ito's Lemma
Proof: Using a Taylor Expansion
dV =
•
∂V
∂V
∂2V
1 ∂2V
1 ∂2V
2
2
(dt)
+
(dx)
+
dxdt + h.o.t.
dt +
dx
+
∂t
2 ∂t2
∂x
2 ∂x2
∂x2
Any term of order
3
(dt) 2
[This is because
tant than
•
Note that
or higher is small relative to terms of order
(∆t)2
is innitely less important than
3
(∆t) 2 ,
dt.
which is innitely less impor-
(∆t)]
2
(dz) = dt.
Since
∆z ∝
√
∆t,
recalling
∆z ≈ h
[there's a Law of Large Numbers
making this true, this is math PhD land and not something Laibson knows or expects us to
know]
Hence we can eliminate:
2
∗ (dt) = h.o.t.
∗ dxdt = a(x, t)(dt)2 + b(x, t)dzdt = h.o.t.
2
∗ (dx) = b(x, t)2 (dz)2 + h.o.t. = b(x, t)2 dt + h.o.t.
Combining these results we have:
∂V
1 ∂2V
∂V
dt +
dx +
b(x, t)2 dt
∂t
∂x
2 ∂x2
dz is a Wiener process that depends only on t through random motion
dx is a Ito process that depends on x and t through random motion
V (x, t) is a value function (and an Ito Process) that depends on a Ito process dx
and directly on
t
Intuition: Drift in V through Curvature of V :
•
Even if we assume
doesn't depend on
•
We still have
If
V
∗
a(x, t) = 0
t]
E(dV ) =
is concave,
V
1 ∂2V
2 ∂x2
[no drift in Ito Process] and assume
is expected to fall due to variation in
0
V (x) = ln x, hence V =
· dx = αxdt
+ σxdz
h
Ex:
•
=⇒
dt
[holding
x
xed,
V
1
x and
V
00
=
x
[convex
→
rise]
− x12
∂V
∂x
b(x, t)dz
Growth in V falls below α due to concavity of V
Intuition for proof : for Ito Processes,
of order
=0
b(x, t)2 dt 6= 0
i
∂V
1 ∂2V
2
dt +
· dV = ∂V
∂t + ∂x a(x, t) + 2 ∂x2 b(x, t)
h
i
2
· = 0 + x1 αx − 2x1 2 (σx) dt + x1 σxdz
· = α − 12 σ 2 dt + σdz
∂V
∂t
(dx)
2
behaves like
b(x, t)2 dt,
so the eect of concavity is
and can not be ignored when calculating the dierential of
V
Because of Jensen's inequality, if you are moving on independent variable (x) axis randomly,
expected value will go down
33
Lecture Summary
Talked about a continuous time process (Brownian motion, Weiner process or its generalization as
an Ito Process) that has these wonderful properties that is kind of like the continuous time analog of
things in discrete time are thought of as random walks. But it is even more general that this:
Has perverse property that it moves innitely far over unit time, but when you look at how it changes
over any discrete interval of time, it looks very natural.
Also learned how to study functions that take Ito Processes as arguments: key tool in these processes
is Ito's Lemma. It uses a 2nd order taylor expansion and throws out higher order terms. So we can
V
characterize drift in that function
very simply and powerfully, which we put to use in next lectures.
Lecture 9
Continuous Time Bellman Equation & Boundary Conditions
Continuous Time Bellman Equation
Final Form:
ρV (x, t)dt = max {w(u, x, t)dt + E [dV ]}
u
Inserting Ito's Lemma
∂V
1 ∂2V
∂V
2
+
a+
b(x, t) dt
ρV (x, t)dt = max w(u, x, t)dt +
u
∂t
∂x
2 ∂x2
Next 1.5 lectures about solving for
V
using this equation with Ito's Lemma. First, Merton. Then,
stopping problem.
•
Intuition
ρV (x, t)dt
| {z }
= max
u
required rate of return







w(u, x, t)dt
| {z }
instantaneous dividend
+
E [dV ]
| {z }




instantaneous capital gain



required rate of return = instantaneous return; instantaneous capital gain = instantaneous
change in the program.
•
Derivation
Towards Condensed Bellman
∗
Let
·
∗
Let
·
w(x, u, t) =
instantaneous payo function
Arguments:
x
0
is state variable,
0
x = x + ∆x and t = t + ∆t .
u
is control variable,
t
is time
We'll end up thinking about what happens as
Just write down Bellman equation as we did in discrete time
34
∆t → 0
n
o
−1
V (x, t) = max w(x, u, t)∆t + (1 + ρ∆t) EV (x0 , t0 )
u
V (x, t) = max
u




w(x, u, t)∆t
|
{z
}
payo ×time



ow
∗
3 simple steps algebra with
(1 + ρ∆t)
−1
+
step
(1 + ρ∆t) EV (x0 , t0 )
|
{z
}







discounted value of continuation
term:
(1 + ρ∆t) V (x, t) = max {(1 + ρ∆t) w(x, u, t)∆t + EV (x0 , t0 )}
u
ρ∆tV (x, t) = max {(1 + ρ∆t) w(x, u, t)∆t + EV (x0 , t0 ) − V (x, t)}
u
n
o
2
ρ∆tV (x, t) = max w(x, u, t)∆t + ρw(x, u, t) (∆t) + EV (x0 , t0 ) − V (x, t)
u
∗
Let
∆t → 0
2
(dt) = 0
. Terms of order
ρV (x, t)dt = max {w(x, u, t)dt + E [dV ]}
u
(*)
Substitute using Ito's Lemma
∗
Substitute for
·
E [dV ]
where
∂V
1 ∂2V
∂V
∂V
2
+
a(x, u, t) +
b(x, u, t) dt +
b(x, u, t)dz
dV =
∂t
∂x
2 ∂x2
∂x
∂V
since E
bdz = 0
∂x
∂V
∂V
1 ∂2V 2
E [dV ] =
+
a+
b
dt
∂t
∂x
2 ∂x2
∗
Substituting this expression into (*) we get
∂V
1 ∂2V
∂V
2
ρV (x, t)dt = max w(u, x, t)dt +
+
a+
b(x,
t)
dt
u
∂t
∂x
2 ∂x2
·
•
which is a partial dierential equation (in
x
and
t)
Interpretation
Sequence Problem:
ˆ
∞
e−ρt w (x(t), u(t), t) dt
t
Interpretation of Bellman (*) before substuting for Ito's Lemma:
ρV (x, t)dt
| {z }
required rate of return
= max
u







w(x, u, t)
| {z }
instantaneous dividend
35
+
E [dV ]
| {z }







instantaneous capital gain
∗
∗
∗
≡
note that instantaneous capital gain
instantaneous change in the program
And when we plug in Ito's Lemma
Instantaneous capital gain
=
∂V
∂V
1 ∂2V
b(x, t)2
+
a+
∂t
∂x
2 ∂x2
Application 1: Merton's Consumption
•
Merton's Consumption
Consumer has 2 assets
∗
∗
Equity: return
Invests share
•
r
r+π
Risk free: return
θ
with proportional variance
σ2
in assets, and consumes at rate c
Ito Process that characterizes dynamics of
x,
her cash on hand:
dx = [rx + πxθ − c] dt + θxσdz
Intuition
∗
∗
rxdt every period on
π (πxθdt). Consuming
assets.
equity premium
at rate c, so depletes assets at rate
a = [rx + πxθ − c]
aka •
θxσdz .
b = θxσ
and
equation of motion for the state variable Distinguishing Variables
x
is a stock (not in the sense of equity, but a quantity)
c
is a ow (ow means rate).
∗
∗
Need to make sure units match (usually use annual in economics)
i.e. spending $1000/48hrs is ~$180,000/year
·
∗ c
He can easily consume at a rate that is above his total stock of wealth
is bounded between
·
•
is getting an additional
Not between
0
0
and
and
x
∞
as it was in discrete time
Bellman Equation
∂V
1 ∂2V
2
ρV (x, t)dt = max w(u, x, t)dt +
[rx + πxθ − c] +
(θxσ) dt
c,θ
∂x
2 ∂x2
Note the
∗
cdt.
Standard Deviation term: cost to holding equities is volatility. Volatility scales with
the amount of assets in risky category:
So,
A fraction
θx
Drift term: earn
maximization over both c and θ, so two FOCs
In world of continuous time, these can be changed instantaneously.
36
·
(In principle, in every single instant, could be picking a new consumption level
and asset allocation level
·
c
θ)
It will turn out that in CRRA world, he will prefer a constant asset allocation
θ,
even though he is constantly changing his consumption
V
doesn't depend directly on
∗
t,
so rst term was removed
If someone was nitely lived, then function would depend on
t
dt can be removed since notationally redundant
Solving with CRRA utility
•
c1−γ
1−γ [Lecture]
It turns out that in this problem, the value function inherits similar properties to the utilty
function:
•
u(c) =
1−γ
V (x) = ψ x1−γ
If given utility function is CRRA
u(c) =
c1−γ
1−γ , then it can be proved that
This is just a scaled version of the utility function (by
1−γ
V (x) = ψ x1−γ
ψ)
This gives us single solution to Bellman equation
•
Solving for Policy Functions [given value function
•
Assume
1−γ
V (x) = ψ x1−γ
We need to know
The restriction
•
1−γ
ρψ
ψ, θ
and optimal consumption policy rule for
1−γ
•
Two FOCs,
θ
c
allows us to do this
, re-write Bellman plugging in for
u(c) =
c1−γ ∂V ∂ 2 V
1−γ , ∂x , ∂x2
n
h
i o
x1−γ
γ
dt = max u(c)dt + ψx−γ [(r + θπ)x − c] − ψx−γ−1 (θσx)2 dt
c,θ
1−γ
2
x1−γ
= max
ρψ
c,θ
1−γ
•
]
: nd values for policy functions that solve Bellman Equation
V (x) = ψ x1−γ
V (x) = ψ x1−γ
Taking
1−γ
V (x) = ψ x1−γ
and
h
i
γ
c1−γ
+ ψx−γ [(r + θπ)x − c] − ψx−γ−1 (θσx)2
1−γ
2
c:
γ
2
F OCθ : ψx−γ πx − 2
F OCc : u0 (c) = c−γ = ψx−γ
ψx−γ−1 θ(σx)2 = 0
Simplifying
1
c = ψ− γ x
θ=
π
γσ 2
Interpretation:
Nice result: Put more into equities the higher the equity premium
with higher RRA
γ
and higher variability of equities
π;
put less into equities
σ2
Why is variance of equities replacing familiar covariance term from Asset Pricing lecture?
In Merton's world, all the stocasticity is coming from equity returns.
37
•
Plugging back in, we get:
1
ψ− γ =
•
r+
+ 1 − γ1
ρ
γ
Special case:
So
γ=1
π2
2γσ 2
implies
M P C ' .05
c = ρx
(Marginal Propensity to Consume)
But we also get
π
γσ 2
θ=
=
0.06
1(0.16)2
= 2.34
Saying to borrow money so have assets of 2.34 in equity [and therefore 1.34 in liabilities]
We don't know what's wrong here. This model doesn't match up with what wise souls think you
should do.
•
Can we x by increasing
With
∗
γ?
Remember no one in classroom had
0.06
5(0.16)2
γ = 5, θ =
γ≥5
= 0.47
Recommends 47% allocation into equities
Typical retiree:
No liabilities; $600,000 of social security (bonds), $200,000 in house, less than $200,000 in 401(k)
401(k) usually 60% bonds and 40% stocks
Equity allocation is
8%
of total assets
Solving with Log utility
u(c) = ln(c) [PS5]
•
Given that guess for value function is
•
Set up is the same:
•
When substitute for
V (x) = ψ ln(x) + φ
n
h
ρV (x, t)dt = maxc,θ w(u, x, t)dt + ∂V
∂x [rx + πxθ − c] +
V (x)
and
−1
ρ [φ + ψ ln(x)] = max ln(c) + [(r + θπ)x − c] ψx
c,θ
•
1
− ψ(θσx)2 ψx−2
2
FOCs:
F OCc : c =
F OCθ : θ =
1
ψx
π
σ2
Plugging back into Bellman
ρφ + ρψ ln(x) = ψr − ln(ψ) − 1 +
•
i o
(θxσ)2 dt
u(c):
•
1 ∂2V
2 ∂x2
ψπ 2
2σ 2
+ ln(x)
Observation to solve for parameters
We can re-write the above with just constants on RHS:
ψπ 2
2σ 2
∗
ρψ ln(x) − ln(x) = ψr − ln(ψ) − 1 +
− ρφ
Given that the RHS is constant, this will only hold if
be increasing in
·
Hence,
x,
ρψ = 1
and
0 = ψr − ln(ψ) − 1 +
The Bellman equation is satised
∗ ρψ = 1
and
ρψ = 1.
[Otherwise LHS would
while RHS won't increase, a contradiction]
∀x
i
ρφ = ψr − ln(ψ) − 1 +
ψπ 2
2σ 2
38
ψπ 2
2σ 2
− ρφ.
This means:
Giving us
•
∗ ψ=
1
ρ
∗ φ=
1
ρ
r
ρ
π2
2ρσ 2
+ ln(ρ) − 1 +
Optimal Policy
Plugging back into FOCs
∗ c = ρx
and
θ=
π
σ2
General Continuous Time Problem
Most General Form
•
Equation of motion of the state variable
dx = a(x, u, t)dt + b(x, u, t)dz ,
Note
•
dx
depends on choice of control
u
Using Ito's Lemma, Continuous Time Bellman Equation
ρV (x, t) = w(x, u∗ , t) +
u∗ = u(x, t) =optimal
∂V
∂t
+
∂V
∂x
a(x, u∗ , t) +
1 ∂2V
2 ∂x2
b(x, u∗ , t)2
value of control variable
Properties: PDE and Boundary Conditions
•
Value function is a 2nd order PDE
• V
•
is the 'dependent' variable;
x
and
t
are the 'independent' variables
PDE will have continuum of solutions
Need structure from economics to nd the right solution to this equationthe solution the
makes economic sensefrom the innite number of solutions that make mathematical sense.
=⇒
These restrictions are called boundary conditions, which we need to solve PDE
=⇒
Need to use economic logic to derive boundary conditions
Name of the game in these problems is to use economic logic to nd right solution, and
eliminate all other solutions
(Any policy generates a solution, we're not looking for any solution that corresponds to any
policy, but for the solution that corresponds to the optimal policy)
•
[In Merton's problem, we exploited 'known' functional form of value function, which imposes
boundary conditions]
Two key examples of boundary conditions in economics
1. Terminal Condition
•
Suppose problem ends at date
T
39
i.e. have already agreed to sell oil well on date
•
Then we know that
Where
Ω(x, t)
V (x, T ) = Ω(x, T ) ∀x
T
[Note this is at date
T]
is (some known, exogenous) termination function. Note that it can be
state dependenti.e. change with the price of oil.
•
Can solve problem using techniques analogous to backward induction
•
This restriction gives enough structure to pin down solution to the PDE
2. Stationary
•
∞−horizon
problem
Value function doesn't depend on
t =⇒ becomes
an
ODE
1
ρV (x) = w(x, u∗ ) + a(x, u∗ )V 0 + b(x, u∗ )2 V 00
2
V0
•
Note the use of
•
We will use Value Matching and Smooth Pasting
since now only one argument for
V,
can drop the
∂
notation
Motivating Example for Value Matching and Smooth Pasting:
•
Consider the optimal stopping problem :
V (x, t) = max w(x, t)∆t + (1 + ρ∆t)−1 EV (x0 , t0 ), Ω(x, t)
•
where
Ω(x, t)
is the termination payo in the stopping problem
the solution is characterized by the stopping rule:
∗
∗
if
x > x∗ (t)
continue; if
x ≤ x∗ (t)
stop
Motivation: i.e. irreversibly closing a production facility
Assume that
dx = a(x, t)dt + b(x, t)dz
In continuation region, value function characterized using Ito's Lemma:
∗ ρV (x, t)dt = w(x, t)dt +
h
∂V
∂t
+
∂V
∂x
a(x, t) +
1 ∂2V
2 ∂x2
i
b(x, t)2 dt
Value Matching
•
Value Matching:
All continuous problems must have continuous value functions
V (x, t) = Ω(x, t)
for all
x, t
such that
x(t) ≤ x∗ (t)
(at boundary and to left of boundary)
40
A discontinuity is not permitted, otherwise one could get a better payo from not following
the policy
Intuition: an optimal value function has to be continuous at the stopping boundary.
∗
If there was a discontinuity, can't be optimal to quit (if
continued so far (if
See graph [remember
V
V
below
V
is above
Ω)
or to have
Ω)
on y axis,
x
on x-axis]
Smooth Pasting
Derivatives with respect to
x
must be equal at the boundary.
Using problem above:
•
Smooth Pasting:
Vx (x∗ (t), t) = Ωx (x∗ (t), t)
Intuition
∗ V
and
·
Ω
must have same slope at boundary.
Proof intuition:
if slopes are not equal at convex kink, then can show that the
policy of stopping at the boundary is dominated by the policy of continuing for
just another instant and then stopping. (It doesn't mean the latter is the optimal
policy, but shows that initial policy of stopping at the boundary is sub-optimal)
∗
∗
x∗ (t),
If value functions don't smooth paste at
then stopping at
x∗ (t)
can't be optimal:
Better to stop an instant earlier or later
·
If there is a (convex) kink at the boundary, then the gain from waiting is in
and the cost from waiting is in
∆t.
41
So there can't be a kink at the boundary.
√
∆t
∗
Think of graph [remember
V
on y axis,
x
on x-axis]
Will need a 3rd boundary condition from the following lecture
Lecture 10
Third Boundary Condition, Stopping Problem and ODE Strategies
Statement of Stopping Problem
•
Assume that a continuous time stochastic process
x(t)
is an Ito Process,
dx = adt + bdz
i.e.
•
x
is the price of a commodity produced by this rm
While in operation, rm has ow prot
`
w(x) = x
(Or
•
w(x) = x − c
if there is a cost of production)
Assume rm can costlessly (permanently) exit the industry and realize termination payo
Ω=0
•
•
Intuitively, rm will have a stationary stopping rule
if
x > x∗
continue
if
x ≤ x∗
stop (exit)
Draw the Value Function (Ω
=0
in below graph)
42
Characterizing Solution as an ODE
•
Overall Value Function
V (x) = max x∆t + (1 + ρ∆t)−1 EV (x0 ), 0
Where
•
and
ρ
is interest rate
In stopping region
•
x0 = x(t + ∆t),
V (x) = 0
In continuation region
General
ρV (x)dt = xdt + E(dV )
Since:
V (x) = x∆t + (1 + ρ∆t)−1 EV (x0 )
=⇒ (1 + ρ∆t)V (x) = (1 + ρ∆t)x∆t + EV (x0 )
=⇒ ρ∆tV (x) = (1 + ρ∆t)x∆t + EV (x0 ) − V (x).
Let
∆t → 0
and multiply out. Terms of order
Substitute for
∗
E(dV )
(dt)2 = 0.
using Ito's Lemma
2nd Order ODE:
ρV = x + aV 0 +
b2 00
V
2
Interpretation
∗
Value function must satisfy this dierential equation in the continuation region
·
We've derived a 2nd order ODE that restricts the form of the value function
V
in
the continuation region
·
2nd order because highest order derivative is 2nd order, ordinary because only
depends on one variable (x),
drift from jensen's inequality
drift in argument
ρV
|{z}
required return
∗
•
=
x
|{z}
dividend
z}|{
aV 0
+
|
z }| {
b2 00
V
2
+
capital gain =how
{z
V is
instantly drifting
}
b2 00
2 V is drift resulting from brownian motion'jiggling and jaggling': it is negative
if concave, and positive if convex (as in this case).
Term
Intuition:
x∗
almost always negative. [don't confuse with
get when stopping; but can decide on some policy of
x
V (x) = Ω = 0. This
x∗ ]
is the value you
for when to stop:
Why? Option value love the ability take the chance at future prots. Even if drift is
negative, will still stay in, since through Brownian motion may end up protable again.
•
And now, we have continuum of solutions to ODE, so need restrictions to get to single solution
43
Third Boundary Condition
Boundary Conditions
∗
*Three variables we need to pin down: two constants in 2nd order ODE, and free boundary (x )
•
Value Matching:
V (x∗ ) = 0
•
Smooth Pasting:
V 0 (x∗ ) = 0
•
Third Boundary Condition for large x
As
∗
x→∞
the option value of exiting goes to zero
It is so unlikely that prots would become negative (and if it happened, would likely
be very negatively discounted) that value of being able to shut down is close to zero
∗
x
As
gets very large, value function approaches that of the rm that does not have the
option to shut down (which would mean it would theoretically need to keep running at
major losses)
Hence
V
converges to the value associated with the policy of never exiting the industry
[equation derived later]
lim
x→∞ 1
ρ
∗
V (x)
=1
x + aρ
Denominator is value function of rm that can't shut down
·
[Or can think of it as
limx→∞ V (x) =
1
ρ
x+
a
ρ
]
We can also write:
lim V 0 (x) =
x→∞
∗
x,
Intuition: at large
·
·
1
ρ
how does value function change with one more unit of price
x?
Use sequence problem:
Having one more unit of price produces same exact sequence shifted up by one unit
all the way to innity. What is the additional value of a rm with price one unit
higher?
ˆ
ˆ
ˆ ∞
1
e−ρt x(t)dt − e−ρt (x0 )(t)dt =
e−ρt · 1 · dt =
ρ
0
{z
} |
{z
}
|
·
old oil well
·
new oil well
So if I raise price by 1, I raise value of rm by
Solving 2nd Order ODEs
Denitions
•
Goal: Solve for
•
Complete Equation
F
with independent variable
x
Generic 2nd order ODE:
44
1
ρ
F 00 (x) + A(x)F 0 (x) + B(x)F (x) = C(x)
•
Reduced Equation
C(x)
is replaced by
0
F 00 (x) + A(x)F 0 (x) + B(x)F (x) = 0
Theorem 4.1
•
Any solution
F̂ (x),
of the reduced equation can be expressed as a linear combination of any two
linearly independent solutions of the reduced equation,
F̂ (x) = C1 F1 (x) + C2 F2 (x)
F1
and
F2
F1 , F2 linearly
provided
independent
Note that two solutions are linearly independent if there do not exist constants
such that
A1 F1 (x) + A2 F2 (x) = 0
∀x
Theorem 4.2
•
The general solution of the complete equation is the sum of
any particular solution of the complete equation
and the general solution of the reduced equation
Solving Stopping Problem
1. State Complete and Reduced Equation
•
Complete Equation
Dierential equation that characterizes continuation region
ρV = x + aV 0 +
•
b2 00
V
2
Reduced equation
0 = −ρV + aV 0 +
b2 00
V
2
2. Guess Form of Solution to nd General Solution to Reduced Equation
45
A1
and
A2
•
First challenge is to nd solutions of the reduced equation
•
Consider class
erx
•
To conrm this is in fact a solution, dierentiate and plug in to nd:
0 = −ρerx + arerx +
b2 2 rx
r e
2
This implies (dividing through by erx )
0 = −ρ + ar +
•
b2 2
r
2
Apply quadratic formula
r=
r+
•
Let
•
General solution to reduced equation
−a ±
represent the positive root and let
r−
p
a2 + 2b2 ρ
b2
represent the negative root
Any solution to the reduced equation can be expressed
+
C + er + C − er
−
3. Find Particular Solution to Complete Equation
ρV = x + aV 0 +
b2 00
2 V
•
Want a solution to the complete equation
•
Consider specic example: payo function of policy never leave the industry
•
The value of this policy is
ˆ
∞
e−ρt x(t)dt
E
0
•
In solving for value of the policy without the expectation, we'll also need to know
ˆ t
E [x(t)] = E x(0) +
dx(t)
0
ˆ t
ˆ t
= E x(0) +
[adt + bdz(t)] = E x(0) + at +
bdz(t)
0
´t
[
0
bdz(t) = 0
0
as it is the Brownian motion term]. Hence:
E [x(t)] = x(0) + at
•
The value of the policy in V
(x) =
form, without the expectation
46
E [x(t)]:
Derived by integration by parts
ˆ
ˆ
udv = uv −
Taking
E
´∞
0
e−ρt x(t)dt:
ˆ
ˆ
∞
e
E
−ρt
ˆ
e−ρt [x(0) + at] dt
0
ˆ
∞
∞
e−ρt [x(0)] dt +
=E
0
e−ρt [at] dt =
0
x(0)
+E
ρ
ˆ
∞
e−ρt [at] dt
0
−ρt
So if
dv = e
and u = at
v = − ρ1 e−ρt and du = adt
´
´∞
´
´ ∞ 1 −ρt
udv = 0 e−ρt (at)dt then uv − vdu = [−at ρ1 e−ρt ]∞
adt
0 − 0 −ρe
Note
− ρ1 e−ρt ]∞
0 =
Think of
Then
= [−at ρ1 e−ρt ]∞
0 −
1
ρ
[ ρ12 e−ρt a]∞
0
= 0 + a ρ12
1
=
ρ
a
x(0) +
ρ
Hence our candidate particular solution takes the form
1
V (x) =
ρ
•
∞
x(t)dt = E
0
•
vdu
a
x+
ρ
Conrm that this is a solution to the complete equation
ρ
ρV = x + aV 0 +
b2 00
2 V
1
a
a
1
b2
x+
=x+ =x+a
+
·0
ρ
ρ
ρ
ρ
2
4. Put Pieces Together
•
General solution to reduced equation
+ r+ x
C e
•
− r− x
+C e
with roots
r ,r =
−a ±
p
a2 + 2b2 ρ
b2
Particular solution to general equation
1
ρ
•
−
+
a
x+
ρ
Hence the general solution to the complete equation is
[Sum of particular solution to complete equation and general solution to reduced equation]
1
V (x) =
ρ
a
x+
ρ
47
+ C + er
+
x
+ C − er
−
x
•
Now just need to apply boundary conditions
5. Apply Boundary Conditions
•
3 Conditions
Value Matching:
V (x∗ ) = 0
Smooth Pasting:
V 0 (x∗ ) = 0
•
limx→∞ V 0 (x) =
This gives us three conditions on the general solution
C+ = 0
∗
•
1
ρ
[from
limx→∞ V 0 (x) =
1
ρ]
Which then implies:
V (x∗ ) =
V 0 (x∗ ) =
1
ρ
1
ρ
x∗ +
a
ρ
+ C − er
+ r − C − er
−
x
∗
−
=0
x∗
=0
[Value Matching] (1)
[Smooth Pasting] (2)
Simplifying from (1) and (2)
Plugging (1) into (2)
∗
(1) implies
·
C − er
−
x∗
= − ρ1 x∗ + aρ
Plugging this into (2) we get
− r− ρ1 x∗ + aρ = 0
=⇒
Simplifying: 1 = r − x∗ + a
ρ
∗
∗
1
ρ
Gives us
x∗ =
Plugging in for
−
1
r−
r =
−
= x∗ +
a
ρ
a
ρ
−a−
We get our solution for
1
r−
√
a2 +2b2 ρ
b2
x∗
x∗ = −
b2
a+
p
a2
+
2b2 ρ
−
a
<0
ρ
(∗)
The general solution to the complete equation can be expressed as:
•
the sum of a particular solution to the complete dierential equation and the general solution to
the reduced equation
Note: this is usually done in search problems by looking for the particular solution of the
policy never leave. Then you can get a general solution, and then use value matching and
smooth pasting to come up with the threshold value.
48
Lecture 12
Discrete Adjustment - Lumpy Investment
Lumpy Investment Motivation
•
Plant level data suggests individual establishments do not smooth adjustment of their capital
stocks.
Instead, investment is lumpy. Ex rms build new plant all at once, not a little bit each year
•
Doms & Donne 1993: 1972-1989:
largest investment episode, rm
total investment, rm i
i
. If investment were spread
1
this ratio would be
18 .
Instead average value was 14 . Hence, on average rms did 25% of investment in 1 year over
18 year period
•
We go from convex to ane cost functions:
smooth convex cost functions assume small adjustments are costlessly reversible
use ane functions, where small adjustments are costly to reverse
Lumpy Investment
Capital Adjustment Costs Notation
•
Capital adjustment has xed and variable costs
Upward Adjustment Cost:
∗ CU
is xed cost,
cU
CU + cu I
is variable cost
Downward Adjustment Cost:
CD + cD (−I).
∗ CD + cD |I| is the general case. Here
CD + cD (−I)
∗ CD is xed cost, cD is variable cost
we usually assume
cD > 0
and
I < 0,
Firm's Problem
•
Firm loses prots if the actual capital stock deviates from target capital stock
Deviations
•
(x − x∗ )
generate instantaneous negative payo
Functional form:
b
b
2
− (x − x∗ ) = − X 2
2
2
where
X = (x − x∗ ).
49
x∗
so we get
• X
is an Ito Process between adjustments:
dX = αdt + σdz
where
dz
are Brownian increments
Value Function Representation
•
Overall Value Function
(ˆ
∞
e−ρ(τ −t)
V (X) = max E
τ =t
A (n) =
)
∞
X
b 2
− Xτ dτ −
e−ρ(τ (n)−t) A (n)
2
n=1
CU + cU In
CD + cD |In |
if
if
In > 0
In < 0
A (n) =
nth
adjustment;
Notation:
τ (n) =
date of
nth
adjustment;
cost of
In =
adjustment
∗
∗
x∗ :
∗
Above x :
Below
Adjust capital at
Adjust capital at
X = U . Adjust to X = u.
X = D. Adjust to X = d.
• Value Function in all 3 Regions
1. Between Adjustments:
Bellman Equation
∗
Using Ito's
1
b
ρV (X) = − X 2 + αV 0 (X) + σ 2 V 00 (X)
2
2
h
i
∂V
∂V
1 ∂2V 2
Lemma: E [dV ] =
dt
∂t + ∂x a + 2 ∂x2 b
Functional form solved for below [PS6 2.d]
2. Action Region Below
[If
X ≤ U]
V (X) = V (u) − [CU + cU (u − X)]
This implies
V 0 (X) = cU
∀X ≤ U
1. Action Region Above
[If
X ≥ D]
V (X) = V (d) − [CD + cD (X − d)]
This implies
V 0 (X) = −cD
50
∀X ≥ D
investment in
nth
Boundary Conditions
1. Value Matching
lim V (X) = V (u) − [CU + cU (u − U )] = V (U ) = lim V (X)
X↑U
X↓U
lim V (X) = V (d) − [CD + cD (D − d)] = V (D) = lim V (X)
X↓D
X↑D
2. Smooth Pasting
lim V 0 (X) = cU = V 0 (U ) = lim V 0 (X)
X↑U
X↓U
lim V 0 (X) = −cD = V 0 (D) = lim V 0 (X)
X↓D
X↑D
First Order Conditions
•
Intuition:
=
When making capital adjustment, willing to move until the marginal value of moving
marginal cost
This gives us two important optimality condtitions:
V 0 (u) = cU
V 0 (d) = −cD
51
Solving for Functional Form of
V (X) in Continuation Region
PS6 2.d
2
σ+2αX
X2
+ 2α
is a solution to the continuation region equaρ +
ρ2
ρ3
b
1 2 00
2
0
tion: ρV (X) = − X + αV (X) + σ V (X).
2
2
Show that this is the expected present value of the rm's payo stream assuming that adjustment
Question: Show that
V (X) = − 2b
costs are innite.
This is guess and check, where we've been provided the guess:
b
V (X) = −
2
Calculate
V 0 (X)
and
X2
σ + 2αX
2α2
+
+
ρ
ρ2
ρ3
V 00 (X):
b
V (X) = −
2
0
2X
2α2
+ 2
ρ
ρ
, V 00 (X) = −
b
ρ
1 2 00
2
0
V (X) = ρ1 −b
2 X + αV (X) + 2 σ V (X) :
1 −b 2
b 2X
2α2
1 2b
b
2αX
σ2
2α 2
2
V (X) =
X −α
+ 2 − σ
=−
X +
+
+ 2
ρ 2
2
ρ
ρ
2 ρ
2ρ
ρ
ρ
ρ
We plug back into the continuous time Bellmann Equation from
52
=−
2α
b X2
σ 2 + 2αX
+
+
2 ρ
ρ2
ρ3
So our guessed solution works as a solution to the continuous Bellmann.
We also know that this works as the expected present value of the rm's payo stream, given that this
is the equivalent to the solution to the continuous time Bellman for the continuation region, because
the innite adjustment costs mean that
u = −∞
and
d = ∞,
hence the solution will hold for
X ∈ R.
Appendix: Math Reminders
Innite Sum of a Geometric Series with
∞
X
|x| < 1:
xk =
k=0
1
1−x
L'Hopital's Rule
If
lim f (x) = lim g(x) = 0
x→c
x→c
and
f 0 (x)
x→c g 0 (x)
lim
and
then
or
±∞
exists
g 0 (x) 6= 0
f (x)
f 0 (x)
= lim 0
x→c g(x)
x→c g (x)
lim
Probability
Expectation of RV:
E(exp(ã)) = exp[E[ã] + 12 V ar[ã]].
When ã is a random variable, or anything
x ∼ N (µ, σ 2 ) −→Ax + B ∼ N (Aµ + B, A2 σ 2 ) −→ E [exp (Ax + B)] = exp Aµ + B + 12 A2 σ 2
1
2
[Since x ∼ N (µ, σ )−→ E(exp(x)) = exp(E(x) + V ar(x));
2
so if standard normal:
= exp(µ + 21 σ 2 )]
In case of Asset pricing
Rit
has stochastic returns
Rit = exp(rit + σ i εit − 12 [σ i ]2 )
i
i
i
where εt has unit variance. Hence εt is a normally distributed random variable: εt ∼ N (0, 1)
1 i 2
1 i 2
i
t
i2
i i
t
From above we see that this implies: σ εt + ri − [σ ] ∼ N (σ ∗ 0 + ri − [σ ] , σ
∗ 12 ) =
2
2
1 i 2
i2
2 [σ ] , σ )
We have:
53
N (rit −
Then:
E[Rit ] = E[exp(rit + σ i εit − 12 [σ i ]2 )] which as we've
= exp(mean + 12 V ar) = exp(rit − 12 [σ i ]2 + 21 σ i2 )
= exp(rit )
seen is now exp(a random variable) so we can use
Covariance:
If A and B are random variables, then
V (A + B) = V (A) + V (B) + 2Cov(A, B)
Log Approximation:
x:
ln(1 + x) ≈ x
1 + x ≈ ex
For small
54