Lecture 4 Exposition of the
Lagrange Method
This lecture is technical. Please
read Chow, Dynamic Economics,
chapters 1 and 2.
1
Brief explanation of the Lagrange method for
dynamic optimization– 3 steps
• 1. Start with the constrained maximization
problem max r(x,u) subject to x=f(u).
• Set up the Lagrange expression
• L = r(x,u) –λ[x-f(u)].
• Differentiate L with respect to x, u and λ to
obtain three first-order conditions.
• Solve these equations for the three variables.
2
step 2 - Generalize above
procedure to many periods
• Objective function is a weighted sum of
r(x(t),u(t)) over time t.
• Constraints are x(t+1) = f(x(t),u(t)).
• We call x the state variable and u the
control variable.
• Set up the Lagrange expression
• L = Σt βt{r(x(t),u(t)) –λt+1[x(t+1)- f(x(t),u(t))]}
and differentiate to obtain first-order
conditions to solve for the u’s and x’s.
3
Step 3 - stochastic
• Model is xt+1 = f(xt, ut, εt), εt stochastic.
• We now have an expectation operator in
front of the previous objective function
• L = E Σ1Tβt {r(xt ,ut) – λt+1[xt+1 - f(xt ,ut)]}
• The first-order conditions can still be
obtained by differentiation after the
summation sign.
• This summarizes all I know after 30 years
of work on dynamic optimization.
4
Solution of a linear-quadratic
optimal control problem
L = Σt=1T { βt(xt –at}’Rt(xt –at}/2– βt λt’[xt – b - A xt-1 – C ut-1 ] }
(3)
Treating a more general case we let xt be a column vector of p state
variables, ut be a column vector of q control variables, at be a vector of
specified targets for xt to approach, Rt be a given symmetric matrix in
the utility function r and λt’ be a row vector of Lagrange multipliers for
period t, with prime denoting transpose. In the dynamic equation for xt,
b is a given column vector of p components, A is a given p by p matrix
and C is a given p by q matrix. Since we are dealing with a vector of p
state variables, we need a vector of p Lagrange multipliers for each
period. Our objective is to maximize L by choosing the optimum policies
u0 to uT-1.
5
We will find the optimum for the control variable ut period by period backward in time
beginning with period T. Given the optimum uT-1, we find the optimum uT-2 that
maximizes the objective function for the last two periods. Given the optimum uT-1 and uT2,
we find the optimum uT-3 that maximizes the objective function for the last three
periods and so forth. In this way maximizing a function of T variables, u0, u1, …, uT-1 is
reduced to maximizing T functions of one variable each, which is much easier than the
first problem. This procedure is valid when the utility function is a sum of utility
functions each for one period or when the utility function is time separable.
6
For period T we maximize
LT = βT(xT –aT)’RT(xT –aT)/2 - βTλT’[xT – b - A xT-1 – C uT-1] (4)
Using the differentiation rules for a vector z and a symmetric matrix A
δ(u’z )/δu = δ(z’u )/δu = z
and δ(u’Au)/δu = 2Au
we set to zero the partial derivatives of β-T times LT with respect to the
(vector) control variable uT-1 and the (vector) state variable xT to obtain
C’λT =0
(5)
RT(xT –aT}- λT = 0
(6)
7
Premultiplying (6) by C’ and using (5) we have
C’RT(b + A xT-1 + C uT-1 –aT}=0
(7)
The solution for the optimum uT-1 as a function of the state xT-1 is
uT-1 = - (C’RTC)-1 C’RT (A xT-1 + b – aT)
= GT xT-1 + gT
(8)
where we have defined
GT = - (C’RTC)-1 C’RTA
gT = - (C’RTC)-1 C’RT(b – aT)
(9)
(10)
8
Under optimal control the state variable xT will be determined by
xT = AxT-1 + CuT-1 + b = (A+CGT)xT-1 + CgT + b
(11)
Using (11) and (6) the optimum solution for the optimum Lagrange
multiplier as a function of the state variable xT is
λT = RT(xT –aT}= RT((A+CGT)xT-1 + CgT + b – aT)
(12)
= HT xT-1 + hT
where we have defined
HT = RT(A+CGT)
(13)
hT = RT(CgT + b – aT)
(14)
9
The next step is to take as given functions the solutions (8) for uT-1 = uT-1(xT1) and (12) for λT = λT(xT-1) for the last period T and to solve the problem for
the last two periods with the Lagrangean
L2 = βT-1(xT-1 –aT-1)’RT-1(xT-1 –aT-1)/2– βT-1 λT-1’[xT-1 – b - A xT-2 – C uT-2]
-βT λT’[xT – b - (A+CGT) xT-1]
(15)
Note that the value of the objective function βT(xT –aT)’RT(xT –aT) for the
last period T has been maximized by choosing the optimum control
function uT-1 = uT-1(xT-1) for uT-1. The remaining problem is to choose uTAnother way to put
2 to maximize this two-period objective function.
this point is that if we set up a Lagrangean involving the utility for the last
two periods and try to find its maximum by differentiating with respect to uT1, xT, uT-2 and xT-1, the first order conditions obtained for the first two
derivatives are now solved. The remaining problem is to obtain and solve the
first order conditions from the second two derivatives.
10
Setting to zero the partial derivatives of L2 with respect to uT-2 and xTrespectively gives
C’ λT-1 = 0
(16)
RT-1(xT-1 –aT-1}- λT-1 + β(A+CGT)’ λT =0
(17)
which correspond to (5) and (6). Pre-multiplying (17) by C’ to make its
second term equal to zero and using (12) to substitute for λT we obtain
C’RT-1(xT-1 –aT-1} + βC’(A+CGT)’(HTxT-1 + hT)
= [C’RT-1 + βC’(A+CGT)’HT](Axt-2 + CuT-2 + b}-C’RT-1aT-1+
βC’(A+CGT)’hT
=[C’RT-1+ βC’(A+CGT)’HT]AxT-2 + [C’RT-1+ βC’(A+CGT)’HT]CuT-2
+ [C’RT-1+ βC’(A+CGT)’HT]b -C’RT-1aT-1+ βC’(A+CGT)’ hT = 0
11
Solving this equation for uT-2 as a function of xT-2, we get
uT-2 = -[C’RT-1C+ βC’(A+CGT)’HTC ]-1{[C’RT-1A+ βC’(A+CGT)’HTA]xT-2
+ [C’RT-1+ βC’(A+CGT)’HT]b -C’RT-1aT-1+ βC’(A+CGT)’ hT]}
= GT-1 xT-2 + gT-1
(18)
where
GT-1 = -[C’RT-1C+ βC’(A+CGT)’HTC ]-1{[C’RT-1A+ βC’(A+CGT)’HTA]
(19)
gT-1 = - [C’RT-1C+ βC’(A+CGT)’HTC]-1{[C’RT-1+ βC’(A+CGT)’HT]b C’RT-1aT-1+ βC’(A+CGT)’hT
(20)
(18) is the optimum control equation for uT-2 analogous to (8).
12
As before under optimum control xT-2 will be determined by the
equation
xT-1 = AxT-2 + CuT-2 + b = (A+CGT-1)xT-2 + CgT-1 + b
To find the optimum λT-1 we use (17) and (12)
λT-1 = RT-1(xT-1 –aT-1} + β(A+CGT)’(HT xT-1 + hT)
= [RT-1 + β(A+CGT)’HT]xT-1 - RT-1aT-1 + β(A+CGT)’ hT
= [RT-1 + β(A+CGT)’HT][GT-1 xT-2 + gT-1] - RT-1aT-1 + β(A+CGT)’ hT
= HT-1xT-2 + hT-1
where
HT-1 = [RT-1 + β(A+CGT)’HT]GT-1
hT-1 = [RT-1 + β(A+CGT)’HT]gT-1 - RT-1aT-1 + β(A+CGT)’ hT
(21)
(22)
(23)
13
To continue with this procedure we maximize the objective function for
the last three periods by forming a Lagrangean L3 including the utility
of period T-2 and differentiating it with respect to uT-3 and xT-2 and so
forth. The time subscripts of all equations from (15) to (23) will be
reduced by 1. By this process we complete the solution to the optimal
control problem of finding the optimum feedback control function ut =
Gt+1xt + gt+1 for t = T-1, T-2, …, 0 and the associated Lagrange function
λt = Htxt-1+ ht for t = T, T-1, ….1. backward in time. To do so we solve a
set of equations for the matrices Gt and the vectors gt and an associated
set for Ht and ht. For the last period GT and gT are given by (9) and (10),
and HT and hT are given by (13) and (14). For the period T-1, GT-1 and
gT-1 are given by (19) and (20) and HT-1 and hT-1 are given by (22) and
(23). For any earlier period t, the last four equations remain valid with
T-1 replaced by t. The process continues until the optimal control rule
for u0 is found for the determination of x1. The equations (19), (20), (22)
and (23) are known as matrix Ricatti equations in control theory. In this
calculation we treat as given the parameters Rt and at of utility function,
A, C and b of the dynamic model and the discount rate β.
14
To illustrate consider the following dynamic equation
xt = A1xt-1 + A2 xt-2 + C1ut-1 + C2ut-2 + b
(24)
By redefining the state variable we can write it as the following firstorder equation
xt
=
A1
A2
C2
xt-1
+
C1
ut-1
b
+
xt-1
I
0
0
xt-2
0
0
ut-1
0
0
0
ut-2
I
0
The above matrix equation can be redefined as xt = Axt-1 + Cut-1 + b for
use in the Lagrangean expression (3). We can thus have both current
and lagged state variables and control variables entering the utility
function for determining the optimum path for CO2 emission et. The
redefinition of a vector of state variables to include these other variables
can be useful in practical applications.
15
The above discussion deals only with a small part of the subject of
optimal control or dynamic optimization. The dynamic model employed
can be in discrete time or in continuous time. It may be deterministic or
stochastic. We have dealt with only one deterministic control problem in
discrete time. A different exposition of the method of this section was
presented in Chow (1975, pp. 158-9). The steady state version of the
matrix Ricatti equations was derived by the Lagrange method in Chow
(1997, beginning with section 2.3, where the need to solve for functional
equations as feedback control equations is clearly pointed out).
Stochastic models in both discrete and continuous time are discussed in
Chow (1997) where it is shown (on p. 145) that the Lagrange method for
obtaining optimal feedback equations for stochastic models in
continuous time is reduced to Pontryagin’s maximum principle as a
special case when the model becomes deterministic.
16
Problems
• Do problems 1 and 3 of Chapter 2 of
Chow, Dynamic Economics
17