Lecture #23 Linear Quadratic Regulator (Apr. 14, 2015) v. 1.0 (R. Fearing)
Ref: K. Ogata, Modern Control Engineering 2002.
Here the infinite horizon, continuous time, Linear Quadratic Regulator is derived. A cost function which
is Quadratic in control and error is used. The considered system is Linear and uses a linear state feedback
control u = −Kx. The Regulator problem considers x(t) → 0. Infinite horizon means considering the total
cost as t → ∞.
Given a system in state-space form:
ẋ = Ax + Bu
y = Cx
(1)
Define instantaneous cost of control u(t)T Ru(t), instantaneous cost of output error y(t)T Qy(t). Q, R are
positive semidefinite, that is xT Qx ≥ 0 ∀x. Assume Q = QT and R = RT .
Define Quadratic cost
Z
∞
(y T Qy + uT Ru)dt.
J=
(2)
0
To combine (1) and (2), we introduce a “cost of state x”, xT P x, where P is positive semidefinite (also
P = P T ) and where P will be derived. The difference between “cost” of final and initial states is given by
Z ∞
d T
(x P x)dt,
(3)
xT (∞)P x(∞) − xT (0)P x(0) = −xT (0)P x(0) =
dt
0
using the assumption of a stable regulator, thus xT (∞)P x(∞) → 0.
Now adding Eq.(2) and Eq.(3 gives
Z ∞
Z ∞
d T
T
T
T
J − x (0)P x(0) =
(y Qy + u Ru) + (x P x)dt =
(y T Qy + uT Ru) + ẋT P x + xT P ẋdt
dt
0
0
(4)
Using (1) we get:
T
∞
Z
xT C T QCx + uT Ru + (xT AT + uT B T )P x + xT P (Ax + Bu)dt
J − x (0)P x(0) =
(5)
0
Grouping quadratic terms
J − xT (0)P x(0) =
Z
∞
xT [C T QC + AT P + P A]x + uT Ru + uT B T P x + xT P Bu dt
(6)
0
Algebraic Riccati Equation (A.R.E.)
For a given N and since A, C, Q are known, the A.R.E. P N P = C T QC + P A + AT P can be solved for P ,
e.g. using Matlab care(). (N is to be determined below.)
Substituting P N P in (6), we get:
Z ∞
T
J − x (0)P x(0) =
xT [P N P ]x + uT Ru + uT B T P x + xT P Bu dt
(7)
0
Considering that for optimal linear state feedback, u(t) → −Kx, the instantaneous “cost” of the difference
between u and Kx can be given by (u + Kx)T R(u + Kx) = 0. Let K = R−1 B T P . Then
(u + Kx)T R(u + Kx)
= xT K T RK T x + uT Ru + uT RKx + xT K T Ru
T
T
T
T
= x P BR
−1
T
T
T
T
T
B P x + u Ru + u B P x + x P Bu
T
T
T
T
= x P N P x + u Ru + u B P x + x P Bu
(8)
(9)
(10)
where N = BR−1 B T . Noting that Eq.(10) is identical to the term inside the integral in Eq.(7), which is 0
for u = −Kx, then the minimum cost is:
J = xT (0)P x(0)
(11)
where P is solution of C T QC + P A + AT P − P BR−1 B T P = 0, the algebraic Riccati equation.