Lecture – 27
Linear Quadratic Regulator (LQR) Design – I
Dr. Radhakant Padhi
Asst. Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
LQR Design:
Problem Objective
z
To drive the state X of a linear (rather
linearized) system X = A X + BU to the origin
by minimizing the following quadratic
performance index (cost function)
tf
1 T
1
J = ( X f S f X f ) + ∫ ( X T Q X + U T RU ) dt
2
2 t0
where
S f , Q ≥ 0 (psdf), R > 0 (pdf)
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
2
LQR Design: Guideline for
Selection of Weighting Matrices
S f ≥ 0 (psdf), Q ≥ 0 (psdf), R > 0 (pdf)
These are usually chosen as diagonal matrices, with
(
s fi = maximum expected/acceptable value of 1/x
2
if
qi = maximum expected/acceptable value of (1/x
2
i
ri = maximum expected/acceptable value of (1/ui2
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
3
)
)
)
LQR Design:
Some Facts to Remember
z
{
z
The pair { A, B} needs to be controllable and the pair
A, Q needs to be detectable
}
S f ≥ 0 (psdf), Q ≥ 0 (psdf), R > 0 (pdf)
(these are usually chosen as diagonal matrices)
z
z
By default, it is assumed that t f → ∞
Constrained problems (state and control inequality
constraints) are not considered here. Those will be
considered later.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
4
LQR Design:
Problem Statement
z
Performance Index (to minimize):
tf
1 T
1 T
J = ( X f S f X f ) + ∫ ( X Q X + U T RU ) dt
2 2
t0 ϕ( X f
)
L ( X ,U )
X = A X + BU
z
Path Constraint:
z
Boundary Conditions: X ( 0 ) = X 0 : Specified
t f : Fixed, X ( t f ) : Free
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
5
LQR Design:
Necessary Conditions of Optimality
z
z
1 T
Terminal penalty: ϕ ( X f ) = ( X f S f X f )
2
1 T
Hamiltonian:
H = ( X Q X + U T RU ) + λ T ( AX + BU )
2
X = AX + BU
z
State Equation:
z
Costate Equation: λ = − ( ∂H / ∂X ) = − ( QX + AT λ )
z
Optimal Control Eq.: ( ∂H / ∂U ) = 0 ⇒ U = − R −1 BT λ
z
Boundary Condition: λ f = ( ∂ϕ / ∂X f ) = S f X f
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
6
LQR Design:
Derivation of Riccati Equation
Guess:
λ (t ) = P (t ) X (t )
Justification:
From functional analysis theory of normed linear space, λ ( t )
lies in the "dual space" of X ( t ) , which is the space consisting
of all continuous linear functionals of X ( t ) .
Reference: Optimization by Vector Space Methods
D. G. Luenberger, John Wiley & Sons, 1969.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
7
LQR Design:
Derivation of Riccati Equation
Guess
λ (t ) = P (t ) X (t )
+ PX
λ = PX
+ P ( AX + BU )
= PX
+ P ( AX − BR −1 BT λ )
= PX
+ P ( AX − BR −1 BT PX )
= PX
− ( QX + AT PX ) = ( P + PA − PBR −1 BT P ) X
T
−1 T
P
+
PA
+
A
P
−
PBR
B P + Q) X = 0
(
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
8
LQR Design:
Derivation of Riccati Equation
z
Riccati equation
P + PA + AT P − PBR −1 BT P + Q = 0
z
Boundary condition
P (t f ) X f = S f X f
(X
f
is free )
P (t f ) = S f
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
9
LQR Design:
Solution Procedure
z
z
z
Use the boundary condition P ( t ) = S and
integrate the Riccati Equation backwards
from t f to t0
Store the solution history for the Riccati
matrix
Compute the optimal control online
f
f
U = − ( R −1 BT P ) X = − K X
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
10
LQR Design:
Infinite Time Regulator Problem
Theorem (By Kalman)
As t f → ∞, for constant Q and R matrices, P → 0 ∀t
Algebraic Riccati Equation (ARE)
PA + AT P − PBR −1 BT P + Q = 0
Note:
z
ARE is still a nonlinear equation for the Riccati matrix. It is not
straightforward to solve. However, efficient numerical methods
are now available.
z
A positive definite solution for the Riccati matrix is needed to
obtain a stabilizing controller.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
11
Example – 1:
Stabilization of Inverted Pendulum
Dr. Radhakant Padhi
Asst. Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
A Motivating Example:
Stabilization of Inverted Pendulum
System dynamics:
θ = ωn2 θ − u, ωn2 = g / L
u
mg
θ L
( Linearized about vertical equilibrium point )
System dynamics (state space form):
Define: x = θ , x = θ
1
2
⎡ x1 ⎤ ⎡ 0 1 ⎤ ⎡ x1 ⎤ ⎡ 0 ⎤
⎢ x ⎥ = ⎢ω 2 0 ⎥ ⎢ x ⎥ + ⎢ −1⎥ u
⎣ ⎦
⎣ 2⎦ ⎣ n
⎣ 2⎦ N
N
⎦ N
X
A
X
B
Performance Index (to minimize):
∞
1 ⎛
1 ⎞
J = ∫ ⎜ θ 2 + 2 u 2 ⎟ dt
2 0⎝
c
⎠
⎡1 0 ⎤
1
=
Q=⎢
R
,
⎥
2
0
0
c
⎣
⎦
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
13
A Motivating Example:
Stabilization of Inverted Pendulum
ARE:
PA + AT P − PBR −1 BT P + Q = 0
⎡ p1 p2 ⎤
Let P = ⎢
⎥ (a symmetric matrix)
p
p
3⎦
⎣ 2
⎡ p2ωn2 p1 ⎤ ⎡ p2ωn2 p3ωn2 ⎤ ⎡ c 2 p22 c 2 p2 p3 ⎤ ⎡1 0 ⎤ ⎡0 0 ⎤
+⎢
⎢
⎥+⎢
⎥−⎢ 2
⎥ = ⎢0 0 ⎥
2
2 2 ⎥
0
0
p
p
c
p
p
c
p
ω
p
p
⎣
⎦ ⎣
⎦
2⎦
3 ⎦
2 ⎦
⎣ 1
⎣ 3 n
⎣ 2 3
Equations:
2 p2ωn2 − c 2 p22 + 1 = 0
u
mg
θ L
1 ⎡ 2
ω ± ωn4 + c 2 ⎤
2 ⎣ n
⎦
c
⇒
p2 =
⇒
p3 = ±
p1 + p3ωn2 − c 2 p2 p3 = 0 (repeated)
2 p2 − c 2 p32 = 0
1
2 p2
c
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
14
A Motivating Example:
Stabilization of Inverted Pendulum
However, p3 is a diagonal term, which needs to be real and positive.
Hence, p2 needs to be positive. Therefore
1 ⎡ 2
4
2 ⎤
ω
ω
c
,
+
+
n
2 ⎣ n
⎦
c
Moreover,
p2 =
p3 =
1
2 p2
c
u
θ L
mg
p1 + p3ωn2 − c 2 p2 p3 = 0
p1 = c 2 p2 p3 − p3ωn2 (not needed in this problem)
Gain Matrix:
K = R −1 BT P = ⎡⎣ −c 2 p2
−c 2 p3 ⎤⎦
Control:
(
u = − K X = c 2 p2θ + p3θ
)
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
15
A Motivating Example:
Stabilization of Inverted Pendulum
Analysis
Open-Loop System:
λ −1
2
2
λI − A =
=
λ
−
ω
=0
n
2
−ωn λ
λ = ± ωn
u
θ L
mg
( right half pole: unstable system )
Define: ω 2 = ωn4 + c 2
Closed-Loop System:
0
⎡
ACL = A − BK = ⎢ 2 2
⎣ωn − c p2
Closed-Loop Poles:
λ I − ACL = 0
1 ⎤
−c 2 p3 ⎥⎦
p2 =
1
2
2
ω
+
ω
(
)
n
c2
1
2 2
2 1/ 2
p3 =
2 p2 = 2 (ωn + ω )
c
c
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
16
A Motivating Example:
Stabilization of Inverted Pendulum
Analysis
Closed-Loop Poles:
λ + 2 (ω + ω
2
λ1,2 = −
( Note:
2
n
)
2 1/ 2
λ +ω = 0
u
θ L
2
1
1
2
2 1/ 2
2
2 1/ 2
j
ω
+
ω
±
ω
−
ω
(
)
(
n
n )
2
2
ω 2 = ωn4 + c 2 > ωn2
)
Both of the closed-loop poles are strictly in the left-half plane.
Hence, the closed-loop is guaranteed to be “asymptotically stable”.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
17
mg
Example – 2:
Finite-time Temperature Control
Dr. Radhakant Padhi
Asst. Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Example: Finite Time
Temperature Control Problem
System dynamics :
θ = −a (θ − θ ) + bu
n
where
a, b : Constants
θ
: Temperature
θ n : Ambient temperature (Constant = 200C)
u : Heat input
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
19
Problem formulations
Case – 1:
Cost Function:
tf
1 2
J = ∫ u dt
20
θ ( t f ) = θ f = 300 C
(Hard constraint)
Case – 2:
Cost Function:
tf
⎡
⎤
2
1
2
J = ⎢ s f (θ f − 30 ) + ∫ u dt ⎥
2 ⎢⎣
⎥⎦
0
s f > 0 : Weightage
i.e. θ ( t f ) ≈ 300 C
(Soft Constraint)
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
20
Solution:
Solution:
x (θ − θ a ) , θ ( 0 ) = θ a
x = −ax + bu , x ( 0 ) = (θ a − θ a ) = 0
1 2
Η = u + λ ( −ax + bu )
2
⎛ ∂Η ⎞
λ = −⎜
⎟ = aλ
⎝ ∂x ⎠
∂Η
= 0 ⇒ u = −λ b
∂u
Necessary conditions
x = −ax + bu
λ = aλ
u = −bλ
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
21
Solution: Case – 1 (Hard constraint)
a (t −t f )
− a (t f −t )
λ =e
λf = e
λf
u = −be
(
− a t f −t
)
λf
x = −ax − b λ f e
2
(
− a t f −t
)
Taking laplace transform:
⎡
⎤
⎢ sX ( s ) − x ( 0 ) ⎥ = −aX ( s ) − b 2λ f e − at f
N⎥
⎢
0
⎣
⎦
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
⎛ 1 ⎞
⎜
⎟
⎝s−a⎠
22
Solution: Case – 1 (Hard constraint)
⎛ 1 ⎞
X ( s ) = −b λ f e ⎜ 2
2 ⎟
⎝s −a ⎠
1
1 ⎞
− at f 1 ⎛
2
= −b λ f e
−
⎜
⎟
2a ⎝ s − a s + a ⎠
− at f 1
2
at
− at
e
e
Hence x(t ) = −b λ f e
−
(
)
N
2a
2
− at f
Unknown
However, x (t f ) = (θ f − θ a ) = 10 C
0
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
23
Solution: Case – 1 (Hard constraint)
x(t f ) = 10 = −b λ f e
2
− at f
⎛ b 2λ f ⎞
−2 at f
10 = − ⎜
1− e
⎜ 2a ⎟⎟
⎝
⎠
−20a
λf = 2
−2 at f
b 1− e
(
(
(
1 at f
− at f
e −e
2a
)
)
)
⎛
−20 a
2 ⎜
x(t ) = − b
⎜ b 2 1 − e −2 at f
⎝
(
)
at
− at
⎞
−
10
e
e
(
)
⎟ e − at f 1 ( e at − e − at ) =
at f
− at f
⎟
2a
e −e
⎠
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
(
)
24
Solution: Case – 1 (Hard constraint)
Note :
x(t f ) =
(
10 e
(e
at f
at f
−e
−e
)
− at f
− at f
)
= 10
(i.e. The boundary condition is "exactly met".)
Controller :
⎡
−20a
− a (t f − t )
⎢
u (t ) = − b e
⎢ b 2 1 − e −2 at f
⎣
(
)
⎤ ⎡
at
20
a
e
−
⎥=⎢
⎥ ⎢ b e at f − e − at f
⎦ ⎣
(
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
)
⎤
⎥
⎥
⎦
25
Solution: Case – 2 (Soft constraint)
θ f → 300 C
⇒ x f → 100 C.
Hence the cost function is
tf
⎡
⎤
2
1
2
J = ⎢ s f ( x f − 10 ) + ∫ u dt ⎥
2 ⎢⎣
⎥⎦
0
⎛ λf
⎞
λ f = s f ( x f − 10 ) ⇒ x f = ⎜ + 10 ⎟
⎜s
⎟
f
⎝
⎠
However, we have
b2
− at f
x ( t ) = − λ f e ( eat − e − at )
2a
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
26
Solution: Case – 2 (Soft constraint)
λf
b2
−2 at f
At t = t f , x ( t f ) = − λ f 1 − e
=
+ 10
sf
2a
(
)
⎡ 1 b2
−2 at f ⎤
i.e. λ f ⎢ +
1− e
⎥ = −10
⎢⎣ s f 2a
⎥⎦
⎡
⎤
−20 s f a
⎥
i.e. λ f = ⎢
⎢ 2a + s f b 2 1 − e −2 at f ⎥
⎣
⎦
⎡
−20 s f a
− a (t f −t )
− a (t f −t )
⎢
Hence λ = e
λf = e
⎢ 2a + s f b 2 1 − e −2 at f
⎣
(
)
(
)
(
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
)
⎤
⎥
⎥
⎦
27
Solution: Case – 2 (Soft constraint)
u ( t ) = −bλ
⎡
⎤
20
−
s
a
− a (t f −t )
f
⎢
⎥
= −be
⎢ 2a + s f b 2 1 − e −2 at f ⎥
⎣
⎦
⎡
at
⎢
10s f abe
− a (t f −t )
⎢
= −be
2
s
b
⎢ at f
at f
− at f
f
⎢⎣ ae + 2 e − e
(
)
(
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
)
⎤
⎥
⎥
⎥
⎥⎦
28
Correlation between hard and soft
constraint results
As s f → ∞,
10abe at
lim u ( t ) S .C . = lim
s f →∞
⎛ 1
⎜⎜
⎝ sf
20ae at
s f →∞
=
(
b e
at f
−e
⎞ at f b 2 at f
− at f
e −e
⎟⎟ ae +
2
⎠
− at f
(
)
)
= u ( t ) H .C .
i.e. The "soft constraint" problem behaves
like the "hard constraint" problem when s f → ∞.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
29
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
30