Optimal Control, Guidance and Estimation
Lecture – 19
SDRE and
θ-D
Designs
Prof. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Topics
State-Dependent Riccati Equation
(SDRE) Design
θ − D Design
Benchmark Examples
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
2
StateState-Dependent Riccati Equation
(SDRE) Design
Dr. Radhakant Padhi
Associate Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
References
J. R. Cloutier, “State-Dependent Riccati Equation Techniques: An
Overview”, Proceedings of the American Control Conference,
Albuquerque, New Mexico, USA, 1997.
J. R. Cloutier and D. T. Stansbery, “The Capabilities and Art of StateDependent Riccati Equation-Based Design”, Proceedings of the
American Control Conference, Anchorage, AK, USA, 2002.
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett, StateDependent Riccati Equation Techniques: Theory and Applications.
Workshop Notes: American Control Conference, June, 1998.
T. Cimen, “State-Dependent Riccati Equation (SDRE) Control: A
Survey”, Proceedings of 17th IFAC World Congress, Seoul, Korea,
2008.
T. Cimen, “Systematic and Effective Design of Nonlinear Feedback
Controllers via the State-Dependent Riccati (SDRE) Method”, Annual
Reviews in Control, Vo. 34, 2010, pp.32-51.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
4
SDRE Design: Usage
Nonlinear suboptimal control design
• Regulator design
• Servo (tracking) design
• Robust control ( H /H ) design
2
∞
Nonlinear suboptimal observer design
Nonlinear suboptimal filters design
(Essentially, wherever Riccati equation appears,
SDRE concept can be brought in)
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
5
SDRE Design: Problem Statement
1
∞
Performance Index: J = ∫ ( X T Q ( X ) X + U T R ( X )U ) dt
2t
(to minimize)
System Dynamics: Xɺ = f ( X ) + B ( X ) U
(control affine)
Conditions:
• f ( X ) , B ( X ) , Q ( X ) , R ( X ) ∈ C k ( k ≥ 1)
• f ( 0) = 0
• B ( X ) ≠ 0 ∀X ∈ Ω (domain of interest)
• J is globally convex ( True when Q ( X ) , R ( X ) > 0 )
• f ( X ) = A ( X ) X , { A ( X ) , B ( X )} is point-wise stabilizable
0
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
6
SDRE Design: Procedure
∞
1
X T Q ( X ) X + U T R ( X ) U ) dt
(
∫
2 t0
Cost Function
J=
Write the system dynamics
in state-dependent
coefficient (SDC) form
Xɺ = A ( X ) X + B ( X ) U
Solve the state-dependent P ( X ) A ( X ) + AT ( X ) P ( X ) + Q ( X )
Riccati equation
− P ( X ) B ( X ) R −1 ( X ) BT ( X ) P ( X ) = 0
Construct the controller
U = −  R −1 ( X ) BT ( X ) P ( X )  X
= −K ( X ) X
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
7
Implementation Issue
Solve the Riccati equation symbolically
by long hand algebra.
Use symbolic software package to solve
the Riccati equation symbolically
(e.g. Maple, Mathematica, Matcad etc.)
Solve the Riccati equation online with a
high speed computer.
Obtain off line point solution and use
gain scheduling.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
8
Reference:
Example - 1
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,
Workshop Notes: American Control Conference, June, 1998.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
9
Reference:
Example - 1
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,
Workshop Notes: American Control Conference, June, 1998.
(0,0)
u
x
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
10
Definitions:
Controllability / Observability
Controllability
A ( X ) is an controllable (stabilizable) parameterization of the
nonlinear system in a region Ω if the pair { A ( X ) , B ( X )}
is point-wise controllable (stabilizable) in the linear sense ∀X ∈ Ω.
Observability
Output: Z = C ( X ) X
A ( X ) is an observable (detectable) parameterization of the
nonlinear system in a region Ω if the pair {C ( X ) , A ( X )}
is point-wise observable (detectable) in the linear sense ∀X ∈ Ω.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
11
SDRE Design: Useful Results
In addition to the conditions mentioned earlier,
if A ( X ) ∈ C k ( k ≥ 1) and it is both a detectable and
stabilizable parameterization, then the SDRE
approach produces a closed loop system that
is “locally asymptotically stable”.
For scalar problems, the resulting SDRE
nonlinear controller satisfies all the necessary
conditions of optimality; i.e. for scalar problems
it always leads to the optimal solution (this is
not true for vector case however).
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
12
SDRE Design: Useful Results
Out of the three necessary conditions,
the optimal control equation ∂H / ∂U = 0 is
always satisfied
However, the costate equation λɺ = − ( ∂H / ∂X )
is satisfied only asymptotically (under
certain additional mathematical
conditions). This is the reason for suboptimality of the controller in general.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
13
Convergence of Costate Equation
Let B ( 0, r ) be an arbitrarily large open ball centred
at the origin with radious r < ∞ . Assume that the functions
A ( X ) , B ( X ) , P ( X ) , Q ( X ) , R ( X ) along with their gradients
Axi ( X ) , Bxi ( X ) , Pxi ( X ) , Qxi ( X ) , Rxi ( X ) , i = 1, … n are
bounded in B ( 0, r ) . Then, in SDRE nonlinear regulation,
under asymptotic stability (i.e. as X → 0), the necessary condition
λɺ = − ( ∂H / ∂X ) is asymptotically satisfied at a quadratic rate.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
14
SDRE Design: Capabilities
Can directly specify and affect
performance through the selection of
appropriate state dependent state and
control weighting matrices
Can incorporate hard bounds on state
and control
Can directly handle unstable and/or nonminimum phase systems
Can preserve beneficial nonlinearities
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
15
SDRE Design: Capabilities
Can be used to design servo (tracking)
control
Can be applied to a broad class of
nonlinear systems
Can incorporate extra degree of freedom
(a design parameter) to enhance
performance of the suboptimal controller
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
16
SDRE Design: Limitations
Can be applied only to a class of
nonlinear problems
Suboptimality of the controller
Non-uniqueness of the parameterization
of the system dynamics
Applicable for infinite-time problems only
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
17
SDRE Design: Limitations
Demands solution of Riccati equation
online, which may not be feasible for
high-dimensional problems (since Riccati
equation is nonlinear)
No analytical guarantee of global stability
for the resulting controller in general.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
18
SDRE Design: Some Useful Tricks
Presence of stateindependent terms
Presence of statedependent terms that
excludes the origin
Uncontrollable and
Unstable but
Bounded State
dynamics
Constant bias:
bɺ = −α b ( 0 < α ≪ 1)
 cos x1 − 1 
cos x1 ⇒ 
 x1 + 1 ( bias )
x1


Add a stabilizing term
( xɺ = ( i ) − α x )
1
1
( 0 < α ≪ 1)
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
19
Extra Degree of Freedom
Claim:
Assume that A1 ( X ) and A2 ( X ) are two SDC parameterizations.
Then another SDC parameterization can be constructed as a
convex combination of these two parameterizations as follows:
A3 ( X ) = α ( X ) A1 ( X ) + 1 − α ( X )  A2 ( X ) ,
0 ≤α ( X ) ≤1
Proof:
{α ( X ) A ( X ) + 1 − α ( X ) A ( X )} X
1
2
= α ( X ) A1 ( X ) X + 1 − α ( X )  A2 ( X ) X
= α ( X ) f ( X ) + 1 − α ( X )  f ( X ) = f ( X )
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
20
Reference:
Example - 2
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,
Workshop Notes: American Control Conference, June, 1998.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
21
Reference:
Example - 2
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,
Workshop Notes: American Control Conference, June, 1998.
x1
x2
u
t
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
22
Reference:
Example - 2
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,
Workshop Notes: American Control Conference, June, 1998.
x1
x2
u
t
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
23
Reference:
Example - 2
α(t)
J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,
State-Dependent Riccati Equation Techniques: Theory and Applications,
Workshop Notes: American Control Conference, June, 1998.
x1
x2
u
t
t
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
24
θ-D
Suboptimal Control Design
Courtesy:
Ming Xin, Department of Aerospace Engineering
Mississippi State University, USA
Dr. Radhakant Padhi
Associate Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
References
1.
Xin M. and Balakrishnan S. N., “A New Method for Suboptimal
Control of a Class of Nonlinear Systems,” Optimal Control
Applications and Methods, 26(2): 55-83, 2005.
2.
Xin M., Balakrishnan S. N., Stansbery D. T., and Ohlmeyer E.J.,
“Nonlinear Missile Autopilot Design with θ - D Technique,” AIAA
Journal of Guidance, Control and Dynamics, 27, 406-417, 2004.
3.
Drake D., Xin, M. and Balakrishnan S. N., “A New Nonlinear Control
Technique for Ascent Phase of Reusable Launch Vehicles,” AIAA
Journal of Guidance, Control and Dynamics, 27(6): 938-948, 2004.
4.
Radhakant Padhi, Ming Xin and S. N. Balakrishnan, “Suboptimal
Control of a One-dimensional Nonlinear Heat Equation Using POD
and θ - D Techniques”, Optimal Control Applications and Methods,
29(3): pp.191-224, 2008.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION Prof.
Radhakant Padhi, AE Dept., IISc-Bangalore
26
Optimal Control Problem
System Dynamics:
xɺ (t ) = f ( x(t ) ) + Bu (t )
control affine form
Objective:
Find a controller u to minimize a cost function
1
J=
2
t f →∞
∫
 xT Q ( x ) x + u T R ( x ) u  dt
t0
This is an infinite-horizon optimal control problem
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
27
Solution to the Optimal Control Problem
u = − R −1 B T
∂V *
∂x
• Solve the Hamilton-Jacobi-Bellman (HJB) equation
∂V * T
1 ∂V *T
∂V * 1 T
f ( x) −
BR −1 B T
+ x Qx = 0
∂x
2 ∂x
∂x 2
where
V * = min J
u
Challenge
• A closed-form solution is very difficult to obtain
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
28
Summary of θ - D Technique
• Make Approximations
J=
∞

1 ∞ T 
i 
T
 x  Q + ∑ Di θ  x + u Ru dt
∫
0
2  
i =1



 A( x )  
xɺ = f ( x ) + Bu = F ( x ) x + Bu =  A0 + θ 
 x + Bu
 θ  

• Solve perturbed Hamilton-Jacobi-Bellman equation
∞
∂V *T
1 ∂V *T
∂V * 1 T 

f ( x) −
BR−1BT
+ x  Q + ∑Diθ i  x = 0
∂x
2 ∂x
∂x 2 
i =1

∂V * ∞
= ∑ Ti ( x , θ )θ i x
∂x
i =0
Assume:
Recall: u = − R −1BT
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
∂V *
∂x
29
Substitute in HJB equation and equate coefficients:
Algebraic Riccati Equation
−1
T
T
T0
T0 A0 + A0 T0 − T0 BR B T0 + Q = 0
Linear Equation with
constant coefficients
T
c0 1
T1 Ac 0 + A T = F1 ( x, T0 ,θ ) − D1
⋮
Ac 0 = A0 − BR −1 BT T0
T1
⋮
Linear Equation with
constant coefficients
Tn Ac 0 + Ac 0TTn = Fn ( x, T1 ,⋯, Tn−1 ,θ ) − Dn
Tn
• Closed-form Optimal Control
n
u = − R −1BT ∑ Ti ( x ,θ )θ i x Note: θ will be cancelled in the final control calculation
i =0
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
30
Construction of Di
Note:
F1 ( x , T0 ,θ ) = −
T0 A( x )
θ
Fn ( x , T1 ,⋯ , Tn −1 ,θ ) = −
Di
−
AT ( x )T0
θ
Tn−1 A( x )
θ
−
AT ( x )Tn−1
θ
n −1
+ ∑ T j BR −1 BT Tn − j
j =1
is constructed as Di = kie−l t Fi ( A( x), T0 ,⋯,Ti −1 , θ )
i
such that
Fi ( A( x ), T0 ,⋯ , Ti −1 ,θ ) − Di = ε i ( t ) ⋅ Fi ( A( x ), T0 ,⋯ , Ti −1 ,θ )
with
ε i ( t ) = 1 − ki e − l t a small number
i
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
31
Motivation of Using Di
ε i ( t ) = 1 − ki e − l t is used to:
i
Prove the convergence of the series.
Guarantee semi-global asymptotic stability
Reduce the initial control level
Adjust the system transient performance
ki and li are primary design parameters to tune the system
performance
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
32
• Systematic Method of Selecting k
and l i Parameters
Ideally, on the optimal path, the Hamiltonian
H =
i
1
2
xT Q x +
1
2
uT Ru +
∂ V *T
f
∂x 
(x) +
B u  = 0
Procedure is as follows
– An initial value of (ki, li) is given.
– Controller is run, computing H value at each time.
– Iteratively change (ki, li) to minimize H in the leastsquare sense.
– This procedure is run offline.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
33
Benchmark Example (Kokotovic,1994)
Scalar Problem:
System dynamics:
Cost function:
xɺ = x − x 3 + u
J=
1 ∞ 2
( x + u 2 )dt
∫
0
2
The optimal solution
u = −( x − x 3 ) − x x 4 − 2 x 2 + 2
Feedback Linearization solution
u fl = x3 − 2 x
Feedback linearization cancels the beneficial nonlinearity
and results in large control effort when the state is large!
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
34
θ − D Solution
• Factorize nonlinear term f(x) as
A0 = 1,
A1 ( x) = − x 2 with Q = 1, R = 1
θ − D solution
T0 = 1 + 2
T1 = −
T2 = −

1  x2 ⋅ (1 + 2)
− D1 
2
θ
2 2

1  2 −1
2 x ⋅
2 2
2 2

2

 x 2 ⋅ (1 + 2)
 1  x 2 ⋅ (1 + 2)

2
−
D
+
2
−
D


1
1  − D2 
2
θ
θ

 8


OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
35
θ − D Solution
Di terms in the θ − D method play key role.
 T 0 A ( x ) A T ( x )T 0 
−
− θ

θ


D1 = 0 .98 e
−2 t
D 2 = 0 .9 8 e
− 0 .9 t
 T1 A ( x ) A T ( x )T1 
−
− θ

θ


OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
36
Figure 3: Scalar problem: x0= [10,10]
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
37
Comparison Between
SDRE and θ - D Methods
SDRE Method
θ - D Method
Suboptimal
Suboptimal
Requires adjustment of
weighting matrices
Requires adjustment of
both weighting matrix and
other design parameters
Needs state dynamics in
terms of state dependant
coefficient form
Same requirement
Solving the Riccati
equation online
Solving a set of Lyapunov
equations online
Higher Computational Time Lesser computational time
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore
38
Thanks for the Attention….!!
OPTIMAL CONTROL, GUIDANCE
AND ESTIMATION Prof. Radhakant
Padhi, AE Dept., IISc-Bangalore
39