Modern Control
Lecture 11
Dr.-Ing. Erwin Sitompul
President University
http://zitompul.wordpress.com
2 0 1 4
President University
Erwin Sitompul
Modern Control 11/1
Chapter 10
Optimal Control
Homework 10
Consider again the control system as given before, described by
0 1 
0
x (t )  
x (t )    u (t )

0 0
1 
y (t )  1 0 x (t )
Assuming the linear control law
u(t )  k x(t )  k1 x1 (t )  k2 x2 (t )
Determine the constants k1 and k2 so that the following performance
index is minimized

J1   x T (t ) x (t )dt  x T (0) P x (0)
0
Consider only the case where the initial condition is x(0)=[c 0]T and
the undamped natural frequency (ωn) is chosen to be 2 rad/s.
• Calculate the transfer function of the
system if compensated with k
• Determine the value of corresponding k (k1
or k2?) to obtain ωn as requested
President University
Erwin Sitompul
Modern Control 11/2
Chapter 10
Optimal Control
Solution of Homework 10
Substituting the state feedback and finding the transfer function,
ˆ b
G( s )  c  s I  A
1
s
 1 0 
 k1
1
1  0 
s  k2  1 
 s  k2 1  0 
1 0 
 1 

k
s
 1
 

s 2  k2 s  k1
1
 2
s  k2 s  k1
k1
1
  2
 k1  s  k2 s  k1
Gain
n2
 2
s  2n s  n2
President University
 k1    4
2
n
Erwin Sitompul
1 
0
ˆ
 A


4

k

2
Modern Control 11/3
Chapter 10
Optimal Control
Solution of Homework 10
T
ˆ
A P  P Aˆ   I
1   1 0 
0 4   p11 p12   p11 p12   0




1  k   p



p
p
p

4

k
0

1



2   12
22 
 12
22  
2
4 p22  p11  k2 p12   1 0 
 4 p12  4 p12

p k p 4p


p

k
p

p

k
p
0

1


 11 2 12
22
12
2 22
12
2 22 
1
5
k2 20
p12  , p22 
, p11  
8
8k 2
8 8k 2
p12   c 
20 
p
2  k2
2

J1  x T (0) P x (0)   c 0  11
 c p11  c 




8
8
k
p
p
0

2 
 12
22   
20 
dJ1
21
 c   2   0  k2  20
dk2
 8 8k2 
President University
Erwin Sitompul
u(t )  k1 x1 (t )  k2 x2 (t )
 4 x1 (t )  20 x2 (t )
Modern Control 11/4
Chapter 10
Optimal Control
Algebraic Riccati Equation
 Consider again the n-dimensional state space equations:
x (t0 )  x 0
x(t )  Ax(t )  Bu(t )
but now with the following performance index to be minimized:



J   x (t )Qx (t )  u (t ) Ru(t ) dt
T
T
0
Q , R : symmetric, positive
semidefinite
 The control objective is to construct a stabilizing linear state
feedback controller of the form u(t) = –Kx(t) that at the same
time minimizes the performance index J.
The state feedback equation u(t) = –K x(t) is also called the “control
law.”
President University
Erwin Sitompul
Modern Control 11/5
Chapter 10
Optimal Control
Algebraic Riccati Equation
 First, assume that there exists a linear state feedback optimal
controller, such that the optimal closed-loop system:
x(t )   A  BK  x(t )
is asymptotically stable.
 Then, there exists a Lyapunov Function V = xT(t)P x(t) with a
positive definite matrix P, such that dV/dt evaluated on the
trajectories of the closed-loop system is negative definite.
 The synthesis of optimal control law involves the finding of an
appropriate Lyapunov Function, or equivalently, the matrix P.
President University
Erwin Sitompul
Modern Control 11/6
Chapter 10
Optimal Control
Algebraic Riccati Equation
 The appropriate matrix P is found by minimizing:
f  u(t )  
dV
T
 x T (t )Qx (t )  u (t ) Ru(t )
dt
 For unconstrained minimization,
df  u(t ) 
d  dV

T
T


x
(
t
)
Qx
(
t
)

u
(
t
)
Ru
(
t
)

 0
d u(t )
d u(t )  dt
 u(t )u* (t )
Optimal Solution
If u(t) = –K x(t) is so chosen that
min{f(u(t)) = dV/dt + xT(t)Qx(t) + uT(t)Ru(t)} = 0
for some V = xT(t)Px(t),
Then the controller using u(t) as control law is an optimal controller.
President University
Erwin Sitompul
Modern Control 11/7
Chapter 10
Optimal Control
Algebraic Riccati Equation
 The differentiation yields:
df  u(t ) 
d  dV

T
T


x
(
t
)
Qx
(
t
)

u
(
t
)
Ru
(
t
)


d u(t )  dt
d u(t )
 u(t )u* (t )


d
T

2 x T (t ) P x (t )  x T (t )Qx(t )  u (t ) Ru(t )
d u(t )
d
T
T
T
T

2
x
(
t
)
P
Ax
(
t
)

2
x
(
t
)
P
Bu
(
t
)

x
(
t
)
Qx
(
t
)

u
(t ) Ru(t )

d u(t )
 2 x T (t ) P B  2u (t ) R
T
0
d T
x (t ) Px (t )   x T (t ) Px (t )  x T (t ) Px (t ) d uT (t ) Ru(t )  2uT (t ) R d u(t )

dt
dt
dt
if P symmetric
d
T
T
 x T (t ) Px(t )  x T (t ) Px(t )
u (t ) Ru(t )  2u (t ) R
d u(t )
 2 x T (t ) P x (t )



President University
Erwin Sitompul

Modern Control 11/8

Chapter 10
Optimal Control
Algebraic Riccati Equation: Control Law
 Hence, incorporating the fact that P and R are symmetric, the
optimal control law can be written as:
1
u (t )   R B Px (t )
*
T
or
u (t )   K x (t )
*
1
KR B P
T
President University
Erwin Sitompul
Modern Control 11/9
Chapter 10
Optimal Control
Algebraic Riccati Equation: Minimum Test
 We now need to perform the “Second Derivative Test” to find out
whether u*(t) is a solution that minimizes f(u(t)).
Second Derivative Test
• If f’(x) = 0 and f”(x) > 0 then f has a local minimum at x
• If f’(x) = 0 and f”(x) < 0 then f has a local maximum at x
• If f’(x) = 0 and f”(x) = 0 then no conclusion can be drawn
 Performing the “Second Derivative Test”,
d 2 f  u(t ) 
d  u(t ) 
2

 dV

T
T

x
(
t
)
Qx
(
t
)

u
(
t
)
Ru
(
t
)

2 

d  u(t )   dt
d2

d
T
2 x T (t ) P B  2u (t ) R
d u(t )
 2R  0


 If the weight matrix R is chosen to be a positive definite matrix,
then the optimal solution u*(t) is indeed a solution that minimizes
f(u(t)).
President University
Erwin Sitompul
Modern Control 11/10
Chapter 10
Optimal Control
Algebraic Riccati Equation: Finding P
 Now, the appropriate matrix P must be found, in order to obtain
the optimal closed-loop system in the form of:
x(t )   A  BK  x(t )

1

x (t0 )  x 0
x (t )  A  BR B P x (t )
T
 The optimal controller with matrix P minimizes the cost function
f(u(t)), and will yield:
dV
dt
 x T (t )Qx (t )  u (t ) Ru (t )  0
*T
*
u ( t )  u* ( t )
x T (t ) Px(t )  x T (t ) Px(t )  x T (t )Qx(t )  u (t ) Ru(t )  0
T

 After some substitutions of x(t)
and later u*(t),


1
x T (t ) A P  P A x (t )  2 x T (t ) P BR B Px (t )
T
1
T
 x T (t )Qx(t )  x T (t ) P BR B Px (t )  0
President University
T
Erwin Sitompul
Modern Control 11/11
Chapter 10
Optimal Control
Algebraic Riccati Equation: Finding P
 After regrouping, we will obtain:

1

x T (t ) A P  P A  Q  P BR B P x (t )  0
T
T
 The equation above should hold for any x(t), which implies that:
1
A P  P A  Q  P BR B P  0
T
T
Algebraic Riccati Equation
(ARE)
 After solving the ARE for P, the optimal control law given by:
1
u (t )   R B Px (t )
*
T
can be applied to the linear system of
x(t )  Ax(t )  Bu(t ),
any x (t0 )  x 0
• The solution of ARE does not depend on
initial conditions
President University
Erwin Sitompul
Modern Control 11/12
Chapter 10
Optimal Control
Solving Optimal Control Problem in Matlab
 The solution of the ARE can be calculated easily in Matlab.
 The command to be used is:
[K,P,E] = lqr(A,B,Q,R,N)
 The performance index to be minimized is:



J   x T (t )Qx (t )  u (t ) Ru(t )  2 x T (t ) N u(t ) dt
T
0
 For the inputs: A and B are the system matrices, while Q, R, and
N are the weight matrices. In our case, N is set to zero.
 For the outputs, K is the optimal gain, P is the solution matrix of
the ARE, while E is the eigenvalues of the optimal system.
President University
Erwin Sitompul
Modern Control 11/13
Chapter 10
Optimal Control
Example 1: Optimal Control
Consider the following model:
x(t )  2u1 (t )  2u2 (t )
along with the performance index:

J    x 2 (t )  ru12 (t )  ru22 (t ) dt
0
Find the optimal control law for the system.
The matrices are:
 r 0
A  0, B   2 2 , Q  1, R  
,

0 r 
The ARE is solved as:
1
AT P  PA  Q  P BR B P  0
T
1 r 0   2
1  P  2 2 
P0



 0 1 r   2
President University
8
 1  P2  0
r
r
P
8
Erwin Sitompul
Modern Control 11/14
Chapter 10
Optimal Control
Example 1: Optimal Control
The optimal closed-loop system
is described by:
The control law is:
1
u (t )   R B Px(t )
*
T
*
1 r 0   2  r
 
x(t )



 0 1 r   2 8
1

2r
x(t )   2 2 u (t )
1
1 x(t )

1
  2 2  
2r
4

x(t )
2r
1
1 x(t )

Some conclusions:
• The system is a 1st order system with 2 inputs
• The optimal location of the system’s pole is at s = –4/ 2r
• Control effort u lightly weighted  r<<  pole location more to the
left  system faster  energy expense increases
• Control effort u heavily weighted  r>>  pole location more to
the right  system slower  energy expense decreases
President University
Erwin Sitompul
Modern Control 11/15
Chapter 10
Optimal Control
Example 2: Optimal Control
Consider the following continuous-time system:
0 1 
0 
x (t )  
x (t )    u (t )

0 a 
1 
y (t )  1 0 x (t )
Design an optimal controller that minimizes

J    x T (t )Qx (t )  u T (t ) Ru (t ) dt
with
0
1 0 
Q
,

0 0 
Rr
Give weight to x1(t)
No restriction for x2(t)
President University
Erwin Sitompul
Modern Control 11/16
Chapter 10
Optimal Control
Example 2: Optimal Control
 P1
1
K  R B P   0 1 
r
 P2
1
T
P2  1
  P2

P3  r
P3 
P is found by solving the ARE:
1
A P  P A  Q  P BR B P  0
T
0 0   P1
1  a   P

 2
T
P2   P1


P3   P2
P2  0 1  1 0 




P3  0 a  0 0 
 P1 P2  0  1
 P1 P2 

0
0 1 




 P2 P3  1  r
 P2 P3 
1 2

1  P2

r

 P  aP  1 P P
2
2 3
 1
r
President University
1

P1  aP2  P2 P3 
0 0
r
=
1 2  0 0 
2( P2  aP3 )  P3
r 
Erwin Sitompul
Modern Control 11/17
Chapter 10
Optimal Control
Example 2: Optimal Control
Three equations can be obtained:
1
1  P22  0
r
1
P1  aP2  P2 P3  0
r
1
2( P2  aP3 )  P32  0
r
P2   r
P32  2arP3  2r r  0
P3  ar  (ar )2  2r r
Thus, the optimal gain is given by:
K
1
 P2
r
1
P3    r
r 
ar  (ar ) 2  2r r 

The requested control law is:
1
*
u (t )   K x(t )    r
r 
President University
 ar  (ar )  2r r  x (t )

2
Erwin Sitompul
Some conclusions:
•…
•…
Modern Control 11/18
Chapter 10
Optimal Control
Example 2: Validation of Answer
From the previous example, let us now set a = 1:
0 1 
0
x (t )  
x (t )    u (t )

0 1
1 
y (t )  1 0 x (t )
Taking the positive out of both plus-minus signs, P2 and P3 can be
obtained. The solution of P will thus be:
 P1
P
 P2
1


2
r

(
r
)(

r

r

2
r
r
)
r
P2  

r


P3  
2

r
r  r  2r r 
( r  2 r )

r


2


r

r

r

2
r
r


President University
Erwin Sitompul
Modern Control 11/19
Chapter 10
Optimal Control
Example 2: Validation of Answer
The optimal feedback gain will be:
1
1 T
KR B P   r
r 
with:
 r  r 2  2r r 

1
*
u (t )   K x(t )    r
r 
r  r 2  2r r  x (t )

The optimal closed-loop matrix is given by:
Aˆ   A  BK 
 0 1  1 0  

2

 
   r r  r  2r r 



 0 1 r 1  

1
 0



 1 r  1  2 r 
1
 0

  1

 12
2

 1  2r 
 r
President University
Erwin Sitompul
Modern Control 11/20
Chapter 10
Optimal Control
Example 2: Validation of Answer
For r = 0.25, for example
1
K  r
r 
 r  r 2  2r r 

  2 1.2361
1
 0

Aˆ    1

 12
2

 1  2r 
 r
1 
0



2

2.2361


President University
• Location of
optimal poles
Erwin Sitompul
Modern Control 11/21
Chapter 10
Optimal Control
Example 2: Optimal State Feedback
Changing the control effort weight r will change the optimal gain K
and the location of the optimal poles E.
President University
Erwin Sitompul
Modern Control 11/22
Example 2: Optimal State Feedback
Using x0 = [2;–1], the response of the system with 3 different
weights r will now be compared.
• Why x1, not x2?
: r = 0.05
: r = 0.25
: r = 1.4
President University
• Smaller r results in larger magnitude of u,
since control effort is weighted less
• Smaller r results in smaller magnitude of
state x1, and faster convergence rate
Erwin Sitompul
Modern Control 11/23
Chapter 10
Optimal Control
Homework 11
The regulator shown in the figure below contains a plant that is
described by
0 1
0 
x (t )  
x (t )    u (t )

 1 2 
1 
y (t )  1 0 x (t )
r(t)  0
x(t )
u(t )
+
–
x = Ax+ bu
y(t )
c
k
and has a performance index
 T  2 0

2
J    x (t ) 
x(t )  u (t )  dt

0 1
0


Determine
a) The Riccati matrix P
b) The state feedback matrix k
c) The closed-loop eigenvalues
President University
Erwin Sitompul
Modern Control 11/24
Chapter 10
Optimal Control
Homework 11A
Consider the system described by the equations
0 1 
0
x (t )  
x
(
t
)

u (t )



0 0
1 
y (t )  1 0 x (t )
(a) Determine the optimal control law u*(t) which minimizes the
following performance index. (Hint: Use Algebraic Riccati
Equation)

J1    y 2 (t )   u 2 (t )  dt
0
(b) Use ρ = 0.1·StID and calculate the numerical result of K. Verify
your answer using Matlab. (Hint: Learn more about command
“lqr”)
 Deadline: 3 December 2014.
President University
Erwin Sitompul
Modern Control 11/25