Modern Control
Lecture 10
Dr.-Ing. Erwin Sitompul
President University
http://zitompul.wordpress.com
2 0 1 4
President University
Erwin Sitompul
Modern Control 10/1
Chapter 8
State Feedback and State Estimators
Homework 9
Refer to the last example.
(a) Calculate the transfer function G(s) of the system.
(b) Calculate the steady-state value of the system to a step input,
using the Final Value Theorem of Laplace Transform.
(c) Determine the gain K so that the steady-state response of
KG(s) has zero error to a step input.
(d) Find out the relation between the transfer function gain K and
the reference gain E.
President University
Erwin Sitompul
Modern Control 10/2
Chapter 8
State Feedback and State Estimators
Solution of Homework 9
(a) Calculate the transfer function of the system in s-Domain.
Y ( s)
U (s)
1
  s 0  2 1 1 
 1 
 1 1  

    4 1  


 0 s   1 1  2
  2
G( s)  c( s I  A  bk ) 1 b 
1
  s 0  2 1  4 1   1 
 1 1  




   2
0
s

1
1
8
2
 
 
  

1
s

2
0

 1 
 1 1 
 2
9
s

1

  
0  1 
s  1
1 1 
 2

9
s

2

 

s 2  3s  2
3s  4
G( s)  2
s  3s  2
President University
Erwin Sitompul
Modern Control 10/3
Chapter 8
State Feedback and State Estimators
Solution of Homework 9
(b) Calculate the steady-state value of the step response of the
system, using the Final Value Theorem of Laplace Transform.
y ()  lim y (t )  lim s Y ( s)  lim s G ( s)U ( s)
t 
s 0
s 0
3s  4 1
 lim s  2

s 0
s  3s  2 s
4

2
 2
President University
Erwin Sitompul
Modern Control 10/4
Chapter 8
State Feedback and State Estimators
Solution of Homework 9
(c) Determine the gain K so that the steady-state response of KG(s)
has zero error to a step input.
y ()  lim s G ( s )U ( s)  2
s 0
y ()  lim s KG ( s)U ( s)  u ()  1(t )
s 0
 K  0.5
(d) Find out the relation between the transfer function gain K and
the reference gain E.
KE
a0
K
b0
President University
Erwin Sitompul
Modern Control 10/5
Chapter 10
Optimal Control
Optimal Control: The Motivation
 Consider a car traveling on a straight
line through a hilly road.
 Case 1: How should the driver press
the gas pedal in order to minimize the
total traveling time while keeping the
fuel consumption economical?
 Case 2: How to travel with as cheap
fuel consumption as possible, without
regarding time?
 Case 3: How if time is very important,
no matter how much the fuel cost is.
 Although the system is the same (i.e. the car), the solution for
various cases can be very different. The way the driver behaves
depends on the priority of time or cost, or the mix of both.
President University
Erwin Sitompul
Modern Control 10/6
Chapter 10
Optimal Control
Optimal Control: The Motivation
 Previously, we already can design state-feedback control or
output-feedback control, by placing the poles of the system in a
certain location.
 We do not consider any constraints until now about the location of
the poles.
 In order to get faster response and converge, the poles can be
simply placed far to the left of imaginary axis, with bigger
negative real parts.
 But, having such poles means that the elements of k have to
be large. This means requirement for large control signal, large
capacity, and more energy.
 Thus, there is a conflict between “fast response” and “low price.”
The question is: how can we balance between good transient
response and small control effort?
President University
Erwin Sitompul
Modern Control 10/7
Chapter 10
Optimal Control
Optimal Control
 Optimal control deals with the problem of finding a control law
(in this case, in the form of feedback gain k) for a given system,
such that a certain optimality on a criterion is achieved.
 The resulting control law is optimal solution of a cost function or
performance index.
 A proper cost function (or performance index) will give a
mathematical expression that includes all variables which need to
be optimized. The variables can be state variables x, control input
u, control error e, time t, etc.
President University
Erwin Sitompul
Modern Control 10/8
Chapter 10
Optimal Control
Performance Indexes
 A performance index is a mean to measure the control
performance in the completion of a control objective.
 Suppose the control objective is to bring a system modeled by:
x(t )  Ax(t )  Bu(t )
y (t )  C x (t )
x (t0 )  x 0
in such a way and in a fixed time interval [to,tf ], so that the
components of the state vector are “small”.
 A suitable performance index to be minimized would be:
tf
J1   x T (t ) x (t )dt
to
President University
Erwin Sitompul
Modern Control 10/9
Chapter 10
Optimal Control
Performance Indexes
 If the control objective is to manipulate the system so that the
components of output y(t) are to be small, then:
tf
J 2   y (t ) y (t )dt
T
to
tf
  x T (t )C C x (t )dt
T
to
tf
  x T (t )Qx (t )dt
to
where Q = CTC is a symmetric positive semidefinite matrix.
A matrix is a positive semidefinite matrix if all the real parts of its
eigenvalues are more or equal to zero, OR, if for all nonzero vector z
with real entries, then zTQ z ≥ 0.
President University
Erwin Sitompul
Modern Control 10/10
Chapter 10
Optimal Control
Performance Indexes
 In order to control the inputs u(t) so that they are not too large,
we use the following performance index:
tf
J 3   u (t )u(t )dt
T
to
tf
J 4   u (t ) Ru(t )dt
T
to
where R is a symmetric positive semidefinite weight matrix.
 Not all performance indexes can be minimized at the same time.
A compromise can be taken by minimizing a convex combination
of the indexes, e.g.:
J 5   J1  (1   ) J 3
tf


   x (t ) x (t )  (1   )u (t )u(t ) dt,
T
T
0   1
to
President University
Erwin Sitompul
Modern Control 10/11
Chapter 10
Optimal Control
Performance Indexes
 The weighting constants can also be accommodated in the weight
matrices Q and R as follows:
tf


J 6   x T (t )Qx (t )  u (t ) Ru(t ) dt
T
to
with

0
Q


0
0

President University

0
0
0
0 
1  

 0 1 

, R  


0
1 
0 




0 1  
 0
Erwin Sitompul
Modern Control 10/12
Chapter 10
Optimal Control
Weight Matrices Q and R
 The weight matrices Q and R can be used to assign different
weighting to each state and each input, in the calculation of the
performance index.
 Some examples are:
y (t )Q y(t )   y1
T
1 0  y1 
2
2

y

5
y
y2  
1
2
 y 
0
5

 2
u (t ) Ru(t )  u1 u2
T
President University
10 0 0   u1 
u3   0 1 0  u2   10u12  u22  0.1u32
 0 0 0.1 u3 
Erwin Sitompul
Modern Control 10/13
Chapter 10
Optimal Control
Some Performance Indexes for Optimal Control Problems
1. Minimum-time problems
tf
J  tf  to   dt
to
2. Terminal control problems
J   x (tf )  x r (tf )  H  x (tf )  x r (tf ) 
T
x (tf ) : final state
x r (tf ) : reference final state
3. Minimum control effort problems
tf
tf
to
to
2
T
J    u (t ) Ru(t )  dt   u(t ) R dt
4. Tracking problems
tf

J   x (t )  x r (t )
to
President University
2
Q
 u(t )
2
R
 x (tf )  x r (tf )
Erwin Sitompul
2
H
 dt
Modern Control 10/14
Chapter 10
Optimal Control
Optimal Control
 In modern optimal control theory, the following cost function is
mostly implemented:
tf


J 6   x T (t )Qx (t )  u (t ) Ru(t ) dt
T
to
 This choice is based on a certain logics. It requires minimization of
the square of input u (which in general means energy required to
control a system), and minimization of the square of the state
variables x.
 Why should we minimize x, not e? The history of optimal control
development starts from the implementation where state variables
x should be minimized. In this implementation, the state is equal
to the deviation from a set point zero.
 Besides, the same thing applies also for linearized system. Here,
the state Δx denotes the deviation from x0, the operating point
where the linearization was taken.
 In electric systems, i.e., the input can be electric
current (i) or electric potential (V), which are
quadratically proportional to electric power (P)
President University
Erwin Sitompul
V2 2
P  Vi 
i R
R
Modern Control 10/15
Chapter 10
Optimal Control
Optimal Control
 An example for such implementation is the
benchmark problem inverted pendulum. The
states of the system: the deviation angle θ
and the position x, should be guided to zero
as fast as possible.
(M  m) x  ml cos   ml 2 sin   F
(l  ml 2 )  mgl sin   mlx cos 
 Or recall again, the water-tank system. The q
i
system is nonlinear due to h1/2.
h  qi A  a1 A 2 gh
yh
V
h
qo
 h 
  12 a1 A 2 g h0  h  1 A  qi
y  h
President University
Erwin Sitompul
v1
h  h0  h
y  y0  y  h
Modern Control 10/16
Chapter 10
Optimal Control
Optimal Control
 The regulation of the state variables to zero can be modified by
shifting the point of origin. Then, the state variables can be
regulated to any constant values.
 We will first discuss the optimal solution to minimize state
variables x along with control effort u.
 Afterwards, we will discuss the solution to minimize control error e
along with control effort u.
President University
Erwin Sitompul
Modern Control 10/17
Chapter 10
Optimal Control
Optimal Control: Minimizing State Variables
 Consider the n-dimensional single-variable state space equations:
x(t )  Ax(t )  bu(t )
y(t )  cx(t )
 The desired state vector is represented xd(t) = 0.
 We will select a feedback controller, so that u(t) is some function
of the measured state variables x(t), and therefore
u(t )  k x(t )
 Substituting into the first equation, we obtain
ˆ (t )
x(t )  Ax(t )  bk x(t )  Ax
President University
Erwin Sitompul
Modern Control 10/18
Chapter 10
Optimal Control
Optimal Control: Minimizing State Variables
 The performance index to be minimized is the one expressed in
terms of the state vector
tf
J1   x T (t ) x (t )dt
to
 In minimizing J1, we let the final time of interest be tf = ∞.
 To obtain the minimum value of J1, we postulate a decreasing
magnitude of the state vector, or equivalently, postulate its
negative first derivation.
 We assume the existence of an exact differential so that
d T
x (t ) Px (t )    x T (t ) x (t )

dt
where P is yet to be determined.
 P is chosen to be symmetric to simplify the algebra without any
loss of generality.
President University
Erwin Sitompul
Modern Control 10/19
Chapter 10
Optimal Control
Optimal Control: Minimizing State Variables
 Then, for a symmetric P, pij = pji, the differentiation on the lefthand side of the last equation can be completed as
d T
x (t ) Px (t )   x T (t ) Px (t )  x T (t ) Px (t )

dt
ˆ
 But x (t )  Ax (t )
thus



T
d T
T
ˆ
ˆ (t )
x
(
t
)
Px
(
t
)

Ax
(
t
)
Px
(
t
)

x
(t ) P Ax


dt
T
T
ˆ
 x (t ) A P x (t )  x T (t ) P Aˆ x (t )



T
ˆ
 x (t ) A P  P Aˆ x (t )
T
T
ˆ
ˆ   I then d  x T (t ) Px (t )    x T (t ) x (t ) as wished.
 If A P  P A
dt
President University
Erwin Sitompul
Modern Control 10/20
Chapter 10
Optimal Control
Optimal Control: Minimizing State Variables
 Further


d T
T
J1   x (t ) x (t ) dt     x (t ) P x (t )  dt   x (t ) P x (t )  x T (0) P x (0)
0
dt
0
0
T
 Therefore, to minimize the performance index J1, the following
two equations must be considered

J1   x T (t ) x (t )dt  x T (0) P x (0)
0
T
ˆ
A P  P Aˆ   I
President University
Erwin Sitompul
Modern Control 10/21
Chapter 10
Optimal Control
Example 1: Optimal State Feedback
Consider the open-loop control system shown in the figure below.
The state variables are x1 and x2. The step response of the system is
quite unsatisfactory due to undamped poles at the origin.
0 1 
0
x (t )  
x (t )    u (t )

0 0
1 
U (s)
1
s
X2
1
s
Y (s)
X1
We will choose a feedback control in the form of
u(t )  k x(t )  k1 x1 (t )  k2 x2 (t )
Therefore, inserting u(t) into the state equation,
1 
 0
ˆ (t )
x (t )  
x(t )  Ax

k1 k2 
President University
Erwin Sitompul
Modern Control 10/22
Chapter 10
Optimal Control
Example 1: Optimal State Feedback
To minimize the performance index J1, we write again the equation
T
ˆ
A P  P Aˆ   I
0 k1   p11
1  k   p

2   12
p12   p11


p22   p12
 k1 p12  p12 k1
 k p  p  k p
 1 22
11
2 12
p12   0
1   1 0 




p22   k1 k2   0 1
k1 p22  p11  k2 p12
  1 0 


p12  k2 p22  p12  k2 p22   0 1
Letting k1 equals an arbitrary stable value, say k1 = 2, then
1
3
k22  6
p12  , p22 
, p11 
4
4k 2
4k 2
President University
Erwin Sitompul
Modern Control 10/23
Chapter 10
Optimal Control
Example 1: Optimal State Feedback
The integral of the performance index is rewritten as

J1   x T (t ) x (t )dt  x T (0) P x (0)
0
We will consider the case where the initial states are given as
xT(0) = [1 2], so that
 p11
J1  1 2 
 p12
p12  1 
 p11  2 p12 
 1 2 




p

2
p
p22  2
 12
22 
 p11  4 p12  4 p22
Substituting the values of P we already have,
k22  4k2  18
k22  6
3

J1 
1
4k 2
4k 2
k2
President University
Erwin Sitompul
Modern Control 10/24
Chapter 10
Optimal Control
Example 1: Optimal State Feedback
To minimize J1 as a function of k2, we take the derivative w.r.t k2 and
set it equal to zero
dJ1 (2k2  4)(4k2 )  (4)(k22  4k2  18)

0
2
dk2
(4k2 )
 4k22  72  0  k2  18
Thus, the system matrix for the compensated system is
1 
0
ˆ (t )
x (t )  
 x (t )  Ax
 2  18 
Checking the characteristic equation of the compensated system,
 s
1  
2
ˆ
a( s )  det( s I  A)  det  

s
 18s  2

 2 s  18  


s1,2  2.121  1.581
• Both poles are now with negative real value
• The system is optimal for the given values of k1 and x(0)
President University
Erwin Sitompul
Modern Control 10/25
Chapter 10
Optimal Control
Example 1: Validation of Answer
Now, suppose we want to stabilize the given system and put the
poles to certain locations by using a non-optimal feedback gain k.
0 1 
0
x (t )  
x (t )    u (t ),

0 0
1 
u(t )    k1 k2  x(t )
1 
 0
 x (t )  
x (t )

 k1 k2 
 s
a(s)  det(s I  ( A  bk ))  det  
  k1
1  
2

s
 k2 s  k1


s  k2  
• Any positive value for k1 and k2 will guarantee the stability
of the system
• With no real input r(t) –the system only have pure state
feedback connection– the output will decay to zero
• The integral of squared states of the system using kopt will
now be compared with any arbitrary chosen k
• Note: k1 = 2 must be kept
President University
Erwin Sitompul
Modern Control 10/26
Chapter 10
Optimal Control
Example 1: Validation of Answer
President University
Erwin Sitompul
Modern Control 10/27
Chapter 10
Optimal Control
Example 1: Validation of Answer
: J1, kopt = [2 4.24264]
: J1, k = [2 2]
: J1, k = [2 10]
: J1, no k
• Without any state
feedback, the output of
the double integrator will
sum up to infinity
President University
• kopt results the lowest
value of cost function J1
Erwin Sitompul
Modern Control 10/28
Chapter 10
Optimal Control
Homework 10
Consider again the control system as given before, described by
0 1 
0
x (t )  
x
(
t
)

u (t )



0 0
1 
y (t )  1 0 x (t )
Assuming the linear control law
u(t )  k x(t )  k1 x1 (t )  k2 x2 (t )
Determine the constants k1 and k2 so that the following performance
index is minimized

J1   x T (t ) x (t )dt  x T (0) P x (0)
0
Consider only the case where the initial condition is x(0)=[c 0]T and
the undamped natural frequency (ωn) is chosen to be 2 rad/s.
• Calculate the transfer function of the
system if compensated with k
• Determine the value of corresponding k (k1
or k2?) to obtain ωn as requested
President University
Erwin Sitompul
Modern Control 10/29
Chapter 10
Optimal Control
Homework 10A
Consider the system described by the equations
0 1 
0
x (t )  
x
(
t
)

u (t )



0 0
1 
y (t )  1 0 x (t )
Determine the optimal control which minimizes the following
performance index. (Hint: You may assume the value of the
feedback gain, where available.)

J1   y 2 (t )dt
0
 Deadline: Wednesday, 26 November 2014.
 Quiz 3: Lecture 8, 9, and 10.
President University
Erwin Sitompul
Modern Control 10/30