SLOVAK UNIVERSITY OF TECHNOLOGY
Faculty of Material Science and Technology in Trnava
AUTOMATIC CONTROL THEORY II
prof. Dr. Ing. Oliver Moravčík
TRNAVA 2007
Automatic Control Theory II
Contents
Optimal control ............................................................................................................. 3
Extremal control of Wiener model processes ............................................................... 8
Homogeneous Nonlinear Systems .............................................................................. 11
Lyapunov Stability Theory ......................................................................................... 26
Adaptive Control (Model reference adaptive control - MRAC) ................................ 32
Modeling Complex Systems ....................................................................................... 38
2
Automatic Control Theory II
Optimal control
Formulation of optimal control problems
There are various types of optimal control problems, depending on the performance
index, the type of time domain (continuous, discrete), the presence of different types of
constraints, and what variables are free to be chosen. The formulation of an optimal
control problem requires the following:

a mathematical model of the system to be controlled,

a specification of the performance index,

a specification of all boundary conditions on states, and constraints to be
satisfied by states and controls,

a statement of what variables are free.
General case with fixed final time and no terminal or path constraints
If there are no path constraints on the states or the control variables, and if the initial and
final times are fixed, a fairly general continuous time optimal control problem can be
defined as follows:
Problem 1: Find the control vector trajectory
to minimize the
performance index:
(1)
subject to:
(2)
where
is the time interval of interest,
is the state vector,
is a terminal cost function,
intermediate cost function, and
is an
is a vector field. Note that
equation (2) represents the dynamics of the system and its initial state condition.
Problem 1 as defined above is known as the Bolza problem. If
problem is known as the Mayer problem, if
problem. Note that the performance index
, then the
, it is known as the Lagrange
is a functional, this is a rule of
correspondence that assigns a real value to each function u in a class. Calculus of
variations is concerned with the optimization of functionals, and it is the tool that is
3
Automatic Control Theory II
used in this section to derive necessary optimality conditions for the minimisation of
J(u).
Adjoin the constraints to the performance index with a time-varying Lagrange
multiplier vector function
augmented performance index
(also known as the co-state), to define an
:
(3)
Define the Hamiltonian function H as follows:
(4)
such that
can be written as:
Assume that
is denoted as
and
are fixed. Now consider an infinitesimal variation in
, that
. Such a variation will produce variations in the state history
and a variation in the performance index
,
:
Since the Lagrange multipliers are arbitrary, they can be selected to make the
coefficients of
and
equal to zero, as follows:
(5)
(6)
This choice of
results in the following expression for
state is fixed, so that
, assuming that the initial
:
For a minimum, it is necessary that
. This gives the stationarity condition:
. (7)
4
Automatic Control Theory II
Equations (2), (5), (6), and (7) are the first-order necessary conditions for a minimum of
J. Equation (5) is known as the co-state (or adjoint) equation. Equation (6) and the
initial state condition represent the boundary (or transversality) conditions. These
necessary optimality conditions, which define a two point boundary value problem, are
very useful as they allow to find analytical solutions to special types of optimal control
problems, and to define numerical algorithms to search for solutions in general cases.
Moreover, they are useful to check the extremality of solutions found by computational
methods. Sufficient conditions for general nonlinear problems have also been
established. Distinctions are made between sufficient conditions for weak local, strong
local, and strong global minima. Sufficient conditions are useful to check if an extremal
solution satisfying the necessary optimality conditions actually yields a minimum, and
the type of minimum that is achieved.
The theory presented above does not deal with the existence of an optimal control that
minimises the performance index J.
The linear quadratic regulator
A special case of optimal control problem which is of particular importance arises when
the objective function is a quadratic function of x and u, and the dynamic equations are
linear. The resulting feedback law in this case is known as the linear quadratic regulator
(LQR). The performance index is given by:
(8)
where
and
are positive semidefinite matrices, and
is a positive definite matrix,
while the system dynamics obey:
(9)
where A is the system matrix and B is the input matrix.
In this case, using the optimality conditions given above, it is possible to find that the
optimal control law can be expressed as a linear state feedback:
(10)
where the state feedback gain is given by:
(11)
and S(t) is the solution to the differential Ricatti equation:
5
Automatic Control Theory II
(12)
In the particular case where
, and provided the pair (A,B) is stabilizable, the
Ricatti differential equation converges to a limiting solution S, and it is possible to
express the optimal control law as a state feedback as in (10) but with constant gain K.
which is given by
where S is the positive definite solution to the algebraic Ricatti equation:
(13)
Moreover, if the pair (A,C) is observable, where
, then the closed loop
system
(14)
is asymptotically stable. This is an important result, as the linear quadratic regulator
provides a way of stabilizing any linear system that is stabilizable. It is worth pointing
out that there are well established methods and software for solving the algebraic Ricatti
equation (13). This facilitates the design of linear quadratic regulators. A useful
extension of the linear quadratic regulator ideas involves modifying the performance
index (8) to allow for a reference signal that the output of the system should track.
Moreover, an extension of the LQR concept to systems with gaussian additive noise,
which is known as the linear quadratic gaussian (LQG) controller, has been widely
applied. The LQG controller involves coupling the linear quadratic regulator with the
Kalman filter using the separation principle.
6
Automatic Control Theory II
Minimum time problems
One special class of optimal control problem involves finding the optimal input u(t) to
reach a terminal constraint in minimum time. This kind of problem is defined as
follows.
Problem 2: Find
and
to minimise:
subject to:
7
Automatic Control Theory II
Extremal control of Wiener model processes
There are many application where it is of interest to position the process output at an
optimum or extremum point. A typical situation is combustion engines where the
emission and efficiency depend on the inputs to the motor such as fuel and air/fuel ratio.
Other examples are control of grinding processes, water turbines, and wind mills. It is
therefore of interest to study processes having an extremum value in the output and to
be able to operate the system as close as possible to the extremum point.
The problem of extremum control can be approached in several ways. Among the first
approaches was the introduction of perturbation signals. A perturbation signal is then
used to get information about the local gradient of the nonlinearity. This is done by
comparing the phase of the perturbation and its influence in the output. The input signal
is then changed using a gradient method to find the extremum point. When a
perturbation signal is introduced the dynamics of the process will influence the response
of the system and this can corrupt the estimation of the gradient.
The perturbation signal method is usually only used to find a constant value of the input
and/or to be able to follow a varying operating point. The process will then behave as an
open loop system around the extremum point. The perturbation signal method has the
advantage that it requires very little information about the process. On the other hand
the convergence of the system and the steady state performance are not very good,
especially in presence of noise. A second approach is to use more advanced
optimization methods. If the nonlinearity is a known function the optimal constant input
might be computed directly. This method has the drawback that the static nonlinearity
and the open loop gain of the process have to be known. Further, the performance at the
extremum point is still as if it were an open loop system. This implies that if the open
loop dynamics of the process is slow then the convergence and recovery after a
disturbance will be slow. There are different classifications of nonlinear systems and we
will discuss two different classes of systems. The first class of models is called
Hammerstein models where the nonlinearity is at the input of a linear dynamic
subsystem. The Hammerstein models have the advantage that the models are linear in
the parameters, which makes it easy to estimate the parameters of the model. In the
second class of models, Wiener models, the system has a linear part followed by a
nonlinearity.
8
Automatic Control Theory II
Problem formulation
We assume that the process is a Wiener model where the linear part is described by the
known discretetime system
(1)
where u(k) is the input signal, z(k) the output of the linear part, and e(k) is Gaussian
distributed white noise with zero mean and standard deviation σ
The model can also be written in polynomial form
(2)
where q is the forward shift operator and deg A = deg C = n and deg B = n − d. Further,
A and C are monic, i.e. the coefficient of the largest power of q is equal to one. The
parameter d is the time delay in the system. The model (1) can also be written in
statespace form.
The nonlinearity is described as a quadratic function of the form
(3)
with γ2 ≠ 0. Other types of nonlinearities can also be assumed. However, we assume, at
least close the optimum point, that the nonlinearity can be described by a quadratic
function. The nonlinearity has an optimum point, maximum or minimum, depending on
the parameter γ . For the sake of simplicity we assume that the extremum point is a
minimum, i.e. γ
2
≠ 0. The minimum of y(k) is obtained for
(4)
The minimum of y(k) is
(5)
Independent of the value of z(k) the output can never be below the value y0.
The control signal, u(k) is allowed to be a function of the process output y(k) and
previous inputs and outputs. In the derivation of some of the controllers we will also
assume that the control signal may be a function of the outputs of the linear system or
its state, i.e. of z(j),
.
9
Automatic Control Theory II
The purpose of the control is to keep the output y(k) as close as possible to the optimum
point y0. The loss function is formally expressed as
(6)
Static controller
Assume that there is no noise acting on the system and assume that the input to (2) is
constant then z(k) = z0 if
(7)
Using (7) on the system (2) gives
The second equality follows since z0 is constant. This implies that the mean value of z is
equal to z0 but the variation around z0 is determined by the open loop noise dynamics
C/A. The output y will thus deviate from the desired value y0. The variable z is a
Gaussian process but the output y is a noncentral 2 distribution. If the open loop system
has slow dynamics then the convergence of z will be slow at the startup or after the
noise process has driven z away from its desired value. The controller (7) can be
regarded as a one step minimization of the quadratic nonlinearity opposed to the use of
the gradient method when using a perturbation signal. Using (7) gives
(8)
This implies that
(9)
where σ2v is the variance of the process v(k), which is the same as the open loop
variance of the process, i.e. the controller gives the correct mean value, but the
stochastic part of the system is not influenced.
10
Automatic Control Theory II
Homogeneous Nonlinear Systems
Corresponding to the real-valued function of n variables hn(t1, . . . ,tn) defined
for ti ε (−∞,∞), i = 1, . . . ,n, and such that hn(t1, . . . ,tn) = 0 if any ti < 0, consider the
input/output relation
(5)
The resemblance to the linear system representations of the previous section is clear.
Furthermore the same kinds of technical assumptions that are appropriate for the
convolution representation for linear systems are appropriate here. Indeed, (5) often is
called generalized convolution, although I won’t be using that term.
Probably the first question to be asked is concerned with the descriptors that can be
associated to a system represented by (5). It is obvious that the assumption that
hn(t1, . . . ,tn) is one-sided in each variable corresponds to causality. The system is not
linear, but it is a stationary system as a check of the delay invariance property readily
shows.
A system represented by (5) will be called a degree-n homogeneous system. The
terminology arises because application of the input αu (t), where α is a scalar, yields the
output αny (t), where y (t) is the response to u (t). Note that this terminology includes
the case of a linear system as a degree-1 homogeneous system. Just as in the linear case,
hn(t1, . . . ,tn) will be called the kernel associated with the system.
For simplicity of notation I will collapse the multiple integration and, when no
confusion is likely to arise, drop the subscript on the kernel to write (5) as
(6)
Just as in the linear case, the lower limit(s) can be replaced by 0 because of the onesided assumption on the kernel. If it is assumed also that the input signal is one-sided,
then all the upper limit(s) can be replaced by t. Finally, a change of each variable of
integration shows that (6) can be rewritten in the form
(7)
11
Automatic Control Theory II
At this point it should be no surprise that a stationary degree-n homogeneous system
will be diagramed as shown in Figure 1.3. Again the system box is labeled with the
kernel.
There are at least two generic ways in which homogeneous systems can arise in
engineering applications. The first involves physical systems that naturally are
structured in terms of interconnections of linear subsystems and simple nonlinearities.
In particular I will consider situations that involve stationary linear subsystems, and
nonlinearities that can be represented in terms of multipliers. For so-called
interconnection structured systems such as this, it is often easy to derive the overall
system kernel from the subsystem kernels simply by tracing the input signal through the
system diagram. (In this case subscripts will be used to denote different subsystems
since all kernels are single variable.)
Example 1.1
Consider the multiplicative connection
of three linear subsystems, shown in Figure 1.4. The linear subsystems can be described
by
and thus the overall system is described by
12
Automatic Control Theory II
Clearly, a kernel for this degree-3 homogeneous system is
A second way in which homogeneous systems can arise begins with a state equation
description of a nonlinear system. To illustrate, consider a compartmental model
wherein each variable xi(t) represents a population, chemical concentration, or other
quantity of interest. If the rate of change of xi(t) depends linearly on other xj(t)’s, but
with a scalar parametric control signal, then xi(t) will contain terms of the form du
(t)xj(t). Nonlinear compartmental models of this type lead to the study of so-called
bilinear state equations
where x (t) is the n x 1 state vector, and u (t) and y (t) are the scalar input and output
signals. Such state equations will be discussed in detail later on, so for now a very
simple case will be used to indicate the connection to homogeneous systems.
Example 1.2
Consider a nonlinear system described by the differential equation
where x (t) is a 2 x 1 vector, u (t) and y (t) are scalars, and
It can be shown that a differential equation of this general form has a unique solution
for all t ≥ 0 for a piecewise continuous input signal. I leave it as an exercise to verify
that this solution can be written in the form
where, of course, the matrix exponential is given by
For the particular case at hand, D2 = 0 so that
13
Automatic Control Theory II
Thus the input/output relation can be written in the form
From this expression it is clear that the system is homogeneous and of degree 2. To put
the input/output representation into a more familiar form, the unit step function
can be introduced to write
Thus, a kernel for the system is
There will be occasion in later chapters to consider homogeneous systems that may not
be stationary. Such a system is represented by the input/output expression
(8)
It is assumed that the kernel satisfies h (t,σ1, . . . ,σn) = 0 when any σi > t so that the
system is causal. Of course, this permits all the upper limits to be replaced by t. If
onesided inputs are considered, then the lower limits can be raised to 0.
As a simple example of a nonstationary homogeneous system, the reader can rework
Example 1.1 under the assumption that the linear subsystems are nonstationary. But
I will consider here a case where the nonstationary representation quite naturally arises
from a stationary interconnection structured system.
14
Automatic Control Theory II
Example 1.3
The interconnection shown in Figure 1.5 is somewhat more complicated than that
treated in Example 1.1. As suggested earlier, a good way to find a kernel is to begin
with the input signal and find expressions for each labeled signal, working toward the
output. The signal v (t) can be written as
Similarly
The output signal is given by
Thus a kernel for this degree-3 system can be written in the form
Because of the usual one-sided assumptions on the linear subsystem kernels, the step
functions might be regarded as superfluous. More importantly, a comparison of
Examples 1.1 and 1.3 indicates that different forms of the kernel are more natural for
different system structures.
15
Automatic Control Theory II
Comparing the representation (8) for nonstationary systems to the representation (7) for
stationary systems leads to the definition that a kernel h (t,σ1, . . . ,σn) is stationary if
there exists a kernel g (t1, . . . ,tn) such that the relationship
(9)
holds for all t,σ1, . . . ,σn. Usually it is convenient to check for stationarity by checking
the functional relationship
(10)
for if this is satisfied, then (9) is obtained by setting
(11)
Therefore, when (10) is satisfied I can write, in place of (8),
(12)
Performing this calculation for Example 1.3 gives a stationary kernel for the system in
Figure 1.5:
In the theory of linear systems it is common to allow impulse (generalized) functions in
the kernel. For example, suppose h (t) = g (t) + g0δ0(t), where g (t) is a piecewise
continuous function and δ0(t) is a unit impulse at t = 0. Then the response to an input
u(t) is
(13)
That is, the impulse in the kernel corresponds to what might be called a direct
transmission term in the input/output relation. Even taking the input u (t) = δ0(t) causes
no problems in this set-up. The resulting impulse response is
(14)
16
Automatic Control Theory II
Unfortunately these issues are much more devious for homogeneous systems of degree
n > 1. For such systems, impulse inputs cause tremendous problems when a direct
transmission term is present. To see why, notice that such a term must be of degree n,
and so it leads to undefined objects of the form δ0n (t) in the response. Since impulsive
inputs must be ruled out when direct transmission terms are present, it seems prudent to
display such terms explicitly. However, there are a number of different kinds of terms
that share similar difficulties in the higher degree cases, and the equations I am
presenting are sufficiently long already. For example, consider a degree-2 system with
input/output relation
(15)
Adopting a loose terminology, I will call both of the latter two terms direct transmission
terms. Allowing impulses in the kernel means that the representation
suffices with
(16)
The dangers not withstanding, impulses will be allowed in the kernel to account for the
various direct transmission terms. But as a matter of convention, a kernel is assumed to
be impulse free unless stated otherwise. I should point out that, as indicated by the
degree-2 case, the impulses needed for this purpose occur only for values of the kernel’s
arguments satisfying certain patterns of equalities.
Example 1.4
A simple system for computing the integral-square value of a signal is shown in Figure
1.6.
This system is described by
17
Automatic Control Theory II
so that a standard-form degree-2 homogeneous representation is
If the input signal is one-sided, then the representation can be simplified to
On the other hand, a simple system for computing the square-integral of a signal is
shown in Figure 1.7.
This system is described by
If the input signal is one-sided, then the representation simplifies to
Comparison of these two systems indicates that direct transmission terms (impulsive
kernels) arise from unintegrated input signals in the nonlinear part of the system.
A kernel describing a degree-n homogeneous system will be called separable if it can be
expressed in the form
(17)
or
(18)
where each vji (.) is a continuous function. It will be called differentiably separable if
each vji (.) is differentiable. Almost all of the kernels of interest herein will be
18
Automatic Control Theory II
differentiably separable. Although explicit use of this terminology will not occur until
much later, it will become clear from examples and problems that separability is a
routinely occurring property of kernels.
The reader probably has noticed from the examples that more than one kernel can be
used to describe a given system. For instance, the kernel derived in Example 1.1 can be
rewritten in several ways simply by reordering the variables of integration. This feature
not only is disconcerting at first glance, it also leads to serious difficulties when system
properties are described in terms of properties of the kernel. Therefore, it becomes
important in many situations to impose uniqueness by working with special, restricted
forms for the kernel. Three such special forms will be used in the sequel: the symmetric
kernel, the triangular kernel, and the regular kernel. I now turn to the introduction of
these forms.
A symmetric kernel in the stationary case satisfies
(19)
or, in the nonstationary case,
(20)
where π(.) denotes any permutation of the integers 1, . . . ,n. It is easy to show that
without loss of generality the kernel of a homogeneous system can be assumed to be
symmetric. In fact any given kernel, say h (t 1, . . . ,tn) in (6), can be replaced by a
symmetric kernel simply by setting
(21)
where the indicated summation is over all n ! permutations of the integers 1 through n.
To see that this replacement does not affect the input/output relation, consider the
expression
(22)
Introducing the change of variables (actually, just a relabeling) τi = σπ(i), i = 1, . . . ,n, in
19
Automatic Control Theory II
every term of the summation in (22) shows that all terms are identical. Thus summing
the n ! identical terms on the right side shows that the two kernels yield the same
input/output behavior.
Often a kernel of interest is partially symmetric in the sense that not all terms of the
summation in (21) are distinct. In this situation the symmetric version of the kernel can
be obtained by summing over those permutations that give distinct summands, and
replacing the n ! by the number of such permutations. A significant reduction in the
number of terms is often the result.
Example 1.5
Consider a degree-3 kernel that has the form
Incidently, note that this is not a separable kernel unless f (t 1+t2) can be written as a
sum of terms of the form f 1(t 1)f 2(t 2). To symmetrize this kernel, (21) indicates that six
terms must be added. However, the first three factors in this particular case are
symmetric, and there are only three permutations that will yield distinct forms of the last
factor; namely f (t 1+t2), f (t 1+t3), and f (t 2+t3). Thus, the symmetric form of the given
kernel is
Again I emphasize that although the symmetric version of a kernel usually contains
more terms than an asymmetric version, it does offer a standard form for the kernel. In
many cases system properties can be related more simply to properties of the symmetric
kernel than to properties of an asymmetric kernel.
The second special form of interest is the triangular kernel. The kernel in (8),
h (t,σ1, . . . ,σn), is triangular if it satisfies the additional property that h (t,σ1, . . . ,σn) = 0
when σi +j > σj for i, j positive integers. A triangular kernel will be indicated by the
subscript "tri" when convenient. For such a kernel the representation (8) can be written
in the form
(23)
Sometimes this special form of the input/output relation will be maintained for
triangular kernels, but often I will raise all the upper limits to ∞ or t and leave
20
Automatic Control Theory II
triangularity understood. On some occasions the triangularity of the kernel will be
emphasized by appending unit step functions. In this manner (23) becomes
(24)
Notice that there is no need to use precisely this definition of triangularity. For example,
if htri (t,σ1, . . . ,σn) = 0 when σj > σi +j, then the suitable triangular representation is
(25)
Stated another way, a triangular kernel
(26)
remains triangular for any permutation of the arguments σ1, . . . ,σn. A permutation of
arguments simply requires that the integration be performed over the appropriate
triangular domain, and this domain can be made clear by the appended step functions.
However, I will stick to the ordering of variables indicated in (23) and (26) most of the
time.
Now assume that the triangular kernel in (26) in fact is stationary . Then let
(27)
so that
(28)
and the input/output relation in (23) becomes
(29)
Or, performing the usual variable change,
(30)
an expression that emphasizes that in (27) triangularity implies gtri (t 1, . . . ,tn) = 0 if ti >
ti +j. But, again, this is not the only choice of triangular domain. In fact, for a degree-n
kernel there are n ! choices for the triangular domain, corresponding to the n !
21
Automatic Control Theory II
permutations of variables in the inequality t π(1) ≥ t π(2) ≥ . . . ≥ t π(n) ≥ 0. So there is
flexibility here: pick the domain you like, or like the domain you pick.
To present examples of triangular kernels, I need only review some of the earlier
examples. Notice that the nonstationary kernel obtained in Example 1.3 actually is in
the triangular form (24). Also the input/output representation obtained in Example 1.2
can be written in the form
This corresponds to the triangular kernel htri (t,σ1,σ2) = δ−1(σ1−σ2) in (24), or to the
triangular kernel gtri (t 1,t 2) = δ−1(t 2−t 1) in (29).
The relationship between symmetric and triangular kernels should clarify the features of
both. Assume for the moment that only impulse-free inputs are allowed. To symmetrize
a triangular kernel it is clear that the procedure of summing over all permutations of the
indices applies. However, in this case the summation is merely a patching process since
no two of the terms in the sum will be nonzero at the same point, except along lines of
equal arguments such as σi = σj , σi = σj = σk, and so on. And since the integrations are
not affected by changes in integrand values along a line, this aspect can be ignored. On
the other hand, for the symmetric kernel hsym(t,σ1, . . . ,σn) I can write the input/output
relation as a sum of n ! n-fold integrations over the n ! triangular domains in the first
orthant. Since each of these integrations is identical, the triangular form is given by
(31)
In the stationary case the symmetric kernel hsym(t 1, . . . ,tn) yields the triangular kernel
corresponding to (30) as
(32)
Of course, these formulas imply that either of these special forms is (essentially)
uniquely specified by the other.
Example 1.6
For the stationary, symmetric degree-2 kernel
a corresponding triangular kernel is
22
Automatic Control Theory II
It is instructive to recompute the symmetric form. Following (21),
Now this is almost the symmetric kernel I started with. Almost, because for t 1 = t 2 the
original symmetric kernel is e 3t1 , while the symmetrized triangular kernel is 2e 3t1 .
This is precisely the point of my earlier remark. To wit, values of the kernel along equal
argument lines can be changed without changing the input/output representation. In fact
they must be changed to make circular calculations yield consistent answers.
Now consider what happens when impulse inputs are allowed, say u (t) = δ0(t). In terms
of the (nonstationary) symmetric kernel, the response is y (t) = hsym(t, 0, . . . , 0), and in
terms of the triangular kernel, y (t) = htri (t, 0, . . . , 0). Thus, it is clear that in this
situation (31) is not consistent. Of course, the difficulty is that when impulse inputs are
allowed, the value of a kernel along lines of equal arguments can affect the input/output
behavior. For a specific example, reconsider the stationary kernels in Example 1.6 with
an impulse input.
Again, the problem here is that the value of the triangular kernel along equal argument
lines is defined to be equal to the value of the symmetric kernel. This can be fixed by
more careful definition of the triangular kernel. Specifically, what must be done is to
adjust the definition so that the triangular kernel gets precisely its fair share of the value
of the symmetric kernel along equal-argument lines. A rather fancy "step function" can
be defined to do this, but at considerable expense in simplicity. My vote is cast for
simplicity, so impulse inputs henceforth are disallowed in the presence of these issues,
and kernel values along lines will be freely adjusted when necessary. (This luxury is not
available in the discrete-time case discussed in Chapter 6, and a careful definition of the
triangular kernel which involves a fancy step function is used there. The reader inclined
to explicitness is invited to transcribe those definitions to the continuous-time case at
hand.)
The third special form for the kernel actually involves a special form for the entire
input/output representation. This new form is most easily based on the triangular kernel.
Intuitively speaking, it shifts the discontinuity of the triangular kernel out of the picture
23
Automatic Control Theory II
and yields a smooth kernel over all of the first orthant. This so-called regular kernel will
be used only in the stationary system case, and only for one-sided input signals.
Suppose htri (t 1, . . . ,tn) is a triangular kernel that is zero outside of the domain
t 1 ≥ t 2 ≥ . . . ≥ tn ≥ 0. Then the corresponding input/output representation can be written
in the form
where the unit step functions are dropped and the infinite limits are retained just to
make the bookkeeping simpler. Now make the variable change from σ1 to τ1 = σ1−σ2.
Then the input/output representation is
Now replace σ2 by τ2 = σ2−σ3 to obtain
Continuing this process gives
(In continuing the process, each variable change can be viewed as a change of variable
in one of the iterated integrals. Thus the Jacobian of the overall change of variables is
unity, as is easily verified. This is a general feature of variable changes in the sequel.)
Letting
(33)
be the regular kernel, I can write
(34)
24
Automatic Control Theory II
where hreg(t 1, . . . ,tn) is zero outside of the first orthant, t 1, . . . ,tn ≥ 0. As mentioned
above, the usual discontinuities encountered along the lines tj −1 = tj , and so on, in the
triangular kernel occur along the edges tj = 0 of the domain of the regular kernel.
It should be clear from (33) that the triangular kernel corresponding to a given regular
kernel is
(35)
Thus (33) and (35), in conjunction with the earlier discussion of the relationship
between the triangular and symmetric kernels, show how to obtain the symmetric kernel
from the regular kernel, and vice versa.
I noted earlier that particular forms for the kernel often are natural for particular system
structures. Since the regular kernel is closely tied to the triangular kernel, it is not
surprising that when one is convenient, the other probably is also (restricting attention,
of course, to the case of stationary systems with one-sided inputs). This can be
illustrated by reworking Example 1.3 in a slightly different way.
25
Automatic Control Theory II
Lyapunov Stability Theory
Basic definitions
Consider a dynamical system which satisfies
(1)
We will assume that f(x, t) satisfies the standard conditions for the existence and
uniqueness of solutions. Such conditions are, for instance, that f(x, t) is Lipschitz
continuous with respect to x, uniformly in t, and piecewise continuous in t. A point
x* є
n
is an equilibrium point of (1) if f(x*, t) ≡ 0. Intuitively and somewhat crudely
speaking, we say an equilibrium point is locally stable if all solutions which start near
x* (meaning that the initial conditions are in a neighborhood of x*) remain near x* for
all time. The equilibrium point x* is said to be locally asymptotically stable if x* is
locally stable and, furthermore, all solutions starting near x* tend towards x* as t → ∞.
We say somewhat crude because the time-varying nature of equation (1) introduces all
kinds of additional subtleties. Nonetheless, it is intuitive that a pendulum has a locally
stable equilibrium point when the pendulum is hanging straight down and an unstable
equilibrium point when it is pointing straight up. If the pendulum is damped, the stable
equilibrium point is locally asymptotically stable.
By shifting the origin of the system, we may assume that the equilibrium point of
interest occurs at x* = 0. If multiple equilibrium points exist, we will need to study the
stability of each by appropriately shifting the origin.
Definition 1. Stability in the sense of Lyapunov
The equilibrium point x* = 0 of (1) is stable (in the sense of Lyapunov) at t = t0 if for
any є > 0 there exists a δ(t0, є) > 0 such that
(2)
Lyapunov stability is a very mild requirement on equilibrium points. In particular, it
does not require that trajectories starting close to the origin tend to the origin
asymptotically. Also, stability is defined at a time instant t0. Uniform stability is a
concept which guarantees that the equilibrium point is not losing stability. We insist that
for a uniformly stable equilibrium point x*, δ in the Definition 1 not be a function of t0,
26
Automatic Control Theory II
so that equation (2) may hold for all t0. Asymptotic stability is made precise in the
following definition:
Definition 2. Asymptotic stability
An equilibrium point x* = 0 of (1) is asymptotically stable at t = t0 if
1. x* = 0 is stable, and
2. x* = 0 is locally attractive; i.e., there exists δ(t0) such that
(3)
As in the previous definition, asymptotic stability is defined at t0. Uniform asymptotic
stability requires:
1. x* = 0 is uniformly stable, and
2. x* = 0 is uniformly locally attractive; i.e., there exists δ independent of t0 for
which equation (3) holds. Further, it is required that the convergence in equation
(3) is uniform.
Finally, we say that an equilibrium point is unstable if it is not stable. This is less of a
tautology than it sounds and the reader should be sure he or she can negate the
definition of stability in the sense of Lyapunov to get a definition of instability. In
robotics, we are almost always interested in uniformly asymptotically stable equilibria.
If we wish to move the robot to a point, we would like to actually converge to that
point, not merely remain nearby. Figure 4.7 illustrates the difference between stability
in the sense of Lyapunov and asymptotic stability.
Definitions 1 and 2 are local definitions; they describe the behavior of a system near an
equilibrium point. We say an equilibrium point x* is globally stable if it is stable for all
initial conditions x0 є
n
. Global stability is very desirable, but in many applications it
can be difficult to achieve. We will concentrate on local stability theorems and indicate
where it is possible to extend the results to the global case. Notions of uniformity are
only important for time-varying systems. Thus, for time-invariant systems, stability
implies uniform stability and asymptotic stability implies uniform asymptotic stability.
27
Automatic Control Theory II
It is important to note that the definitions of asymptotic stability do not quantify the rate
of convergence. There is a strong form of stability which demands an exponential rate
of convergence:
Definition 3. Exponential stability, rate of convergence
The equilibrium point x* = 0 is an exponentially stable equilibrium point of (1) if there
exist constants m, α > 0 and є > 0 such that
(4)
for all ||x(t0)|| ≤ є and t ≥ t0. The largest constant α which may be utilized in (4) is called
the rate of convergence.
Exponential stability is a strong form of stability; in particular, it implies uniform,
asymptotic stability. Exponential convergence is important in applications because it
can be shown to be robust to perturbations and is essential for the consideration of more
28
Automatic Control Theory II
advanced control algorithms, such as adaptive ones. A system is globally exponentially
stable if the bound in equation (4) holds for all x0 є
n
. Whenever possible, we shall
strive to prove global, exponential stability.
The direct method of Lyapunov
Lyapunov’s direct method (also called the second method of Lyapunov) allows us to
determine the stability of a system without explicitly integrating the differential
equation (1). The method is a generalization of the idea that if there is some “measure of
energy” in a system, then we can study the rate of change of the energy of the system to
ascertain stability. To make this precise, we need to define exactly what one means by a
“measure of energy.” Let Bє be a ball of size around the origin,
Bє = {x є
n
: ||x|| < є}.
Definition 4. Locally positive definite functions (lpdf)
A continuous function V :
n
×
+
→
is a locally positive definite function if for some
є > 0 and some continuous, strictly increasing function α :
+
→ ,
(5)
A locally positive definite function is locally like an energy function. Functions which
are globally like energy functions are called positive definite functions:
Definition 5. Positive definite functions (pdf)
A continuous function V :
n
×
+
→
is a positive definite function if it satisfies the
conditions of Definition 4 and, additionally, α(p) → ∞ as p→∞.
To bound the energy function from above, we define decrescence as follows:
Definition 6. Decrescent functions
A continuous function V :
n
×
+
→
is decrescent if for some є > 0 and some
continuous, strictly increasing function β :
+
→ ,
(6)
Using these definitions, the following theorem allows us to determine stability for a
system by studying an appropriate energy function. Roughly, this theorem states that
.
when V (x, t) is a locally positive definite function and V (x, t) ≤ 0 then we can
29
Automatic Control Theory II
conclude stability of the equilibrium point. The time derivative of V is taken along the
trajectories of the system:
.
.
In what follows, by V we will mean V |x˙=f(x,t).
Theorem 1. Basic theorem of Lyapunov
.
Let V (x, t) be a non-negative function with derivative V along the trajectories of the
system.
.
1. If V (x, t) is locally positive definite and V (x, t) ≤ 0 locally in x and for all t,
then the origin of the system is locally stable (in the sense of Lyapunov).
.
2. If V (x, t) is locally positive definite and decrescent, and V (x, t) ≤ 0 locally in x
and for all t, then the origin of the system is uniformly locally stable (in the
sense of Lyapunov).
.
3. If V (x, t) is locally positive definite and decrescent, and − V (x, t) is locally
positive definite, then the origin of the system is uniformly locally
asymptotically stable.
.
4. If V (x, t) is positive definite and decrescent, and − V (x, t) is positive definite,
then the origin of the system is globally uniformly asymptotically stable.
The conditions in the theorem are summarized in Table 4.1.
Theorem 1 gives sufficient conditions for the stability of the origin of a system. It does
not, however, give a prescription for determining the Lyapunov function V (x, t). Since
30
Automatic Control Theory II
the theorem only gives sufficient conditions, the search for a Lyapunov function
establishing stability of an equilibrium point could be arduous. However, it is a
remarkable fact that the converse of Theorem 1 also exists: if an equilibrium point is
stable, then there exists a function V (x, t) satisfying the conditions of the theorem.
However, the utility of this and other converse theorems is limited by the lack of a
computable technique for generating Lyapunov functions.
Theorem 1 also stops short of giving explicit rates of convergence of solutions to the
equilibrium. It may be modified to do so in the case of exponentially stable equilibria.
Theorem 2. Exponential stability theorem
.
x* = 0 is an exponentially stable equilibrium point of x = f(x, t) if and only if there
exists an є > 0 and a function V (x, t) which satisfies
for some positive constants α1, α2, α3, α4, and ||x|| ≤ є.
The rate of convergence for a system satisfying the conditions of Theorem 2 can be
determined from the proof of the theorem. It can be shown that
are bounds in equation (4). The equilibrium point x* = 0 is globally exponentially stable
if the bounds in Theorem 2 hold for all x.
31
Automatic Control Theory II
Adaptive Control (Model reference adaptive control - MRAC)
Introduction
Model reference adaptive control (MRAC) is one of the main approaches to adaptive
control. The basic structure of a MRAC scheme is shown in Figure 6.1. The reference
model is chosen to generate the desired trajectory, ym, that the plant output yp has to
follow. The tracking error e1 = yp - ym represents the deviation of the plant output from
the desired trajectory. The closed-loop plant is made up of an ordinary feedback control
law that contains the plant and a controller C(θ) and an adjustment mechanism that
generates the controller parameter estimates θ(t) on-line.
The purpose of this chapter is to design the controller and parameter adjustment
mechanism so that all signals in the closed-loop plant are bounded and the plant output
yp tracks ym as close as possible.
MRAC schemes can be characterized as direct or indirect and with normalized or
unnormalized adaptive laws. In direct MRAC, the parameter vector θ of the controller
C(θ) is updated directly by an adaptive law, whereas in indirect MRAC θ is calculated
at each time t by solving a certain algebraic equation that relates θ with the on-line
estimates of the plant parameters. In both direct and indirect MRAC with normalized
adaptive laws, the form of C(θ), motivated from the known parameter case, is kept
unchanged. The controller C(θ) is combined with an adaptive law (or an adaptive law
and an algebraic equation in the indirect case) that is developed independently. This
design procedure allows the use of a wide class of adaptive laws that includes gradient,
least-squares and those based on the SPR-Lyapunov design approach. On the other
hand, in the case of MRAC schemes with unnormalized adaptive laws, C(θ) is modified
to lead to an error equation whose form allows the use of the SPR- Lyapunov design
approach for generating the adaptive law. In this case, the design of C(θ) and adaptive
law is more complicated in both the direct and indirect case, but the analysis is much
simpler and follows from a consideration of a single Lyapunov-like function.
32
Automatic Control Theory II
Simple Direct MRAC Schemes (Scalar Example: Adaptive Regulation)
Consider the following scalar plant:
(1)
where a is a constant but unknown. The control objective is to determine a bounded
function u = f(t; x) such that the state x(t) is bounded and converges to zero as t → ∞ for
any given initial condition x0. Let -am be the desired closed-loop pole where am > 0 is
chosen by the designer.
Control Law If the plant parameter a is known, the control law
(2)
with k* = a + am could be used to meet the control objective, i.e., with (2), the closedloop plant is
whose equilibrium xe = 0 is e.s. in the large.
Because a is unknown, k* cannot be calculated and, therefore, (2) cannot be
implemented. A possible procedure to follow in the unknown parameter case is to use
the same control law as given in (2) but with k* replaced by its estimate k(t), i.e., we
use
(3)
and search for an adaptive law to update k(t) continuously with time.
Adaptive Law The adaptive law for generating k(t) is developed by viewing the
problem as an on-line identification problem for k*. This is accomplished by first
33
Automatic Control Theory II
obtaining an appropriate parameterization for the plant (1) in terms of the unknown k*
and then using a similar approach to estimate k* on-line. We illustrate this procedure
below.
We add and subtract the desired control input -k*x in the plant equation to obtain
Because a – k* = -am we have
or
(4)
Equation (4) is a parameterization of the plant equation (1) in terms of the unknown
controller parameter k*. Because x; u are measured and am > 0 is known.. It turns out
that the adaptive laws developed for (4) using the SPR-Lyapunov design approach
without normalization simplify the stability analysis of the resulting closed-loop
adaptive control scheme considerably. Therefore, as a starting point, we concentrate on
the simple case and deal with the more general case that involves a wide class of
adaptive laws in later sections.
Because
is SPR we can proceed with the SPR-Lyapunov design approach and

generate the estimate x of x as
(5)
where the last equality is obtained by substituting the control law u=-kx. If we now

choose x (0) = 0, we have
, which implies that the estimation error

є1 defined as є1 = x - x is equal to the regulation error, i.e., є1 = x, so that (5) does not

have to be implemented to generate x . Substituting for the control u = -k(t)x in (4), we
~
obtain the error equation that relates the parameter error k = k – k* with the estimation
error є1 = x, i.e.
(6)
or
34
Automatic Control Theory II
The error equation (6) is in a convenient form for choosing an appropriate Lyapunov
function to design the adaptive law for k(t). We assume that the adaptive law is of the
form
(7)
where f1 is some function to be selected, and propose
(8)
for some γ > 0 as a potential Lyapunov function for the system (6), (7). The time
derivative of V along the trajectory of (6), (7) is given by
(9)
Choosing f1 = γ є1 x, i.e.,
(10)
we have
(11)
.
Analysis Because V is a positive definite function and V ≤ 0, we have
. Because є1 = x, we also have that
which implies that
,
and
therefore all signals in the closed-loop plant are bounded.
Furthermore,
and
(which follows from (6) ) imply
that є1(t) = x(t) → 0 as t → ∞. From x(t) → 0 and the boundedness of k, we establish
.
that k (t) → 0; u(t) → 0 as t → ∞:
We have shown that the combination of the control law (3) with the adaptive law (10)
meets the control objective in the sense that it forces the plant state to converge to zero
while guaranteeing signal boundedness.
We cannot establish that k(t) converges to k*, i.e., that the pole of the closed-loop plant
converges to the desired one given by -am. The lack of parameter convergence is less
crucial in adaptive control than in parameter identification because in most cases, the
35
Automatic Control Theory II
control objective can be achieved without requiring the parameters to converge to their
true values.
The simplicity of this scalar example allows us to solve for є1 = x explicitly, and study
the properties of k(t); x(t) as they evolve with time. We can verify that
(12)
where c2 = γx20 + (k0 - a)2, satisfy the differential equations (6) and (10) of the closedloop plant. Equation (12) can be used to investigate the effects of initial conditions and
adaptive gain γ on the transient and asymptotic behavior of x(t); k(t). We have
limt→∞ k(t) = a + c if c > 0 and limt→∞ k(t) = a - c if c < 0, i.e.,
(13)
Therefore, for x0 ≠ 0; k(t) converges to a stabilizing gain whose value depends on γ and
the initial condition x0; k0. It is clear from (13) that the value of k∞ is independent of
whether k0 is a destabilizing gain, i.e., 0 < k0 < a, or a stabilizing one, i.e., k0 > a, as
long as (k0 - a)2 is the same. The use of different k0, however, will affect the transient
behavior as it is obvious from (12). In the limit as t → ∞, the closed-loop pole
converges to -(k∞ - a), which may be different from -am. Because the control objective is
to achieve signal boundedness and regulation of the state x(t) to zero, the convergence
of k(t) to k* is not crucial.
36
Automatic Control Theory II
Implementation The adaptive control scheme developed and analyzed above is given
by the following equations:
(14)
where x is the measured state of the plant. A block diagram for implementing (14) is
shown in Figure 6.2.
The design parameters in (14) are the initial parameter k0 and the adaptive gain γ > 0.
For signal boundedness and asymptotic regulation of x to zero, our analysis allows k0; γ
to be arbitrary. It is clear, however, from (12) that their values affect the transient
performance of the closed-loop plant as well as the steady-state value of the closed-loop
pole. For a given k0; x0 ≠ 0, large γ leads to a larger value of c in (12) and, therefore, to
a faster convergence of x(t) to zero. Large γ, however, may make the differential
equation for k “stiff” (i.e., k large) that will require a very small step size or sampling
period to implement it on a digital computer. Small sampling periods make the adaptive
scheme more sensitive to measurement noise and modeling errors.
Remark In the proceeding example, we have not used any reference model to describe
the desired properties of the closed-loop system. A reasonable choice for the reference
model would be
(15)
which, by following exactly the same procedure, would lead to the adaptive control
scheme
where e1 = x - xm. If xm0 ≠ x0, the use of (15) will affect the transient behavior of the
tracking error but will have no effect on the asymptotic properties of the closed-loop
scheme because xm converges to zero exponentially fast.
37
Automatic Control Theory II
Modeling Complex Systems
What is a complex system?
A relatively recent area of scientific inquiry is the exploration of the dynamics of
complex systems. A defining characteristic of complex systems is their tendency to selforganize globally as a result of many local interactions. In other words, organization
occurs without any central organizing structure or entity. Such self-organization has
been observed in systems at scales from neurons to ecosystems.
A complex adaptive system has the following characteristics: it persists in spite of
changes in the diverse individual components of which it is comprised; the interactions
between those components are responsible for the persistence of the system; and the
system itself engages in adaptation or learning. To say that a system is complex is to say
that it moves between order and disorder without becoming fixed in either state. To say
that such a system adapts is to say that it responds to information by changing.
Such systems abound. Not only the ant colony and the human body as a whole, but also
various systems within the body such as the nervous system and the immune system fall
into this category. These are systems that persist in spite of the continual changes of
individual components, maintaining coherence and adapting in response to a
phenomenal amount of information throughout the lifetime of the organism in which
they function.
Adaptation and Finding Excellent Solutions
A complex adaptive system receives a stream of data about itself and its surroundings.
In that stream, it identifies particular regularities and compresses them into a concise
"schema", one of many possible ones related by mutation or substitution. In the
presence of further data from the stream, the schema can supply descriptions of certain
aspects of the real world, predictions of events that are to happen in the real world, and
prescriptions for behavior of the complex adaptive system in the real world. In all these
cases, there are real world consequences: the descriptions can turn out to be more
accurate or less accurate, the predictions can turn out to be more reliable or less reliable,
and the prescriptions for behavior can turn out to lead to favorable or unfavorable
outcomes. All these consequences then feed back to exert "selection pressures" on the
competition among various schemata, so that there is a strong tendency for more
38
Automatic Control Theory II
successful schemata to survive and for less successful ones to disappear or at least to be
demoted in some sense.
Thus, a complex adaptive system:
1. interacts with the environment
2. creates schemata, which are compressed and generalized regularities
experienced in those interactions
3. behaves in ways consistent with these schemata
4. incorporates feedback from the environment to modify and adapt its schemata
for greater success
Self-Organization in Complex Systems
The process by which a complex system achieves maximum fitness results in selforganization by the system, that is, agents acting locally, unaware of the extent of the
larger system of which they are a part, generate larger patterns which result in the
organization of the system as a whole. This concept can be seen at work in ant and
termite colonies, beehives, market economies, and can even be modeled on one's home
computer using free software. The idea that an ant colony is a system that organizes
itself without any leader is intriguing. Each individual ant, acting with limited
information, contributes to the emergence of an organized whole. This new way of
looking at organization as an emergent property of complex systems calls into question
some fundamental assumptions about organization in general, and about learning in
particular.
Not every system is a complex adaptive system; certain conditions must be met in order
for a system to self-organize. First of all, the system must include a large number of
agents. In addition, the agents must interact in a nonlinear fashion. If there aren't enough
components or they are prevented from interacting, you won't see patterns emerge or
evolve. The nature of the interactions must be nonlinear. This constitutes a major break
with Sir Isaac Newton, who said in Definitions II of the Principia: "The motion of the
whole is the sum of the motion of all the parts." For us, the motion of the whole is not
only greater than, but different than the sum of the motions of the parts, due to nonlinear
interactions among the parts or between the parts and the environment.
39
Automatic Control Theory II
Complex Adaptive Systems Summarized
From this discussion, the following characteristics of complex adaptive systems can be
extracted:
1. Complex adaptive systems involve agents whose local, non-linear interactions
result in self-organization by the system as a whole.
2. Complex adaptive systems exist in a mixed condition between order and chaos
that enables them to achieve stability and flexibility simultaneously.
3. The agents in a complex adaptive system thrive by devising excellent solutions
to difficult problems, rather than by finding best or perfect solutions.
4. Complex adaptive systems find excellent solutions by creating schemata based
on regularities identified as successful, behaving in ways consistent with these
schemata, and incorporating feedback to adapt the schemata for greater success.
Modeling Complex Systems
One way to examine what may be happening in self-organizing complex systems is
through the use of computer simulations. The three main components of the modeling
environment are turtles, patches, and the observer. The individual agents in the system
are called turtles, although they can represent any kind of agent from a molecule to a
person. The environment in which the turtles operate is divided into patches. Patch size
and movement by turtles within and between patches is determined by the program
designer. Patches are not necessarily passive but may be, and typically are, active
components of the system. Commands may apply either to turtles or to patches. The
third component, the observer, can issue commands that affect both patches and turtles.
Variables within a model may be set up as sliders, and in many models the sliders can
be manipulated while the model is running. This feature allows the user to alter
variables and search for excellent solutions within the constraints identified by the
model designer. For example, a simple model of an ecosystem might include agents
identified as predators, other agents called prey and patches with food for the prey in
varying amounts. The interactions between the two different kinds of agents, as well as
between the agents and the patches, can be defined by simple commands that identify
when predators eat prey, when prey eat food, under what conditions new agents are
"born" and "die," and so on. If such a model is designed with sliders to control the
number of predators and prey, as well as the proportion of food available, the user can
40
Automatic Control Theory II
experiment to try to determine how a change in one part of the system affects the system
as a whole and how a system might adapt in order to survive or thrive.
The beauty of these modeling tools with regard to building the scientific mind is that
they provide the user with a dynamic visual and interactive medium through which to
explore the concepts of complex systems. They are simple enough to be used by
students in middle or high school, while at the same time they have the potential
sophistication required of graduate level research. As such, the use of these free
modeling tools opens up the world of complex systems to a broad audience, including
those without advanced understanding of science and mathematics. The medium itself
can describe and explain, through color, pattern and motion, concepts that previously
might have been incomprehensible.
41