Transactions of the Institute of Measurement and Control 23,1 (2001) pp. 51–68
Control of bounded dynamic
stochastic distributions using square
root models: an applicability study in
papermaking systems
H. Wang1, H. Baki and P. Kabore
Department of Paper Science, UMIST, Manchester M60 1QD, UK
Following the recent developments on the modelling and control algorithms of the shape of
the output probability density function for general dynamic stochastic systems (Wang, 1998a,
Proceedings of the IFAC Workshop on AARCT, Cancun, pp. 95–99), this paper presents a square
root approximation-based control algorithm, where the B-splines function expansion is used
to approximate the square root of the output probability density function in order to guarantee
its positiveness. It has been shown that with such an approximation, the system is generally
nonlinear. This is true even when the dynamic part of the system is linear. As such, a nonlinear
control algorithm has been developed to control the output probability density function of the
system. A simulated example is used to demonstrate the use of the algorithm and encouraging
results have been obtained.
Key words: B-spline neural networks; dynamic stochastic systems; nonlinear control algorithms; papermaking systems; probability density function.
1. Introduction
Over the past several decades, research into the control of stochastic systems has
been concentrated on the control of certain stochastic properties of the system
Address for correspondence: Dr H. Wang, Department of Paper Science, UMIST, Manchester M60
1QD, UK. E-mail: Hong.Wang얀umist.ac.uk
1
Dr Hong Wang is also an affiliated member of the Control Systems Centre at UMIST.
 2001 The Institute of Measurement and Control
0142-3312(01)TM013OA
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
52 Bounded dynamic stochastic distribution
output. In most cases, the control of the variance and the mean values of the
output of stochastic systems are set as the design target, where the purpose of
control system design is to select a control strategy so as to realize the optimal
tracking of the mean and variance of the system output with respect to their
desired values. This leads to the well established minimum variance control
(Åström, 1970) and the Markovian jump system control (Gajic and Losada, 1998).
In these approaches, it is normally assumed that the variables of the system obey
Gaussian distribution. Indeed, this assumption plays a key part in the development of many fundamental theories in modern control systems. Based on this
assumption, well known techniques, such as Kalman filtering and LQG algorithms, are obtained and used successfully in many real systems.
However, this assumption is restrictive for some applications. Also, the control
of only the mean and the variance of the system output may not be enough.
Typical examples are most control processes in the wet end of general papermaking machines, where most probability density functions of process variables, such
as fibre length and filler sizes, cannot be described by Gaussian distributions
(Wang, 1998c). Indeed, the pore size distribution of the fibrious network on the
formation wire table is always approximated by a truncated ⌫-distribution, rather
than Gaussian-type distribution.
Because of this, several algorithms have been recently developed by Wang
(1998c; 1999a,b), where the control algorithm design has been directly focused on
the control of the total shape of the output probability density functions of stochastic systems. In these approaches, the stochastic system considered has its outputs
taken as the measured probability density functions of the system output and its
inputs as a set of deterministic variables. As shown in Figure 1, these variables
affect dynamically the shape of the probability density functions of the system
output. The purpose here is to design a control strategy u(k), k = 0, 1, 2, % so that
the output probability density function of the considered stochastic system follows
the given distribution as closely as possible.
In these algorithms (Wang et al., 1997, Wang, 1999a,b), the well known B-spline
approximations have been used to approximate the probability density function
of the system output directly. Once all the basis functions have been selected, the
weight of the approximation can be regarded as only being related to the control
input. For dynamic systems, the relationship between the weights and the control
input u(k) is expressed by a set of difference equations. When these difference
equations are linear, a very simple solution to the probability density function
control can be obtained using well established linear control theory. In this Bspline model (Wang, 1998c) the natural constraint that the integration of the probability density function over its definition domain must be equal to 1.0 is guaran-
Figure 1
The considered stochastic system
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
53
teed after some simple formulations of the weights in the B-spline expansion.
Although several simulated examples have been tried successfully for linear systems, in a recent study (Dodson and Wang, 1999) on information geometry of ⌫family distributions, it has been found that the weight training trajectories sometimes lead to a partly negative probability density function. As such, alternative
approximations to the output probability density functions need to be developed
so that the positiveness of the output probability density function can be guaranteed during the closed-loop control of the stochastic systems.
This leads to the current work, where instead of approximating the probability
density function directly, its square root is approximated by the B-spline functions.
With such an approximation, it has been shown that the output equation used by
Wang (1998b,c) becomes nonlinear regardless of the dynamic part of the system.
As a result, a nonlinear control algorithm is developed and shown to be able to:
1) stabilize the closed loop system, and
2) produce a desired tracking performance.
2. Preliminaries and model representation
Similarly to the work presented by Wang (1998c), denote v(k) 苸 [a, b] as a uniformly bounded random process variable defined on k = 0, 1, 2, % and assume
that v(k) represents the output of a stochastic system. For example v(k) can be
used to represent the pore size of the fibrious network in the wire section of a
paper machine (Smook, 1992). Denote u(k) 苸 Rm as the control input vector which
controls the distribution of v(k), then at each sample time k, v(k) can be characterized by its probability density function ␥(y, u(k)) which is defined by
P(a ⱕ v(k) ⬍ ␰, u(k)) =
冕
␰
␥(y, u(k))dy
(1)
a
where P(a ⱕ v(k) ⬍ ␰, u(k)) represents the probability of output v(k) lying inside
the interval [a, ␰ ] when u(k) is applied to the system. This means that the probability density function ␥(y, u(k)) of v(k) is controlled by u(k). For example, u(k)
can be regarded as a retention aid when the retention system (Wang, 1998c) in
papermaking is considered.
Assuming the interval [a, b] is known and the probability density function ␥(y,
u(k)) is continuous and bounded, then using the well known B-spline neural network, the following square root approximation is obtained
冑␥(y, u) = 冘 w (u(k))B (y) + e
n
i
i
0
(2)
i=1
where wi are the weights which depends on u(k), Bi(y) are the pre-specified basis
functions and e0 represents the approximation error. Indeed, the above approximation is realizable in practice. This is largely due to the development of sensor
techniques, where, in papermaking, probability density functions can now be easily measured (Moore, 1997).
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
54 Bounded dynamic stochastic distribution
To simplify the formulation, it is assumed that e0 = 0. This means that only the
following equality
冑␥(y, u) = 冘 w (u(k))B (y)
n
i
(3)
i
i=1
will be considered. Since Equation (2) means that
冉冘
冊
n
␥(y, u(k)) =
2
wi(u(k))Bi(y)) ⱖ 0 ∀ y 苸 [a, b]
i=1
(4)
it can be seen that the positiveness of ␥(y, u(k)) can be automatically guaranteed. Denote:
C0(y) = (B1(y), B2(y), %, Bn−1(y)) 苸 Rn−1
V(k) = (w1, w2, %, wn−1 )T 苸 R1×(n−1)
(5)
then it can be shown that at sample time k, the square root of the output probability density function becomes
冑␥(y, u(k)) = [C (y) B (y)]
0
n
冋 册
V(k)
wn
(6)
However, since ␥(y, u(k)) is a probability density function, the equality
冕
b
␥(y, u(k))dy =
a
冕 冉冑
b
冊
␥(y, u(k))
a
2
dy = 1
(7)
should always be satisfied. Using Equation (6), it can be seen that the following
equality
冕
b
(C0(y)V(k) + wnBn(y))2dy = 1
(8)
a
should hold for any set of weights and basis functions. This leads to
1 = VT(k)
冋冕
b
+
a
冋冕
册
b
a
册
冋冕
b
CT0 (y)C0(y)dy V(k) + 2
册
C0(y)Bn(y)dy V(k)wn
a
B2n(y)dy w2n
(9)
As a result, it can still be seen that only n − 1 weights are independent and wn is
nonlinearly related to V(k). Denote
a1 = VT(k)
a2 = 2
冋冕
冋冕
册
(10)
C0(y)Bn(y)dy V(k)
(11)
b
a
b
a
CT0 (y)C0(y)dy V(k) − 1
册
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
a3 =
冋冕
b
册
B2n(y)dy 苷 0
a
55
(12)
then Equation (9) can be further expressed as
a3w2n + a2wn + a1 = 0
(13)
By solving this quadratic equation, it can be obtained that
wn =
冑
1
(−a2 ± a22 − 4a1a3 ) = h(V(k))
2a3
(14)
provided
a22 − 4a1a3 ⱖ 0
In Equation (14), h(V(k)) stands for a known nonlinear function of V(k). Using
Equations (10)–(12), the above condition can be transferred to:
a3 − Vⳕ(k)QabV(k) ⱖ 0
where it can be shown that
Qab = −cT1 c1 + a3
c1 =
冕
冕
(15)
b
CT0 (y)C0(y)Tdy
a
b
C0(y)Bn(y)dy
(16)
a
Since all the basis function are pre-specified, matrix Qab is well defined and known.
Indeed, it can be shown that for the basis function selected recursively as given
in Brown and Harris (1994), Qab is always positive definite. Indeed, the Qab matrix
can be written as follows
Qab = −
冋冕
b
a
册 冋冕
T
b
C0(y)Bn(y)dy
a
册 冕
b
C0(y)Bn(y)dy +
冕
b
B2n(y)dy
a
CT0 (y)C0(y)dy
a
(17)
The integral expressions can be written in a function norm form as follows
Qab = − 储C0(y)Bn(y)储2 + 兩兩Bn(y)储2储C0(y)储2
(18)
Since for all the norms, the following inequality
储C0(y)Bn(y)储2 ⱕ 兩兩Bn(y)储2储C0(y)储2
(19)
holds, it can be concluded that Qab ⱖ 0.
On the other hand, Equation (15) also leads to following inequality
a3 ⱖ 储V(k)储2Qab ∀ k
(20)
Let ␭max be the largest eigenvalue of Qab, then this inequality can be satisfied if:
a3 ⱖ ␭max兩兩V(k)储2 ⱖ 储V(k)储2Qab
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
(21)
56 Bounded dynamic stochastic distribution
holds. Therefore, the domain in which the weight will evolve can be given in the
following inequality
储V(k)储 ⱕ
冪␭
a3
(22)
max
By substituting wn in Equation (14) into Equation (3), it can be seen that
冑␥(y, u(k)) = C (y)V(k) + h(V(k))B (y) y 苸 [a, b]
0
n
(23)
In comparison with the models developed by Wang (Wang et al., 1997; Wang,
1999a), it can be seen that the square root approximation generally leads to a
nonlinear relationship between the measured output probability density function
and the weight vector V(k). In the rest of the paper, we will only use V(k) to
formulate the control solution.
Equation (6) gives an instantaneous expression of the considered probability
density function at sample time k. However, in many systems, the actual probability density function of the system output is dynamically related to the control
input u(k). This can be justified by considering a typical example in a paper
machine shown in Figure 2, where, at its wet end, there are normally several
sections in which the initial paper web is formed.
These sections are referred to as the headbox approaching units, the head box
and the moving wire table. In the approaching units and the head box, fibres,
fillers and other chemical additives are mixed. This mixture generally consists of
5% solids and 95% water. As such, when this mixture is injected onto the moving
wire table, some water is drained through the wire nets into a white water pit
underneath the wire table, leaving a fibre-dominant network on the wire table.
This network is regarded as paper web. Due to the random nature of fibre length
and filler particles, the density distribution of the paper web is random. When an
image analysis-based sensor is used, such a density of the paper web distribution
can be measured and represented as a grey level distribution. This grey distri-
Figure 2
A paper machine wet end
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
57
bution is a two-dimensional random process having both machine and cross directions (Smook, 1992), and generally controlled by the thick stock input and chemical input before the headbox. To achieve a good web formation, all the control
inputs should be selected to make the network density distribution as close as
possible to a uniform distribution.
In this case, the ideal probability density function should be a uniform one
defined on [a, b], with a and b being very close to each other. Indeed, the headbox
can be regarded as a tank. Using the mass balance principle, it can be shown that
at least a first order dynamic exists between the input (i.e., thick stock) and the
output distribution (i.e., the density distribution of the paper web). Since the output probability density function has been represented by Equation (3), this
dynamic relation can only be realized between the input and the weights. As a
result, for dynamic systems, let us assume that the following state space form
x(k + 1) = Gx(k) + Hu(k)
V(k) = Ex(k) + Ju(k)
(24)
can be used to represent the dynamic relationship between measured input u(k)
苸 Rm and V(k), where x(k) 苸 Rl is the unmeasurable state vector, V(k) is taken
as the output vector, l is a positive integer, {G, H, E, J} are parameter matrices
which have appropriate dimensions in accordance with the state vector, input and
the weights vector.
Also, due to the existence of the dynamic relationship between u(k) and V(k),
it has been shown (Wang, 1998c) that the measured output probability function
␥(y, u(k)) at sample time k should only depend on the past values of the input
sequences u(k − i), i = 0, 1, 2, %. This means that
␥(y, u(k)) = ␥(y, U(k))
(25)
with U(k) = (u(k − 1), u(k − 2), %, u(0)).
Following the discussions in section 1, it can be seen that the overall stochastic
system can be expressed by Equations (24) and (25), where the input to the system
is a crispy valued variable u(k) and the output is the measured output probability
density function of the system. This is clearly presented in Figure 1. To this end,
the purpose of control algorithm design is to select u(k) such that the measured
␥(y, u(k)) follows a targeted distribution function, g(y), as closely as possible.
Similar to the algorithms developed by Wang (1998c), we can consider two
approaches. In the first approach, the control algorithm is obtained based upon
the direct use of the measured probability density function, whilst in the second
approach, the control input is based upon the weights in Equation (24) and the
weights are calculated at each sample time from the measured probability density
function. In this study, we will only consider the second approach. In this
approach, we will further simplify Equation (24) as follows:
V(k + 1) = AV(k) + Bu(k)
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
(26)
58 Bounded dynamic stochastic distribution
3. The control algorithm
As discussed in Section 1, the purpose of the control algorithm design is to choose
control sequence {u(k)} such that the actual probability density function of the
system output is made as close as possible to a pre-specified continuous probability density function g(y), which is defined on [a, b] and is independent of
{u(k)}. This is equivalent to choosing {u(k)} such that √␥(y, U(k)) is made as close
as possible to √g(y). As a result, the purpose of selecting u(k) is to minimize the
following performance function.
冕冑
冉 冕冑
J=
b
1
2
冑
( ␥(y, u(k + 1)) − g(y))2dy + Ru2(k)
a
= 1−
b
冑
冊
␥(y, u(k + 1)) g(y)dy + Ru2(k)
a
(27)
Based upon Equation (25), ␥(y, u(k+1)) is only related to (u(k), u(k − 1), %, u(0)).
This means that J is a function of (u(k), u(k − 1), %, u(0)). Assuming the current
sample time is k, then (u(k − 1), u(k − 2), %, u(0)) are available. As such, J can be
regarded as only related to u(k).
To minimize J, u(k) should be selected from:
⭸J
=0
⭸u(k)
(28)
After following the steps presented in appendix A.2, an optimal control strategy
can be found as follows:
冉
u(k) = u(k − 1) + 2R −
⭸F
⭸u
冊
−1
(u(k−1),V(k))
冉冊
⭸F
⭸u
⌬V
(29)
(u(k−1),V(k))
where ⌬V = Vref − V(k) and u0 = c2 /2R.
In the formulations presented later in appendix A.2, we have assumed that
a22 − 4a1a3 ⬎ 0 for all V 苸 Rn−1. This imposes a condition on the norm of V(k) as
shown in Equation (22) as matrix Qab is a positive definite matrix. Indeed, if the
condition in Equation (22) is not satisfied for the optimal controller considered,
then the weights of the basis function becomes complex numbers due to the square
root operation in Equation (55). As a result, the control strategy should be
obtained by performing a constrained optimization which guarantees the constraint given in Equation (22).
In this case, to keep weights in the region specified by Equation (22), 储V(k + 1)储
should satisfy:
储V(k + 1)储2 = [AV(k + 1) + Bu(k)]T[AV(k + 1) + Bu(k)]
= VT(k)ATAV(k) + 2VT(k)ATBu(k) + BTBu2(k)
ⱕ
冪␭
a3
max
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
(30)
(31)
Wang et al.
59
where u(k) is a scalar. Denote
f(u(k)) = ␭1(V(k)) + 2␭2(V(k))u(k) + ␭3(V(k))u2(k)
(32)
where
␭1(V(k)) = VT(k)ATAV(k) −
冪␭
a3
max
␭2(V(k)) = VT(k)ATB
␭3(V(k)) = BTB
(33)
Then it can be seen that to satisfy the constraint given in Equation (22), f(u(k))
must be kept less than zero. If Equation (32) is solved in terms of u(k) for V(k),
the lower and upper limits of the control are obtained as follows:
u1(k), u2(k) =
冑
−␭2 ± ␭22 − ␭1␭3
␭3
(34)
This equation implies that ␭22 − ␭1␭3 ⱖ 0 in order to obtain a real controller input
which is able to keep the weights in the specified region given by Equation (22).
The following theorem states that if 储A储2 ⬍ 1, then there is always a real input
that can keep the weights inside the region specified by Equation (22).
Theorem 3.1
Let a stable linear time invariant discrete-time model be given as
V(k + 1) = AV(k) + Bu(k)
(35)
Provided that 储V(k)储2 ⱕ b and 储A储2 ⱕ 1, then, there exists a u(k) 苸 R such that 储V(k +
1)储2 ⱕ b is guaranteed.
The proof of this theorem will be supplied in appendix A.3. If 储A储 is not less
than one, then the norm of A-matrix can be made less than 1 by using a linear
transformation. This can be shown true because we have assumed that all of the
eigenvalues ␭ of the A-matrix lie within the unit circle (i.e., 兩␭兩 ⬍ 1).
This theorem also provides the condition for the stability of the closed loop
system, as V(k) is always bounded when control input u(k) in Equation (34) is
applied.
The algorithm summary for real systems can be given as follows:
step 1 – at sample time k, take a population of the system output to find ␥(y, u(k)).
step 2 – use ␥(y, u(k)) and Equation (49) to find out the weights V(k) and wn(k).
step 3 – calculate u(k) from Equation (34) and apply it to the system given in
Equations (26) and (45).
step 4 – increase k by 1 and go back to step 1.
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
60 Bounded dynamic stochastic distribution
4. Simulations results
In order to show the applicability of the method proposed in this paper, an
example is considered, and an optimal control strategy developed in the previous
section is applied to this example. Simulations are carried out in a MATLAB
environment (Baki et al., 1999), and the results are presented using the parametric
uncertain toolbox (PUT; toolbox Baki and Munro, 1997) in the two- and threedimensional plots. In this example, a truncated and bounded normal distribution
is considered and one of the parameters of this distribution is dynamically related
to the control input. Here, the basis functions are chosen as third-order B-spline
functions, and they cover the interval completely.
In this example, a truncated and Gaussian-like distribution is considered and
four basis functions are used for the approximation purpose. To construct such a
probability density function, let us choose
␥0(y) =
1
冑2␲␴
(y−m)2
e−
2␴2
y 苸 [a, b]
(36)
where ␴ ⬎ 0 is a constant and m is a parameter which is dynamically related to
the control input. Define the following integration function
F(y) =
冕
y
␥0(b)db
(37)
a
Since a and b are bounded numbers, ␥0(y) is not a proper probability density
function. To make it a probability density function, we define
F*(y) =
F(y) − F(a)
y 苸 [a, b]
F(b) − F(a)
(38)
This leads to the following probability density function
(y−m)2
dF*(y)
1
−
␥(y) =
=
e 2␴2 y 苸 [a, b]
dy
(F(b) − F(a)) 2␲␴
冑
(39)
This model represents some real systems in practice. For example, in papermaking
systems, one of the pulping methods (i.e., producing required fibres from wood
chips) is called thermo-mechanical pulping (TMP, see Smook, 1992). In this process, fibre length distribution can be approximated as a truncated and Gaussianlike distribution as shown in Equation (39), where parameter m of such a distribution is dynamically related to the disc gap (i.e., u(k)).
For this example, the dynamical relationship between parameter m and the control input u(k) is chosen as follows:
m(k + 1) = 0.9 m(k) + u(k), and m(0) = 1
(40)
and to simulate the system, ␴, a and b are set to 1, −3 and +3, respectively. The
square root of the bounded-probability density function is expressed in terms of
basis functions as follows:
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
冑␥(y, u) = B (y)w
1
1
+ B2(y)w2 + B3(y)w3 + B4(y)w4
61
(41)
where
B1(y) = [y2 + 6y + 9]I1 + [−y2 − 3y − 1.5]I2 + [y2 ]I3
B2(y) = [y2 + 4y + 4]I2 + [−y2 − 1y − 0.5]I3 + [y2 − 2y + 1]I4
B3(y) = [y2 + 2y + 1]I3 + [−y2 + 1y − 0.5]I4 + [y2 − 4y + 4]I5
B4(y) =
[y2 ]I4 + [−y2 + 3y − 1.5]I5 + [y2 − 6y + 9]I6
(42)
and Ii are the intervals and defined as follows
Ii(y) =
再
1 y 苸 [␭i, ␭i+1 )
0
(43)
elsewhere
and ␭i = i − 4, (i = 1, 2, 3, 4, 5, 6). In Figure 3, all the basis functions and an
approximated probability density function are shown.
In order to express a dynamical relationship in terms of the A and B matrices
which represent the linear and dynamical relationship between the control input
and weights, the example model given by Equations (39) and (40) is excited by a
square-wave input and the corresponding probability density function is obtained.
Hence, weights for every m(k) are calculated via the population of output samples
at each sample time k.
After collecting enough data (square-wave inputs versus weights), an identification algorithm using least-square estimation has been employed. This algorithm
produces the following A and B matrices:
冤
0.8713
0.0185 −0.0291
A = −0.0260
0.9470 −0.0459
−0.0177
−0.0239
0.9167
冥
冤
−0.2157
and B = −0.0122
0.0134
冥
(44)
As we discussed in the introduction of this paper, a population of outputs is
Figure 3
Approximation using B-spline basis functions
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
62 Bounded dynamic stochastic distribution
generally available rather than its probability density function. Therefore, to simulate the system we need to produce an output population at each sample time k
after obtaining the probability function using A and B matrices in Equation (44).
This can be achieved using the cumulative function given in Equation (1). A cumulative function at sample time k is shown in Figure 4. From this cumulative function, satisfactory numbers of uniformly distributed points (in our case 5000 points)
are chosen between 0 and 1 along vertical axis, and then the corresponding
sampled values of the system output (between a and b) can be obtained through
the projection principle with respect to this cumulative function. These 5000 output samples constitute a population of the system output at sample time k. In the
simulation, these 5000 output samples represent the system output distribution
when it is subjected to the control input. The procedure for generating a population of the system output using an output cumulative distribution function is
illustrated in Figure 4.
Once these outputs are obtained, it can be used to find the corresponding probability density function. In our example, we used a function written in MATLAB
for this purpose, and found the corresponding weights (so-called ‘measured
weights’ since they only become available through a measurement process in a
real application) using the formulae in Equation (49). A block diagram of this
simple example controlled by the optimal controller designed earlier is shown in
Figure 5.
To this end, the system given by Equations (39) and (40) can be equivalently
represented by Equations (26) and (41). The initial values of the weights are
chosen as w = [1, 0.3, 0.5, 0.18461], and the targeted probability density function
has a set of weights given by w = [0.7500, −0.2000, 0, 1.1580]. The simulation
results are presented in Figures 6 and 7. In these figures, the role of the control
action in producing the desired probability density function can clearly be seen.
It can be concluded that desired simulation results have been obtained.
Since the purpose of control is to make the actual shape of the probability density function track that of a given probability density function, the dynamic change
Figure 4 Producing
function
output
population
using
cumulative
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
63
Figure 5 A block diagram of the model controlled by the
optimal controller
Figure 6 Action of optimal control in probability density
function
of actual distribution can best be illustrated by using three-dimensional graphic
where time, output variable and probability density function are chosen as the X,
Y and Z axes, respectively. The three-dimensional surface plot of the set of output
probability density functions controlled by the optimal control for this example
is shown in Figure 6. The initial and the reference (target) probability density
functions are given in Figure 7(a) together with the basis functions chosen. The
response of the control input is shown in Figure 7(b).
In this simulation, there is a square root operation applied to the probability
density function. For this reason, 兩√␥(y, u(k))兩 will be obtained rather than
√␥(y, u(k)) itself. Therefore, there will be some nonderivable points in
兩 √␥(y, u(k))兩. These points constitute important information in retrieving the actual
shape of √␥(y, u(k))兩. The positive values of 兩√␥(y, u(k))兩 are changed with negative
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
64 Bounded dynamic stochastic distribution
Figure 7
Initial and reference probability density functions
counterparts between these nonderivable points and the correct weights, which
represent the approximation to √␥(y, u(k)), can then be easily found.
5. Conclusions
In this paper, the bounded distribution control problem has been addressed and
an optimal control strategy has been developed using the square root of the
bounded probability density function. An example model is considered and the
optimal control is applied to this model successfully. In the approximation part
of the probability density function, B-spline basis functions have been used. Therefore, optimal strategy has been developed such that the weights for the B-spline
basis functions always stay inside a region specified by the requirement of the
probability density function (i.e., the integral from the lower bound to the upper
bound is equal to 1.0). This imposes that the control input must also be limited
by certain boundary values given by Equation (34), hence, the convergence speed
of the optimal control is reduced by this operation.
As an extension to this paper, future work can be performed to develop new
optimal control strategies. Here, in order to maintain the weights inside the region
specified, the control input is sandwiched by two boundary values. Another idea
might be that these constraints can be considered in the performance index as
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
65
done in the Lagrange multiplier case. Therefore, restrictions on the weights can
be expressed as another state variable, and control strategy can be developed
accordingly. In our case, using the optimal control sandwiched by two boundary
values, the desired weights have been achieved, as we can see from Figure 6.
Acknowledgement
The authors would like to thank the financial support from the UK EPSRC
grant (GB/K97721).
References
Åström, K.J. 1970: Introduction to stochastic control theory. New York: Academic Press.
Baki, H., Kabore, P. and Wang, H. 1999:
Bounded distribution control toolbox
(MATLAB). Manchester: Department of
Paper Science, UMIST.
Baki, H. and Munro, N. 1997: A MATLAB toolbox for parametric uncertain models. In: Colloquium in Robust Control: Theory, Software and
Applications, Savoy Place, London, October,
London: IEE, pp. 6/1–6/3.
Brown, M. and Harris, C. 1994: Neurofuzzy
adaptive modelling and control. HemelHempstead: Prentice-Hall.
Dodson, C. and Wang, H. 1999: Information
geometry for statistical parameter estimation
in process optimisation algorithms for stochastic systems. Submitted to Journal of Statistical Inference for Stochastic Processes.
Gajic, Z. and Losada, R. 1998: Monotonicity of
algebraic Lyapunov iterations for optimal
control of jump parameter linear systems. In:
Proceedings of the 1998 American Control Conference. Philadelphia: Omnipress, Volume 1,
746–50.
Moore, M. 1997: Practical applications of a
novel web monitor for paper machines. In:
Proceedings of the 1997 APPITA Conference,
Melbourne. Sydney: Australia Tappi.
Smook, G. 1992: Handbook for pulp and paper
technologists. Vancouver: Angus Wilde Publications.
A.
Wang, H., Wang, A.P. and Duncan, S. 1997:
Advanced process control for paper and board
making. Leatherhead, Surrey: Pira International Ltd.
Wang, H. 1998a: Control of the output probability density functions for a class of nonlinear stochastic systems. In: Proceedings of the
IFAC Workshop on AARCT, Cancum, Mexico.
Oxford:Elsevier Science, 95–99.
—— 1998b: Detect unexpected changes of the
particle size distribution in paper making
white water systems. In: Proceedings of the
IFAC Workshop on Online Fault Detection and
Supervision in the Chemical Process Industries,
Lyon. Oxford: Elsevier Science, 56–60.
—— 1998c: Robust control of the output probability density functions for multivariable
stochastic systems. In: Proceedings of the 37th
IEEE Conference on Decision and Control,
Tampa, Florida. Philadelphia: Omnipress,
1305–10.
—— 1999a: Robust control of the output probability density functionss for multivariable
stochastic systems with guaranteed stability.
IEEE Transaction on Automatic Control 44,
2103–07.
—— 1999b: Model reference adaptive control
of the output stochastic distributions for
unknown linear stochastic systems. International Journal of Systems Science 30, 707–15.
Appendices
A.1 Finding weights from the probability density function
The square root of the probability density function, √␥(y, u(k), can be written in
terms of basis functions and weights as follows
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
66 Bounded dynamic stochastic distribution
√␥(y, u(k)) = [C0(y) Bn(y)]
冋
V(k)
h(V(k))
册
(45)
where C0(y) is defined in Equation (5).
By multiplying Equation (45) by [CT0 (y) Bn(y)]T, the following equation is
obtained.
冋 册
C0T(y)
Bn(y)
√␥(y, u(k)) =
冋 册
CT0 (y)
Bn(y)
[C0(y) Bn(y)]
After some manipulations, this equation becomes
冋 册
CT0 (y)
Bn(y)
冋
√␥(y, u(k)) =
C0T(y)C0(y) CT0 (y)Bn(y)
Bn(y)C0(y)
2
n
B (y)
冋
V(k)
h(V(k))
册冋
册
(46)
册
V(k)
h(V(k))
(47)
To find the weights from the probability density function, we integrate both sides
of Equation (47) from a and b. This leads to the following equation
冋
兰ab CT0 (y) √␥(y, u(k))dy
兰ab Bn(y) √␥(y, u(k))dy
册 冋
=
兰ab C0T(y)Co(y)dy 兰ab CT0 (y)Bn(y)dy
兰ab Bn(y)C0(y)dy
兰ab B2n(y)dy
册冋
V(k)
h(V(k))
册
(48)
Hence, the weights can be found by performing a matrix inversion to give
冋
where
V(k)
h(V(k))
−1
A1 = Qab
冕
册 冋
=
A1 A2
AT2 A3
册冋 册
C2
(49)
C3
b
B2n(y)dy
a
−1
A3 = Qab
冕
b
A2 = − Q−1
ab
冕
冋
Bn(y)C0(y)dy
a
b
CT0 (y)C0(y)dy
a
Qab = det
C2 =
冕
冕
b
兰ab CT0 (y)Co(y)dy 兰ab C0T(y)Bn(y)dy
兰ba Bn(y)C0(y)dy
兰ba B2n(y)dy
册
CT0 (y) √␥(y, u(k))dy
a
C3 =
b
Bn(y) √␥(y, u(k))dy
a
Equation (49) reveals the relationship between the weights and the measured
probability density function, where the weights are clearly defined once the output
probability density function is measured.
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
Wang et al.
67
A.2 Derivation of optimal control strategy
Using Equation (27) and (28), it can be shown that:
−
冕
b
√g(y)
a
⭸√␥(y, u(k + 1))
dy + 2Ru(k) = 0
⭸u(k)
(50)
and
⭸ √␥(y, u(k + 1))
⭸h
= (C0(y) + Bn(y)
(V(k + 1)))B
⭸uk
⭸V(k + 1)
(51)
By substituting Equation (51) into Equation (50), it can be further obtained that:
冉冕
−
b
√g(y)C0(y)Bdy −
a
冕
b
√g(y)Bn(y)
a
⭸h
(V(k + 1))Bdy) + 2Ru(k) = 0
⭸V(k + 1)
(52)
To simplify the formulation, let us define
F(u(k), V(k)) = −
冕
b
√g(y)C0(y)Bdy −
a
冕
b
√g(y)Bn(y)
a
⭸h
Bdy
⭸V(k + 1)
(53)
To minimize J at each sample time k, one has to solve the following nonlinear
algebraic equation:
F(u(k), V(k)) + 2Ru(k) = 0
(54)
For this purpose, using c1 and Qab in Equation (16), it can be shown that
冉
⭸h
1
1
− c1 ±
[8c1V(k + 1)c1
=
2
⭸V(k + 1) 2a3
2 √a2 − 4a1a3
−8a3Vⳕ(k + 1)
=
冉
冕
冕
b
Cⳕ
0 (y)C0(y)dy])
a
1
4
− c1 ± 2
Vⳕ(k + 1) [c1ⳕc1
2a3
√a2 − 4a1a3
− a3
b
Cⳕ
0 (y)C0(y)dy])
(55)
a
a22 − 4a1a3 = 4(a3 − Vⳕ(k + 1) QabV(k + 1))
a2 = 2c1V
(56)
(57)
This leads to the following expression for F(u(k), V(k))
F(u(k), V(k)) = c2 ±
= c2 ±
2
a3
冕
b
a
1
√g(y) Bn(y) Vⳕ(k + 1)QabBdy
P
2 Vⳕ(k + 1)QabB
a3
P
冕
b
√g(y) Bn(y)dy
a
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016
(58)
68 Bounded dynamic stochastic distribution
where
P = √a3 − Vⳕ(k + 1)Qab(k + 1)
冕冋
b
c2 =
√g(y) (−C0(y)B + Bn(y)
a
册
c1B
dy
2a3
V(k + 1) = AV(k) + Bu(k).
(59)
(60)
(61)
Using this expression, it remains for Equation (54) to be solved. For this purpose,
let us put it in the following format:
2Ru(k) − c2 = ±
Vⳕ(k + 1)QabB
c3 = F(V(k), u(k))
P
(62)
where c3 = 兰ab √[g(y)]Bn(y)dy and F(V(k), u(k)) is obtained by calculating V(k + 1)
in terms of V(k) and u(k) using the linear relationship given in A and B. By differentiating Equation (62), it can be obtained that:
冉
2R −
冊
⭸F
⭸F
⌬u −
⌬V = 0
⭸u
⭸V
(63)
A.3 The proof of Theorem 3.1
In Equation (34), the condition that makes ⌬ = ␭22 − ␭1 ␭3 ⱖ 0 should be searched.
The square root expression in this equation is given as
⌬ = VT(k)ATBBTAV(k) − V(k)TATV(k)BTB + b2BTB
= VT(k) [(ATB) (ATB)T − ATABTB]V(k) + b2BTB
(64)
Note that B B is a scalar. Since V (k)V(k) ⱕ b , it can be shown that
T
T
2
⌬ ⱖ VT(k)[(ATB) (ATB)T − ATABTB]V(k) + VT(k)V(k)BTB
= VT(k)((ATB) (ATB)T − ATABTB + BTB)V(k)
= VT(k) [(ATB) (ATB)T + BTB(I − ATA)]V(k)
(65)
From this last expression, if 储A A储 ⱖ 1, then ⌬ ⱖ 0. This is because
T
储ATA储2 ⱕ 储AT储2 储A储2 = 储A储22 ⬍ 1
(66)
This is given as a condition in the theorem. Therefore, 储A储2 ⬍ 1 is a sufficient
condition to have a real input that keeps the state vector V(k + 1) inside the ball.
Downloaded from tim.sagepub.com at PENNSYLVANIA STATE UNIV on September 16, 2016