c
ICIC International °2008
ISSN 1881-803X
ICIC Express Letters
Volume 2, Number 4, December 2008
pp. 371-377
SUB-OPTIMAL RISK-SENSITIVE FILTERING FOR THIRD DEGREE
POLYNOMIAL STOCHASTIC SYSTEMS
Ma. Aracelia Alcorta-G., Michael Basin
Juan J. Maldonado O., Sonia G. Anguiano R.
Department of Physical and Mathematical Sciences
Autonomous University of Nuevo Leon
Cd. Universitaria, San Nicolas de los Garza, Nuevo Leon, 66450, Mexico
{ aalcorta; mbasin }@fcfm.uanl.mx
{ matematico one; srostro } @hotmail.com
Received June 2008; accepted September 2008
Abstract. The risk-sensitive filtering problem with respect to the exponential meansquare criterion is considered for stochastic Gaussian systems with third degree polynomial drift terms and intensity parameters multiplying diffusion terms in the state and
observations equations. The closed-form suboptimal filtering algorithm is obtained linearizing a nonlinear third degree polynomial system at the operating point and reducing
the original problem to the optimal filter design for a first degree polynomial system.
The reduced filtering problem is solved using quadratic value functions as solutions to
the corresponding Hamilton-Jacobi-Bellman equation. The performance of the obtained
risk-sensitive filter for stochastic third degree polynomial systems is verified in a numerical example against the mean-square optimal third degree polynomial filter and extended
Kalman-Bucy filter, through comparing the exponential mean-square criteria values. The
simulation results reveal strong advantages in favor of the designed risk-sensitive algorithm for large values of the intensity parameters.
Keywords: Risk-sensitive filtering, Stochastic systems
1. Introduction. The optimal mean-square filtering theory was initiated by Kalman and
Bucy for linear stochastic systems, and then continued for nonlinear systems in a variety
of papers (see for example [1-7]). More than thirty years ago, Mortensen [8] introduced
a deterministic filter model which provides an alternative to stochastic filtering theory.
In this model, errors in the state dynamics and the observations are modeled as deterministic ”disturbance functions,” and a mean-square disturbance error criterion is to be
minimized. Special conditions are given for the existence, continuity and boundlessness
of a drift f (x) in the state equation and a linear function h(x) in the observation one. A
concept of the stochastic risk-sensitive estimator, introduced more recently by McEneaney
[9], in regard to a dynamic system including nonlinear drift f (x), linear observations, and
intensity parameters multiplying diffusion terms in both, state and observation, equations.
Again, the exponential mean-square (EMS) criterion, introduced in [10] for deterministic
systems and in [11] for stochastic ones, is used instead of the conventional mean-square
criterion to provide a robust estimate, which is less sensitive to parameter variations in
noise intensity. This paper presents a solution to the risk-sensitive filtering problem with
respect to the exponential mean-square criterion for stochastic third degree polynomial
systems including intensity parameters multiplying diffusion terms in both, state and
observation, equations. The closed-form suboptimal filtering algorithm is obtained linearizing a nonlinear third degree polynomial system at the operating point and reducing
the original problem to the optimal filter design for a first degree polynomial (affine)
system. The reduced filtering problem is solved seeking quadratic value functions as solutions to the corresponding Hamilton-Jacobi-Bellman equation. Undefined parameters
371
372
M. A. ALCORTA, M. BASIN, J. J. MALDONADO AND S. G. ANGUIANO
in the value functions are calculated through ordinary differential equations composed by
collecting terms corresponding to each power of the state-dependent polynomial in the
HJB equation. The closed-form risk-sensitive filter equations are explicitly obtained. The
performance of the obtained risk-sensitive filter for stochastic third degree polynomial
systems is verified in a numerical example against the mean-square optimal third degree
polynomial filter and extended Kalman-Bucy filter, through comparing the exponential
mean-square criteria values. The simulation results reveal strong advantages in favor of
the designed risk-sensitive algorithm for large values of the intensity parameters multiplying diffusion terms in state and observation equations. Tables of the criteria values and
simulation graphs are included.
2. Problem Statement and Preliminaries.
2.1. Optimal risk-sensitive filtering problem. Consider the following stochastic diffusion model, for the state process Xt :
√
(1)
dXt = f (Xt )dt + ²dWt ,
where f (xt ) represents the nominal dynamics. The observation process Yt satisfies the
√
equation: dYt = h(xt )dt + ²dW̃t . Here, ² is a parameter and W and W̃ are independent
Brownian motions, which are also independent of the initial condition X0 . X0 has probability density k² exp(−²−1 φ(x0 )) with a certain constant k² . The exponential mean-square
cost function to be minimized over possible estimates mt is given by:
Z
1 T
J = ²logE{exp
(xt − mt )T (xt − mt )dt/Yt },
(2)
² 0
where E(ξ/Yt ) is the conditional expectation of a random variable ξ with respect to the
observations process Yt . In the rest of the paper, the assumptions (A1)-(A4) from [12]
hold. Let q(T, x) denote the unnormalized conditional density of XT , given observations
Yt for 0 ≤ t ≤ T. It satisfies the Zakai stochastic PDE, in a sense made precise in [13],
Sec. 7. Since the normalizing constant k² above is unimportant for q, it is assumed that
q(0, x) = exp(−²−1 φ(x)),
q(s, x) = p(s, x)exp[²−1 Yt · h(x)],
(3)
where p(s, x) is called pathwise unnormalized filter density. Then, p satisfies the following
linear second-order parabolic PDE with coefficients depending on YT
K
∂p
= (Ls )∗ p + p,
∂s
²
(4)
where, for any g ∈ Rn , let
1
1
²
tr(gxx ) + f · gx , K(t, x) = (Yt · h)x · (Yt · h)x − L(Yt · h) − |h|2 . (5)
Lg =
2
2
2
L denote the differential generator of the Markov diffusion Xt in (1). By assumptions
(A1) and (A3) in [12], K is bounded and continuous. Since Y0 = 0, p(0, x) = q(0, x). The
initial condition for (4) is given by (3). We rewrite (4) as follows:
∂p
B
1
= tr(pxx ) + A · px + p,
(6)
∂s
2
²
where A = −f (x) + (Yt · h(x))x , B(t, x) = −²div[f (x) − (Yt · h(x))x ] + K(t, x), (divax )j =
Σni,j=1 (aij )xi , j = 1, ..., n. Taking log transform Z(T, x) = ²logp(T, x), the nonlinear
parabolic PDE is obtained
∂Z
∂s
=
1
²
tr(Zxx ) + A · Zx + Zx · Zx + B,
2
2
(7)
ICIC EXPRESS LETTERS, VOL.2, NO.4, 2008
373
with initial data Zx (0, x) = −φ(x). The risk-sensitive optimal filter problem consists
in finding the estimate CT of the state xt , verifying that Z(s, x) = 12 (x − Cs )T Qs (x −
Cs ) + ρs − Yt · h(xt ), is a viscosity solution of (7). The notation for all the variables is
x(t) = xt , xt ∈ Rn , Wt ∈ Rm , yt ∈ Rp , f, h ∈ Rn , where fx , hx are assumed bounded. Here,
hx is the matrix of partial derivatives of h. The same notation holds for Zx .
3. Risk-sensitive Suboptimal Filter. Taking f (xt ) = At +A1t xt +A2 (t)xT x+A3 (t)xxxT ,
h(xt ) = Et + E1t xt , where At ∈ Rn , A1t , ∈ Mn×n , A2 (t) is a tensor of dimension n × n ×
n, A3 (t) is a tensor of dimension n × n × n × n, Et ∈ Rp , E1t ∈ Mn×p , the following system
of stochastic equations is obtained:
√
dXt = At + A1t Xt + A2 (t)X T X + A3 (t)X T XX + ²dB̂t , x0 = x0 ,
(8)
√
dYt = Et + E1t Xt + ²dB̃t .
Regarding that f (xt ) is a third degree polynomial and linearizing the expansion of f (xt )
in Taylor series around the equilibrium point ξ, the suboptimal filtering algorithms are
obtained and presented in the following theorem.
Theorem 3.1. The suboptimal solution to the filtering problem for the system (8) with
the exponential mean-square criterion (2) takes the form:
Ċs = f 0 (ξ)ξ + f 0 (ξ)T Cs − Q−1 E1 (dy − E1t Cs − Et ),
(9)
T
E1t ,
Q̇s = −f 0 (ξ)T Qs − Qs f 0 (ξ) + QTs Qs − E1t
where ξ is the equilibrium point of f (xt ).
Proof: The value function is proposed
1
Z(s, X) = (Xt − Cs )T Qs (Xt − Cs ) + ρs − Yt · (Et + E1t xt ),
2
where Zx (0, x) = −φ(x), Cs , Qs , ρs are functions of s ∈ [0, T ], Cs ∈ Rn , Qs is a symmetric
matrix of dimension n × n and ρs is a scalar function) as a viscosity solution of the
nonlinear parabolic PDE (7). Zx , Zxx are the partial derivatives of Z respect to x, and
∇Z is the gradient of Z. The partial derivatives of Z are given by:
1
1
1
∂Z
= (Xt − Cs )T Q̇s (Xt − Cs ) + ρ̇s − ĊsT Qs (Xt − Cs ) − (Xt − Cs )T Qs Ċs − dYt · (Et
∂s
2
2
2
∂Z
1
∂2Z
1
+E1t xt )
= Qs (X − Cs ) + (X − Cs )T Qs − Yt E1t ,
= Qs .
∂x
2
2
∂x∂x
Substituting (10) and the expressions for A, B in (7), the following expression is obtained:
1
1
1
(Xt − Cs )T Q̇s (Xt − Cs ) + ρ̇s − ĊsT Qs (Xt − Cs ) − (Xt − Cs )T Qs Ċs − dYt · (Et + E1t xt ) =
2
2
2
²
1
0
0
(q11 + q22 + .. + qnn ) + [−f (ξ)ξ − f (ξ)x + YT Ė1 ][QT (x − Cs ) − YT E1 ] + [QT (x − CT ) −
2
2
1
1
0
0
YT E1 ][QT (x − CT ) − YT E1 ] − ²A1 + YT E1 · YT E1 − (f (ξ)ξ + f (ξ)x)YT E1 − (E + E1 x)2
2
2
Collecting the second degree terms, equalizing them to zero, and doing it again for the
terms of first degree, the risk-sensitive filtering equations (9) are obtained. In similar
manner, collecting the zero degree terms, the equation for ρs is obtained.¦
Here, QT is a symmetric negative definite matrix, and the initial condition Q0 = q0 is
derived from initial conditions for Z. If φ(xt ) = xTt Kxt , Q(0) = −K. It is easy to verify
that if Q = −P −1 , where P is the covariance matrix, those equations are equivalent to
the Kalman-Bucy filtering equations, as was shown in [9].
374
M. A. ALCORTA, M. BASIN, J. J. MALDONADO AND S. G. ANGUIANO
4. Example. Consider the pendulum equations with friction [14] for the dynamical system with state and output equations:
√
√
g
k
ẋ1 = x2 , ẋ2 = − Senx1 − x2 + ²ψ1 (t), xi0 = xio , yt = x1 + ²ψ2 (t),
l
m
(10)
where x ∈ R2 , g = 9.8m/seg 2 is the gravity constant, l = 0.5m is the length of the
road, m = 0.25kg denotes the mass of the bob, θ is the angle subtended by the road
and the vertical axes through the pivot point, k = 0.001 denotes the friction coefficient.
White Gaussian noises ψ1 , ψ2 are the derivatives of the Brownian motion Wi , which are
independent of each other and the initial condition xi0 = xio , ² is a varying parameter.
Note that x1 is the measured variable, while x2 is observable but not measured. Expanding
in Taylor series around the origin, this system is reduced to the third degree polynomial
√
√
x3
k
form: ẋ1 = x2 , ẋ2 = − gl (x1 − 3!1 ) − m
x2 + ²ψ1 (t), xi0 = xi , yt = x1 + ²ψ2 (t). Further
linealization yields the linear dynamical system:
√
√
g
k
dx1t = x2 , dx2 = − x1 − x2 dt + ²ψ1 (t), xi0 = xi , yt = x1 dt + ²ψ2 (t),
l
m
(11)
Substituting the corresponding values from (10) into (9), the equations for the suboptimal
risk-sensitive filtering algorithms are given by:
g
1
(q22 (ẎT − C1 ) + q12 C2 ),
Ċ1 = − C2 + 2
l
q12 − q22 q11
k
1
Ċ2 = C1 − C2 − 2
(q21 (ẎT − C1 ) + q11 C2 ),
m
q12 − q22 q11
(12)
where q12 , q22 , q11 are the solutions of the following symmetric Riccati matrix equation :
g
g
k
2
2
+ q12
− 1, q12
˙ T = q22 − q11 + q12 + q11 q12 + q12 q22 ,
q11
˙ T = 2 q21 + q11
l
l
m
2k
2
2
+ q22
− 1.
q22
˙ T =
q22 − 2q21 + q12
m
(13)
The last equations (12) and (13) are simulated using Simulink in M atLab7. The initial conditions for the simulation are x0 = 0, q11 (0) = −8, q12 (0) = −1.75, q22 (0) =
−1.5, C1 (0) = 1, C2 (0) = 1, T = 10. The graphs of the difference between the state xt ,
and the estimate CT , that is, ei = xi − Ci , for i = 1, 2, with ² = 1000, are shown in Figure
1. Applying the extended Kalman-Bucy filter algorithms [15] to the state equations (11),
the equations for the estimate vector m(t) and symmetric covariance matrix P (t) are
obtained:
p11
g
k
p12
(dY (t) − m1 (t)dt), dm2 (t) = − m1 (t)dt − m2 (t)dt +
(dY (t)
²
l
m
²
p2 (t) + p212 (t)
g
k
+ ², ṗ12 (t) = − p11 (t) − p12 (t) + p22 (t) −
−m1 (t)dt), ṗ11 (t) = −2p12 (t) − 11
²
l
m
g
k
p2 (t) + p222
p11 (t)p12 (t) + p22 (t)p12 (t)
, ṗ22 (t) = −2 p12 (t) − 2 p22 (t) − 12
+ ².
²
l
m
²
dm1 (t) = m2 (t)dt +
This system of equations is simulated with the initial conditions: m1,2 (0) = 1, p11 (0) =
0.1678321, p12 (0) = −0.1958041, p22 (0) = 0.89510. The graph of the difference between
state xi , and the estimate mi (t), that is, ei = xi − mi , for ² = 1000, can be observed in
ICIC EXPRESS LETTERS, VOL.2, NO.4, 2008
375
Table 1. Comparison of mean-square criterion (2) for R-s filter, third
degree polynomial and K-B extended filter for certain values of ².
² value JR − S J − polynomial Jext.K − B
0.001 8.10669
0.5528
0.5647
0.01
8.1077
0.9272
1.4173
0.1
8.1105
7.1198
15.9131
1
8.12
10.7299
51.7838
6
10
8.255
1.285 × 10
70.8296
100
9.3933
8.25 × 109
69.6641
13
1000
21.4763
1.42 × 10
62.0612
Figure 3. The third degree polynomial filtering equations from [1] are given by
p11
g
k
g
(Ẏ − m1 (t)), ṁ2 (t) = − m1 (t) − m2 (t) + (3p11 m1 (t) + m31 (t))
²
l
m
6l
1 2
g
k
g
p21
(t) + p22 + p211 (t) +
+ (Ẏ − m1 (t)), ṗ11 (t) = 2p12 + ² − p11 , ṗ12 (t) = − p11 (t) −
²
²
l
m 12
2l
2
2g
2k
g
g
g
(t) + p11 (t)p12 (t) + p12 (t)m21 (t).
p11 (t)m21 (t) − p11 p12 , ṗ22 (t) = − p12 (t) −
2l
²
l
m 22
l
l
The graph of the difference between state xi , and the third degree polynomial estimate
mi (t), that is, ei = xi −mi , for ² = 1000, can be observed in Figure 2. Table 1 presents some
values of the exponential mean-square cost function corresponding to the risk-sensitive,
extended Kalman-Bucy and third degree polynomial filters. It can be observed that the
Jr−s values are less that the Jext.K−B and Jpolynomial values for large values of the parameter
².
ṁ1 (t) = m2 (t) +
Figure 1. Graphs of the absolute values of the difference between the state
xt and the risk-sensitive estimate CT , for ² = 1000.
5. Conclusions. This paper presents the suboptimal solutions to the risk-sensitive optimal filtering problem for stochastic third degree polynomial systems with Gaussian white
noises, an exponential-quadratic criterion to be minimized, and intensity parameters multiplying the white noises, using Taylor series for linearizing the third degree polynomial
in the state equation. Numerical simulations are conducted to compare performance of
the obtained risk-sensitive filter algorithms against third degree polynomial filtering algorithms and extended Kalman-Bucy filter, through comparing the exponential mean-square
criteria values. The simulation results reveal strong advantages in favor of the designed
376
M. A. ALCORTA, M. BASIN, J. J. MALDONADO AND S. G. ANGUIANO
Figure 2. Graphs of the absolute values of the difference between the xt
and the third degree polynomial estimate m(t), for ² = 1000.
Figure 3. Graphs of the absolute values of the difference between the xt
and the Kalman-Bucy extended estimate m(t), for ² = 1000.
risk-sensitive suboptimal algorithms in regard to the final criteria values, corresponding
to large values of the intensity parameters multiplying diffusion terms in the state and
observation equations.
Acknowledgment. The first author thanks the UCMEXUS-CONACyT Foundation for
financial support under Posdoctoral Research Fellowship Program and Grant 52930. The
second author thanks the Mexican National Science and Technology Council (CONACyT)
for financial support under Grants 55584 and 52953.
REFERENCES
[1] M. V. Basin and M. A. Alcorta-Garcia, Optimal filtering and control for third degree polynomial
systems, Dynamics of Continuous Discrete and Impulsive Systems, vol.10, pp.663-680, 2003.
[2] M. V. Basin and M. A. Alcorta-Garcia, Optimal filtering for bilinear systems and its application to
terpolymerization process state, Proc. of the IFAC 13th Symposium System Identification, pp.467472, 2003.
[3] M. V. Basin, J. Perez and R. Martinez-Zuniga, Optimal filtering for nonlinear polynomial systems
over linear observations with delay, International Journal of Innovative Computing, Information and
Control, vol.2, pp.357-370, 2006.
ICIC EXPRESS LETTERS, VOL.2, NO.4, 2008
377
[4] F. L. Lewis, Applied Optimal Control and Estimation, Prentice Hall Englewood Cliffs, New Jersey,
EUA, 1992.
[5] V. S. Pugachev and I. N. Sinitsyn, Stochastic Systems Theory and Applications, World Scientific,
Singapore, 2001.
[6] S.-T. Yau, finite-dimensional filters with nonlinear drift. I: A class of filters including both KalmanBucy and Benes filters, J. Math. Systems Estimation and Control, pp.181-203, 1994.
[7] H. Zhang, M. V. Basin and M. Skliar, Optimal state estimation for continuous, stochastic, state-space
system with hybrid measurements, International Journal of Innovative Computing, Information and
Control, vol.2, pp.863-874, 2006.
[8] R. E. Mortensen, Maximum likelihood recursive nonlinear filtering, J. Optim. Theory Appl., pp.386—
94, 1968.
[9] W. M. McEneaney, Robust H∞ filtering for nonlinear systems, Systems and Control Letters, pp.315325, 1998.
[10] M. D. Donsker and S. R. S. Varaqdhan, Asymptotic evaluation of certain Markov process expectations for large time, I, II, III, Comm. Pure Appl. Math., vol.28, pp.1-45, pp.279-301, 1975; vol.29,
pp.389-461, 1976.
[11] W. H. Fleming and W. M. McEneaney, Risk-sensitive control and differential games, Lecture Notes
in Control and Info. Sci., vol.184, Springer-Verlag, New York, pp.185-197, 1992.
[12] W. H. Fleming, et al., Robust limits of risk sensitive nonlinear filters, Math. Control Signals and
Systems, vol.14, pp.109-142, 2001.
[13] D. L. Lukes, Optimal regulation of nonlinear dynamic systems, SIAM J. Control Opt., vol.7, pp.75100, 1969.
[14] H. K. Khalil, Nonlinear Systems, Third Edition, Prentice Hall, New Jersey 1996.
[15] A. H. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, Inc., New York, 1970.