Discretizing stochastic dynamical systems
using Lyapunov equations
Niklas Wahlström, Patrik Axelsson,
Fredrik Gustafsson
Division of Automatic Control
Linköping University
Linköping, Sweden
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Motivation
2(15)
Stochastic dynamical systems are important in state
estimation, system identification and control.
System models are often provided in continuous time, while a
major part of the applied theory is developed for discrete-time
systems.
Discretization of continuous-time models is hence
fundamental.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Discretization
Continuous-time model
ẋ(t) = Ax(t) + Bw(t)
T
E[w(t)w(τ ) ] = Sδ(t − τ )
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
3(15)
Discrete-time model
xk+1 = Fxk + wk
E[wk wTl ] = Qδkl
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Discretization
3(15)
Discrete-time model
Continuous-time model
xk+1 = Fxk + wk
ẋ(t) = Ax(t) + Bw(t)
E[wk wTl ] = Qδkl
T
E[w(t)w(τ ) ] = Sδ(t − τ )
This gives the relations
F=e
ATs
,
Q=
Z Ts
T
eAτ BSBT eA τ dτ
0
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Discretization
3(15)
Discrete-time model
Continuous-time model
xk+1 = Fxk + wk
ẋ(t) = Ax(t) + Bw(t)
E[wk wTl ] = Qδkl
T
E[w(t)w(τ ) ] = Sδ(t − τ )
This gives the relations
F=e
ATs
,
Q=
Z Ts
T
eAτ BSBT eA τ dτ
0
Problem
How do we solve this integral in a numerically good manner
for arbitrary A, B, S and Ts ?
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Analytical solution - System has only integrators
If the system has only integrators, A is
nilpotent and eAτ can be expressed
exactly with a finite Taylor series
eAτ = I + Aτ + · · · +
4(15)
Im
1 p−1 p−1
A τ .
p!
Re
Then, the integral has an analytical
solution
Z Ts
Q=
T
eAτ BSBT eA τ dτ
0
p−1 p−1
=
i+j+1
Ts
∑ ∑ i!j!(i + j + 1) Ai S(Ai )T .
Figure : The poles of the system
i=0 j=0
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Theorem
5(15)
Theorem - Contribution 1
The solution to the integral
Z Ts
T
eAτ BSBT eA τ dτ
Q=
(1)
0
satisfies the Lyapunov equation
AQ + QAT = −BSBT + eATs BSBT eA
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
T
Ts
(2)
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Theorem
5(15)
Theorem - Contribution 1
The solution to the integral
Z Ts
T
eAτ BSBT eA τ dτ
Q=
(1)
0
satisfies the Lyapunov equation
AQ + QAT = −BSBT + eATs BSBT eA
T
Ts
(2)
Idea
Solve (2) to find solution for (1).
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Theorem
5(15)
Theorem - Contribution 1
The solution to the integral
Z Ts
T
eAτ BSBT eA τ dτ
Q=
(1)
0
satisfies the Lyapunov equation
AQ + QAT = −BSBT + eATs BSBT eA
T
Ts
(2)
Idea
Solve (2) to find solution for (1).
Lemma Eq. (2) has a unique solution if and only if
λi (A) + λj (A) 6= 0 ∀i, j
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Theorem
5(15)
Theorem - Contribution 1
The solution to the integral
Z Ts
T
eAτ BSBT eA τ dτ
Q=
(1)
0
satisfies the Lyapunov equation
AQ + QAT = −BSBT + eATs BSBT eA
T
Ts
(2)
Idea
Solve (2) to find solution for (1).
Lemma Eq. (2) has a unique solution if and only if
Note: This is not
fulfilled if the system
λi (A) + λj (A) 6= 0 ∀i, j
has integrators!
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Lyapunov equation - System with only strictly stable
poles
6(15)
Im
If the system has only strictly stable
poles, the Lyapunov equation has a
unique solution
T
T
ATs
T A Ts
AQ + QAT = −
| BSB + e{z BSB e },
−V
Re
which is identical to the solution of the
integral
Q=
Z Ts
T
eAτ BSBT eA τ dτ.
0
Figure : The poles of the system
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Summary so far
Im
7(15)
Im
Re
p−1 p−1
Q=
∑∑
i=0 j=0
i+j+1
Ts
Ai S(Ai )T
i!j!(i + j + 1)
Re
AQ + QAT = −V
How do we find Q if we have both integrators and strictly stable poles?
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
8(15)
Im
Consider a system on the following block
triangular form
A11 A12
A=
0 A22
Re
where all integrators are collected in
A22 , i.e.
λi (A11 ) 6= 0 ∀i
λj (A22 ) = 0 ∀j
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
Figure : The poles of the system
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
9(15)
The corresponding Lyapunov equation for this system reads
A11 A12
0 A22
Q11 Q12
Q11 Q12 AT11 0
V11 V12
+ T
=− T
,
QT12 Q22
Q12 Q22 AT12 AT22
V12 V22
which gives
A11 Q11 + Q11 AT11 = −V11 − A12 QT12 − Q12 AT12 ,
A11 Q12 + Q12 AT22 = −V12 − A12 Q22 ,
T
A22 QT12 + QT12 AT11 = −V12
− Q22 AT12 ,
A22 Q22 + Q22 AT22 = −V22
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
9(15)
The corresponding Lyapunov equation for this system reads
A11 A12
0 A22
Q11 Q12
Q11 Q12 AT11 0
V11 V12
+ T
=− T
,
QT12 Q22
Q12 Q22 AT12 AT22
V12 V22
which gives
A11 Q11 + Q11 AT11 = −V11 − A12 QT12 − Q12 AT12 ,
A11 Q12 + Q12 AT22 = −V12 − A12 Q22 ,
T
A22 QT12 + QT12 AT11 = −V12
− Q22 AT12 ,
A22 Q22 + Q22 AT22 = −V22
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
9(15)
The corresponding Lyapunov equation for this system reads
A11 A12
0 A22
Q11 Q12
Q11 Q12 AT11 0
V11 V12
+ T
=− T
,
QT12 Q22
Q12 Q22 AT12 AT22
V12 V22
which gives
A11 Q11 + Q11 AT11 = −V11 − A12 QT12 − Q12 AT12 ,
A11 Q12 + Q12 AT22 = −V12 − A12 Q22 ,
T
A22 QT12 + QT12 AT11 = −V12
− Q22 AT12 ,
A22 Q22 + Q22 AT22 = −V22
Proposed solution:
- contribution 2
1. Find Q22 by using the analytical solution.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
9(15)
The corresponding Lyapunov equation for this system reads
A11 A12
0 A22
Q11 Q12
Q11 Q12 AT11 0
V11 V12
+ T
=− T
,
QT12 Q22
Q12 Q22 AT12 AT22
V12 V22
which gives
A11 Q11 + Q11 AT11 = −V11 − A12 QT12 − Q12 AT12 ,
A11 Q12 + Q12 AT22 = −V12 − A12 Q22 ,
T
A22 QT12 + QT12 AT11 = −V12
− Q22 AT12 ,
A22 Q22 + Q22 AT22 = −V22
Proposed solution:
- contribution 2
1. Find Q22 by using the analytical solution.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
9(15)
The corresponding Lyapunov equation for this system reads
A11 A12
0 A22
Q11 Q12
Q11 Q12 AT11 0
V11 V12
+ T
=− T
,
QT12 Q22
Q12 Q22 AT12 AT22
V12 V22
which gives
A11 Q11 + Q11 AT11 = −V11 − A12 QT12 − Q12 AT12 ,
A11 Q12 + Q12 AT22 = −V12 − A12 Q22 ,
T
A22 QT12 + QT12 AT11 = −V12
− Q22 AT12 ,
A22 Q22 + Q22 AT22 = −V22
1. Find Q22 by using the analytical solution.
Proposed solution:
2. Find Q12 by solving a Sylvester equation.
- contribution 2
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Systems with integrators + strictly stable poles
9(15)
The corresponding Lyapunov equation for this system reads
A11 A12
0 A22
Q11 Q12
Q11 Q12 AT11 0
V11 V12
+ T
=− T
,
QT12 Q22
Q12 Q22 AT12 AT22
V12 V22
which gives
A11 Q11 + Q11 AT11 = −V11 − A12 QT12 − Q12 AT12 ,
A11 Q12 + Q12 AT22 = −V12 − A12 Q22 ,
T
A22 QT12 + QT12 AT11 = −V12
− Q22 AT12 ,
A22 Q22 + Q22 AT22 = −V22
1. Find Q22 by using the analytical solution.
Proposed solution:
2. Find Q12 by solving a Sylvester equation.
- contribution 2
3. Find Q11 by solving a Lyapunov equation.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Van Loan’s method (standard method in literature)
1. Form the 2n × 2n matrix
10(15)
Im
A BSBT
H=
.
0 −AT
2. Compute the matrix exponential
HTs
e
Re
M11 M12
=
.
0
M22
3. Then Q is given as
Q = M12 MT11 .
Van Loan, C.F. (1978). Computing integrals involving the matrix
exponential.
Figure : The poles of the system
IEEE Transactions on Automatic Control, 23(3), 395-404.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Numerical Evaluation
11(15)
Im
Marginally stable systems with
4 stable poles and 2
integrators are considered.
γfast
The 2D region
Re
γslow , γfast ∈ [10−1 , 101 ] is
divided into 25 × 25 bins.
In total 100 systems are
randomly generated for each
bin.
Consider the sampling time
Ts = 1.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
γslow
Figure : The poles of the system
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Numerical Evaluation - Proposed method
Proposed method
Im
γslow
10−5
10−1 −1
10
10−6
10−7
100
γfast
Numerical error
101
100
12(15)
γfast
Re
101
γslow
If γslow is small, the condition
is closer to be violated
λi (A11 ) + λj (A11 ) 6= 0 ∀i, j
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
Figure : The poles of the system
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Numerical Evaluation - Van Loan’s method
Van Loan’s method
Im
γslow
10−5
10−6
10−1 −1
10
10−7
100
γfast
Numerical error
101
100
13(15)
γfast
Re
101
γslow
If γfast is large, the matrix
exponential is ill-conditioned
exp
A BSBT
T
0 − AT s
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
Figure : The poles of the system
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Numerical Evaluation - Comparison
Proposed method
14(15)
Van Loan’s method
101
γslow
γslow
10−5
100
10−1 −1
10
100
γfast
101
100
10−1 −1
10
10−6
Numerical error
101
10−7
100
γfast
101
The proposed method performs better if the slowest pole is fast.
Van Loan’s method performs better if the fastest pole is slow.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Conclusions
15(15)
An algorithm for computing an integral involving the matrix
exponential common in optimal sampling has been proposed.
The algorithm is based on a Lyapunov equation and is
justified with a novel theorem.
Numerical evaluations showed that the proposed algorithm has
advantageous numerical properties for slow sampling and
fast dynamics in comparison with a standard method in the
literature.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Analytical solution - Example
16(15)
Consider a double integrator (two zero eigenvalues of A)
0
0
1
w(t), E[w(t)w(τ )] = γδ(t − τ ).
x(t) +
ẋ(t) =
1
0
0
| {z }
|{z}
B
A
Only zero eigenvalues ⇔ nilpotent, so
Aτ
e
1
= I + Aτ =
0
0
0
+
1
0
1
τ
0
Q can be computed analytically
Q=
Z Ts
"
Aτ
T AT τ
e BSB e
0
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
dτ = γ
Ts3
3
Ts2
2
Ts2
2
Ts
#
.
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET
Discretization
17(15)
Continuous-time model
Discrete-time model
ẋ(t) = Ax(t) + Bw(t)
T
E[w(t)w(τ ) ] = Sδ(t − τ )
xk+1 = FT xk + wk
E[wk wTl ] = QT δkl
...or more strict
dx(t) = Ax(t) + Bdw(t)
E[dw(t)dw(t)T ] = Sdt
where w(t) a Brownian motion.
Niklas Wahlström, Patrik Axelsson, Fredrik Gustafsson
Discretizing stochastic dynamical systems using Lyapunov equations
AUTOMATIC CONTROL
REGLERTEKNIK
LINKÖPINGS UNIVERSITET