DISCRETE AND CONTINUOUS
DYNAMICAL SYSTEMS
SUPPLEMENT 2007
Website: www.AIMSciences.org
pp. 286–293
THEORETICAL OPTIMIZATION OF FINITE DIFFERENCE
SCHEMES
Claire David and Pierre Sagaut
Université Pierre et Marie Curie-Paris 6
Institut Jean Le Rond d’Alembert, UMR CNRS 7190
Boı̂te courrier n0 162, 4 place Jussieu, 75252 Paris, cedex 05, France
Abstract. The aim of this work is to develop general optimization methods
for linear finite difference schemes used to approximate linear differential equations, on the basis of a matrix equation, which enables to determine the optimal
value of a parameter for a given scheme.
1. Introduction: Scheme classes. Finite difference schemes used to approximate linear differential equations induce numerical errors, which partly depend on
given parameters (as, for instance, the number of cells per wavelength). The usual
process consists in testing several values for those coefficients, until one finds the
optimal one.
We here propose a completely different approach, which consists in determining the
optimal value of a given parameter, which minimizes the L2 norm of the error for
a given finite difference scheme.
Consider the transport equation:
∂u
∂u
+c
=0
∂t
∂x
with the initial condition u(x, t = 0) = u0 (x).
(1)
A linear finite difference scheme for this equation can be written under the form:
α ui n+1 + β ui n + γ ui n−1 + δ ui+1 n + ε ui−1 n + ζ ui+1 n+1
+ η ui−1 n−1 + θ ui−1 n+1 + ϑ ui+1 n−1 = 0
(2)
where:
ul m = u (l h, m τ )
(3)
l ∈ {i − 1, i, i + 1}, m ∈ {n − 1, n, n + 1}, j = 0, ..., nx , n = 0, ..., nt , h, τ
denoting respectively the mesh size and time step.
The Courant-Friedrichs-Lewy number (cf l) is defined as σ = c τ /h .
A numerical scheme is specified by selecting appropriate values of the coefficients
α, β, γ, δ, ε, ζ, η, θ and ϑ in equation (2). Values corresponding to numerical
schemes retained for the present works are given in Table 1.
2000 Mathematics Subject Classification. 65.
Key words and phrases. Linear finite difference schemes, Sylvester equation.
286
THEORETICAL OPTIMIZATION OF FINITE DIFFERENCE SCHEMES
287
Table 1. Numerical scheme coefficient.
Name
Leapfrog
Lax
α
Lax-Wendroff
Crank-Nicolson
1
τ
1
2τ
1
τ
1
τ
+
c
h2
β
0
0
γ
δ
²
−1
2τ
c
2h
−c
2h
2
−1
+ ch2τ
τ
−1
+ hc2
τ
0
0
0
−1
c
+ 2h
2τ
(1−σ)c
2h
−c
h2
−1
c
− 2h
2τ
−(1+σ)c
2h
−c
h2
ζ
0
0
0
0
η
0
0
θ
0
0
ϑ
0
0
0
0
−c
h2
−c
h2
0
0
The number of time steps will be denoted nt , the number of space steps, nx . In
general, nt À nx .
The paper is organized as follows. The equivalent matrix equation is exposed in
section 2. The determination of the optimal value of a given parameter presented
in section 3.
2. The Sylvester equation.
2.1. Matricial form of the finite differences problem. Let us introduce a
rectangular matrix defined by:
U = [ui n ] 1≤i≤nx −1, 1≤n≤nt
(4)
The finite difference scheme (2) can be written under the following matricial form:
M1 U + U M2 + L(U ) = M0
(5)
where M1 , M2 and M0 are square matrices respectively nx − 1 by
given by:



β δ
0 ...
0
0 γ
0



.
.
.
.
..
..
..
..
 ε β

 α 0






.
.
.
.
.
..
..
..
.. ..
M1 =  0
M2 =  0
0 



 . .


.
.. ... β
 ..
 .. . . . . . .
δ 
0 ... 0
ε β+δ
0 ... 0
nx − 1, nt by nt ,
...
..
.
..
.
..
.
α

0
.. 
. 


0 


γ 
0
(6)
and where L is a linear matricial operator which can be written as:
L = L1 + L2 + L3 + L4
(7)
where L1 , L2 , L3 and L4 are given by:

u22
u32
...
un2 t
0
..
..
.
.
..
..
.
.
unntx −1 0
0
0
0
0
. . . un1 t −1
. . . un2 t −1
..
..
.
.

 u23
u33
...


.
..
..
L1 (U ) = ζ 
..
.
.

3
 u2
u
.
.
.
nx −1
nx −1
0
0
...

0
0
...
1
 0
u
u21
1


u01
u11
L2 (U ) = η  0
 ..
..
..
 .
.
.
0 u1nx −2 u2nx −2 . . .
unntx−1
−2















(8)
288
CLAIRE DAVID, PIERRE SAGAUT




L3 (U ) = θ 


0
u21
u22
..
.
...
u31
u32
..
.
...
...
...
..
.
...
un1 t
un2 t
..
.
0
0
0
..
.







u2
u3
. . . unntx −2 0
 nx −2 1 nx −2 2

0
u2
u2 . . . un2 t −1
 0
u13
u23 . . . un3 t −1 


 ..

.
..
..
..
..
L4 (U ) = ϑ  .

.
.
.


 0 u1
. . . . . . unt −1 
0
nx −1
0
...
(9)
nx −1
...
0
The second member matrix M0 bears the initial conditions, given for the specific
value n = 0, which correspond to the initialization process when computing loops,
and the boundary conditions:




M0 = 



−γ u01 − ε u10 − η u00 − θ u20 − ϑ u02
−γ u02 − η u01 − ϑ u03
.
.
.
0
−γ unx −2 − η u0nx −2 − ϑ u0nx −1
−γ u0nx −1 − η u0nx −2 − ζ u2nx − ϑ u0nx
−ε u20 − η u10 − θ u30
0
.
.
.
0
−ζ u3nx − ϑ u1nx
...
...
.
.
.
...
...
...
...
.
.
.
...
...
n
n −1 
−ε u0 t − η u0 t

0


.

.

.


0
t −1
−ϑ un
nx
(10)
When non-Dirichlet boundary conditions are taken into account, and when the
solution at i = nx is obtained by interpolation, setting unnx = unnx −1 , M0 has to be
changed in:

mod
M0



=



−γ u01 − ε u10 − η u00 − θ u20 − ϑ u02
−γ u02 − η u01 − ϑ u03
.
.
.
0
−γ unx −2 − η u0nx −2 − ϑ u0nx −1
0
−γ unx −1 − η u0nx −2 − ϑ u0nx
and a corrective term, of
matrix equation, where:

0 0
0

.
..
 0 0


D =  0 ... ...

 . .
.. ...
 ..
0 ... ...
−ε u20 − η u10 − θ u30
0
.
.
.
0
0
...
...
.
.
.
...
...
...
...
.
.
.
...
...
n
n −1
−ε u0 t − η u0 t
0
.
.
.
0
0








(11)
the form: D.U.M3 , has to be taken into account in the
...
..
.
..
.
0
0


0
0

.. 
 ζ
. 




,
M
=

 0
3
0 


 .

 ..
0
1
0
ϑ
0
..
.
0
..
..
.
..
..
.
.
.
...
0
...
..
.
..
.
..
.
ζ

0
.. 
. 


0 


ϑ 
0
(12)
The new matrix equation is then:
M1 U + U M2 + L(U ) + D.U.M3 = M0mod
(13)
Denote by uexact the exact solution of the transport equation (1).
The Uexact corresponding matrix will be:
Uexact = [Uexacti n ] 0≤i≤nx −1, 0≤n≤nt
(14)
THEORETICAL OPTIMIZATION OF FINITE DIFFERENCE SCHEMES
289
where:
Uexact ni = Uexact (xi , tn )
(15)
M1 U + U M2 + L(U ) = M0
(16)
with xi = i h, tn = n τ .
U is then the solution of:
We will call error matrix the matrix defined by:
E = U − Uexact
(17)
Let us consider the matrix F defined by:
F = M1 Uexact + Uexact M2 + L(Uexact ) − M0
(18)
The error matrix E satisfies then:
M1 E + E M2 + L(E) = F
(19)
2.2. The matrix equation.
2.2.1. Theoretical formulation. Minimizing the error due to the approximation induced by the numerical scheme is equivalent to minimizing the norm of the matrices
E satisfying (19).
Since the linear matricial operator L appears only in the Crank-Nicholson scheme,
we will restrain our study to the case L = 0. The generalization to the case L 6= 0
can be easily deduced: denote


0 1
0 ... 0

. 
 0 0 . . . . . . .. 




Σ =  0 ... ... ... 0 
(20)


 . .

.. ... 0 1 
 ..
0 ... 0 ... 0
we have:
L1 (U ) = ζ Σ.U.T Σ , L2 (U ) = η T Σ.U.Σ , L3 (U ) = θ T Σ.U.T Σ , L4 (U ) = ϑ Σ.U.Σ
(21)
where T denotes the transposed matrix, which yields:
L(U ) = (ζ Σ + θ T Σ).U.T Σ + (η T Σ + ϑ Σ).U.Σ
(22)
The problem is then the determination of the minimum norm solution of:
M1 E + E M2 + (ζ Σ + θ T Σ).E.T Σ + (η T Σ + ϑ Σ).E.Σ = F
(23)
which is a specific form of the generalized Sylvester equation:
AX + BXC = D
(24)
where A and B are m by m matrices, C and F n by n matrices, and X, a m by n
matrix.
290
CLAIRE DAVID, PIERRE SAGAUT
The solving of the Sylvester equation is generally based on Schur decomposition:
for a given square n by n matrix A, n being an even number of the form n = 2 p,
there exists a unitary matrix U and an upper triangular block matrix T such that:
A = U ∗T U
(25)
∗
where U denotes the (complex) conjugate matrix of the transposed matrix T U .
The diagonal blocks of the matrix T correspond to the complex eigenvalues λi of
A:


T1 0 . . . . . . 0

. 
 0 . . . . . . . . . .. 




T =  ... . . . Ti . . . ... 
(26)


 . .

.
.
..
..
.. 0 
 ..
0
0 . . . 0 Tp
where the block matrices Ti , i = 1, ..., p are given by:
µ
¶
Re [λi ]
Im [λi ]
− Im [λi ] Re [λi ]
(27)
Re being the real part of a complex number, and Im the imaginary one.
Due to this decomposition, the Sylvester equation require, to be solved, that the
dimensions of the matrices be even numbers. We will therefore, in the following,
restrain our study to nx and nt being even numbers. So far, it is interesting to
note that the Schur decomposition being more stable for higher order matrices, it
perfectly fits finite differences problems.
Complete parametric solutions of the Sylvester equation (24) is given in [2], [3].
As for the determination of the solution of the Sylvester equation, it is a major
topic in control theory, and has been the subject of numerous works (see [1], [6],
[8], [9], [10], [11], [12]).
In [1], the method is based on the reduction of the observable pair (A, C) to an
observer-Hessenberg pair (H, D), H being a block upper Hessenberg matrix. The
reduction to the observer-Hessenberg form (H, D) is achieved by means of the staircase algorithm (see [4]).
In [9], in the specific case of B being a companion form matrix, the authors propose
a very neat general complete parametric solution, which is expressed in terms of
the controllability of the matrix pair (A, B), a symmetric matrix operator, and a
parametric matrix in the Hankel form.
We recall that a companion form, or Frobenius matrix is one of the following kind:


0 ... ... ... 0
−b0
 1 0 ... ... 0
−b1 




..
..


0 ... .
.
B= 0 1
(28)

 .

.
.
.
.
..
..
..
..
 .. 0

0 0 . . . 1 0 −bp−1
These results can be generalized through matrix block decomposition to a block
companion form matrix, which happens to be the case of our matrix M2 in the
THEORETICAL OPTIMIZATION OF FINITE DIFFERENCE SCHEMES
specific case of nx and nt being even numbers:

1
M2B
0
... ...
2

M2B
0 ...
 0

..
..

.
.
M2 =  0
0
 .
..
..
 .
.
.
 .
0
0
0
... 0
0
0
..
.
0
k
M2B
291









(29)
p
the M2B , 1 ≤ p ≤ k being companion form matrices.
Another method is presented in [14], where the determination of the minimumnorm solution of a Sylvester equation is specifically developed.
The accuracy and computational stability of the solutions is examined in [5].
2.2.2. Existence condition of the solution. Equation (24) has a unique solution if
and only if A and B have no common eigenvalues, which is presently the case.
(19) has thus a unique solution, which accounts for the consistency of the given
problem.
3. Optimization: determination of the optimal value of a specific parameter for a given scheme. Advect a sinusoidal signal
u = Cos [
2π
(x − c t) ]
λ
(30)
through the Lax scheme, where:
λ = nλ dx
(31)
nλ denotes the number of cells per wavelength.
Let nλ remain unknown.
Equation (23) can thus be normalized as:
M1 E + E M2 = F
(32)
where

h cf l
 M1 =
c M1
l
(33)
M2 = h cf
c M2

h cf l
F =
c F
We deliberately choose a small value for the number of steps: nt = nx = 20, starting
from the point that if the error is minimized for a small value of this number, it will
be the same as this number increases.
Figure 1 displays the L2 norm of the isovalues of the error as a function of nλ (maximums are in white, minimums in black; larger values are shown lighter). It clearly
appears that, if one chooses a value of nλ in a darker zone, the error is minimized.
Calculations have been done with the LyapunovSolve package of Mathematica,
which presents the advantage of dealing with symbolic parameters, as it is the
case for our a priori unknown coefficient nλ .
292
CLAIRE DAVID, PIERRE SAGAUT
h
1
0.8
0.6
0.4
0.2
0
5
15
10
20
Figure 1. Isovalues of the L2 norm of the error as a function of
the number of cells per wavelength nλ
The L2 norm of the error, for two significative values of the number of cells per
wavelength np (one corresponding to a minimum of the error, and one corresponding
to a maximum of the error), is displayed in Figure 2:
0.175
0.15
0.125
0.1
0.075
0.05
0.025
20
40
60
80
t
€€€€€€
100 Τ
Figure 2. L2 norm of the error for different values of the number
of cells per wavelength nλ . Case 1 (in black): np = 6.75. Case 2
(in grey): np = 20.
The above results ensure the faster convergence of the scheme.
4. Conclusion. The matrix form of the linear finite difference problem enables us
to minimize the symbolic expression of the L2 norm of the error as a function of one
of the scheme coefficient (as the number of cells per wavelength, presented as an
example). Thus, the optimal value of a given coefficient, i. e. one that minimizes
the error, can be easily determined.
REFERENCES
[1] P. Van Dooren, Reduced order observers: A new algorithm and proof, Systems Control Lett.,
4 (1984), 243–251.
[2] A. Berman and R. J. Plemmons, “Nonnegative Matrices in the Mathematical Sciences,” SIAM,
Philadelphia, 1994.
[3] H. R. Gail, S. L. Hantler and B. A. Taylor, Spectral Analysis of M/G/1 and G/M/1 type
Markov chains, Adv. Appl. Probab., 28 (1996), 114–165.
THEORETICAL OPTIMIZATION OF FINITE DIFFERENCE SCHEMES
293
[4] D. L. Boley, “Computing the Controllability algorithm / Observability Decomposition of a
Linear Time-Invariant Dynamic System, A Numerical Approach,” PhD. thesis, Report STANCS-81-860, Dept. Comp. i, Sci., Stanford University, 1981.
[5] A. S. Deif, N. P. Seif and S. A. Hussein, Sylvester’s equation: accuracy and computational
stability, Journal of Computational and Applied Mathematics, 61 (1995), 1–11.
[6] J. Z. Hearon, Nonsingular solutions of T A − BT = C, Linear Algebra and its applications, 16
(1977), 57–63.
[7] C. H. Huo, Efficient methods for solving a nonsymmetric algebraic equation arising in stochastic fluid models, Journal of Computational and Applied Mathematics, (2004), 1–21.
[8] C. C. Tsui, A complete analytical solution to the equation T A − F T = LC and its applications,
IEEE Trans. Automat. Control AC, 32 (1987), 742–744.
[9] B. Zhou and G. R. Duan, An explicit solution to the matrix equation AX − XF = BY , Linear
Algebra and its applications, 402 (2005), 345–366.
[10] G. R. Duan, Solution to matrix equation AV + BW = EV F and eigenstructure assignment
for descriptor systems, Automatica, 28 (1992), 639–643.
[11] G. R. Duan, On the solution to Sylvester matrix equation AV + BW = EV F and eigenstructure assignment for descriptor systems, IEEE Trans. Automat. Control, AC-41 (1996)(4),
276–280.
[12] P. Kirrinnis, Fast algorithms for the Sylvester equation AX − XB T = C, Theoretical Computer Science, 259 (2000), 623–638.
[13] M. Konstantinov, V. Mehrmann and P. Petkov, On properties of Sylvester and Lyapunov
operators, Linear Algebra and its applications, 312 (2000), 35–71.
[14] A. Varga, TA numerically reliable approach to robust pole assignment for descriptor systems,
Future Generation Computer Systems, 19(7) (2003), 1221–1230.
[15] G. B. Witham, “Linear and Nonlinear Wave,” Wiley-Interscience, 1974.
[16] S. Wolfram, “The Mathematica book,” Cambridge University Press, 1999.
Received September 2006; revised March 2007.
E-mail address: [email protected]