The next notion is the order (Konsistenzordnung) of a method:
Definition 2.2 Order
The integration method is of order p if
kτ (h)k = O(hp ) .
After several steps, local errors are accumulated, and we need another concept to describe this process:
Definition 2.3 Global error
The global error of an integration method is
en (X) = y(X) − yn
where X = xn fixed, n variable.
The method is convergent if
lim ken (X)k = 0 .
n→∞
Convergence thus means:
Für feiner werdendes Gitter strebt der globale Fehler
The following theorem can be shown:
Theorem 2.1 Convergence of one-step methods
Assume f Lipschitz-continuous with constant L from (1.11). Furthermore,
let the one-step method be of order p, i.e.,
kτ (h)k = O(hp ) .
Then, the global error satisfies
ken (X)k ≤
eL|X−x0|
p
chmax
L
where hmax = max hi .
i
16
−1
Discussion
1) Ordnung globaler Diskretisierungsfehler = Ordnung lokaler Diskretisierungsfehler. Man sagt bei ESV auch kurz: Konsistenz =⇒ Konvergenz
2) Variable Schrittweiten sind zugelassen. Der Verstärkungsfaktor
1 L(x−x0 )
− 1) ist jedoch von den Schrittweiten unabhängig.
L (e
4) Manchmal findet man die Schreibweise e(h, x) statt en (x) für den globalen Diskretisierungsfehler, um die Abhängigkeit von der Schrittweite
zu betonen. Oft besitzt e(h, x) eine asymptotische Entwicklung in h.
Einschrittverfahren sind demnach einfach zu analysieren: Man ermittelt
den lokalen Fehler (durch Taylorreihenentwicklung) und bekommt aus der
Konsistenz bereits globale Konvergenz.
17
2.2
Runge–Kutta Methods
Runge–Kutta methods are the most popular class of one-step methods. In
each step, they evaluate the ”slope” of y in intermediate values, the so-called
stages.
Example 2.2:
Method of Heun
K2
y
1
K1
y0
x
x
0
1
Figure 4: Idea of Heun’s method
Discretization scheme
with stages
y1 = y0 + 12 h(K1 + K2 )
K1 = f (x0 , y0 ),
K2 = f (x1 , y0 + h · K1 ).
Increment function Φ
y1 = y0 + h · Φ(x0 , y0 , h) ,
Φ(x, y, h) = 12 [f (x, y) + f (x + h, y + hf (x, y))]
= 12 (K1 + K2 )
General notation for one step of a Runge-Kutta method
(Runge–Kutta–Verfahren):
y1 = y0 + h(b1 K1 + b2 K2 + . . . + bsKs )
(2.6)
with stage values
Ki = f
x0 + ci h, y0 + h
i−1
X
j=1
18
aij Kj
!
,
i = 1, . . . , s.
(2.7)
The method is fully specified by the number of stages s, the coefficients
bi , ci for i = 1, . . . , s and aij for i = 1, . . . , s and j ≤ i − 1.
Example 2.3: Heun’s method revisited
Number of stages s = 2, coefficients
1
1
b1 = , b2 = , c1 = 0, c2 = 1, a21 = 1.
2
2
Example 2.4:
Modified Euler method
mit
y1 = y0 + h · K2
K1 = f (x0 , y0 ),
K2 = f x0 + h2 , y0 +
Thus s = 2 and b1 = 0, b2 = 1, c1 = 0, c2 = 12 , a21 = 12 .
h
2
· K1 .
Butcher–Tableau
c1 0
c2 a21
.. ..
. .
cs as1
b1
...
...
0
...
i.e.,
. . . ass−1 0
b2 . . . bs
c
A
bT
Examples
0
1 1
0
Heun
1
2
1
2
1
2
1
2
mod. Euler
0 1
0
1
2
1
2
1
2
“Classical Runge–Kutta method”
(Runge 1895, Kutta 1901)
0 12
1 0 0 1
1
6
2
6
2
6
1
6
19
Determining the Coefficients
Criteria for the coefficients:
Wie kommt man auf die Koeffizienten? Die Koeffizienten sind zunächst
freie Parameter des Verfahrens. Sie werden so bestimmt, dass das Verfahren “möglichst gut” ist, d.h. eine möglichst hohe Ordnung erreicht.
Wesentliches Hilfsmittel bei der Bestimmung sind Taylorreihenentwicklungen der exakten Lösung und der numerischen Näherung.
Consider the case s = 2 with
y1 = y0 + h(b1K1 + b2K2) ,
K1 = f (x0 + c1 h, y0 ) ,
K2 = f (x0 + c2 h, y0 + ha21 K1).
By Taylor expansion
K1 = f (x0, y0) + fx (x0, y0 ) · c1 · h + O(h2 )
K2 = f (x0, y0) + fx (x0, y0 ) · c2 · h + fy (x0, y0)ha21K1 + O(h2 )
= f (x0, y0) + fx (x0, y0 ) · c2 · h + fy f (x0, y0)ha21 + O(h2 )
=⇒ y1 = y0 +h(b1 + b2)f (x0, y0)
+h2 (b1c1 + b2 c2 )fx (x0, y0 )
+h2 b2 a21fy f (x0, y0) + O(h3 )
Compare with exact solution
1
y(x0 + h) = y0 + hf (x0, y0 ) + h2 (fx + fy f )(x0, y0) + O(h3 )
2
Order conditions for p = 2
1
1
b1 + b2 = 1, b1 c1 + b2c2 = , b2a21 = .
(2.8)
2
2
Particular solutions of (2.8) are Heun’s method and modified Euler.
Note: it is impossible to achieve p = 3 with s = 2 stages!
For higher order, we thus need more stages, and the procedure for determining the coefficients becomes tedious. Remedy: study only ODEs in
20
autonomous form. Compare y ′ = f (x, y) and
!
y
Y ′ = F (Y ), Y :=
, F (Y ) :=
x
f (x, y)
1
!
.
A numerical method should be invariant under this transformation, i.e.,
should give the same results. This requirement leads to the conditions
ci =
i−1
X
aij ,
(2.9)
j=1
since in the last row of F we have x0 + ci h = x0 + h
i−1
P
aij · 1.
j=1
Summarizing, it is sufficient to determine the coefficients aij , bi in the autonomous case, and then the nodes ci follow from (2.9).
Theorem 2.2 Order conditions for p = 4
Let f ∈ C p(D) and denote by A, b, c the method coefficients. The RungeKutta method is of order p if (2.9) holds and
p=1:
s
X
bi = 1
(1)
bi ci = 1/2
(2)
i=1
p=2:
p=3:
p=4:
s
X
i=1
s
X
bic2i
i=1
s
X
= 1/3,
bic3i = 1/4,
i=1
s
X
i,j=1
bi aij c2j = 1/12,
s
X
i,j=1
s
X
and (1)
bi aij cj = 1/6 (3) and (1) − (2)
bi ci aij cj = 1/8,
i,j=1
s
X
biaij ajk ck = 1/24 and (1) − (3)
i,j,k=1
21
Constructing a Method
We try now to construct a method of order p = 4 with s = 4 stages.
P
Assuming ci = aij , we have to determine 10 unknown coefficients
b1, b2, b3, b4, a21, a31, a32, a41, a42, a43
such that Theorem 2.2 holds.
P
The condition bi = 1 looks like a quadrature rule, and thus we interprete
b1, . . . , b4 as weights of a quadrature formula. Also, the ci can be viewed as
nodes in the interval [0, 1]. From
X
X
X
2
bi ci = 1/2,
bi ci = 1/3,
bi c3i = 1/4,
we get that this quadrature formula is exact for all polynomials in P3.
We select Newton’s 3/8-rule where
c = (0, 1/3, 2/3, 1)T , b = (1/8, 3/8, 3/8, 1/8)T .
P
P
P
and compute from
biaij cj = 1/6,
bici aij cj = 1/8 and
bi aij ajk ck =
1/24 the equations
a32 = 1,
a42 c2 + a43 c3 = 1/3,
b4 a43a32 c2 = 1/24.
Consequently, a43 = 1 and a42 = −1. From the definition of ci we also get
a21 = 1/3, a31 = −1/3, a41 = 1.
The result is Kutta’s 3/8-rule
0
1
3
2
3
1
3
− 13 1
1 1 −1 1
1
8
3
8
22
3
8
1
8
Table on the relation between number of stages and maximum order:
Number of stages s 1 2 3 4 5 6 7 8
max. order p
1 2 3 4 4 5 6 6
Error Estimation and Stepsize Control
So far, the grid
x0 < x1 < . . . < xn−1 < xn
has been kept fixed, usually with constant stepsize hi = xi+1 − xi.
Disadvantages:
Goals for an adaptive algorithm:
(i) eine vorgegebene Genauigkeit der numerischen Lösung erreicht wird,
(ii) der dazu notwendige Rechenaufwand möglichst minimal wird.
23
Gesamtfehler
p
e (x)< O(h )
n
Rundungsfehler
h
Figure 5: Global error and influence of round-off
y
x
große Schrittweiten
kleine Schrittweiten
Figure 6: Grid adapted to solution behavior
Objective of adaptive stepsize control:
Choose hi such that the next discrete solution point yi+1 satisfies a given
tolerance criterion.
In practice, this means that we only control the local error and not the
global error! I.e., we compute hi such that
ku(xi+1)|u(xi )=yi − yi+1k ≤ ǫ
(2.10)
with ǫ given.
Since the exact solution u(x) for the initial value u(xi) = yi in (2.10) is
unknown, it is replaced by an approximation ŷi+1. For this reason, we
speak of error estimation (Fehlerschätzung).
24
In case of Runge–Kutta methods, the following approach is widespread:
Combine a method of order p (for yi+1) with a method of order p + 1 (for
ŷi+1).
We call ŷi+1 the embedded method (eingebettetes Verfahren).
Butcher–table
c
A
bT
b̂T
−→ y1 = y0 + h
−→ ŷ1 = y0 + h
s
P
i=1
ŝ
P
bi K i
b̂iKi
i=1
This idea goes back to Fehlberg, and often these schemes with embedded
error estimator are called Runge–Kutta–Fehlberg methods. Below we will
study the computation of ŷi+1 with order p + 1 in more detail. For the
moment, we assume that ŷi+1 is available and derive next a general adaptive
time-stepping algorithm.
Given yi+1 and ŷi+1, we compute the optimal stepsize hopt. in the following
way: It holds
.
kŷi+1 − yi+1k = c · hp+1
(yi+1 order p).
The desired stepsize hopt. satisfies
c · hp+1
= ǫ.
opt.
By division we obtain
and thus
hp+1
= hp+1
opt.
ǫ
kŷi+1 − yi+1k
r
ǫ
.
kŷi+1 − yi+1k
hopt. = h p+1
25
(2.11)
The relation (2.11) between actual and desired stepsize is the basis of all
variable stepsize algorithms. Note that it only makes sense as long as h is
”small enough”.
In practice, additional heuristics is required to make the control algorithm
efficient and robust. In the algorithm below, we denote the method by Φ
(order p) and by Φ̂ (order p + 1).
Algorithm 2.1 Adaptive one-step method
Start: x0, y0 , h0 , ǫ, xend given;
i = 0;
while xi < xend
x = xi + hi ;
y = yi + hi Φ(xi, yi, hi );
ŷ = yi + hi Φ̂(xi, yi, hi );
e = kŷ − yk;
r
ǫ
% safety factor ρ
hopt. = h p+1 · ρ;
e
hopt. = min(α · hi , max(βhi , hopt. )); % bounds α, β
if e ≤ ǫ
xi+1 = x;
yi+1 = y;
hi+1 = min(hopt., xend − xi+1);
i = i + 1;
else
hi = hopt. ;
% repeat step
end
end
26
Remarks:
1) The constants ρ, α, β depend on the method. For higher order methods,
the bounds for the change in h may be less restrictive.
2) A good norm for computing the error is
v
u X
2
u1 n
ŷ
−
y
j
j
ERR = t
, W T = |ŷ|
n j=1 AT OL + RT OL · W Tj
with absolute and relative tolerances AT OL, RT OL. Instead of e ≤ ǫ
we then compare the error with 1, ERR ≤ 1.
3) In the algorithm, we proceed with yi+1 as new solution. However, ŷi+1
is more accurate, and thus it has become common sense to proceed
with ŷi+1. In this case, we actually do not compute an error estimator
but an error indicator.
Embedded Methods
As example, we construct an embedded method for the classical RK-method
with p = 4. The Butcher table reads then
0
1
2
1
2
1
2
0 12
1 0 0
1
1
6
2
6
2
6
1
6
b̂1 b̂2 b̂3 . . .
b̂ŝ
First approach: We assume ŝ = 4 and try to determine the 4 coefficients
b̂1, . . . , b̂4 such that
ŷ1 = y0 + h(b̂1 K1 + . . . + b̂4K4)
27
is of order 3.
The order conditions from Theorem 2.2 yield
b̂1 + b̂2 + b̂3 + b̂4 = 1,
b̂2/4 + b̂3 /4 + b̂4 = 1/3,
b̂2 /2 + b̂3/2 + b̂4 = 1/2,
b̂3/4 + b̂4/2 = 1/6,
with aij and ci inserted from above. There is only one solution b̂ = b, which
means that with ŝ = 4, it is impossible to construct an embedded method!
Second approach: The choice ŝ = 5 gives more degrees of freedom with
respect to the coefficients. However, we don’t want to spend extra computational effort for evaluating another stage K5.
Fehlberg’s trick: Determine c5 , a51, . . . , a54 such that
K5(old step) = K1(new step).
(2.12)
This scheme is called FSAL (FirstSameAsLast). For our method, this means
X
X
K5(o.s.) = f (x0+c5 h, y0+h
a5j Kj ) = f (x0+h, y0 +h
biKi ) = K1(n.s.).
Hence c5 = 1 and a5j = bj for j = 1 : 4. An analysis of the order conditions
for p = 3 now shows:
There is a family of weights b̂ leading to order 3. Example:
0
1
2
1
2
1
2
0 21
1 0 0 1
1 16 26 26
1
6
1
6
2
6
2
6
2
6
2
6
1
6
1
6
0
0
1
6
The evaluation of ŷ − y reduces to the simple formula
ŷ − y = (K1(n.s.) − K4(o.s.))/6 ,
which means that ŷ is never really computed.
28
Runge-Kutta Methods in MATLAB
ode45
This code is based on an explicit Runge-Kutta (4,5) formula, the DormandPrince pair. In general, ode45 is the best function to apply as a first try for
most problems.
This method is sometimes also called DOPRI5(4). It has s = 6 and ŝ = 7
stages and uses FSAL for the extra stage. Coefficients:
0
1
5
3
10
4
5
8
9
1
1
1
5
3
40
44
45
19372
6561
9017
3168
35
384
35
384
5179
57600
9
40
−56
15
−25360
2187
−355
33
0
0
0
32
9
64448
6561
46732
5247
500
1113
500
1113
7571
16695
−212
729
49
176
125
192
125
192
393
640
−5103
18656
−2187
6784
−2187
6784
−92097
339200
11
84
11
84
187
2100
0
1
40
ode23
This is an implementation of an explicit Runge-Kutta (2,3) pair of Bogacki
and Shampine. It may be more efficient than ode45 at crude tolerances and
in the presence of moderate stiffness.
29
Chapter
3
Multistep Methods
3
Multistep methods use data from previous steps to compute a new discrete
.
approximation. More precisely, a k–step method computes yi+1 = y(xi+1)
from
(xi−k+1, yi−k+1), (xi−k+2, yi−k+2), . . . , (xi−1, yi−1), (xi, yi ).
The data stems from old steps or from a starting procedure (Anlaufrechnung).
3.1
Adams and BDF Methods
In this section, we introduce the most popular classes of multistep methods.
Adams methods. This method class is derived by integrating y ′ = f (x, y)
from xi to xi+1,
xi+1
Z
y(xi+1) = y(xi) +
f (x, y(x)) dx .
(3.1)
xi
The integrand in (3.1) depends on the unknown solution y, and therefore
we replace it by an interpolation polynomial p.
We distinguish between two cases:
30
a) The polynomial p interpolates the data fi−k+j = f (xi−k+j , yi−k+j ) , j =
1, . . . , k
f
fi−k+1
f i−1
x i−k+1
x i−1
11111 p(x)
00000
00000
11111
fi
00000
11111
00000
11111
Integration
00000
11111
00000
11111
00000
11111
00000
11111
xi
x
x
y
i+1
i+1
Figure 7: Idea of Adams-Bashforth methods
The Lagrange formula gives
p(x) =
k
X
fi−k+j · li,j (x) with li,j (x) =
j=1
k
Y
ν=1
ν6=j
x − xi−k+ν
.
xi−k+j − xi−k+ν
Inserting p(x) in the integral in (3.1), we obtain
yi+1 = yi + hi
k
X
βi,j fi−k+j ,
βi,j
j=1
1
:=
hi
Zxi+1
li,j (x) dx.
xi
In the equidistant case xi = x0 +i·h, the coefficients βi,j =: βj−1 , j = 1, . . . , k
do not depend on the index i,
Zxi+1
Z1 Y
k
li,j (x) dx = h
0
xi
= h
ν=1
ν6=j
Z1 Y
k
0
|
ν=1
ν6=j
x0 + ih + sh − (x0 + (i − k + ν)h)
ds
x0 + (i − k + j)h − (x0 + (i − k + ν)h)
k−ν +s
ds.
j−ν
{z
}
=:βj−1
Summarizing, the method reads
yi+1 = yi + h(β0 fi−k+1 + β1 fi−k+2 + . . . + βk−1fi )
31
(3.2)
E.g.,
k=1
k=2
k=3
yi+1 = yi + hfi,
yi+1 = yi + h − 12 fi−1 + 32 fi ,
5
yi+1 = yi + h 12
fi−2 − 16
12 fi−1 +
23
12 fi
.
This class of multistep methods is called Adams–Bashforth methods.
b) The alternative approach requires that p interpolates fi−k+j for j =
1, . . . , k, k+1. Since p depends now on yi+1, this leads to an implicit method.
f
p(x)
fi
f i+1
xi
x i+1
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
x i−k+1
x
Figure 8: Idea of Adams-Moulton methods
An analogous construction of the interpolation polynomial yields
p(x) =
k+1
X
fi−k+j ·
∗
li,j
(x),
∗
li,j
(x)
j≡1
=
k+1
Y
ν=1
ν6=j
x − xi−k+ν
.
xi−k+j − xi−k+ν
In the equidistant case, this simplifies to
∗
βj−1
=
Z1 k+1
Y
0
ν=1
ν6=j
k−ν +s
ds,
j−ν
j = 1, . . . , k + 1
and the method reads
∗
yi+1 = yi + h(β0∗ fi−k+1 + . . . + βk−1
fi + βk∗fi+1).
32
(3.3)
E.g.,
yi+1 = yi + h 12 fi + 12 fi+1
trapezoidal rule,
1
8
5
yi+1 = yi + h − 12
fi−1 + 12
fi + 12
fi+1 .
k=1
k=2
This class of implicit multistep methods is called Adams–Moulton–methods.
Note: k = 0 makes sense here and results in
yi+1 = yi + hfi+1
implicit Euler.
Adams methods are usually implemented as predictor–corrector method
(Prädiktor–Korrektor–Schema):
(0)
(i) yi+1 = yi + h(β0 fi−k+1 + . . . + βk−1fi)
(start with explicit Adams–Bashforth)
(l+1)
(l)
(ii) yi+1 = Ψ(yi+1), l = 0, 1, . . .
Correct with fixed point iteration based on implicit Adams–Moulton
where
∗
Ψ(y) := hβk∗f (xi+1, y) + yi + h(β0∗fi−k+1 + . . . + βk−1
fi ).
The corrector iteration converges for h small to a fixed point y = Ψ(y) due
to
kΨ′ (y)k = h|βk∗ | kfy (xi+1, y)k ≤ M < 1 .
33
BDF methods. The BDF (Backward Difference Formulas) are based on
differentation instead of integration, as in the case of Adams methods.
Determine the polynomial q(x) that interpolates
(xi−k+1, yi−k+1), . . . , (xi, yi ), (xi+1, yi+1)
and satisfies
q ′ (xi+1) = f (xi+1, q(xi+1)) = f (xi+1, yi+1).
From the Lagrange formula, we get
q(x) =
k+1
X
∗
yi−k+j li,j
(x)
′
⇒
q (xi+1) =
j=1
k+1
X
∗,′
yi−k+j li,j
(xi+1).
j=1
In the equidistant case,

1d
d ∗
li,j (x)|x=xi+1 = 
dx
h ds
|
k+1
Y
ν=1
ν6=j

k − ν + s
 |s=1
j−ν
{z
}
=:αj−1
The coefficients α0 , . . . , αk are independent of step i, and hence the k-step
BDF method can be written as
α0 yi−k+1 + . . . + αk−1 yi + αk yi+1 = hfi+1.
(3.4)
BDF are implicit and important for solving stiff ODEs.
Examples:
k=1
yi+1 − yi = hf (xi+1, yi+1)
k=2
3
1
2 yi+1 − 2yi + 2 yi−1 = hfi+1
11
3
1
6 yi+1 − 3yi + 2 yi−1 − 3 yi−2
k=3
34
(implicit Euler)
(BDF-2)
= hfi+1
(BDF-3)
General notation for multistep methods. Instead of
α0 yi−k+1 + . . . + αk yi+1 = h(β0 fi−k+1 + . . . + βk−1fi + βk fi+1)
we write a linear k-step method in the form
α0 yi + α1 yi+1 + . . . + αk yi+k = h(β0 fi + . . . + βk fi+k )
or in short
k
X
αl yi+l = h
l=0
βl fi+l
(3.5)
l=0
with method coefficients αl and βl .
3.2
k
X
Order and Convergence
The local error of a multistep method is defined as defect (Defekt), compare
Def. 2.1.
Definition 3.1 Local error of a multistep method
Let y(x) be the solution of the IVP y ′ = f (x, y), y(x0) = y0. The local error
of the multistep method (3.5) with constant stepsize h is defined as
!
k
k
X
X
1
τ (h) :=
αl y(xi+l ) − h
βl f (xi+l, y(xi+l )) .
h
l=0
l=0
The error τ (h) follows thus from inserting the exact solution into (3.5).
We say that the multistep method has order p if
kτ (h)k = O(hp ) .
To simplify the notation somewhat, we set αk := 1 in the following.
Why is this possible?
35
Theorem 3.1 Order conditions
P
P
The multistep method
αl yi+l = h βl fi+l has order p if
k
X
l=0
αl = 0 and
k
X
k
X
q
αl l = q
l=0
βl lq−1
for q = 1, . . . , p.
l=0
Example 3.1: Consider the method
yi+2 + 4yi+1 − 5yi = h(4fi+1 + 2fi ).
Thus α2 = 1, α1 = 4, α0 = −5, β1 = 4, β0 = 2
X
X
p=1:
αl = 0,
αl · l − βl = (−5 · 0 − 2) + (4 · 1 − 4) + (1 · 2) = 0
X
p=2:
αl · l2 − 2βl l = (−5 · 0 − 2 · 0) + (4 · 1 − 8) + (1 · 4) = 0
X
p=3:
αl · l3 − 3βl l2 = (−5 · 0 − 2 · 0) + (4 · 1 − 12) + (1 · 8) = 0
Overall, we have a method of order p = 3.
Order of method classes from above:
k–step Adams–Bashforth Adams–Moulton BDF
p
k
k+1
k
What is the maximum order that we can achieve?
36
(3.6)