A Global Convergence Theorem for a Class of
Parallel Continuous Explicit Runge-Kutta Methods
and Vanishing Lag Delay Differential Equations
C.T.H. Baker & C.A.H.Paul
Numerical Analysis Report No. 229 (revised)
May 1994
University of Manchester/UMIST
Manchester Centre for Computational Mathematics
Numerical Analysis Reports
DEPARTMENT OF MATHEMATICS
Reports available from:
Department of Mathematics
University of Manchester
Manchester M13 9PL
England
And by anonymous ftp from:
vtx.ma.man.ac.uk
(130.88.16.2)
in pub/narep
A Global Convergence Theorem for a Class of Parallel Continuous
Explicit Runge-Kutta Methods and Vanishing Lag Delay
Differential Equations
Christopher T.H. Baker∗ and Christopher A.H. Paul∗
May 23, 1994
Abstract
Iterated continuous extensions (ICEs) are continuous explicit Runge-Kutta methods developed for
the numerical solution of evolutionary problems in ordinary and delay differential equations (DDEs).
ICEs have a particular rôle in the explicit solution of DDEs with vanishing lags. They may be regarded
as parallel continuous explicit Runge-Kutta (PCERK) methods, as they allow advantage to be taken
of parallel architectures. ICEs can also be related to a collocation method.
The purpose of this paper is to provide a theorem giving the global order of convergence for
variable-step implementations of ICEs applied to state-dependent DDEs with and without vanishing
lags. Implications of the theory for the implementation of this class of methods are discussed and
demonstrated. The results establish that our approach allows the construction of PCERK methods
whose order of convergence is restricted only by the continuity of the solution.
Key words. Parallel continuous explicit Runge-Kutta methods, iterated continuous extensions,
delay differential equations, vanishing lag.
AMS subject classifications. 65L06 65L70 65Q05 65Y05
1
Introduction
The outline of this paper is as follows: We state the equations defining a class of methods [1] for solving
delay differential equations (DDEs). Our main aim, achieved in Section 3, is to establish the global
convergence and order of convergence for our class of methods. Section 2 provides preliminary results to
this end. In Section 4 a variant method is introduced, and we show how the analysis of Section 3 can be
modified to analyse this method. Practical considerations are addressed and numerical results are provided
in subsequent sections.
The presentation covers variable-step, varying smoothness and the possibility of a vanishing statedependent lag. The paper provides a systematic, self-contained and rigorous analysis valid under these
conditions. We assume familiarity with the concept of a continuous explicit Runge-Kutta (CERK) method
(see [5] p.176 et seq.) for initial-value problems in ordinary differential equations (ODEs),
y (t) = f (t, y(t)) for t ≥ t0
and y(t0 ) = y0 .
(1)
The CERK methods outlined in this paper are suitable for solving DDEs of the form
u (t) = F (t, u(t), u(γ(t, u(t)))) for t0 ≤ t ≤ tN ≤ T,
∗ Mathematics
Department, Victoria University of Manchester, Manchester M13 9PL, England.
1
(2)
with γ(t, u(t)) ≤ t and u(t) = Ψ(t) for t ≤ t0 , where Ψ(t) is a prescribed initial function. In particular,
these methods are suited to DDEs which have a singular or vanishing lag, for which γ(t, u(t)) → t∗ as
t → t∗ . Other work relating to the explicit numerical solution of vanishing lag DDEs is referenced by
Neves [8]. It is a complicating feature of (2) that the solution u(t) is liable (depending on the initial
function Ψ(t)) to suffer jump discontinuities in its derivatives at an ordered set of points {t∗j }; however, on
successive intervals [t∗j , t∗j+1 ] the solution is smooth. A recent paper [11] describes a strategy for calculating
the points {t∗j } for a system of DDEs; see also [4]. A CERK method can be associated with the tableau
c
A
θ
T
(3)
,
b (θ)
where A is strictly lower triangular, and with the points
{tn , {tni }} with tn+1 = tn + Hn
and
tni = tn + ci Hn
(Hn > 0).
For our adaptation of a ν-stage CERK method to the DDE (2), we shall employ the parameters
c = [ci ]νi=1 with c1 = 0, ci ∈ [0, 1], A = [aij ]νi,j=1 , b(θ) = [bi (θ)]νi=1 (a vector of polynomials in θ) and
B = [bj (ci )]νi,j=1 . (The matrix B corresponds to a collocation method with abscissae {ci }.) We use
these parameters in an iterative method for computing an approximate solution u
(t), for which a result
ρ
(t)| = O(H ) holds. For compactness of notation we call (3) a CERK triple
of the form supt<T |u(t) − u
{c, A, b(θ)}. Central to the idea of a CERK method is the following concept:
Definition 1.1 A continuous extension v(t), based on the polynomials {bi (θ)} and the values {
v (tn ), {Vni
}}
(at the points {tn , {tni }}), is a function defined on the interval [tn , tn + Hn ] by
v(tn + θHn ) = v(tn ) + Hn
ν
bi (θ)Vni
for 0 ≤ θ ≤ 1.
i=1
This concept is more general than might appear at first, and is often implicit rather than explicit in
the discussion of CERK methods. In the CERK solution of an ODE (1), the values {
v(tn )} are “full
step values” approximating {y(tn )}, and the values {Vni
} are “internal-stage derivatives” approximating
{y (tn + ci Hn )}. They have a similar interpretation in CERK methods for DDEs. The basic concept of
the continuous order of a CERK formula is associated with its application to an ODE:
Definition 1.2 The CERK triple {c, A, b(θ)} with step H > 0 has continuous order p∗ := p∗ (c, A, b(θ))
y (t) −
when used to solve the initial-value problem (1), if the approximation y(t) satisfies supt∈[t0 ,t0 +H] |
p∗ +1
y(t)| = O(H
) as H 0 whenever y(t) and f (t, y) have sufficiently smooth derivatives.
The continuous order is governed by conditions on the CERK triple (see [1], p.374). We shall assume that
the derivative function F (t, u1 , u2 ) and the solution u(t) of (2) are such that the following holds:
Hypothesis 1.3 There exists an integer p such that the CERK method applied to the initial-value problem
y (t) = f (t, y(t)) for t ∈ [tn , tn+1 ],
with
y(tn ) = u(tn )
and
f (t, y(t)) = F (t, y(t), u(γ(t, u(t)))),
yields an approximation y(t) that satisfies supt∈[tn ,tn+1 ] |
y (t) − y(t)| = O(Hnp+1 ) as Hn 0 uniformly for
each n = 0, 1, . . . , N − 1.
The value of p is specific to the problem (2), with the given choice of Ψ(t). If u(t), γ(t, u) and F (t, u1 , u2 )
are all sufficiently smooth then p ≥ p∗ (in general p = p∗ ). Henceforth we assume the following:
2
Hypothesis 1.4 The following additional assumptions hold:
1. F (t, u1 , u2 ) has continuous partial derivatives upto order p∗ ,
2. u(r−1) (t) ∈ Lip[tn , tn+1 ] for all n = 0, 1, . . . , N − 1, and
3. u(s−1) (t) ∈ Lip[tn , tn+2 ] for all n = 0, 1, . . . , N − 2.
The iterative methods discussed here are defined by iterated continuous extensions (‘ICEs’), which were
introduced into the open literature in [1]. The practical motivation for such methods was illustrated by
Paul & Baker in [10]. ICEs are defined as follows:
Definition 1.5 In the numerical solution of (2), employing the CERK triple {c, A, b(θ)}, an initial ap[0]
(t) for t > tn defines: (i) starting values {
u[0] (tni )}νi=1 satisfying
proximation zn (t) to u
[0]
u
(tni ) =
u
(tn ) + Hn
i−1
j=1
[0]
aij F (tnj , u
[0] (tnj ), zn[0] (
γnj )),
(4)
and (ii) for m ≥ 1, an m-th ICE u
[m] (tn + θHn ), on setting k = m in the recurrence:
u
[k] (tn + θHn ) =
where zn[l] (t) =
u
(tn ) + Hn


(t)
 u
[0]
zn (t)

 [l]
u
(t)
ν
i=1
[k−1]
bi (θ)F (tni , u
[k−1] (tni ), zn[k−1] (
γni
)),
(5)
t ≤ tn ,
t > tn and l = 0,
t > tn and l ≥ 1.
if
if
if
[k]
[k]
Here u
(t) = u
[m] (t) if t0 ≤ t ≤ tn , u
(t) = Ψ(t) if t < t0 and γ
ni = γ(tni , zn (tni )).
According to this definition, u
[k] (t) (for k ≥ 1 and t ∈ [tn , tn+1 ]) is a continuous extension based on the
[k−1] [k−1]
[k]
values {
u(tn ), {F (tni , u
[k−1] (tni ), zn
(
γni ))}}. Note also that zn (t) ≡ u
[k] (t) for t ≥ tn and k ≥ 1.
[0]
For each interval [tn , tn+1 ], given an initial approximation zn (t), the m-th ICE is uniquely defined by
the choice of CERK triple and “step” Hn . Two possible initial approximations are: Case 1 u
(tn ), and Case 2
[m−1] [m−1]
(tn−1 ) + Hn−1 νi=1 bi (θ∗ )F (tn−1,i , u
[m−1] (tn−1,i ), zn−1 (
γn−1,i ))
u
(tn−1 + θ∗ Hn−1 ) for θ∗ ≥ 1 – that is, u
extrapolated from the preceding interval [tn−1 , tn ]. In the latter case, when n = 0, there is no “preceding
[0]
interval”; we then recommend the choice z0 (t) = u
(t0 ).
Remark: The m-th ICE may be regarded as an approximate solution to the equation
(tn ) + Hn
u
(tn + θHn ) = u
ν
bi (θ)F (tni , u
(tni ), γ(tni , u
(tni ))).
(6)
i=1
2
Assumptions and lemmas
The aim of this paper is to obtain a result for the m-th ICE of the form
sup |u(t) − u
(t)| = O(H ρ ).
t<T
[0]
Clearly ρ is dependent on the choice of initial approximation zn (t), on the number of iterations m, on
the continuity properties of the problem (2) and its solution, and on properties of the CERK triple. We
will establish the global order of convergence for ICEs for the two initial approximations suggested above.
First, however, we state some assumptions and lemmas:
3
Lemma 2.1 Suppose that hn > 0 and αn = 1 + Cn hn , where 0 ≤ Cn ≤ C, for n = 0, 1, . . . , N − 1 and
N −1
i=0 hi ≤ T . If {Dn } and {en } are non-negative sequences with
en
≤
αn en−1 + Dn hn for n = 0, 1, . . . , N − 1
en
≤
(e−1 + max {Dk }T ) exp(CT ).
(7)
and e−1 ≥ 0, then
Proof. Denote by Πk,n the quantity
0≤k≤n
n
j=k
αj . From the recurrence (7), by induction,
en ≤ Π0,n e−1 + Π1,n D0 h0 + Π2,n D1 h1 + · · · + Dn hn
(n ≥ 0).
n
Note that αn ≤ 1 + Chn ≤ exp(Chn ). Thus Πk,n ≤ exp(C j=k hj ) ≤ exp(CT ) for k = 0, 1, . . . , n and
n = 0, 1, . . . , N − 1. Also D0 h0 + D1 h1 + · · · + Dn hn ≤ max{Dk }T for n = 0, 1, . . . , N − 1, and the result
k
follows.
We also require the following assumption:
Hypothesis 2.2 The sequence of steps {Hn } is assumed chosen such that u(r−1) (t) ∈ Lip[tn , tn+1 ] (in
particular, u(r) (t) ∈ C[tn , tn+1 ]) and Hn+1 /Hn = ηn ≤ η for n = 0, 1, . . . , N − 1. The derivative function
F (t, u1 , u2 ) in (2) satisfies uniform Lipschitz conditions (with Lipschitz constants L2 and L3 ) with respect
to its second and third arguments. In the case of a state-dependent lag, we also require that γ(t, u) satisfies
a uniform Lipschitz condition (with Lipschitz constant Lγ ) with respect to u, and the solution u(t) satisfies
a uniform Lipschitz condition (with Lipschitz constant Lu ). It is further assumed that γ is evaluated to
[l]
[l]
ensure, for every computed approximation zn (t) to u(t) with t ∈ [tn , tn+1 ], that γ(t, zn (t)) ≤ tn+1 .
Notation: In the theorems that follow we shall refer to the Lipschitz constants L2 , L3 , Lγ and Lu , and the
constants K1 = L3 + L3 Lu Lγ and K2 = L2 + L3 + L3 Lu Lγ .
b(θ)}
The m-th ICE is related to the CERK method defined by the CERK triple {
c, A,
indicated
below:
0
c A 0 ··· ···
..
..
. B 0
.
.
.
.
.
..
..
..
.. 0
c
A
(8)
≡
..
..
..
..
.. ..
T (θ)
θ b
.
.
.
.
.
.
c 0 ··· 0 B
0
θ 0T · · · · · · 0T bT (θ) ,
where dim(
c) = m dim(c). This tableau allows standard Runge-Kutta (RK) analysis to be applied to
the discrete RK method [1] obtained by setting θ = 1. The discrete RK method has a Butcher tableau
corresponding to a class of “parallel RK methods” [6]; ICEs, when applied to either an ODE or a DDE,
can be implemented efficiently on a parallel computer (see Section 5.3). The continuous order conditions
determining p∗ (see Definition 1.2) for the tableau (8) applied to the ODE (1) (and similarly for the DDE
(2) when γ(tni , u(tni )) ≤ tn for all i) were discussed by Paul & Baker [10].
Hypothesis 2.3 The parameters {c, b(θ)} define the continuous quadrature order q ∗ := q ∗ (c, b(θ)) of the
CERK triple, for which the continuous extension based on the values {u(tn ), {u (tni )}} satisfies: (i)
ν
∗
max u(tn + θHn ) − u(tn ) + Hn
bi (θ)u (tni ) ≤ CHnmin{q ,r}+1
(9)
0≤θ≤1 i=1
4
when u(r−1) (t) ∈ Lip[tn , tn+1 ], and (ii)
ν
∗
max u(tn + θHn ) − u(tn ) + Hn
bi (θ)u (tni ) ≤ Cηn Hnmin{q ,s}+1
1≤θ≤1+ηn (10)
i=1
when u(s−1) (t) ∈ Lip[tn , tn+2 ].
The CERK triple is said to provide a continuous quadrature extension u(tn ) + Hn νi=1 bi (θ)u (tni ) to
u(t). Using standard Peano-type quadrature theory, the CERK triple {c, A, b(θ)} satisfies (9) and (10)
when
ν
θj+1
for j = 0, . . . , q ∗ − 1.
bi (θ)cji =
(11)
j
+
1
i=1
Note that these conditions are independent of ηn .
b(θ)}
Remarks: The continuous order p∗ of the CERK triple {
c, A,
(8) cannot exceed q ∗ , and if m
∗
is sufficiently large it equals q . In the preceding hypothesis, we have introduced the generally distinct
integers r and s ≤ r related to the differentiability of u(t). However, a more precise insight can be obtained
by introducing integers pn , rn and sn such that u(rn −1) (t) ∈ Lip[tn , tn+1 ] and u(sn −1) (t) ∈ Lip[tn , tn+2 ].
In fact, r = min0≤i≤N −1 {ri } and s = min0≤i≤N −2 {si }. The determination of pn (corresponding to p
in Hypothesis 1.3) requires information on the behaviour of the lag, since γ(t, u(t)) can traverse several
previous intervals as t traverses the interval [tn , tn+1 ].
Lemma 2.4 (Differenced F -values). Assuming that Hypothesis 2.2 is valid,
|F (t, v, zn[l] (γ(t, zn[l] (t)))) − F (t, w, u(γ(t, u(t))))| ≤ L2 |v − w| + K1 sup |
zn[l] (x) − u(x)|
x≤tn+1
for t ∈ [tn , tn+1 ].
[l]
Proof. Writing γ(t, zn (t)) as γz , we use the triangle inequality
|F (t, v, zn[l] (γz )) − F (t, w, u(γ(t, u(t))))|
|F (t, v, zn[l] (γz )) − F (t, v, u(γz ))| +
≤
|F (t, v, u(γz )) − F (t, w, u(γz ))| +
|F (t, w, u(γz )) − F (t, w, u(γ(t, u(t))))|,
[l]
and exploit the restriction (by hypothesis) that γ(x, zn (x)) ≤ tn+1 for x ∈ [tn , tn+1 ]. In particular
|F (t, v, zn[l] (γz )) − F (t, v, u(γz ))|
≤ L3 |
zn[l] (γz ) − u(γz )|
≤ L3 |
zn[l] (x∗ ) − u(x∗ )| for some x∗ ≤ tn+1 ,
and
|F (t, w, u(γz )) − F (t, w, u(γ(t, u(t))))|
≤ L3 |u(γz ) − u(γ(t, u(t)))|
≤ L3 Lu Lγ |
zn[l] (t) − u(t)|.
Notation: For n ≥ 0, let
ξn[k] :=
sup
t∈[tn ,tn+1 ]
|u(t) − zn[k] (t)|
[0]
and en :=
sup
t0 ≤t≤tn+1
|u(t) − u
(t)| = max{en−1 , ξn[m] }.
[0]
Lemma 2.5 (Bounds on ξn ). Case 1. If zn (t) := u
(tn ), then
ξn[0] ≤ en−1 + Lu Hn .
[0]
Case 2. If zn (t) := u
(tn−1 + θ∗ Hn−1 ) for n ≥ 1, then
[m−1]
min{q∗ ,s}+1
ξn[0] ≤ en−2 + Mη Hn−1 ξn−1 + Cη Hn−1
for positive constants Cη and Mη .
5
[0]
Proof. Case 1. From the definition of zn (t),
ξn[0]
=
sup
t∈[tn ,tn+1 ]
|u(t) − u
(tn )|
≤
en−1 +
≤
en−1 + Lu Hn .
sup
t∈[tn ,tn+1 ]
|u(t) − u(tn )|
Case 2. Suppose that u(s−1) (t) ∈ Lip[tn−1 , tn+1 ], and the continuous quadrature order of the CERK
triple {c, A, b(θ)} is q ∗ . We introduce the continuous extension U (tn−1 + θ∗ Hn−1 ) based on the values
{u(tn−1 ), {u (tn−1,i )}}, namely
U (tn−1 + θ∗ Hn−1 )
= u(tn−1 ) + Hn−1
ν
bi (θ∗ )u (tn−1,i ).
i=1
Thus, by Hypothesis 2.3,
u(tn + θHn )
min{q∗ ,s}+1
≤ U (tn−1 + θ∗ Hn−1 ) + Cη Hn−1
,
[0]
where θ∗ = 1 + θHn /Hn−1 ∈ [1, 1 + η]. From the definition of zn (t) and applying Lemma 2.4,
ξn[0]
≤
min{q∗ ,s}+1
sup
θ ∗ ∈[1,1+η]
|u(tn−1 + θ∗ Hn−1 ) − u
(tn−1 + θ∗ Hn−1 )| + Cη Hn−1
≤ en−2 + Hn−1
≤ en−2 +
where Mη = K2
ν
sup
θ ∗ ∈[1,1+η] i=1
ν
sup
θ ∗ ∈[1,1+η]
min{q∗ ,s}+1
[m−1]
|bi (θ∗ )|K2 ξn−1 + Cη Hn−1
i=1
[m−1]
min{q∗ ,s}+1
Mη Hn−1 ξn−1 + Cη Hn−1
,
|bi (θ∗ )|.
Notation: Let
N = sup
ν
0≤θ≤1 i=1
|bi (θ)|
K2
.
1 − L2 ||A||∞ Hn
(12)
[1]
Lemma 2.6 (A bound on ξn ). Given the CERK triple {c, A, b(θ)} and the value p in Hypothesis 1.3,
there exists a positive constant U such that for L2 ||A||∞ Hn < 1,
[1]
p+1
.
ξn[1] ≤ N Hn ξn[0] + d[1]
n where dn ≤ (1 + N Hn )en−1 + UHn
(13)
Proof. Consider the internal-stage derivatives
[0]
fni
= F (tni , u
(tn ) + Hn
i−1
j=1
[0]
aij fnj , zn[0] (γ(tni , zn[0] (tni )))),
and the related quantities
fni
= F (tni , u(tn ) + Hn
i−1
aij fnj , u(γ(tni , u(tni )))).
j=1
Using Lemma 2.4 and K2 = L2 + K1 ,
[0]
K2 en−1 + K1 ξn
.
1 − L2 ||A||∞ Hn
[0]
max |fni − fni | ≤
i
(14)
Now
[1] (tn + θHn ) =
zn[1] (tn + θHn ) ≡ u
u
(tn ) + Hn
ν
i=1
u(tn + θHn ) =
u(tn ) + Hn
ν
i=1
6
[0]
bi (θ)fni ,
bi (θ)fni + O(Hnp+1 ),
by Hypothesis 1.3. Differencing these equations, employing (14), and bounding the order term using a
suitable constant U, we find that
ξn[1]
ν
[0]
en−1 + Hn sup
≤
N Hn ξn[0] + (1 + N Hn )en−1 + UHnp+1 ,
0≤θ≤1 i=1
|bi (θ)|
K2 en−1 + K1 ξn
+ UHnp+1
1 − L2 ||A||∞ Hn
≤
since K1 ≤ K2 .
[k]
The proof of a similar result for ξn with k ≥ 2 follows different lines:
[k]
Lemma 2.7 (A bound on ξn for k ≥ 2). Assuming that Hypothesis 2.3 is valid and that L2 ||A||∞ Hn < 1,
then
[k]
min{q∗ ,r}+1
for k ≥ 2.
ξn[k] ≤ N Hn ξn[k−1] + d[k]
n where dn ≤ (1 + N Hn )en−1 + CHn
Proof. We have by Hypothesis 2.3 that
u(tn + θHn ) ≤ u(tn ) + Hn
ν
bi (θ)F (tni , u(tni ), u(γ(tni , u(tni )))) + CHnmin{q
∗
,r}+1
i=1
and, from the definition of the k-th ICE,
u
[k] (tn + θHn ) =
u
(tn ) + Hn
ν
i=1
[k−1]
bi (θ)F (tni , u
[k−1] (tni ), zn[k−1] (
γni
)).
Thus, differencing these equations and using the triangle inequality,
ξn[k]
≤
en−1 + Hn sup
ν
0≤θ≤1 i=1
[k−1]
|bi (θ)||∆fni
| + CHnmin{q
∗
,r}+1
,
where for k ≥ 2, using Lemma 2.4 and K2 = L2 + K1 ,
[k−1]
|∆fni
| =
≤
[k−1]
|F (tni , u(tni ), u(γ(tni , u(tni )))) − F (tni , u
[k−1] (tni ), zn[k−1] (
γni
))|
L2 ξn[k−1] + K1 (en−1 + ξn[k−1] ).
Therefore, since K1 ≤ K2 ,
ξn[k] ≤ QHn ξn[k−1] + (1 + QHn )en−1 + CHnmin{q
where Q = K2 sup
ν
0≤θ≤1 i=1
3
∗
,r}+1
,
|bi (θ)| = (1 − L2 ||A||∞ Hn )N < N .
The convergence result
We now present the main result, concerning the order of convergence of ICEs when applied to vanishing
lag state-dependent DDEs.
We assume that the CERK triple {c, A, b(θ)} is given, and let H := maxn {Hn }. We suppose that
Hypothesis 2.2 is valid (so that u(r−1) (t) ∈ Lip[tn , tn+1 ] for all n) and assume that the conditions of
(t)
Hypotheses 1.3, 1.4 and 2.3, in terms of q ∗ and r, hold for the CERK triple and the problem (2). If u
is computed from the m-th ICE using (8), then the following results hold:
Theorem 3.1 (Global order of convergence).
[0]
(tn ) for each interval [tn , tn+1 ], then
Case 1. If zn (t) := u
(t)| = O(H ρ1 ) where ρ1 = min{m, q ∗ , r}.
sup |u(t) − u
t0 ≤t<T
7
[0]
Case 2. If zn (t) := u
(tn−1 + θ∗ Hn−1 ) for each interval [tn , tn+1 ] with n ≥ 1, u(s−1) (t) ∈ Lip[tn−1 , tn+1 ]
and e0 = O(H σ ), then
sup |u(t) − u
(t)| = O(H ρ2 ) where ρ2 = min{m + p − 1, m + s, q ∗ , r, σ}.
t0 ≤t<T
[0]
[0]
(tn ) or zn (t) := u
(tn−1 + θ∗ Hn−1 ) (the choice varying
Case 3. If, for each [tn , tn+1 ], either zn (t) := u
with n), then
(t)| = O(H ρ3 ) where ρ3 = min{ρ1 , ρ2 }.
sup |u(t) − u
t0 ≤t<T
[m]
Proof. We recall that en = max{en−1 , ξn } for n ≥ 0 and observe that e−1 = 0. Suppose that
[k]
[k−1]
[k]
+ dn for k = 1, 2, . . . , m. Thus,
L2 ||A||∞ H < 1, then from Lemmas 2.6 and 2.7, ξn ≤ N Hn ξn
ξn[1]
≤ N Hn ξn[0] + (1 + N Hn )en−1 + UHnp+1
ξn[k]
≤ N Hn (N Hn ξn[k−2] + d[k−1]
) + d[k]
n
n
k−2
k [0]
k−1 [1]
l
≤ (N Hn ) ξn + (N Hn )
dn +
(N Hn )
max {d[l]
n }.
and for k ≥ 2,
(15)
2≤l≤k
l=0
Suppose that N H < 12 , so that (1 + N Hn )/(1 − N Hn ) = 1 + (2N Hn )/(1 − N Hn ) ≤ 1 + 4N Hn for all n.
[l]
Substituting the values dn into (15), we deduce that for k ≥ 1,
ξn[k]
≤
(N Hn )k ξn[0] + N k−1 UHnk+p + (1 + N Hn )
≤
(N Hn )k ξn[0]
+ (1 + 4N Hn )en−1 + max{N
[0]
k−1
(N Hn )l en−1 +
l=0
k−1
min{q∗ ,r}+1
CHn
1 − N Hn
U, 2C}Hnmin{k+p,q
∗
+1,r+1}
.
[0]
(16)
[0]
Case 1. For zn (t) := u
(tn ), from Lemma 2.5, ξn = en−1 + Lu Hn . Putting k = m, substituting ξn into
(16), and using (N Hn )m ≤ N Hn gives
en ≤ (1 + 5N Hn )en−1 + DHnmin{m+1,m+p,q
∗
+1,r+1}
,
(17)
where D = max{N m Lu , N m−1 U, 2C}. Applying Lemma 2.1 to this result yields
sup |u(t) − u
(t)| ≤ DT exp(5N T )H ρ1 , where ρ1 = min{m, m + p − 1, q ∗ , r}.
t0 ≤t<T
Also note that (17) gives a bound on e0 for Cases 2 and 3,
e0 ≤ DHnmin{m+1,m+p,q
∗
+1,r+1}
.
[0]
Case 2. For zn (t) := u
(tn−1 + θ∗ Hn−1 ), we shall obtain a bound on1
[m−1]
ên = max {el , N Hl ξl
0≤l≤n
} for 0 ≤ n ≤ N − 1.
From Lemma 2.5,
ξn[0]
1 See
[m−1]
min{q∗ ,s}+1
≤ en−2 + Mη Hn−1 ξn−1 + Cη Hn−1
Mη
[m−1]
min{q∗ ,s}+1
N Hn−1 ξn−1 + Cη Hn−1
≤ en−1 +
N
Mη
min{q∗ ,s}+1
)ên−1 + Cη Hn−1
≤ (1 +
.
N
Remark 5 following this proof.
8
(18)
Substituting this result into (16), putting k = m and using Hn ≤ ηHn−1 and Hn ≤ H for all n, gives
ξn[m] ≤ (N Hn )m (1 +
∗
Mη
)ên−1 + (1 + 4N Hn )en−1 + Hn−1 Eη H min{m+s,m+p−1,q ,r} ,
N
where Eη = max{Cη N m , ηN m−1 U, 2ηC}. Similarly, putting k = m − 1 and multiplying by N Hn <
1
2
gives
∗
Mη
1
)ên−1 + (1 + 4N Hn )en−1 + Hn−1 Eη H min{m+s,m+p−1,q ,r} .
N
2
N Hn ξn[m−1] ≤ (N Hn )m (1 +
[m]
[m−1]
[m]
Thus, since ên = max{ên−1 , ξn , N Hn ξn
} and (N Hn )m ≤ N Hn < 1, the bounds on ξn
[m−1]
N Hn ξn
give
∗
Mη
1 + 5N Hn + (N Hn )m
ên ≤
ên−1 + Hn−1 Eη H min{m+s,m+p−1,q ,r} .
N
≤ (1 + RHn )ên−1 + Hn−1 Eη H min{m+s,m+p−1,q
∗
,r}
,
and
(19)
where R = 5N + Mη . Applying Lemma 2.1 (re-indexing the subscripts) to this result yields
sup |u(t) − u
(t)| ≤
t0 ≤t<T
sup
0≤n≤N −1
|ên | ≤ (ê0 + Eη T H min{m+s,m+p−1,q
∗
,r}
) exp(RT ).
Thus supt0 ≤t<T |u(t) − u
(t)| = O(H ρ2 ), where ρ2 = min{σ, m + s, m + p − 1, q ∗ , r}.
[0]
Case 3. We seek an inequality relating successive values of ên as above, but valid for either choice of zn (t).
We therefore return to the proof of Case 1: In addition to (17) we obtain, by putting k = m − 1 in (16)
and multiplying by N Hn < 12 , the bound
∗
1
(1 + 6N Hn )en−1 + DHnmin{m+1,m+p,q +1,r+1} .
2
Thus we deduce a result analogous to (19), namely
N Hn ξn[m−1] ≤
ên ≤ (1 + 5N Hn )ên−1 + DHnmin{m+1,m+p,q
∗
+1,r+1}
.
(20)
Since either (19) or (20) is valid, we have for n ≥ 1
ên
≤
(1 + RHn )ên−1 + max{D, Eη } max{Hn−1 , Hn }H min{m+s,m+p−1,m,q
so that by Lemma 2.1 with hn =
ên
≤
1
2
∗
,r}
,
max{Hn−1 , Hn }
(ê0 + F T H min{m+s,m+p−1,m,q
∗
,r}
) exp(RT ),
where F = max{2D, 2Eη }. Thus
sup |u(t) − u
(t)| = O(H ρ3 ) where ρ3 = min{σ, m + s, m + p − 1, m, q ∗ , r}.
t0 ≤t≤T
Remarks:
1. The preceding theorem provides not only a convergence proof, but an estimate of the rate of convergence as H 0. In particular, it shows the rôle of the order of the error e0 in Case 2. Implementing
the method in Case 2 requires a starting procedure for which σ does not restrict the order of accuracy.
[0]
This is most easily accomplished by employing z0 (t) := u(t0 ) and using m = m0 on the interval
[t0 , t1 ], so that σ = min{m0 , q ∗ , r} + 1 by (18).
2. It may sometimes be overlooked that: (i) a quantity that is (for ρ ≥ 1) O(H ρ ) as H 0 is also
O(H ρ−1 ), and (ii) for a particular choice of H = H ∗ , we may have 01 (H ∗ ) > 02 (H ∗ ) even though
01 (H)/02 (H) → 0 as H 0. Irrespective of order arguments, it may be appreciated that our
technique is based upon a predictor-corrector type iteration for the solution of equation (6). Thus,
equation (6) may be solved more accurately by increasing the value of m, although the order of the
method remains the same.
9
3. Theorem 3.1 indicates that, for smooth problems, the order of our PCERK methods is limited by the
continuous quadrature order q ∗ and not by the continuous order p∗ of the CERK triple {c, A, b(θ)}.
Thus arbitrarily high-order PCERK methods can be constructed by ensuring that the CERK triple
satisfies only the appropriate continuous quadrature order conditions (11) (cf. Theorem 2.4 in [1]).
4. Although the main application of Theorem 3.1 is for ICEs, by considering only one iteration (m = 1),
Theorem 3.1 can be applied to a typical continuous RK method adapted for solving DDEs of type
(2).
[m−1]
}. The proof of the theorem
5. Theorem 3.1 is proved by considering ên = max0≤l≤n {el , N Hl ξl
[m−1]
might be attempted using the more natural definition of ên = max1≤l≤n {el , N Hl−1 ξl−1 }, or at[m−1]
, but the manipulation becomes rather more involved.
tempting to eliminate terms in ξl
6. In our discussion of ICEs we have, for simplicity, fixed m and taken values of p, r and s which
suffice for every interval. However, identifying each value of pn , rn and sn (cf. Remarks following
Hypothesis 2.3) suggests a basis for an adaptive strategy for varying the value of m (mn , say) for
each interval.
7. Theorem 3.1 refers to the order of convergence for a general sequence of steps {Hn } with Hn+1 /Hn ≤
η and H := maxn {Hn } 0. The choice of steps {Hn } governs the values of p, r and s which occur
in the theorem, and hence governs the value of the order of convergence ρ. In general, associated
with any particular problem, there will be “favoured sequences” of steps
H
H
H
..
.
:=
:=
:=
..
.
{H1 , H2 , H3 , . . .}
{H1 , H2 , H3 , . . .}
{H1 , H2 , H3 , . . .}
..
.
with corresponding stepsizes H := maxn {Hn }, H := maxn {Hn }, . . . for which the values of p, r
and s are increased, so that the value of ρ is improved.
Corollary 3.2 (Optimum order). There is a “preferred strategy” for choosing the steps {Hn } so that the
values of p, r and s in Theorem 3.1 are optimized, and thus the order of convergence is maximized as
H 0 through a sequence of compatible stepsizes.
We consider that our proof of the preceding theorem has intrinsic mathematical interest and may
provide a template for the analysis of restricted classes of equations (exploiting additional hypotheses) and
[0]
of other methods. We mention one such possibility: ICEs depend on the initial approximation zn (t) and
starting values {
u[0](tni )} that are associated with A and c; however, it is possible to use any suitable
starting values {
u[0] (tni )}, and, in particular, to avoid any reference to A in their definition. (An obvious
[0]
[0]
example is to take u
[0] (tni ) = zn (tni ); indeed, the case of using A = 0 and zn (t) = u
(tn ) is already covered
by Case 1 in Theorem 3.1.) In order to analyze the convergence of general methods of this type, equation
(14) must be amended, the effect on Lemma 2.6 determined, and subsequent changes to Theorem 3.1
made. This suggests a wider significance for our style of proof, as we demonstrate in the next result and
the analysis of the next section.
[0]
Theorem 3.3 (A special case). If γ
ni ≤ tn for all i and n, then Case 1 and Case 2 are equivalent, and
(t)| = O(H ρ4 ) where ρ4 = min{m + p − 1, q ∗ , r}.
sup |u(t) − u
t0 ≤t≤T
10
[0]
[0]
[0]
[0]
[0]
Proof. For all i, if γ
ni ≤ tn then zn (
γni ) = u
(
γni ) (independent of the initial approximation zn (t)).
[0]
[0]
Thus the result of Lemma 2.5 becomes ξn = en−1 . Substituting ξn into (16), putting k = m and using
(N Hn )m ≤ N Hn gives
ξn[m]
≤ (N Hn )m en−1 + (1 + 4N Hn )en−1 + max{N m−1 U, 2C}Hnmin{m+p,q
∗
+1,r+1}
,
so that
en
≤ (1 + 5N Hn )en−1 + max{N m−1 U, 2C}Hnmin{m+p,q
∗
+1,r+1}
.
The result then follows as for Case 1 in Theorem 3.1.
Remark: Theorem 3.3 gives an improved result for the order of convergence for ICEs applied to ODEs
and non-vanishing lag DDEs.
4
A first-same-as-last strategy
In the case that cν = 1 we have the opportunity of using a first-same-as-last (FSAL) strategy. Using an
FSAL strategy alters the starting values
u
[0] (tni ) =
u
(tn ) + Hn
i−1
j=1
[0]
[0]
[0]
aij fnj , where fnj = F (tnj , u
[0] (tnj ), zn[0] (
γnj )),
[0]
[0]
[m−1]
by replacing fn1 with f˘n1 = fn−1,ν for n ≥ 1. Thus the starting values now read
i−1
[0]
[0]
u
[0] (tni ) = u
(tn ) + Hn (ai1 f˘n1 +
aij F (tnj , u
[0] (tnj ), zn[0] (
γnj ))),
(21)
j=2
and we save a derivative function evaluation on each interval. The iteration for the m-th ICE (5) uses the
new starting values, but otherwise remains unchanged.
Definition 4.1 The FSAL iterated continuous extension u
[m] (t) is defined by replacing (4) by (21) in
Definition 1.5.
Theorem 4.2 (Global order of convergence). Using the FSAL strategy, the orders of convergence stated
in Theorems 3.1 and 3.3 are preserved when m ≥ 2.
Proof. First we amend Lemma 2.6, using the additional bound
[0]
[m−1]
[m−1]
|f˘n1 − fn1 | ≤ L2 ξn−1 + K1 (en−2 + ξn−1 ),
which yields the result
[1]
[0]
[m−1]
ξn ≤ N Hn ξn + (1 + N Hn )en−1 + N Hn ξn−1 + UHnp+1 .
[1]
[m−1]
Thus (13) holds, but with dn = (1+N Hn )en−1 +N Hn ξn−1 +UHnp+1 . In order to prove Theorem 4.2, we
[m]
[m−1]
obtain bounds on values {ên } by considering ξn and N Hn ξn
for m ≥ 2 (cf. the proof of Theorem 3.1,
Case 2). Thus the FSAL strategy in Case 1 yields the bound
ên
≤
(1 + (5 + η)N Hn )ên−1 + DHnmin{m+1,m+p,q
∗
+1,r+1}
,
and the result then follows. Similarly, in Case 2,
ên
≤
(1 + (5N + ηN + Mη )Hn )ên−1 + Hn−1 Eη H min{m+s,m+p−1,q
and again the result follows. The proof of Case 3 is now straightforward.
11
∗
,r}
,
The corresponding proof for Theorem 3.3 is immediate, as the result of Lemma 2.5 is unchanged when
using the FSAL strategy.
[0]
Remark: If γ
ni ≤ tn for all i and n, Theorem 4.2 is easily proved when m = 1.
5
Algorithmic considerations
5.1
The ICE recurrence
The basis of our method is the recurrence
(tn ) + Hn
u
[k] (tn + θHn ) = u
ν
i=1
[k−1]
bi (θ)F (tni , u
[k−1] (tni ), zn[k−1] (
γni
)).
(22)
[k−1]
The iteration from u
[k−1] (t) to u
[k] (t) proceeds by first evaluating the quantities {
γni }νi=1 . Evaluating
[k−1] [k−1] ν
[k−1]
(
γni )}i=1 involves evaluating Ψ(t) if γni
< t0 , or evaluating an expression of the form
the terms {
zn
[k−1]
[k−1]
ν
(22) if γ
ni
≥ t0 . Next the values {
u
(tni )}i=1 are computed by setting θ = ci in (22), and finally the
[k−1] [k−1]
[k−1] (tni ), zn
(
γni ))}νi=1 are computed. (Note that, since c1 = 0,
derivative function values {F (tni , u
[k]
the value fn1 = F (tn , u
(tn ), u
(γ(tn , u
(tn )))) remains unchanged in the iteration, and thus need not be
recomputed.)
5.2
Parallel processing
The computation of an ICE falls naturally into two parts, the evaluation of the starting values {
u[0] (tni )}νi=1
and the subsequent iteration. We indicate how ICEs may be implemented in parallel and how this affects
the choice of parameters.
The computation of the starting values is in general sequential. However, one possible choice for A for
parallel implementation of ICEs is the ‘trivial predictor’ (A = 0). This defines a primitive2 scheme with
p∗ = 1, but we do not consider it further here. A more suitable choice for A is c[1, 0, . . . , 0], for which
[0]
p∗ = 2. With the latter choice, we first compute fn1 (a step that is avoided using the FSAL variant), so
that {
u[0] (tni )}νi=2 can be evaluated in one parallel stage on ν − 1 processors.
Each further iteration can be evaluated in one parallel stage that consists of evaluating (simultaneously)
[k−1]
[k−1] [k−1] ν
zn
(
γni )}i=1 and {
u[k−1] (tni )}νi=1 , and finally evaluating the
the values {
γni }νi=1 , then computing {
[k−1] ν
derivative function values {fni }i=1 . Thus, after m iterations (and m + 1 parallel “stages”), the global
order of the solution is min{m + 1, q ∗ , r, σ} by Theorem 3.1, Case 2.
5.3
Transforming theory into practice
The analysis presented above is intended to contribute to the theoretical understanding of ICEs under
general conditions. As such, it can be applied to a wide variety of implementations of algorithms, embodying various step-change strategies and varying continuity conditions. In a practical algorithm, the
aim is to control the actual error by adjusting the parameters of the method, usually the step and possibly
the order. The construction of robust codes which achieve this purpose is part art and part science. In
our opinion, mathematical analysis can often only give insight and confidence in an algorithm, despite its
rigorous nature. Thus we note that statements concerning the order of convergence of a method are only
a guide to the actual performance of a method, since (i) such arguments are asymptotic as the stepsize
H 0, and (ii) controlling bounds on the local error is different from controlling the global error. Baker et
al. [3] recently addressed a number of the practical issues that arise in the numerical treatment of DDEs.
i−1
2
It fails to satisfy the “simplifying assumption”
j=1
aij = ci .
12
Whilst we cannot expect in this paper to give a definitive statement about the best practice in the numerical treatment of DDEs, we include in an Appendix some observations on practical aspects of solving
DDEs numerically.
6
Numerical results
We seek to demonstrate the consistency of numerical results with our theorems.
6.1
A practical illustration of order
Theorems 3.1, 3.3 and 4.2 imply the existence of a stepsize H ∗ > 0 such that
(t)| ≤ MT H ρ uniformly for 0 < H ≤ H ∗ ,
sup |u(t) − u
t0 <t<T
(23)
when u
(t) ≡ u
H (t) is computed using a sequence of steps {Hn }, with H = maxn {Hn }. Equation (23)
holds for some MT ≡ MT (c, A, b(θ)) which is independent of H.
In the context of Corollary 3.2, this bound is valid for an optimum value of ρ (and a corresponding particular choice of MT ) only for a restricted set of “compatible” stepsizes H ∈ (0, H ∗ ], that may correspond
to special step selection strategies.
For a fixed-step Hn = H, one might hope for a stronger form of the theory to hold, namely that there
exists a bounded function 0(t) independent of H such that
sup |u(t) − u
(t) − H ρ 0(t)| → 0 as H 0,
t0 <t<T
(24)
or even supt0 <t<T |u(t) − u
(t) − H ρ 0(t)| = O(H ρ+1 ). Numerical results which illustrate (24) convincingly
also demonstrate that (23) is valid with MT = 20(T ). We include some numerical results using a fixed
step H that are consistent 3 with (24).
The possible variations in behaviour of an error that is consistent with (23) make it difficult to provide
figures which always appear convincing. To persuade the reader of the difficulty, it is sufficient to consider
the behaviour of (for example)
25
0(H) := (1 + (−1)[ H ] )H 2 + H 3 , where [X] = the integer part of X,
(25)
as a function of H > 0. Plotting the graphs of 0(H) against H for (i) H = 18 , 28 , . . . , 1, (ii) H = 19 , 29 , . . . , 1
1
2
and (iii) H = 20
, 20
, . . . , 1 using Matlab [7] produces Figure 1.
3 Numerical
results cannot be said to prove (24).
13
Illustration of non-monotonic second order convergence
Illustration of non-monotonic second order convergence
3
3
+ - H = [0:1/8:1]
2.5
2.5
o - H = [0:1/9:1]
: - H = [0:1/20:1]
2
Sequence values
Sequence values
2
1.5
1.5
1
1
0.5
0.5
0
0
0.2
0.4
0.6
Stepsize H
0.8
0
0
1
Figure 1
0.2
0.4
0.6
Stepsize H
0.8
1
Figure 2
The construction of the function 0(H) is such that for a “compatible” discrete sequence of values of H
3
2
(those for which [ 25
H ] is an odd integer) 0(H) behaves like H , although 0(H) = O(H ) for the continuous
variable H ∈ R+ . (This illustrates the requirement for “compatible” values of H in Corollary 3.2.) The
function 0(H) satisfies H 3 ≤ 0(H) ≤ 2H 2 + H 3 , and these bounding curves are identifiable in the graph.
Figure 1 illustrates how the order of the function 0(H) is difficult to identify conclusively when one selects
a restricted set of stepsizes. In our view it is preferable to opt for a random sample of stepsizes, and in
Figure 2 we display the effect of taking 50 pseudo-randomly distributed values of H ∈ [0, 1] and joining the
resulting points. By choosing a random distribution, one hopes to determine the true order of convergence;
that all the computed values satisfy |0(H)| ≤ 2H 2 + H 3 is evidence of O(H 2 ) behaviour.
We have introduced this discussion of 0(H): (i) as a general warning against deducing the order of
convergence by selecting atypical values of the parameter H (similar remarks also apply in connection
with (24)), and (ii) because the behaviour of the error in Figure 6 is similar to the graph in Figure 2. In
connection with the warning in (i), we believe it best to select a random sequence of values of H when
providing numerical evidence, unless one seeks to make statements concerning “preferred” step selection
strategies.
6.2
Choice of parameters
In order to illustrate the order of convergence of ICEs, we compare results computed using: (i) a continuous
version of the classical RK method (26) (which can be regarded as an ICE with m = 1), and (ii) the mth ICE using A = [0, 12 , 1]T [1, 0, 0] and taking m = 2 (27). For an ICE applied to a general DDE (cf.
[0]
[0]
Theorem 3.3) we take z0 (t) := u(t0 ), and on subsequent intervals zn (t) := u(tn−1 + θ∗ Hn−1 ). Both
methods have global order of convergence three by Theorem 3.1 when applied to sufficiently smooth
functions. Also, both formulae can use the FSAL strategy to economize on derivative function evaluations.
The classical RK and ICE methods have four and five sequential stages respectively. (On the first
[0]
interval, the use of z0 (t) := u(t0 ) increases the number of RK stages in the ICE to seven.) In a 3processor parallel implementation, the general step may be implemented in parallel in four and three stages
respectively, and the FSAL variants may be implemented in three and two parallel stages respectively.
14
0
0
1
2
1
2
1
2
1
1
0
0
1
6
0
1
2
0
1
3
0
1
0
1
3
1
6
b1 (θ)
=
b2 (θ)
=
b3 (θ)
=
b4 (θ)
=
2 3
3 2
3θ − 2θ +
− 32 θ3 + θ2
− 32 θ3 + θ2
2 3
1 2
3θ − 2θ .
θ
(26)
The classical Runge-Kutta method with p∗ = q ∗ = 3.
0
0
1
2
1
2
1
0
1
2
1
1
1
0
0
0
0
0
0
5
24
1
6
1
3
2
3
−1
24
1
6
0
0
0
b1 (θ) =
0
b2 (θ) =
0
0
0
0
0
1
6
2
3
1
6
b3 (θ) =
2 3
3 2
3θ − 2θ +
− 43 θ3 + 2θ2
2 3
1 2
3θ − 2θ .
θ
(27)
The second-ICE with p∗ = 2 and q ∗ = 3.
b(1)}
The tableaux (26) and (27) correspond to the triples {
c, A,
with the coefficients {bi (θ)} shown
alongside.
6.3
Results for a test equation
In [1] we illustrated our methods with examples in which the lag vanished for some argument. Here, we
demonstrate the numerical performance of the classical RK method and the ICE by solving the DDE
u (t) = 0.2u(t − 2)(2 − u(t)),
Ψ(t) = cos(t)
on the interval [0, 10], using a selection of fixed stepsizes H. Due to the size of the delay, when H ≤ 2 the
order of convergence is given by Theorem 3.3. The solution u(t) is smooth on each interval [0, 2], [2, 4],
[4, 6], . . ., but the left and right derivatives u (0± ) do not agree, and thus there are jump discontinuities in
derivatives at t = 0, 2, 4, . . .. However, note that u(t) is Lipschitz continuous at t = 0. If we use a fixed-step
H of the form H = 2/N for N ∈ Z, all the functions involved are smooth on [tn , tn+1 ] for n = 0, 1, 2, . . .
(Hn = H for all n). To demonstrate that our numerical results are consistent with (24) we plot graphs of
(t) /H ρ 0(t)
EH (t) := u(t) − u
(28)
with a suitable value of ρ and a sequence of diminishing stepsizes H = 1/16, 1/32 and 1/64; the results
are given in Figures 3 and 4.
15
-4
-5
Iterated continuous extension (m=2)
x 10
x 10
2
FSAL classical Runge-Kutta method (m=1)
5
1.5
: - H = 1/16
0
x - H = 1/32
Scaled error
Scaled error
1
0.5
-5
o - H = 1/64
-10
0
: - H = 1/16
x - H = 1/32
-15
-0.5
o - H = 1/64
-1
0
2
4
6
8
t
-20
0
10
Figure 3
2
4
6
8
t
10
Figure 4
The “actual solution” u(t) was obtained using a fifth-order RK DDE method (given in [9]) at a high
accuracy. Because the choices of H ensure that continuity is not a problem, we take ρ = 3 (as predicted by
Theorems 3.1, 3.3 and 4.2), and the displayed graphs of the scaled error EH (t) are almost coincident. Thus
the figures suggest that as H 0 through a sequence of values H = 1/2r , r = 4, 5, 6, . . ., the global order
of convergence is three (consistent with (24)) for both the classical RK method and the ICE (whether or
not the FSAL strategy is used). In the corresponding plots (not shown) for a general fixed-step, the effect
of the jump in u (t) at t = 0 is clearly visible at t = 2 and observable at t = 4; in Figures 3 and 4 only the
effect at t = 2 is visible.
Figures 5 and 6 display the maximum error supn |en | when using the ICE (m = 2) with the FSAL
strategy. There is a subsequence of stepsizes that give unrestricted r and O(H 3 ) convergence, as shown
in Figure 5. However, for a random fixed-step we have p = 1 and r = 2, and the error is O(H 2 ), being
bounded by (5.6 × 10−4)H 2 . Figure 6 shows that for 1000 pseudo-random fixed stepsizes with H ∈ [0, 0.2],
the maximum error satisfies supn |en | ∈ [(1.8 × 10−4 )H 3 , (5.6 × 10−4 )H 2 ].
-6
x 10
-5
FSAL iterated continuous extension (m=2)
x 10
1.6
1.8
1.4
1.6
1.4
1
1.8E-4*H
Maximum error over [0,10]
Maximum error over [0,10]
1.2
3
0.8
0.6
0.4
5.6E-4*H
2
1.2
1
1.8E-4*H
3
0.8
0.6
0.4
0.2
0
0
FSAL iterated continuous extension (m=2)
0.2
0.05
0.1
Stepsize H
0.15
0
0
0.2
Figure 5
0.05
0.1
Stepsize H
0.15
0.2
Figure 6
Recalling Remark 2 in Section 3, it is encouraging to observe that, given the continuity properties of
u(t), F (t, u1 , u2 ), etc., our numerical results (and those reported in [1] for vanishing lag DDEs) indicate
16
that for the ICEs under consideration, both with special and arbitrary choices of H, the order predicted
in our theorems is realistic.
Acknowledgements
This paper arose, in part, from academic exchanges with Prof. John Butcher, whose hospitality to Prof.
C.T.H. Baker during an academic visit to Auckland University (funded in part by the Royal Society of
London) is gratefully acknowledged. The authors wish to express their thanks to the referees and the editor
for encouraging comments. The authors also thank Dr. Ruth Thomas for reading an initial manuscript and
for her suggestions. The research of Dr. C.A.H. Paul was funded by the Science & Engineering Research
Council under grant GR/H59237. A visit of Prof. Butcher to the University of Manchester was funded by
the Science & Engineering Research Council under grant GR/H04688.
References
[1] C. T. H. Baker and C. A. H. Paul, Parallel continuous Runge-Kutta methods and vanishing lag
delay differential equations, Adv. Comp. Math., 1 (1993), pp. 367–394.
[2] C. T. H. Baker and C. A. H. Paul, Computing stability regions – Runge-Kutta methods for delay
differential equations, IMA J. Numer. Anal., 14 (1994) pp. 347–362
[3] C. T. H. Baker, C. A. H. Paul, and D. R. Willé, Issues in the numerical solution of evolutionary delay differential equations, Numer. Anal. Technical Report 248, Department of Mathematics,
Manchester University, England, 1994.
[4] M. A. Feldstein and K. W. Neves, High order methods for state-dependent delay differential
equations with non-smooth solutions, SIAM J. Numer. Anal., 21 (1984), pp. 844-864.
[5] E. Hairer, S. P. Nørsett and G. Wanner, Solving Ordinary Differential Equations – I, SpringerVerlag, Berlin, 1987.
[6] K. R. Jackson, A survey of parallel numerical-methods for initial-value problems for ordinary differential equations, IEEE Trans. Magnetics, 27 (1991), pp. 3792–3797.
[7] Matlab,The MathWorks Inc., Natick, MA 01760.
[8] K. W. Neves, Numerical solution of functional differential equations with state dependent lags, Doctoral Thesis, Department of Mathematics, Arizona State University, Arizona, AZ, 1974.
[9] C. A. H. Paul, Runge-Kutta methods for functional differential equations, Ph.D. thesis, Department
of Mathematics, Manchester University, England, 1992.
[10] C. A. H. Paul and C. T. H. Baker, Explicit Runge-Kutta methods for the numerical solution of
singular delay differential equations, Numer. Anal. Technical Report 212, Department of Mathematics,
Manchester University, England, 1992.
[11] D. R. Willé and C. T. H. Baker, The propagation of derivative discontinuities in systems of delay
differential equations, Appl. Num. Math., 9 (1992), pp. 223-234.
17
appendix
Practical implementation of ICEs
Despite a long history, research into the design of numerical codes for the somewhat simpler case of initialvalue problems for ODEs is still active. The case of DDEs is similar; for a discussion of some of the issues
see [3]. Whilst we do not attempt, in this paper, to give a definitive statement about the best practice in
the numerical treatment of DDEs, we conclude with some brief comments and pointers on the practical
implementation of ICEs. Our comments fall into two categories: (i) those relating to the implementation
of a general method for solving DDEs, and (ii) those relating specifically to ICEs.
(i) Generalities:
1. Tracking discontinuities: The view expressed in [3] was that it is (in general) preferable to track the
propagation of derivative discontinuities in the solution, by locating the points {t∗j } referred to in Section 1.
The technique of tracking discontinuities can be implemented in such a way that one has guaranteed lower
bounds on the values {rn } such that u(rn −1) (t) ∈ Lip[tn , tn+1 ], and the analogous values {sn }.
2. Stepping over discontinuities: Even when the position of a derivative discontinuity is known, it may be
necessary to “step over” it. (This can occur when the derivative discontinuity is ill-conditioned so that its
position is difficult to locate accurately, when the position of a derivative discontinuity is not a computer
number and when the points of discontinuity have a cluster point.)
3. Estimating the local error: Local error estimators are generally only valid for small stepsizes and regions
where the solution has sufficiently smooth derivatives. Strategies for controlling the local error in high-order
methods applied to DDEs (and ODEs) with non-smooth solutions is an issue requiring more study, since in
this situation asymptotically correct expressions for the error are harder to obtain than error bounds.
4. Adjusting the stepsize: Stepsize strategies should take into account the position of derivative discontinuities, local error estimates and stability considerations. A range of tactics may be found in ODE-solvers, but
their performance for DDEs, particularly when the lag vanishes, may be less robust.
5. The effect of past error: It is impossible to predict with any certainty the local accuracy that will be
required to achieve a specified global tolerance, because past solution values can be referenced by the delay
term with an arbitrarily large weighting factor.
(ii) Specifics:
1. Choice of RK triple: The use of repeated abscissae may permit a higher-order A matrix for a general
RK method, as in the case of the classical RK method (26), but they offer no improvement in the maximum
order of an ICE. Also, the definition of N in (12) suggests that it might be advantageous if ||A||∞ is kept
small.
2. Adjusting the stepsize: In Case 2 (and the general Case 3) care should be taken to restrict the ratio ηn of
successive stepsizes by a quantity η of reasonable size. This is because the size of η affects the size of terms
such as Cη , Eη , . . ., and the bound on the values {ên } in Theorem 4.2.
3. Changing m: In a practical code, it could prove advantageous to vary m as the smoothness of the solution
and the nature of the lag changes. If the smoothness of F (t, u1 , u2 ) or u(t) is limited, the order of the scheme
cannot be increased upto q ∗ by increasing m. However, if the positions {t∗j } of derivative discontinuities are
tracked along with estimates of the smoothness of u(t), the order of the scheme can be monitored and the
accuracy hopefully controlled by adjusting the value of m.
4. Stability: ICEs are explicit methods, thus they cannot be “highly stable”. When applied to the ODE
stability equation y (t) = λy(t) with λ ∈ C, the stability region of the m-th ICE tends to a semi-circle in the
left-half λH-plane with radius 1/ρ(B) as m → ∞ (see [1]).
18