Variable Step Marching Method
(and PDEs?)
Thomas Shores
April 8, 2016
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
We’ve seen issues of stability pop up in preceding presentations.
Let’s pin a name on a particular type of stability which revolves
around a specific absurdly simple model IVP
′
y = λy , y (0) = 1.
The solution is equally absurdly simple: y (t) = e λt . OK, if
ℜ (λ) < 0 we expect iterates of a marching method to converge to
zero. Let’s solve this ODE numerically via Euler, Trapezoidal and
pencil and paper.
Euler, stepsize h: yn+1 = yn + hλyn , so
yn+1 = yn (1 + hλ) = y0 (1 + hλ)n+1
Trapezoidal, stepsize h: yn+1 = yn + h2 (λyn + λyn+1 ), so
!
!
"
"n+1
1 + 12 hλ
1 + 12 hλ
= y0
yn+1 = yn
1 − 12 hλ
1 − 12 hλ
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
We’ve seen issues of stability pop up in preceding presentations.
Let’s pin a name on a particular type of stability which revolves
around a specific absurdly simple model IVP
′
y = λy , y (0) = 1.
The solution is equally absurdly simple: y (t) = e λt . OK, if
ℜ (λ) < 0 we expect iterates of a marching method to converge to
zero. Let’s solve this ODE numerically via Euler, Trapezoidal and
pencil and paper.
Euler, stepsize h: yn+1 = yn + hλyn , so
yn+1 = yn (1 + hλ) = y0 (1 + hλ)n+1
Trapezoidal, stepsize h: yn+1 = yn + h2 (λyn + λyn+1 ), so
!
!
"
"n+1
1 + 12 hλ
1 + 12 hλ
= y0
yn+1 = yn
1 − 12 hλ
1 − 12 hλ
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
Conclusions:
Clearly Euler iterates converge to zero only if |1 + z| < 1,
where z = hλ is a complex number.
Exercise: Trapezoidal iterates converge to zero for all z with
ℜ (z) < 0.
More generally, define the linear stability region of a marching
method applied to the model problem to be the locus of
z = hλ for which limn→∞ yn = 0. A method is A-stable if its
stability region includes the entire left half-plane of complex
numbers.
Fact: No explicit multistep (this includes RK) method can be
A-stable.
Fact: No multistep method of order greater than 2 can be
A-stable.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
Conclusions:
Clearly Euler iterates converge to zero only if |1 + z| < 1,
where z = hλ is a complex number.
Exercise: Trapezoidal iterates converge to zero for all z with
ℜ (z) < 0.
More generally, define the linear stability region of a marching
method applied to the model problem to be the locus of
z = hλ for which limn→∞ yn = 0. A method is A-stable if its
stability region includes the entire left half-plane of complex
numbers.
Fact: No explicit multistep (this includes RK) method can be
A-stable.
Fact: No multistep method of order greater than 2 can be
A-stable.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
Conclusions:
Clearly Euler iterates converge to zero only if |1 + z| < 1,
where z = hλ is a complex number.
Exercise: Trapezoidal iterates converge to zero for all z with
ℜ (z) < 0.
More generally, define the linear stability region of a marching
method applied to the model problem to be the locus of
z = hλ for which limn→∞ yn = 0. A method is A-stable if its
stability region includes the entire left half-plane of complex
numbers.
Fact: No explicit multistep (this includes RK) method can be
A-stable.
Fact: No multistep method of order greater than 2 can be
A-stable.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
Conclusions:
Clearly Euler iterates converge to zero only if |1 + z| < 1,
where z = hλ is a complex number.
Exercise: Trapezoidal iterates converge to zero for all z with
ℜ (z) < 0.
More generally, define the linear stability region of a marching
method applied to the model problem to be the locus of
z = hλ for which limn→∞ yn = 0. A method is A-stable if its
stability region includes the entire left half-plane of complex
numbers.
Fact: No explicit multistep (this includes RK) method can be
A-stable.
Fact: No multistep method of order greater than 2 can be
A-stable.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
Conclusions:
Clearly Euler iterates converge to zero only if |1 + z| < 1,
where z = hλ is a complex number.
Exercise: Trapezoidal iterates converge to zero for all z with
ℜ (z) < 0.
More generally, define the linear stability region of a marching
method applied to the model problem to be the locus of
z = hλ for which limn→∞ yn = 0. A method is A-stable if its
stability region includes the entire left half-plane of complex
numbers.
Fact: No explicit multistep (this includes RK) method can be
A-stable.
Fact: No multistep method of order greater than 2 can be
A-stable.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
This is not to say that every explicit multistep is severely limited in
the face of stability. Some have larger stability regions than, say,
Euler. The problem is with a choice of a fixed time step h. Enter
variable step methods.
TEXT: A First Course in the Numerical Analysis of Differential
Equations, Arieh Iserles, Cambridge University Press, 1996.
What is the point of variable step and how does it get done?
Dynamic step size choice could be less expensive than
choosing a fixed step size that accommodates the worst part
of the problem as well as the easiest part of the problem.
The key requirement for a variable step method is a way of
estimating (not calculating, that would be an oxymoron!) error
resulting from a given step.
In addition, user must select maximum and minimum tolerable
errors per unit stepsize, emax and emin , in addition to a
minimum tolerable stepsize hmin .
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
This is not to say that every explicit multistep is severely limited in
the face of stability. Some have larger stability regions than, say,
Euler. The problem is with a choice of a fixed time step h. Enter
variable step methods.
TEXT: A First Course in the Numerical Analysis of Differential
Equations, Arieh Iserles, Cambridge University Press, 1996.
What is the point of variable step and how does it get done?
Dynamic step size choice could be less expensive than
choosing a fixed step size that accommodates the worst part
of the problem as well as the easiest part of the problem.
The key requirement for a variable step method is a way of
estimating (not calculating, that would be an oxymoron!) error
resulting from a given step.
In addition, user must select maximum and minimum tolerable
errors per unit stepsize, emax and emin , in addition to a
minimum tolerable stepsize hmin .
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
This is not to say that every explicit multistep is severely limited in
the face of stability. Some have larger stability regions than, say,
Euler. The problem is with a choice of a fixed time step h. Enter
variable step methods.
TEXT: A First Course in the Numerical Analysis of Differential
Equations, Arieh Iserles, Cambridge University Press, 1996.
What is the point of variable step and how does it get done?
Dynamic step size choice could be less expensive than
choosing a fixed step size that accommodates the worst part
of the problem as well as the easiest part of the problem.
The key requirement for a variable step method is a way of
estimating (not calculating, that would be an oxymoron!) error
resulting from a given step.
In addition, user must select maximum and minimum tolerable
errors per unit stepsize, emax and emin , in addition to a
minimum tolerable stepsize hmin .
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
The key question to be answered here is how we can estimate the
error without knowing the solution. Runge Kutta methods, both
implicit (IRK) or explicit (ERK), offer a nice solution:
Starting at yn , which we assume to be acceptible, and using an
RK method of order p, we have
yn+1 = yn + chp+1 + O(hp+2 )
where the coefficient c is independent of h
Now any method of order p + 1 would yield
ỹ n+1 = yn + O(hp+2 ).
Ignore the higher order terms and subtract to obtain an
estimate of the local error κ to order p + 1, namely
κ ≈ ∥yn+1 − ỹ n+1 ∥
which is exactly what we need.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
The key question to be answered here is how we can estimate the
error without knowing the solution. Runge Kutta methods, both
implicit (IRK) or explicit (ERK), offer a nice solution:
Starting at yn , which we assume to be acceptible, and using an
RK method of order p, we have
yn+1 = yn + chp+1 + O(hp+2 )
where the coefficient c is independent of h
Now any method of order p + 1 would yield
ỹ n+1 = yn + O(hp+2 ).
Ignore the higher order terms and subtract to obtain an
estimate of the local error κ to order p + 1, namely
κ ≈ ∥yn+1 − ỹ n+1 ∥
which is exactly what we need.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Detour into ODEs
The key question to be answered here is how we can estimate the
error without knowing the solution. Runge Kutta methods, both
implicit (IRK) or explicit (ERK), offer a nice solution:
Starting at yn , which we assume to be acceptible, and using an
RK method of order p, we have
yn+1 = yn + chp+1 + O(hp+2 )
where the coefficient c is independent of h
Now any method of order p + 1 would yield
ỹ n+1 = yn + O(hp+2 ).
Ignore the higher order terms and subtract to obtain an
estimate of the local error κ to order p + 1, namely
κ ≈ ∥yn+1 − ỹ n+1 ∥
which is exactly what we need.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Embedded RK method
Recall the Butcher tableau for an RK method of order p:
c
A
bT
Here A = [aij ] is p × p and vectors c = [ci ], b = [bi ] are p × 1. The
resulting method is given by
f˜i = f (tn + ci h, yn + h
p
#
aij f˜j ), i = 1, . . . p
j=1
and
yn+1 = yn + h
p
#
bi f˜i .
i =1
In the case of an explicit method, A is strictly lower triangular, so
the sum in the equation defining the f˜i has an upper limit of i − 1
with the understanding that the empty sum results in
f˜1 = f (tn , yn ).
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Simple Example
Consider these ERK tableaux
2
3
0
0
0
2
3
1
4
3
4
2
3
2
3
2
3
0
1
4
2
3
3
8
2
3
2
3
3
8
2
3
0
1
4
1
4
2
3
3
4
3
8
3
8
The first tableau is embedded in the second and they are combined
in the third. The resulting formulas for both the second (yn+1 ) and
third y$n+1 order:
f˜1 =
f (tn , yn )
%
&
f˜2 =
f tn + 23 h, yn + 23 hf˜1
%
&
f˜3 =
f tn + 23 h, yn + 23 hf˜2
%
&
yn+1 =
yn + h 41 f˜1 + 43 f˜2
&
%
y$n+1 = yn + h 41 f˜1 + 38 f˜2 + 83 f˜3
Thomas Shores
Variable Step Marching Method (and PDEs?)
Conclusions
The resulting error estimate for the second order method is
'
3 '
'˜ ˜'
κ = ∥yn+1 − ỹ n+1 ∥ = h 'f2 − f3 ' .
8
This error estimate only required one extra function
evaluation.Embedded RK methods can significantly reduce
function evaluations and achieve acceptible results.
However variable step methods are not a panacea for all
problems, and in particular, not necessarily the best solution
for stiffness issues.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Conclusions
The resulting error estimate for the second order method is
'
3 '
'˜ ˜'
κ = ∥yn+1 − ỹ n+1 ∥ = h 'f2 − f3 ' .
8
This error estimate only required one extra function
evaluation.Embedded RK methods can significantly reduce
function evaluations and achieve acceptible results.
However variable step methods are not a panacea for all
problems, and in particular, not necessarily the best solution
for stiffness issues.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Conclusions
The resulting error estimate for the second order method is
'
3 '
'˜ ˜'
κ = ∥yn+1 − ỹ n+1 ∥ = h 'f2 − f3 ' .
8
This error estimate only required one extra function
evaluation.Embedded RK methods can significantly reduce
function evaluations and achieve acceptible results.
However variable step methods are not a panacea for all
problems, and in particular, not necessarily the best solution
for stiffness issues.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Simple Examples
Consult the flow chart for an outline of algorithm construction.
Let’s finish the detour with a few examples, Exercise 1 of this NA
seminar and its solution
′
y = −ty 2 , y (0) = 1, 0 ≤ t ≤ 6,
(
)
y (t) = 2/ t 2 + 2 .
and the Curtiss-Hirschfelder equation, along with its solution:
′
y = −50 (1 − cos (t)) y , y (0) = 1, 0 ≤ t ≤ 6,
2500
50
1 −50t
cos (t) +
sin (t) +
e
.
2501
2501
2501
To test these examples, examine scripts exerVarRHS.m,
exerVarSoln.m and exerVarTest.m, and run the script VarStepTest.
y (t) =
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Solution Looking for a Problem
Without further ado, here is the solution:
u (x, t) = sin (x) e t cos(x) , 0 ≤ x ≤ π, t ≥ 0.
To see some profiles running out to t = 6, execute the file
plotMarchSoln8.m. So what kind of problem are we looking for? It
should:
Give an opportunity to explore stiffness issues.
Allow for diffusion, so that some of the methods of this
seminar are applicable.
Be sufficiently nontrivial that we don’t get bored solving it, but
not so hard that we can’t.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Solution Looking for a Problem
Without further ado, here is the solution:
u (x, t) = sin (x) e t cos(x) , 0 ≤ x ≤ π, t ≥ 0.
To see some profiles running out to t = 6, execute the file
plotMarchSoln8.m. So what kind of problem are we looking for? It
should:
Give an opportunity to explore stiffness issues.
Allow for diffusion, so that some of the methods of this
seminar are applicable.
Be sufficiently nontrivial that we don’t get bored solving it, but
not so hard that we can’t.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Solution Looking for a Problem
Without further ado, here is the solution:
u (x, t) = sin (x) e t cos(x) , 0 ≤ x ≤ π, t ≥ 0.
To see some profiles running out to t = 6, execute the file
plotMarchSoln8.m. So what kind of problem are we looking for? It
should:
Give an opportunity to explore stiffness issues.
Allow for diffusion, so that some of the methods of this
seminar are applicable.
Be sufficiently nontrivial that we don’t get bored solving it, but
not so hard that we can’t.
Thomas Shores
Variable Step Marching Method (and PDEs?)
A Problem Looking for a Solution
And here it is:
ut = Duxx + c (x, t) u
This is a simple diffusion problem with a not so simple linear
transport term whose coefficient is time and space dependent:
(
)
c (x, t) = cos (x) + D 1 + 3t cos (x) − t 2 sin2 (x)
Since our solution doesn’t satisfy a pure diffusion equation,
some sort of additional term like linear transport or forcing
term is mandatory, and it should give us the result of piping
our solution through the linear operator ut − Duxx . Of course
we’ll have to supply initial and boundary conditions as well.
But first, let’s pause and have a look at a few profiles of our
solution, to see what’s so interesting about it. Here are sample
parameters that we might use: T = 6; numProfiles = 10. It’s
automated in plotMarchSoln8.m. Grab a copy of
VarStepLecture from DropBox, fire up Matlab or Octave and
cd to that directory. Now run it. Interesting...
Thomas Shores
Variable Step Marching Method (and PDEs?)
Marching Methods of Solution for the Model PDE
If you plan to program along, start by opening the files
marchRHS8.m and marchTest8.m and move them to a corner.
Since we’re comparing methods as well as solutions, let’s
do the simplest explicit method (Euler) followed by the
more sophisticated explicit method (ERK4) and the
variable step method (ERK23) on each run.
The parameters: Following professor Larios’ lead, Let’s
select a small diffusive influence, D = 0.006, and we will
run the time out to T = 6 (examine the solution and you
see that quite a large change occurs in that time interval.)
Try N = 64, numSteps = 100, numProfiles = 8. Double
numSteps. Then double N. Comments?
Could diffusion be the problem? Let’s eliminate it entirely.
Reset D to 0.0 in the file marchRHS8.
Take note of the function evaluations in our experiments.
It gets expensive to improve things.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Marching Methods of Solution for the Model PDE
If you plan to program along, start by opening the files
marchRHS8.m and marchTest8.m and move them to a corner.
Since we’re comparing methods as well as solutions, let’s
do the simplest explicit method (Euler) followed by the
more sophisticated explicit method (ERK4) and the
variable step method (ERK23) on each run.
The parameters: Following professor Larios’ lead, Let’s
select a small diffusive influence, D = 0.006, and we will
run the time out to T = 6 (examine the solution and you
see that quite a large change occurs in that time interval.)
Try N = 64, numSteps = 100, numProfiles = 8. Double
numSteps. Then double N. Comments?
Could diffusion be the problem? Let’s eliminate it entirely.
Reset D to 0.0 in the file marchRHS8.
Take note of the function evaluations in our experiments.
It gets expensive to improve things.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Marching Methods of Solution for the Model PDE
If you plan to program along, start by opening the files
marchRHS8.m and marchTest8.m and move them to a corner.
Since we’re comparing methods as well as solutions, let’s
do the simplest explicit method (Euler) followed by the
more sophisticated explicit method (ERK4) and the
variable step method (ERK23) on each run.
The parameters: Following professor Larios’ lead, Let’s
select a small diffusive influence, D = 0.006, and we will
run the time out to T = 6 (examine the solution and you
see that quite a large change occurs in that time interval.)
Try N = 64, numSteps = 100, numProfiles = 8. Double
numSteps. Then double N. Comments?
Could diffusion be the problem? Let’s eliminate it entirely.
Reset D to 0.0 in the file marchRHS8.
Take note of the function evaluations in our experiments.
It gets expensive to improve things.
Thomas Shores
Variable Step Marching Method (and PDEs?)
Marching Methods of Solution for the Model PDE
If you plan to program along, start by opening the files
marchRHS8.m and marchTest8.m and move them to a corner.
Since we’re comparing methods as well as solutions, let’s
do the simplest explicit method (Euler) followed by the
more sophisticated explicit method (ERK4) and the
variable step method (ERK23) on each run.
The parameters: Following professor Larios’ lead, Let’s
select a small diffusive influence, D = 0.006, and we will
run the time out to T = 6 (examine the solution and you
see that quite a large change occurs in that time interval.)
Try N = 64, numSteps = 100, numProfiles = 8. Double
numSteps. Then double N. Comments?
Could diffusion be the problem? Let’s eliminate it entirely.
Reset D to 0.0 in the file marchRHS8.
Take note of the function evaluations in our experiments.
It gets expensive to improve things.
Thomas Shores
Variable Step Marching Method (and PDEs?)