MA254
Theory of ODEs
Christoph Ortner
[email protected]
Zeeman Building, University of Warwick, Coventry CV4 7AL, UK
April 27, 2014
1
2
Contents
1 Introduction
1.1 Literature & Acknowledgements . . . . . . . . . . . .
1.2 Prerequisites . . . . . . . . . . . . . . . . . . . . . . .
1.2.1 RN . . . . . . . . . . . . . . . . . . . . . . . .
1.2.2 Review of matrix norms . . . . . . . . . . . .
1.2.3 Spectral decomposition . . . . . . . . . . . . .
1.2.4 Convexity . . . . . . . . . . . . . . . . . . . .
1.2.5 Continuous functions, uniform convergence . .
1.3 Crash Course in Differentiation . . . . . . . . . . . .
1.3.1 Motivation and definition . . . . . . . . . . .
1.3.2 Jacobi matrix and gradient . . . . . . . . . . .
1.3.3 Some useful properties . . . . . . . . . . . . .
1.3.4 Differentiation and integration of f : R → RN
1.3.5 Higher derivatives . . . . . . . . . . . . . . . .
1.4 Definition of ODEs and their solutions . . . . . . . .
2 Motivating Examples
2.1 Newton’s equations . . . . .
2.1.1 Hamiltonian systems
2.1.2 Molecular dynamics .
2.2 Damping, Gradient Flows .
2.3 Population Dynamics . . . .
2.4 Other examples . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5
6
6
7
7
8
9
9
10
10
12
12
13
14
15
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
15
15
17
18
18
19
19
3 Systems of Linear ODEs
3.1 Revision of the 1D theory . .
3.2 Examples . . . . . . . . . . .
3.3 The exponential of a matrix .
3.4 Duhamel’s Formula . . . . . .
3.5 Uniqueness via energy method
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
21
21
22
23
26
28
.
.
.
.
.
.
.
.
29
29
32
32
33
36
37
39
40
4 Picard’s Theorem
4.1 Quantifying continuity . . . . . . . . . . . . . .
4.2 Picard’s theorem: local existence and uniqueness
4.2.1 IVPs as fixed point problems . . . . . .
4.2.2 Picard’s Theorem . . . . . . . . . . . . .
4.2.3 Well-Posedness . . . . . . . . . . . . . .
4.3 Maximal existence interval and blowup . . . . .
4.3.1 Global existence . . . . . . . . . . . . . .
4.4 Linear IVPs again . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5 Stability
41
5.1 Gronwall’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.2 Well-posedness of the general IVP . . . . . . . . . . . . . . . . . . . . 45
5.3 Regular perturbation theory . . . . . . . . . . . . . . . . . . . . . . . 46
3
6 Numerical Simulation using Euler’s Method
6.1 Construction of Euler’s method and some experiments
6.1.1 Convergence for linear systems . . . . . . . . . .
6.2 Convergence Result . . . . . . . . . . . . . . . . . . . .
6.3 Long-time behaviour . . . . . . . . . . . . . . . . . . .
6.3.1 Region of Stability . . . . . . . . . . . . . . . .
6.3.2 Energy identity for Hamiltonian systems . . . .
6.4 Peano’s theorem . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
49
49
52
53
55
56
56
58
7 Error Growth in (Damped) Hamiltonian Systems
58
8 Autonomous Systems
8.1 Maximal solutions . . . . . . . . . . . . . .
8.2 Remark on dynamical systems . . . . . . .
8.3 Stable equilibria . . . . . . . . . . . . . . .
8.4 The method of Lyapunov . . . . . . . . . .
8.4.1 Basin of attraction . . . . . . . . .
8.5 Stability in Damped Hamiltonian Systems
8.6 Examples of planar autonomous systems .
58
59
61
62
66
69
69
71
4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
Introduction
Ordinary differential equations arise in virtually all sciences, not only physics, chemistry and engineering, but also economics, ecology or social sciences. They generally
are (part of) a model for some physical process, describing the change of an object
in time or space (or both, these would be partial differential equations).
This lecture course focuses on the mathematical structure of differential equations rather than applications and mathematical modeling. Nevertheless, we will try
to remember throughout that the purpose of studying DEs is to eventually use this
to better understand mathematical models and make “real-world predictions.”
Given a mathematical model, phrased in terms of a differential equation, we
first want to know whether it is well-posed: Does a solution exist? In what sense?
If a solution does exist, is it unique? If we know the model parameters (data) only
approximately, can we still rely on the solution of the DE? If a DE is not well-posed
then we need not waste time trying to find a solution: either it does not exist, or
even if it does exist, then it is likely useless to make a prediction.
Next, once we know that a model is well-posed we want to know how to compute
its solution. The current trend in applied mathematics strongly points towards numerical approximations, which provide a straightforward way for a “user” to study
a differential equation. Hence, we will spend some time studying Euler’s method,
the most basic discretisation of initial value problems. We will also touch upon perturbation theory, which provides an analytical approach to approximately solving
DEs. Perturbation techniques can also be used for a qualitative study of solutions
to DEs.
We will then introduce some further techniques for the qualitative study of ODEs:
stability of equilibria and of trajectories, phase plane analysis, equilibria and local
phase portraits, limit cycles, attractors. This section could form the basis for further
modules on dynamical systems.
The theory of DEs is heavily technique-based. Therefore, this module will necessarily be focused around analytical techniques and how they are applied. Examples
are: “energy methods” for uniqueness and stability, weak solutions, fixed point theorems, compactness.
[[ NOTES ON 2014 EXAM: Throughout these notes there are paragraphs of text
enclosed in square brackets (as the present paragraph) and in the electronic version
also printed in red. In these, I clarify to what extent the content of the notes is examinable. Any section or result without such a comment is examinable. Any exercise
or example (or part thereof ) that is marked with ’*’ or ’**’ in this version is not
examinable. Any part of an exercise, example, result or proof marked with (NE) ,
is not examinable.
To make the meaning of ‘not examinable’ precise: it simply means that this material is not considered as part of ‘bookwork’ in the exam. However, unless stated
otherwise, similar material may be asked in the exam, for example as a simple special
case and/or with some simplifying assumptions.
The comments in this manuscript apply only to the 2014 exam.
Undoubtedly there are still mistakes in the manuscript. If you find a mistake,
please email me at [email protected] and I will post a list of corrections
5
on the module website
http://www2.warwick.ac.uk/fac/sci/maths/undergrad/ughandbook/resources/ma254.
I have also completely removed some sections, that I did not cover in 2013/14.
On the problem sheets:
P1: On Q3(ii), despite the (*), this item is useful to look at.
P2: Something of the form Q3(a) will not be in the exam; Q3(d) is not examinable
P3: Q5, Q6 are not examinable
P4: Q1 is only examinable with C = I, Q 6 is not examinable
P5: Q6 is not examinable
P7: Q3 is examinable, Q4 is not.
P8: Q3(c) is not examinable, but it is a fun example, so you should look at it
anyhow.
P9: Q3 (ii) is examinable only for C = I; Q4 is not examinable.
]]
1.1
Literature & Acknowledgements
As opposed to theory of PDEs, the basic theory of ODEs is now more or less settled
and therefore the syllabus for a module on “Theory of ODEs” is largely standardised.
Nevertheless, there is a little space to bring in new ideas. In compiling these lecture
notes, I tried to emphasize constructive ideas and ideas that would also be useful for a
study of PDEs later on. As a result there is a strong focus on analysis techniques, and
a more substantial component on Euler’s method than is typical in an introductory
course.
I have let a recent textbook by Gerald Teschl [Tes12] heavily inspire various
sections of these lecture notes. Although this book is published as an AMS graduate
text, large portions of it are well suited for introductory undergraduate module. The
book is available online [Tes12] and I strongly recommend it for further reading.
I have moreover drawn ideas from the lectures notes of Inge Troch (my own
ODE teacher) and from the lecture notes Analysis III by Barbara Niethammer. In
particular parts of the section on autonomous systems (up to Lyapunov’s method)
is based on the latter.
Many of the exercises compiled in the notes are drawn from these sources, sometimes without much modification.
1.2
Prerequisites
[[ The material of this section is understood as background and can/should be used
in the exam without proofs. ]]
We briefly review some fact from analysis and linear algebra, and in particular
convince ourselves that some concepts such as continuity, differentiability and uniform convergence of functions f : (a, b) → R carry over without changes to functions
f : (a, b) → RN . (upon applying these concepts separately to each element)
Note also that all results for vector values functions f : (a, b) → RN carry over to
matrix-valued functions f : (a, b) → RM ×N by simply identifying RM ×N with RM N .
6
1.2.1
RN
The majority of the analysis in this module is for general N -dimensional systems
of ODEs. We will use the structure of RN as a vector space, supplied with the
Euclidean norm and inner product
T
x · y := x y =
N
X
xi y i ,
and |x| :=
√
x·x
i=1
Key inequalities that we will use on a regular basis are the triangle inequality
for x, y ∈ RN ,
|x + y| ≤ |x| + |y|
(1)
the Cauchy–Schwarz inequality
for x, yıRN ,
(2)
for a, b ∈ R,
for a, b ∈ R, ε > 0.
(3)
(4)
|x · y| ≤ |x||y|
and Cauchy’s inequalities,
ab ≤ 12 a2 + 21 b2
1 2
b
ab ≤ 2ε a2 + 2ε
Exercise 1.1. (i) Prove (3) and (4). (ii) Use (3) to prove (2). (iii) Use (2) to prove
(1).
1.2.2
Review of matrix norms
Given a matrix A ∈ RN ×M , we define its operator norm
kAk := sup |Ah| =
h∈RN
|h|=1
|Ah|
.
h∈RN \{0} |h|
sup
Proposition 1.1. (a) k · k is a norm on RN ×M :
(i) kAk ≥ 0
(ii) kAk = 0 if and only if A = 0
(iii) kA + λBk ≤ kAk + |λ|kBk
(b) k · k is an operator norm:
(iv) |Ah| ≤ kAk|h|
(v) kABk ≤ kAk kBk
∀A ∈ RN ×M , h ∈ RM ,
and
∀A ∈ RN ×M , B ∈ RM ×P .
7
(5)
Exercise 1.2. Prove Proposition 1.1.
In addition to the canonical operator norm k · k we also define the Frobenius
norm
1/2
X
N X
M
2
for A ∈ RN ×M .
kAkF :=
|Aij |
i=1 j=1
Then we have the following relationship between k · k and k · kF . It is clear from its
definition that k · kF is a norm on RN ×M . In addition it is also an operator norm:
Lemma 1.2. The Frobenius norm satisfies the following properties:
(i) |Ah| ≤ kAkF |h| for all h ∈ RN .
(ii) kABkF ≤ kAkF kBkF for all A ∈ RN ×M , B ∈ RM ×P .
(iii) kAk ≤ kAkF for all A ∈ RM ×N .
Exercise 1.3. Prove Lemma 1.2.1
p
Exercise 1.4. (i) Show that kAkF = trace(AT A).
(ii) Show that (A, B) 7→ trace(AT B) is an inner product on RM ×N (in fact, it is
the standard Euclidean inner product).
1.2.3
Spectral decomposition
Let A ∈ RN ×N be symmetric, then there exists an orthonormal basis (ONB)
{v1 , . . . , vN } of RN and λi ∈ R, i = 1, . . . , N
Avi = λi vi .
The λi are called the eigenvalues of A and vi the associated eigenvectors.
Remark 1.3. In general, a matrix A ∈ CN ×N is unitarily diagonalisable if and
only if it is normal: A∗ A = AA∗ , where A∗ = (Ā)T .
Exercise 1.5. (i) Prove the spectral decomposition theorem for symmetric matrices.
(ii**) Prove the spectral decoposition theorem for normal matrices.
We call a matrix A ∈ RN ×N diagonalisable if there exists a basis {v1 , . . . , vN } of
CN (not necessarily ONB) and eigenvalues λi ∈ C such that Avi = λi vi .
We call a matrix A ∈ RN ×N positive definite if xT Ax > 0 for all x ∈ RN \ {0}.
A symmetric and positive matrix is simply called spd.
1
HINT: Show (i), using the Cauchy–Schwarz inequality on RN . (ii), (iii) are corollaries of (i).
8
Not all matrices are diagonalisable, but all matrices have a Jordan normal form.
That is, there exists a basis {v1 , . . . , vN } of CN such that, writing V = (v1 , . . . , vN )
we have


λ
1
i




J1 0 · · ·
λi 1






−1
.
.
0
J
0
·
·
·
.
.
2
V AV = 
.
 where Ji = 
.
.


..
..

.
. 0
λi 1 
λi
The Ji are called Jordan blocks.
Exercise 1.6. Prove that a matrix C ∈ RN ×N is spd if and only if it is symmetric
and all its eigenvalues are positive.
1.2.4
Convexity
A set U ⊂ RN is convex if, for x, y ∈ U, λ ∈ [0, 1], the point (1 − λ)x + λy ∈ U as
well.
If U is convex and f : U → R, then f is convex if, for x, y ∈ U, λ ∈ [0, 1], we
have
f (1 − λ)x + λy ≤ (1 − λ)f (x) + λf (y).
(6)
Exercise 1.7. (i) Prove that the functions f (x) = |x|n , n ∈ N, f (x) =
exp(x), f (x) = min(0, x) are convex.
(ii) Prove that f (x) = −x2 , f (x) = cos(x), f (x) = max(0, x) are non-convex.
(iii) Prove that, if C ∈ RN ×N and b ∈ RN , then f (x) = 12 xT Cx − bT x is convex.
1.2.5
Continuous functions, uniform convergence
Recall from the introductory analysis modules the definitions of continuous functions
and of uniform convergence. Here, we define the spaces, for an interval A ⊂ R,
C(A) := f : A → Rf is continuous on A ,
C(A; RN ) ≡ C(A)N := f : A → RN fj ∈ C(A) for j = 1, . . . , N .
If A is compact (A = [a, b] for a, b ∈ R), then C(A; RN ) is complete when equipped
with the sup-norm
kf k∞,A := sup |f (x)|.
x∈A
We will often write kf k∞ if it is clear over which set the supremum is taken. Note
also that A need not be compact in the definition of k · k∞,A .
Recall that C(A; RN ) being complete means that each Cauchy sequence has
a limit in that space. A sequence (fj )j∈N ⊂ C(A; RN ) is a Cauchy sequence if
kfj − fk k∞ → 0 as j, k → ∞.
9
Recall that a sequence (fj )j∈N , fj : A → R is said to converge uniformly to a
function f if, for all > 0 there exists j0 = j0 () such that |fj (x) − f (x)| ≤ for all
j ≥ j0 and for all x ∈ A.
For vector-valued functions the extension is obvious: If f, fj : A → RN then
fj → f uniformly on A if, for all > 0, there exists j0 such that |fj (x) − f (x)| ≤ for all j ≥ j0 and for all x ∈ A. But now, | · | means the Euclidean norm.
(j)
(j)
is the i-th
Exercise 1.8. Let f (j) , f : A → RN , with f (j) = (fi )N
i=1 , i.e., fi
(j)
(j)
(j)
component of f . Show that f → f uniformly if and only if fi → fi uniformly
for all i = 1, . . . , N .
We shall rarely use the -criterion for uniform convergence, but instead work
with the max-norm.
Exercise 1.9. (i) Let A ∈ R be an interval. Prove (or recall) that for f, (fj )j∈N ⊂
C(A), fj → f uniformly on A if and only if kfj − f k∞,A → 0.
(ii) Extend the statement and argument to C(A; RN ).
(iii) For A compact, deduce the statement that the space C(A; RN ), equipped
with the norm k · k∞,A is complete. (You may use results from ANALYSIS without
proof.)
(iv)
M-Test) Let A ⊂ R be an P
interval and f (j) ∈ RN ×N such
P∞(Weierstrass
(j)
converges uniformly on
that j=1 kf (j) k∞ < +∞. Deduce that the series ∞
j=1 f
A.
1.3
Crash Course in Differentiation
[[ The material of this section is understood as background and can/should be used
in the exam without proofs. ]]
This section is intended as a rapid introduction into and reference for differentiation of multi-variate and/or vector valued functions as required in the remainder
of the lecture notes.
1.3.1
Motivation and definition
Univariate: Recall first the definition of differentiability in 1D. Let f : (a, b) → R
and t ∈ (a, b). Then we say that f is differentiable at t if there exists T ∈ R such
that
f (t + h) = f (t) + T h + o(h),
where o(h)/h → 0 as h → 0. We write T = f 0 (t). In words: f can be “well approximated” in a small neighbourhood of t by an affine function.
For vector valued functions with scalar argument, nothing changes. f : (a, b) →
N
R is differentiable at t if there exists T ∈ RN such that
f (t + h) = f (t) + T h + o(h),
and we call T = f˙(t). Clearly this is equivalent to saying that fi is differentiable at
t for all i = 1, . . . , N .
10
We define spaces of continuously differentiable functions. For an interval A ⊂ R
and j ≥ 1,
C j (A; RN ) = f ∈ C(A; RN ) f is j times differentiable in int(A)
and f (k) ∈ C(A; RN ) for k = 0, . . . , j .
(Note that if A contains one of its endpoints, then by f (k) ∈ C(A; RN ) we mean that
the derivative f (k) first defined on int(A) can be continuously extended to A. This is
subtle point that can in fact be ignored for the purpose of these lecture notes.)
Multi-variate: With this idea of differentiation in mind,
functions with vectorial arguments is immediate. Let U ⊂
f : U → RM . We say that f is (Fréchet) differentiable at x
matrix T ∈ RM ×N (recall that every linear function from RN
h 7→ T h for some T ∈ RM ×N ) such that
f (x + h) = f (x) + T h + o(|h|),
the generalisation to
RN be open and let
∈ U if there exists a
to RM is of the form
as h → 0,
and we write T = ∂f (x).
There is also a second notion of differentiability: We say that f is Gateaux
differentiable at x ∈ U if there exists T ∈ RM ×N such that, for all h ∈ RN ,
f (x + sh) = f (x) + sT h + o(s)
as s → 0.
We still write T = ∂f (x). The subtle difference of letting the perturbation tend to
zero in a straight line, or in an arbitrary fashion makes Gateaux differentiability a
strictly weaker notion of differentiability.
(WARNING: some authors define Gateaux derivative slightly differently. It is
always useful to check what is meant. By contrast Frechet differentiability seems
universally accepted as stated above.)
Exercise 1.10 (Frechet versus Gateaux). (i) Show that if f is Frechet differentiable in x then it is continuous in x as well as Gateaux differentiable in x.
(ii) Conversely, consider f : R2 → R,
(
2
x2 y
, (x, y) 6= (0, 0),
4 +y 2
x
f (x, y) =
0,
(x, y) = (0, 0),
and show that f is Gateaux differentiable at (0, 0) but not even continuous.
(iii) In other respects, Gateaux and Fréchet differentiability are not so far from
one another: If f is Gateaux differentiable in U and ∂f is continuous at x ∈ U ,
then f is Fréchet differentiable at x. (HINT: If you don’t see this immediately, do
Exercise 1.11 first, from which this results follows.)
REMINDER: If we say f is differentiable, then we mean the Frechet sense!
Definition 1.4. If U ⊂ RN , with int(U ) 6= ∅, then we say that f ∈ C 1 (U ; RM ) if
f : U → RM is differentiable at each point x ∈ int(U ) and ∂f ∈ C(U ; RM ×N ).
REMARK: From Exercise 1.10(iii) it follows immediately that f ∈ C 1 (U ; RM )
if and only if f is Gateaux differentiable at each point in U and ∂U ∈ C(U ; RM ×N ),
that is, the definition of C 1 is in fact independent of which of the two notions of
differentiability we employ.
11
1.3.2
Jacobi matrix and gradient
To actually compute the derivative ∂f (x), we simply test the definition
f (x + sh) = f (x) + s∂f (x)h + o(s)
with h = ej (ej is a canonical basis vector). Hence, we obtain that
∂fi (x)
fi (x + sej ) − fi (x)
= lim
= ∂f (x)ej i = ∂f (x) ij .
s→0
∂xj
s
That is, ∂f (x) is simply the matrix of partial derivatives, or, Jacobi matrix,
 ∂f1 ∂f1
∂f1 
· · · ∂x
∂x1
∂x2
N
 ∂f2 ∂f2 · · · ∂f2 

∂xN 
∂f (x) =  ∂x. 1 ∂x. 2 .
.
.. 
.
.
.
 .
.
.
. 
∂fM
∂x2
∂fM
∂x1
···
∂fM
∂xN
If M = 1, i.e. f : U ⊂ RN → R and f is (Frechet or Gateaux) differentiable at
x ∈ U then we call
∇f (x) := ∂f (x)T
the gradient of f at x. Thus, in this case we can alternatively write Gateauxdifferentiability in the form
f (x + sh) = f (x) + s∇f (x) · h + o(s),
and similarly for F-differentiability.
[[ removed non-examinable section ]]
Exercise 1.11 (Partial derivatives and differentiability). We conclude
this discussion with a remark on the relationship between partial derivatives and
Frechet differentiability, which extends Exercise 1.10 (iii):
Suppose that f : U ⊂ RN → RM , U open, has well-defined partial derivatives
∂f (x) at each point x ∈ U . (I.e., t 7→ f (x + tei ) is differentiable at t = 0 for all
x ∈ U .) that ∂f : U → RM ×N is continuous at a point x ∈ U . Then f is Frechet
differentiable at x.
1.3.3
Some useful properties
Exercise 1.12. All the usual linearity properties of differentiation hold for multivariate differentiation as well:
If f, g are differentiable at x, then f + λg is differentiable at x for any λ ∈ R. The only other property we need is the chain rule. For scalar arguments, this
reads
df
dg
d
f (g(t)) =
(g(t)) · (t) or (f ◦ g)0 = (f 0 ◦ g)g 0 .
dt
dg
dt
Note also that the statement for scalar-valued functions immediately implies the
same statement for vector-valued functions.
12
The multi-variate version is formally identical,
∂ f (g(x)) = (∂f )(g(x)) ∂g(x), or ∂(f ◦ g) = (∂f ) ◦ g ∂g
(7)
Another, possibly more memorable way to write this is
∂(f (g(x)))
∂f (g) ∂g(x)
=
∂x
∂g g=g(x) ∂x
Proposition 1.5. Let f : U ⊂ RN → RM and g : V ⊂ RK → RN . Suppose that g is
differentiable at x ∈ V and f is differentiable at g(x) ∈ U , then f ◦ g is differentiable
at x and (7) holds.
The same result holds if we replace “differentiable” with “Gateaux differentiable”.
Proof.
f (g(x + h)) = f g(x) + ∂g(x)h + o(|h|)
= f (g(x)) + ∂f (g(x)) ∂g(x)h + o(|h|) + o |∂g(x)h + o(|h|)|
= f (g(x)) + ∂f (g(x)) ∂g(x)h + o(|h|).
The proof for the Gateaux case is analogous.
An important special case is the following: if f ∈ C 1 (U ; RN ), U ⊂ RN open, and
g ∈ C 1 (a, b; U ), then
d
f (g(t)) = ∂f (g(t))ġ(t).
dt
In particular, if N = 1, then
d
f (g(t)) = ∇f (g(t)) · ġ(t).
dt
1.3.4
Differentiation and integration of f : R → RN
We will not be concerned with integrating over RN , but we do need some results on
integrating vector-valued functions defined on an interval.
Let f : [a, b] → RN be continuous, then we define its integral componentwise:
Z
b
Z
f (t) dt =
a
a
b
N
fi (t) dt
.
i=1
Clearly, we get the fundamental theorem of calculus again: if f ∈ C 1 ([a, b]; RN ),
then
Z b
f˙(t) dt = f (b) − f (a).
a
But other results can fail:
Exercise 1.13. The integral mean value theorem is an example of a result that
does not translate (and hence the differential MVT does not either):
13
(i) Vector-valued, scalar argument: Show the integral MVT fails for f : (0, 2π) →
R2 , f (t) = (cos t, sin t)T .
[[ removed non-examinable section ]]
An important consequence of the fundamental theorem are path integrals: If
γ ∈ C 1 ([a, b]; RN ), then the intregal of f along its path is
Z b
f (γ(t))γ̇(t) dt.
a
Exercise 1.14. (i) Show that, if E ∈ C 1 (U ; R) and γ ∈ C 1 ((a, b); RN ), then
d
E(γ(t)) = ∇E(γ(t)) · γ̇(t).
E ◦ γ ∈ C 1 ((a, b); R) and dt
(ii) Therefore, if f (x) = ∇E(x), then the integral
Z b
f (γ(t)) · γ̇(t) dt = E(γ(b)) − E(γ(a))
a
depends only on the end-points but not on the entire path.
[[ removed non-examinable section ]]
1.3.5
Higher derivatives
The notation for higher derivatives can initially be confusing and we will therefore
avoid their use as much as possible. However, there are some simple definitions and
results that we require.
For U ⊂ RN with int(U ) 6= ∅ and for k ≥ 2, we define (recursively)
C k (U ; RM ) := f ∈ C 1 (U ; RM )∂f ∈ C k−1 (U ; RM ×N ) .
A useful result is the following:
Exercise 1.15. Let f ∈ C 2 (U ; RM ) where U ⊂ RN is convex and closed, then
∂f : U → RM ×N is Lipschitz continuous, in the sense that
k∂f (x) − ∂f (y)kF ≤ L|x − y|
for x, y ∈ U.
There is a particularly important case: Let U ⊂ RN be open and let E ∈
C (U ; R), then we define the Hessian matrix
2
∂ E(x)
2
∇ E(x) :=
.
∂xi ∂xj i,j=1,...,N
2
Exercise 1.16. Let E ∈ C 2 (U ), where U ⊂ RN is open. Prove the following
statements:
(i) ∇2 E(x) = ∂∇E(x).
(ii) E(x + h) = E(x) + ∇E(x) · h + 12 hT ∇2 E(x)h + o(|h|2 ).
(iii) If E ∈ C 3 (U ), then E(x+h) = E(x)+∇E(x)·h+ 21 hT ∇2 E(x)h+O(|h|3 ). 14
1.4
Definition of ODEs and their solutions
Definition 1.6 (Definition of an ODE). Let F : R × (RN )n+1 , then an equation
of the form
F (t, x, ẋ, . . . , x(n) ) = 0
(8)
is called an N -dimensional ordinary differential equation (ODE) of order n.
Let I ⊂ R be an open interval, then a function x ∈ C n (I) satisfying
F (t, x(t), ẋ(t), . . . , x(n) (t)) = 0
∀t ∈ I
(9)
is called a solution of (8) on I.
The ODE is said to be linear if F is an affine function of x, ẋ, . . . , x(n) .
The ODE is said to be autonomous if F does not depend on t.
The ODE is said to be explicit if
F (t, x, ẋ, . . . , x(n) ) = x(n) − f (t, x, ẋ, . . . , x(n−1) ),
for some f : R × (RN )n .
Throughout this module we shall only consider explicit ODEs, and even then
primarily first- and at most second-order, hence we can immediately forget this
abstract definition.
2
2.1
Motivating Examples
Newton’s equations
Newton’s second law of motion states that the force acting on a particle equals to
the acceleration times mass of the particle.
Consider a system of N particles that are at positions xi (t) ∈ Rd , i = 1, . . . , N ,
at time t. Let the particle at xi (t) have mass mi . Moreover, suppose that there exist
functions Fi : (Rd )N → Rd , i = 1, . . . , N such that Fi (x1 , . . . , xN ) is the force acting
on xi , then we can state Newton’s equations of motion as
mi ẍi (t) = Fi x1 (t), . . . , xN (t) for i = 1, . . . , N, t ∈ R.
More typically, we will define x = (x1 , . . . , xN ), M = diag(m1 , . . . , mN ) and F =
(F1 , . . . , FN ), and write
M ẍ(t) = F (x(t)),
(10)
which we will also write in the form
M ẋ(t) = p(t)
ṗ(t) = F (x(t)).
One of the first ODEs one normally encounters is of this form: standing on the
surface of the earth, consider a coordinate system where x3 is normal to the surface,
15
then the gravitational force acting on a particle with x3 > 0 is (0, 0, −mg), which
leads to
 
0

mẍ(t) = −mg 0  ,
(11)
1
or upon replacing x3 = R + h, where R is the earth’s radius,
ḧ = −g,
and we easily deduce that
g
h(t) = h(0) + ḣ(0)t − t2 , or
2 
0
g
x(t) = x(0) + ẋ(0)t −  0  t2 .
2
1
That is, we can determine the entire trajectory (the past and the future) of the
particle by specifying the initial location x(0) and the initial velocity ẋ(0).
This last statement is only true within our model. Note that the model will of
course break down at the latest when the particle hits the earth’s surface, but it will
also be inaccurate if the particle is far away from the earth’s surface.
A more accurate model of gravitational force is
F (x) = −GmM
x
,
|x|3
where m is still the mass of the particle at x, M is the mass of the earth and G the
graviational constant. Here, x is now the distance vector from the centre of earth.
It is no longer obvious whether the resulting equation,
mẍ = −GmM
x
|x|3
(12)
has a solution, or if it does, whether it is unique or whether is can be obtained in
closed form.
While (12) is still a model of course, it is far more accurate (or, has a much
wider region of validity) than the simplified model (11). Can we say anything about
how accurately the solution of (11) approximate the solution to (12) (provided that
latter exists and is unique).
Exercise 2.1. (a) Derive the simplified model (11) from the “exact” model (12)
by a formal expansion in h0 /R, where h0 is initial distance from the earth’s surface.
Show that the expected accuracy of the model is of the order O(h0 /R). 2
(b) Suppose an object falls from height h0 towards the earth. Use both models
to compute the impact velocity as it hits the earth’s surface. Show that the error in
the two predictions is consistent with what we expect from (a). 3
2
3
HINT: Write x1 (t) = R + h(t), x2 = x3 = 0, where R is the radius of the earth.
HINT: Reduce (12) to a 1D model, r̈(t) = −GM/r2 , multiply by ṙ and integrate.
16
More generally, the famous N -body problem can be stated as follows: given initial
values xj (0), ẋj (0) with xj (0) 6= xi (0) for i =6= j, and masses mj , j = 1, . . . , N , find
the solution of system of ODEs
mj ẍj (t) = −G
X mj mi (xj − xi )
,
3
|x
j − xi |
i6=j
j = 1, . . . , N.
(13)
It is considered impossible to solve this system “in closed form” in general. (Only
the case N = 2 is well-understood; already the case N = 3 contains many open
questions.) Substantial effort has therefore been spent on numerical simulation
methods to study large N -body systems, in particular in physical cosmology to study
the evolution of galaxies or even larger cosmological structures.
2.1.1
Hamiltonian systems
F is a conservative force field if F (x) = −∇E(x) for some potential energy E :
(Rd )N → R. In this case, Newton’s equations become
M ẍ = −∇E(x)
(14)
Conservative force-fields generally give a lot of additional structure to an equation
that can be exploited.
Exercise 2.2. We call a force field conservative if its total work over a closed curve
equals zero: if C = {γ(s)|s ∈ [0, S]}, γ ∈ C([0, S]), γ(0) = γ(S), then
Z
Z
O F (x) · dx =
C
S
F (γ(s)) · γ̇(s) ds = 0.
(15)
s=0
(a) Suppose that F (x) = −∇E(x) for some potential E ∈ C 1 (RN ; R). Prove
that F satisfies (15).
(b*) Suppose that F ∈ C 0 (RN ; RN ) satisfies (15). Prove that there exists E ∈
1
C (RN ; R) such that F = −∇E. 4
Exercise 2.3. Show that (13) are Hamiltonian.
Exercise 2.4. Let x ∈ C 2 be a solution of the Hamiltonian system (14). Prove
that it satisfies the energy balance
d 1 T
ẋ
M
ẋ
+
E(x)
≡ 0,
dt 2
that is, kinetic plus potential energy are conserved.
4
HINT: Let −E(x) :=
R1
s=0
F (sx) · x ds.
17
2.1.2
Molecular dynamics
These are just the Newtonian equations of motion
M ẍ = −∇V (x),
(16)
The potential energy V is now so complex that there is very little analytic qualitative theory about these systems, and there is simply no possibility to solve these
equations in closed form or even approximate them by simpler models. MD problems
are almost exclusively solved by numerical simulation.
2.2
Damping, Gradient Flows
A damping force is a force that acts opposite to the direction of the velocity of a
particle, e.g. a viscous damper F = −cẋ where c is a damping coefficient. This force
can, e.g., be used as an approximation to friction. Sometimes, it is also used as a
“magical” dissipation term that just accounts for various forms of energy that we
don’t want to keep track of.
If we add viscous damping to a general Hamiltonian system, then we obtain the
ODE
M ẍ + C ẋ + ∇E(x) = 0,
(17)
where C ∈ RN ×N should normally be positive definite. (see Exercise 16)
For simplicity, let us consider a constant scalar mass and a scalar friction coefficient, then we get the equation
mẍ + cẋ = −E 0 (x)
Suppose now that the friction coefficient is much larger than the mass, to be more
precise suppose that c2 m, then rescaling time s := c−1 t, d/dt = c−1 d/ds, we
obtain
m d2 x dx
+
= −∇E(x)
c2 ds2
ds
We can now formally drop the acceleration term and obtain the first-order gradient
flow equation (also replacing s with t again)
ẋ(t) = −∇E(x(t)).
For more general friction operators the gradient flow equation reads
C ẋ(t) = −∇E(x(t)),
(18)
where C ∈ RN ×N should normally be symmetric and positive definite (spd); see 16
Exercise (c).
Exercise 2.5 (Energy Balance for Gradient Flows). (a) Show, formally,
that solutions of (18), with C = I, satisfy the following energy balance law:
d
E(x(t)) = −|ẋ(t)|2 = −|∇E(x(t))|2 .
dt
18
(b*) Show the corresponding result for general spd C:
d
E(x(t)) = −ẋT C ẋ = −∇E(x(t))T C −1 ∇E(x(t)).
dt
(REMARK: Note that |x|C := (xT Cx)1/2 is a Euclidean norm and |y|C −1 its dual
norm.)
(d) Now consider the general damped Hamiltonian system (17). Show that, if
d
C is positive semi-definite, then dt
E(x(t)) ≤ 0 along trajectories. If C is positive
d
E(x(t)) < 0 (damping). [[ examinable only for C = I. ]]
definite, then dt
Exercise 2.6 (Mathematical Pendulum 1). Consider a pendulum consisting
of a massless rigid rod and a mass m fixed to one end. Let g be the gravitation acting
on the mass and ϕ : [0, ∞) the angle subtended between the rod and its vertical
position. Moreover, taking damping into account, let k be a friction coefficient, then
ϕ satisfies the equation
g
k
(19)
ϕ̈ + ϕ̇ + sin(ϕ) = 0,
m
l
supplied with initial conditions ϕ(0) = ϕ0 , ϕ̇(0) = 0.
(i) Let |ϕ(0)| < π/2. Suppose there exists a solution ϕ ∈ C 2 [0, ∞). Show that
|ϕ(t)| ≤ |ϕ0 |
∀t ≥ 0.
(ii) Rescale the equation by setting ψ := ϕ/ε, where ε := |ϕ0 | and expand the
nonlinear term in (19) in ε. Hence, derive the linear pendulum equation
ψ̈ +
g
k
ψ̇ + ψ = 0,
m
l
with initial condition ψ(0) = ϕ0 /|ϕ0 |. Note in particular, that this equation is independent of the scale ε = |ϕ0 | of the initial displacement.
What order of magnitude do you expect the error ϕ − εψ to be? Justify your
answer.
2.3
Population Dynamics
[[ removed non-examinable section ]]
2.4
Other examples
• Duffing equation
ẍ + δ ẋ + αx + βx3 = γ cos(ωt)
where α, β, γ, δ, ω are model parameters. This is an example of a dynamical
system that exhibits chaos.
19
• FitzHugh–Nagumo model (a simplified variant of the Hodgkin–Huxley model)
is a prototype of an excitable system. It models, e.g., the firing of neurons.
V̇ = V (V − α)(1 − V ) − W + I
Ẇ = (V − γW ),
where I is the magnitude of the stimulus current and , α, γ parameters; in
particular 1.
• Van der Pol’s oscillator
ẍ − ε(1 − x2 )ẋ + x = 0.
is a simple model used to describe stable oscillations in electric circuits.
Exercise 2.7. Convert the (i) Duffing equation; and (ii) the Van der Pol equation
into explicit first-order systems.
20
3
Systems of Linear ODEs
Virtually all linear problems share one common feature: we have a far more complete
understanding of their mathematical structure than we do for (most) nonlinear
problems. A thorough understanding of linear problems can often give us some
(limited) intuition for nonlinear problems, or techniques and tools that may also be
applied to nonlinear problems. For example, nonlinear problems can sometimes be
linearised and if done in a sufficiently immaginative way, this can give interesting
information about the original nonlinear problem as well.
Throughout this section, we shall study the initial value problem (IVP)
ẋ(t) = Ax(t) + b(t),
x(0) = x0 ,
(20)
where A ∈ RN ×N , b ∈ C(R; RN ), x0 ∈ RN . (It would of course be sufficient to
require that b is defined only on a finite interval, say, [0, T ], but for simplicity we
ignore this.)
3.1
Revision of the 1D theory
For 1D systems (N = 1) we have a fairly good understanding of linear IVPs from
MA133 Differential Equations. To get going, we briefly review some of the results.
All of these formulas can be readily derived from the 1D techniques, or checked
through direct computations.
1. If ẋ(t) = ax(t), where a ∈ R is a constant, then by separation of variables, we
get x(t) = x(0)eat . Note, that this ODE is sometimes taken as the definition
of the exponential.
2. If ẋ(t) = a(t)x(t), then x(t) = x(0)e
Rt
0
a(s) ds
; again by separation of variables.
3. ForR the ODE ẋ(t) = a(t)x(t) + b(t), by multiplying with the integrating factor
t
e− 0 a(s) ds , we get
Z t R
Rt
t
a(s) ds
0
x(t) = x(0)e
+
e s a(r) dr b(s) ds.
0
This formula is a 1D version of Duhamel’s Formula.
There is another important class of linear ODE, which can be solved fairly easily:
aN x(N ) + aN −1 x(N −1) + · · · + a2 ẍ + a1 ẋ + a0 x = 0.
(21)
In this case, we make the ansatz x(t) = eλt and obtain the algebraic equation
aN λN + aN −1 λN −1 + · · · + a2 λ2 + a1 λ + a0 = 0.
If λ1 , . . . , λn are the roots of this polynomial, then the general solution of (21) is
x(t) =
N
X
cn e λ n t .
n=1
This formula requires suitable variations according to whether we want to solve the
ODE in R or in C, whether the roots are real of complex, and whether or not the
roots are repeated. We will revisit this ODE in Exercise 3.5.
21
3.2
Examples
Example 3.1 (Radioactive Decay). Suppose we have a radioactive substance
S1 , which decays with rate λ1 into substance S2 , which decays with rate λ2 into
substance S3 , and so forth. Finally, substance SN is stable.
To model this, let sn (t) be some (dimensional or non-dimensional) measure of
the amount of substance Sn at time t, then
ṡ1 (t) = −λ1 s1 (t)
ṡn (t) = λn−1 sn−1 (t) − λn sn (t),
ṡN (t) = 0.
n = 2, . . . , N − 1,
and
A natural condition (if at time t = 0 we only have substance S1 ) is s1 (0) = s1,0 and
sn (0) = 0 for n = 2, . . . , N .
This is a vectorial IVP, which can also be written in matrix form, with s =
(s1 , . . . , sN ),


−λ1
0 ···
···
0
 
.. 

s1,0
..
.
 λ1 −λ2
.



 0 
.. 
... ...
...

ṡ = 
s,
s(0)
=

. .
.
 0
 .. 
 .

... ...
 ..
−λN −1 0
0
0
···
0
λN −1 0
But this is maybe not necessary for this simple model, as we can just solve it recursively: for simplicity, let us suppose that s1,0 = 1 and λi 6= λj 6= 0 for i 6= j,
then
s1 (t) = e−λ1 t
λ1 −λ1 t
−λ2 t
s2 (t) =
e
−e
,
λ2 − λ1
and so forth.
This can be carried out in principle, but it gets tedious very quickly, so we may
want to find an algebraically more effective means of solving this IVP, that can
maybe be analysed efficiently, or for which efficient numerical evaluation techniques
can be employed.
Example 3.2 (Pond Pollution). Consider three ponds A, B, C, connected
by streams. Pond A also has a pollution source. We want to determine how the
spreading of pollutant through the ponds. (See Figure 1.)
The pollutant flow rate into A is f (t). The pollutant flux is the flow rate times
the concentration. E.g., the flux out of pond A is fA xA /VA , where fA is the flow rate
for the stream from A to B, xA the amount of pollutant (in lb) in pond A and VA
22
fA
VA xA
B
A
Pollutant
f (t)
fB
VB xB
fC
VC xC
C
Figure 1: Illustration of Example 3.2: spreading of pollutant through ponds connected by streams.
the volume of pond A. This yields the system
fA
fC
xC −
xA + f (t)
VC
VA
fA
fB
ẋB =
xA −
xB
VA
VB
fC
fB
xB −
xC .
ẋC =
VB
VC
ẋA =
If the ponds are pristine at time t = 0, then we may choose the initial condition
xA (0) = xB (0) = xC (0) = 0. To describe that pollutant is poured into pond A for an
initial time period t1 , and afterwards the system is left intact, we define f (t) = f1
for 0 ≤ t ≤ t1 and f (t) = 0 for t1 < t.
Can this IVP still be solved analytically / “by hand”?
3.3
The exponential of a matrix
We have shown that solutions to (20), if they exist, are unique. We now turn to the
question of existence.
To establish existence of solutions to (20), we begin by considering the ODE
ẋ(t) = Ax(t),
where A ∈ RN ×N is a constant. A natural question to ask is, whether the solution
can be written, as in 1D, in the form
x(t) = etA x(0)?
Suppose, for example, that A = XΛX −1 , where Λ = diag(λ1 , . . . , λN ), then an easy
calculation gives
x(t) = Xdiag(etλ1 , . . . , etλN )X −1 .
23
Moreover, since etλ =
P∞
n=0
Xdiag(e
tλ1
(tλ)n
,
n!
,...,e
we observe that
tλN
)X
−1
= Xdiag
∞
X
(tλ )n i
n!
n=0
∞
X
tn n −1
=X
Λ X
n!
n=0
=
∞
X
tn
n=0
=
n!
∞
X
tn
n=0
n!
X −1
(XΛX −1 )n
An .
Can we use this definition also for general (non-diagonalisable) matrices?
Suppose A ∈ RN ×N and we define eA , formally as in 1D, by
eA :=
∞
X
1 n
A ,
n!
n=0
(22)
then we would, still formally, get that
∞
d X tn n d tA
e x(0) =
A x(0)
dt
dt n=0 n!
∞
X
ntn−1 n =
A x(0)
n!
n=0
∞
X
tn n =A
A x(0)
n!
n=0
tA
= A e x(0) .
This calculation indicates, that (22) is indeed promising. We should now show that
all steps were justified:
Theorem 3.3. (a) Let A ∈ RN ×N , then the series on the right-hand side of (22)
is absolutely convergent. That is, eA ∈ RN ×N is well-defined.
(b) Let E(t) := etA , t ∈ R, then E ∈ C ∞ (R; RN ×N ) and Ė(t) = AE(t).
Proof. (a) Let Dn :=
1 n
A ,
n!
then using the results of § 1.2.2 we obtain
(Dn )ij ≤ kDn kF = 1 kAn kF ≤ 1 kAknF .
n!
n!
Hence,
∞
∞
X
X
1
(Dn )ij ≤
kAknF = ekAkF ,
n!
n=0
n=0
which concludes the proof of (a).
24
(b) Let Em (t) :=
tn n
n=0 n! A ,
Pm
then Em ∈ C 1 (RN ×N ) and
Ėm (t) =
m
X
ntn−1
n!
An
n=0
m−1
X n
=A
n=0
t n
A
n!
= AEm−1 (t).
Thus, if we can show that Em (t) converges uniformly, then we also get that Ėm (t)
converges uniformly, hence lim Em (t) = E(t) is differentiable, and we get the stated
formula Ė = AE.
To see this, we apply the Weierstrass M-Test. We fix some T > 0 and replace A
n
with tA in (a), for some t ≤ T , and define Dn (t) = tn! An , then we get
∞
∞
∞
X
X
X
ktAknF
|T |n
(Dn (t))ij ≤
≤
kAknF = e|T |kAkF .
n!
n!
n=0
n=0
n=0
Thus, Em indeed converges uniformly in [−T, T ] for all T > 0.
Since E ∈ C 1 and Ė = AE it follows inductively that in fact E ∈ C ∞ . This
concludes the proof.
Having now established both existence and uniqueness of solutions to IVPs, we
are now also in a position to construct the general solution of the ODE ẋ = Ax
without initial condition. (superposition principle, dimensionality)
Corollary 3.4 (Homogeneous Solution Space). Let A ∈ RN ×N , then the space
S := x ∈ C 1 (R; RN )ẋ = Ax
is a linear vector space over R with dimension N .
Exercise 3.1. Prove Corollary 3.4.
Exercise 3.2. (a) Compute the matrix exponentials of
2 −3
3 1
(i)
and
(ii)
1 6
1 3
(a’) If you use Matlab, Mathematica, or similar, then find out how to compute
the matrix exponential numerically and check your result.
(b) Using these exponentials, or otherwise, solve the IVPs


 ẋ1 = 2x1 − 3x2 + e−2t
 ẋ1 = 3x1 + x2 + e3t
ẋ2 = x2 + 3x2 + e2t
ẋ2 = x1 + 6x2 + e−2t
(i)
and
(ii)


x1 (0) = 3, x2 (0) = −1.
x1 (0) = x2 (0) = 0.
25
Exercise 3.3. (a) If A, B ∈ RN ×N commute, then eA+B = eA eB . Show that
eA+B = eA eB is in general false.
(b) Conclude that eA ∈ GL(N ), and (eA )−1 = e−A .
(c) Show that if S is symmetric, then (eS )T = eS .
If K ∈ RN ×N is skew-symmetric (K T = −K), then (eK )T = e−K . Conclude that,
if K is skew, then eK is a rotation (orthogonal with determinant 1).
(d) If A is a projection (A2 = A), then eA = I + √
(e − 1)A.
A
kAk
A
kAkF
(e) Show that ke k ≤ e
and ke kF ≤ e
+ N − 1.
Exercise 3.4. (a+) If A ∈ RN ×N , then det(eA ) = etrace(A) .5
(b) Deduce that exp : RN ×N → GL(N ) is not surjective.
Exercise 3.5. (a) Consider the scalar constant coefficient ODE (21). Convert it
into a first order system of the form ż = Az, where A is called the companion
matrix. Show that the characteristic polynomial of A is precisely the characteristic
polynomial of the ODE (21).
(b) Assuming that A has P
N distinct eigenvalues {λn |n = 1, . . . , N }, show that
λn t
.
the general solution is x(t) = N
n=1 cn e
(c*) Using the Jordan normal form, find the general solution for general matrices
A, and try to understand the connection to the construction you learned in MA133
Differential Equations.
3.4
Duhamel’s Formula
Having constructed a solution to the constant coefficient problem, we can now return
to the general linear IVP (20). Based on the 1D result, and inspired by our success in
the constant coefficient case, we can again test whether the natural generalisation of
Duhamel’s formula holds. We achieve this by considering a matrix-valued integrating
factor,
ẋ = Ax + b,
−At
e
−At
ẋ − Ae x = e−At b
d −At e c = e−At b.
dt
Integrating yields
tA
Z
x(t) = e x0 +
t
e(t−s)A b(s) ds.
(23)
0
(Although we could have just “guessed” and checked (24) directly, the above calculation yields also uniqueness.)
Proposition 3.5. (a) Let A ∈ RN ×N and b ∈ C k ([0, T ); RN ) and let x be given by
(24). Then x ∈ C k+1 ([0, T ); RN ), and it is the unique solution of the IVP (20).
5
HINT: Bring A into Jordan normal form A ∼ D + E, where D is diagonal. Show that, if B is
upper triangular and C strictly upper triangular, then BC and CB are strictly upper triangular.
Use this to prove that e(D+E) = eD + E 0 where E 0 is strictly upper triangular.
26
(b) The set of all solutions of the ODE
ẋ = Ax + b,
without initial condition, is given by
Z t
e(t−s)A b(s) ds + S,
x(t) =
0
where x0 ∈ RN is arbitrary, and S is the homogeneous solution space defined in
Corollary 3.4.
Remark 3.6. It is also possible to treat the case of time-dependent matrix A(t),
but the solutions are not as easily characterised anymore. We will return to this case
again in §4.4, as a corollary to Picard’s theorem.
Exercise 3.6. (a) Prove Proposition 3.5.
Rt
(b) Show that, if A ∈ C([0, T ); RN ×N ) and 0 A(s) ds commutes with A(t), then
the following generalisation holds:
Z t R
Rt
t
A(s)
ds
x(t) = e 0
x0 +
e s A(r) dr b(s) ds.
(24)
0
Show that this is false in general.
REMARK: Compare with Exercise 3.5.
Exercise 3.7. (a) Show that, if A ∈ RN ×N is symmetric and positive definite,
then e−tA → 0 as t → ∞.
(b) Let x(t) ∈ C ∞ ([0, ∞)) be the solution of the IVP
ẋ + Ax = b,
x(0) = x0 ,
where A ∈ RN ×N is spd and b ∈ RN . Show that x(t) → x̄ as t → ∞, where x̄ is the
stead state solution: Ax̄ = b.
(c) Are the results (a, b) still true for general positive definite A (not necessarily
symmetric)? Rigorously justify your answer.
Exercise 3.8. Consider the second-order IVP
mẍ(t) + cẋ(t) + ax = b(t),
x(0) = x0 , ẋ(0) = ẋ0 .
Construct a solution of the form
Z
x(t) = x0 S0 (t) + ẋ0 S1 (t) +
t
S1 (t − s)b(s) ds.
0
Exercise 3.9. Consider again the problem of pollutant spreading in a pond network from Example 3.2. Let the unit for time be hours. Let fi /Vi = 1, i ∈ {A, B, C},
let t1 = 50 hours, and f1 = 10 lb / hour.
How much pollutant is in each pond after 50 hours, after 100 hours, and at
t = ∞?
27
3.5
Uniqueness via energy method
It is sometimes the case that proving uniqueness of solutions is achieved through entirely different means than existence. Here we look at a simple argument for uniqueness of solutions to (20) that even generalises to many PDE. The “testing mechanism” is generally a useful concept; see, e.g., our analysis of (damped) Hamiltonian
systems.
Let y, z ∈ C 1 ([0, T ]; RN ) both be solutions to (20) and let x := y − z, then
ẋ(t) = A(t)x(t),
x(0) = 0.
(25)
Thus, as is typical for linear problems, if we can show that the homogenous version
(25) of the IVP (20) has only the trivial solution x ≡ 0, then we can conclude that
solutions to (25) are unique: y ≡ z.
To prove this we take the inner product of the equation with x, to obtain
x · ẋ = x · (Ax)
⇔
⇒
d 1
|x|2
dt 2
d
|x|2
dt
= x · (Ax)
≤ 2kA(t)kF |x|2 .
(26)
Let ξ(t) := |x(t)|2 and a(t) := 2kA(t)kF , then it is easy to see that ξ ∈ C 1 ([0, T ]; R),
a ∈ C([0, T ]; R), and we have just shown that
˙ ≤ a(t)ξ(t).
ξ(t)
We multiply (26) with the integrating factor e−
Rt
0
a(s) ds
to obtain
d − R t a(s) ds
e 0
ξ(t) ≤ 0.
dt
Since ξ(0) = 0 and ξ(t) ≥ 0, it follows that ξ ≡ 0 in [0, T ]. Thus, we have just proven
the following result:
Theorem 3.7. There exists at most one solution x ∈ C 1 ([0, T ]; RN ) of the linear
IVP (20).
Exercise 3.10 (Gronwall type inequality).
This exercise introduces a simple version of one of the most important inequalities
in ODE theory. We will return to it in § 5.
(a) Prove the following variant of Gronwall’s inequality: Let ξ ∈ C 1 ([0, T ); R),
˙ ≤ aξ + b, then
a, b ∈ R and suppose that ξ(t)
ξ(t) ≤ eat ξ(0) +
b(eat − 1)
.
a
(b) Continuous dependence on initial condition: Consider the IVPs, i = 1, 2,
ẋi (t) = A(t)xi (t) + b(t)
xi (0) = xi,0 ,
28
(27)
where A ∈ C([0, T ); RN ×N ), b ∈ C([0, T ); RN ) and xi,0 ∈ RN . Suppose that
kA(t)kF ≤ a for all t ≥ 0. Prove that, if xi ∈ C 1 ([0, T ); RN ) are solutions to (27)
then
|x1 (t) − x2 (t)| ≤ eat |x1,0 − x2,0 |
[[ removed non-examinable section ]]
4
Picard’s Theorem
Recall that an explicit first-order initial value problem is a problem of the form
ẋ(t) = f (t, x)
x(0) = x0
(28)
Our goal in this section is to investigate under which conditions this system has solutions, when they are unique, and for which time interval [0, T ] (or, more generally,
[−T1 , T2 ]) they exist.
4.1
Quantifying continuity
Our standard definition of a solution to (28) requires that x ∈ C 1 . In particular,
ẋ ∈ C and this indicates that we must probably require at the very least that f is
continuous in a neighbourhood of (0, x0 ). Unfortunately, continuity alone is rarely
a “good” analytical concept, and this is also the case here, as demonstrated by the
classical example
ẋ(t) = |x|α (t),
x(0) = 0,
(29)
for some α > 0.
Exercise 4.1. Show that, if α ∈ (0, 1), then (29) has an infinite number of solutions. What can you say (without employing Picard’s theorem) about α ≥ 1?
Not only in the analysis of ODEs, but also in many other applications of analysis one requires stronger, quantitative variants of continuity. The most common
condition of this kind, used in the theory of ODEs is the following.
Definition 4.1 (Lipschitz Continuity). Let U ⊂ RN and let f : U → RM . We
say that f is Lipschitz continuous in U if there exists L ≥ 0 such that
|f (x) − f (y)| ≤ L|x − y|
∀x, y, ∈ U.
(30)
(The smallest L for which this holds is called the Lipschitz constant: L =
inf x6=y∈U |f (x) − f (y)|/|x − y|.)
If U ⊂ RN is open and f is Lipschitz continuous in each compact (closed and
bounded) subset of U , then f is said to be locally Lipschitz continuous in U .
29
We have already seen in Analysis that Lipschitz continuity is a crucial condition
in the theory of ODEs. It is a “practical” condition only if it is easy to check whether
a given function is Lipschitz. In the following, we look at some simple examples.
Exercise 4.2. (a) Is f (x) = ex locally Lipschitz in R? Is it Lipschitz in R? What
is the Lipschitz constant on (−∞, 0]?
(b) Is f (x) = (1 + x2 )−1 Lipschitz in R?
(c) Is f (x) = x2 sin(x−1 ) Lipschitz in [−1, 1]?
(d) Is f (x) = | cos(x)| Lipschitz in R?
(e) Show that the function f (x) = |x|α is Lipschitz continuous in [−1, 1] if
and only if α ≥ 1, but continuous for all α ≥ 0. In particular, conclude that (i)
Lipschitz continuity is strictly stronger than continuity and (ii) strictly weaker than
differentiability.
The previous exercises, and indeed the definition of the Lischitz constant, should
have indicated that there is some connection between Lipschitz continuity and differentiability. The following simplified version of a classical result in real analysis
(Rademacher’s Theorem) provides a practical criterion to test for Lipschitz continuity.
Proposition 4.2. Let U ⊂ RN be open and convex and f ∈ C 1 (U ). Then f is
Lipschitz continuous in U if and only if k∇f k∞,U < +∞. Moreover, the Lipschitz
constant for U is given by L = k∇f k∞,U .
If f ∈ C 2 (U ; RN ), then the Lipschitz constant is L = supx k∂f (x)k, but we only
have that the Lipschitz constant is bounded above by supx k∂f (x)kF .
Proof. It is easy to see that L ≤ k∇f k∞ , using the convexity of U :
Z 1
|f (x) − f (y)| = ∇f (1 − s)y + sx · (x − y) ds
s=0
Z 1 ≤
∇f (1 − s)y + sx · (x − y) ds
Zs=0
1 ∇f (1 − s)y + sx |x − y| ds
≤
s=0
≤ k∇f k∞ |x − y|.
To prove the converse, let ε > 0 and x ∈ U such that |∇f (x)| ≥ k∇f k∞ − ε. If
∇f (x) = 0 (independent of the choice of ε, then f is constant and we have nothing
to prove.
Otherwise, let h = ∇f (x), then by the definition of (Gateaux) differentiability
of f at x and the fact that U is open, we have for all s sufficiently small that
|f (x + sh) − f (x)| = s|∇f (x)h| + o(s) = s|∇f (x)||h| + o(s).
Dividing through by s|h| we obtain that
L ≥ sup
s
|f (x + sh) − f (x)|
|f (x + sh) − f (x)|
≥ lim
= |∇f (x)|.
s→0
s|h|
s|h|
Thus L ≥ k∇f k∞ − ε for all ε > 0, which concludes the proof for N = 1.
The extension to N > 1 is straightforward.
30
U
U1
U2
U2
U3
U4
U5
U6
U7
U8
U3
U1
U4
U5
U9
(b)
(a)
Figure 2: Examples of domains satisfying the conditions of Proposition 4.4
As a “rule of thumb” a function is Lipschitz if it is continuous, differentiable
except on a set of measure zero (e.g., piecewise differentiable) and ∇f is bounded.
To make this precise, we need to introduce weaker forms of derivatives and other
heavier tools of real analysis. See e.g. modules on Measure Theory (MA359), Theory
of PDEs (MA3G1), Advanced PDEs (MA4A2), Advanced Real Analysis (MA4J0).
However, some weaker results can be proven with elementary techniques. The
essence is that it is sufficient if f is continuous, piecewise C 1 and the gradients are
bounded.
Example 4.3. Suppose that f ∈ C(R2 ; RN ), U± := {x ∈ R2 | ± x2 > 0} and
f ∈ C 2 (U± ; RN ). Then f is Lipschitz in R2 if and only if k∂f k∞,U± < ∞.
Proof: First of all, we know from Prop. 4.2 that, if either of k∂f k∞,U± are
infinite, then f is not Lipschitz.
To prove that f is Lipschitz if they are bounded, choose two points x, y ∈ R2 . If
both points lie entire in either U+ or U− then there is nothing to prove. If x ∈ U−
and y ∈ U+ , let z := (1 − t∗ )x + t∗ y ∈ ∂U+ ∩ ∂U− then
|f (y) − f (x)| ≤ |f (y) − f (z)| + |f (z) − f (x)|,
and we can now proceed as in the proof of Prop. 4.2 to get the upper bound.
More generally, one could formulate results along the following lines: [[ The following result is not examinable in its general form, but simple special cases may
need to be derived. Moreover it will be required to understand that continuous and
piecewise C 1 -functions with bounded derivatives are Lipschitz. ]]
Proposition 4.4. (NE) Let U ⊂ RN be closed
S and convex and suppose there exist
disjoint open sets U1 , . . . , UM ⊂ U such that M
m=1 Ūi = U and for any line segment
` = {(1 − s)x + sy|s ∈ [0, 1]}, x, y ∈ U , there exist finitely many points s1 , . . . sK
such that each sub-segment {(1 − s)x + sy|s ∈ [sk , sk+1 ]} ⊂ Ūm for some m. (see
Figure 2 for examples)
31
T
1
M
Suppose that f ∈ C(U ; RM ) ∩ M
m=1 C (Um ; R ), then f is Lipschitz with Lipschitz constant L = maxm=1,...,M supx∈Um k∇f (x)k.
Exercise 4.3. (i)** Prove Proposition 4.4. 6 (NE)
(ii) Prove that f : R2 → R, f (x1 , x2 ) = | cos(x1 + x2 )| is Lipschitz continuous.
(iii) Prove that |x1 | + |x2 |2 is locally Lipschitz in R2 .
[[ removed non-examinable section ]]
4.2
4.2.1
Picard’s theorem: local existence and uniqueness
IVPs as fixed point problems
A common approach to solving nonlinear problems, or just establishing the existence
of solutions to nonlinear problems, is to rephrase them as fixed point problems. For
example, we have a nonlinear equation (system)
F(x) = 0,
where F : RN → RN , then we can always achieve this, by replacing it with
x − τ F(x) =: T (x) = x,
(31)
for some parameter τ > 0. But this only rarely helps finding solutions. Another
more promising generic approach is to try and identify a structure that allows us to
rewrite the nonlinear equation F(x) = 0 in the form
A(x)x = g(x),
where A : RN → RN ×N and g : RN → RN , and in this case a natural fixed point
problem is
T (x) = x, where T (x) = A(x)−1 g(x).
(32)
There are many other, often highly problem dependent, approaches.
Once we have achieved reformulating a problem as a fixed point problem, a number of abstract theorems exist that establish existence and uniqueness of solutions.
Possibly the most elementary of these (but still very powerful) is Banach’s Fixed
Point Theorem (or, Contraction mapping theorem): 7
Lemma 4.5 (Banach’s Fixed Point Theorem). Let (X , k·k) be a Banach space,
8
U ⊂ X be closed, and T : U → U a contraction:
kT (x) − T (y)k ≤ kkx − yk
6
∀x, y ∈ U,
(33)
HINT: L ≤ is easy once you decode the complicated statement. L = can be done similarly as
in Theorem 4.2.
7
Other examples of important fixed point theorems are Brouwer’s FPT (non-constructive; any
continuous function from the closed unit ball in Rd to itself must have a fixed point) or Schauder’s
FPT (generalises Brouwer to infinite dimensional spaces, typically used for existence theorems in
PDE theory, requires some notion of compactness).
8
Complete normed vector space (cf. Analysis, Metric Spaces; it would in fact sufficient to assume
that X is a complete metric space.)
32
then there exists a unique x ∈ U such that T (x) = x.
Exercise 4.4. Prove Lemma 4.5.
The idea behind BFPT is highly constructive. One defines an arbitrary starting
guess x0 ∈ U and then constructs a sequence
xn+1 := T (xn ),
n = 0, 1, 2, . . . .
(34)
If the sequence xn converges (we don’t know this yet, but let us just assume it
does to see what would follow), say xn → x, and T is continuous (follows from the
contraction property), then taking the limit on both sides of (34), we obtain that
x = T (x). The contraction property provides a simple requirement under which (xn )
can be shown to be a Cauchy sequence.
In the case of the ODE (28), making the abstract idea (32) concrete gives the
fixed point map9
ẏ = f (t, x)
T (x) := y, where
,
(35)
y(0) = y0 .
which is more commonly written as
Z
t
f (s, x(s)) ds.
T (x)(t) := x0 +
(36)
0
The corresponding fixed point iteration (34) is called the Picard iteration and reads
Z t
f (t, xn (t)) dt.
(37)
xn+1 (t) := x0 +
0
Of course, all of this is purely formal and we have to fill in some technical details to
make these ideas precise.
Exercise 4.5. Apply the Picard iteration (37), with x0 (t) ≡ x0 , to the linear IVP
ẋ = Ax, x(0) = x0 , for some constant matrix A ∈ RN ×N . Show that
n
X
tm m
A .
xn (t) =
m!
m=0
4.2.2
Picard’s Theorem
Our strategy for establishing the existence and uniqueness of solutions to (28) is
now clear: we want to prove that the fixed point map (36) (let us call it the Picard
map) is a contraction in some suitable subset of a Banach space. Determining the
“suitable space” and the “suitable subset” are typically a part of establishing such
results.
9
This is only one, generic way to doing this, that just happens to be sufficient. Note that
we are updateing the “higher-order” term ẏ, with input from the previous iterate given in the
“lower-order” term f (t, x).
33
As a rough rule of thumb, one normally chooses spaces that are “as large as
possible”. Because the definition of the Picard map (36) only requires that x ∈ C, it
is natural to work in the space of continuous functions.10 Since we require a complete
space, we equip it with the max-norm: X := C([0, T ]; RN ) for some T > 0 that we
still need to determine, and k · k := k · k∞ .
Let us assume, for the moment, that f satisfies a global Lipschitz condition,
|f (t, x) − f (t, y)| ≤ L|x − y|
∀t ∈ R, x, y ∈ RN .
Then we can estimate
Z t
f (s, x(s)) − f (s, y(s)) ds
|T (x)(t) − T (y)(t)| = 0
Z t
≤
f (s, x(s)) − f (s, y(s)) ds
0
Z t
≤L
|x(s) − y(s)| ds
0
≤ Ltkx − yk∞,(0,t) .
That is, if we choose T < 1/L and set k = LT , then we obtain that
kT (x) − T (y)k∞,(0,T ) ≤ kkx − yk∞,(0,T )
∀x, y ∈ C([0, T ]; RN ).
We still need to show that given x ∈ C([0, T ]; RN ), then T (x) ∈ C([0, T ]; RN ),
but this is relatively easy. Then BFPT gives us indeed existence and uniquess of
solutions to the fixed point formulation of the IVP:
Z t
x(t) = x0 +
f (s, x(s)) ds
∀t ∈ (0, T ].
(38)
0
We can also think of this as a weak formulation (or, integral formulation), since it
only requires that x ∈ C, whereas the strong formulation (28) requires that x ∈ C 1 .
But it is easy to see that, in fact, any solutionof the weak formulation is in fact a
solution of the strong formulation. This is called a regularity result.
Proposition 4.6 (Regularity of Weak Solutions). Let x ∈ C([0, T ]; RN ) and
suppose that f ∈ C([0, T ] × U ) where x(t) ∈ U for all t ∈ [0, T ]. If x is a solution of
(38), then x ∈ C 1 ([0, T ]; RN ) and x also solves the strong form (28) of the IVP.
Proof. Since f and x are continous, it follows that Rϕ(s) := f (s, x(s)) is continuous.
t
By the fundamental theorem of calculus, Φ(t) := 0 ϕ(s) ds is continuously differentiable, with Φ̇(t) = ϕ(t). Hence, x(t) = x0 + Φ(t) belongs to C 1 ([0, T ]; RN ) and
ẋ(t) = Φ̇(t) = f (t, x(t)). Moreover, setting t = 0 in (38) it follows that x(0) = x0 .
Thus, x solves the IVP (28).
An even larger natural function space would be L1 ((0, T ); RN ), which is the Banach space of
integrable functions, but to pursue this we would first need a generalised notion of differentiability.
10
34
The global Lipschitz assumptions is too strong for applications, as it excludes far
too many important ODEs. (Of the examples shown in § 2.4, none would fit into the
framework.) But since solutions of the fixed point equations are continuous, they
must be bounded on finite intervals (and indeed arbitrarily close to x0 on arbitrarily
short time intervals), and we should therefore expect that it is enough for f (t, ·) to be
locally Lipschitz continuous. This observation gives the famous Picard Theorem:11
Theorem 4.7 (Picard’s Theorem). Suppose that there exist T1 , R > 0, such that
f ∈ C([0, T ] × BR (x0 ); RN ) and satisfies the Lipschitz condition
|f (t, x) − f (t, y)| ≤ L|x − y|
∀t ∈ [0, T1 ], x, y ∈ BR (x0 ).
Then there exists T > 0 and a unique x ∈ C([0; T ]; RN ) satisfying the weak form
(38) of the IVP.
Sketch of the proof. We want to apply BFPT. Motivated by the foregoing discussions, we choose X := C([0, T ]; RN ) for some 0 < T ≤ T1 and
U := x ∈ X kx − x0 k∞ ≤ R .
We define the map T : U → X through (36). We now need to prove the following:
1. T indeed maps elements of U to X = C([0, T ]; RN ), i.e., it generates continuous
functions.
2. T in fact maps U to U. For this to be true, we will need to choose T sufficiently
small.
3. T is a contraction. This will give a further restriction on the size of T .
After we have established 1.–3., applying BFPT completes the proof.
Corollary 4.8 (Slight Variation/Generalisation of Picard’s Theorem). Suppose that there exist T0 , T1 , T2 , R > 0, such that T1 ≤ T0 ≤ T2 , f ∈ C([T1 , T2 ] ×
BR (x0 ); RN ) and satisfies the Lipschitz condition
|f (t, x) − f (t, y)| ≤ L|x − y|
∀t ∈ [T1 , T2 ], x, y ∈ BR (x0 ).
Then there exists T > 0 and a unique x ∈ C(I; RN ), where I = [T1 , T2 ] ∩ [T0 −
T, T0 + T ], satisfying the weak form (38) of the IVP backward and/or forward in
time.
Exercise 4.6. Carry out the details of the proof of Picard’s Theorem, including
its corollary.
Exercise 4.7. Consider the IVP
ẋ(t) = 2tx(t),
11
x(0) = x0 .
or, Picard–Lindelöf theorem, or Cauchy–Lipschitz theorem, and possibly there are various
other names for it.
35
(i) Compute the first three iterates of the Picard iteration: x0 , . . . , x2 .
(ii) By induction, show that the nth iterate is given by
xn (t) = x0
X
n
i=0
t2i
.
i!
Hence, write the solution in terms of elementary functions.
Exercise 4.8 (Inverse Function Theorem*). Another important result that
follows fairly easily from Banach’s fixed point theorem is the Inverse Function Theorem:
Let U ⊂ RN be open, x0 ∈ U , and let F ∈ C 1 (U ; RN ) in the sense of Frechet
satisfy F(x0 ) = 0 and F 0 (x0 ) is invertible. Then there exists ε, δ > 0 such that the
following holds: For all g ∈ RN with |g| ≤ ε there exists a unique xg ∈ U with
|x0 − xg | ≤ δ such that F(xg ) = g. 12
Exercise 4.9 (Monotone Systems of Equations*). (a) Let F : RN → RN
be Lipschitz continuous and strongly monotone: there exists λ > 0 such that
F(x) − F(y) · (x − y) ≥ λ|x − y|2 .
Prove that, for τ sufficiently small, the fixed point map T defined by (31) is a
contraction. 13
(b) You can deduce from this that F(x) = 0 has a unique solution, but can you
immediately see this from the strong monotonicity condition?
The fixed point map is still useful to compute solutions of F(x) = 0. Based on
the information you are given, compute the optimal parameter τ . Now estimate,
after how many iterations of the fixed point map the initial error is reduce by a
factor 100: |xn − x̄| ≤ 100−1 |x0 − x̄|. 14
4.2.3
Well-Posedness
In typical applications, the initial datum x0 and physical or other parameters entering the definition of the right-hand side f are only known approximately. However,
one would expect that small changes in the data only cause small changes in the solution. This is known as well-posedness. Usually this is strongly related to uniqueness
of solutions. Indeed, most well-posedness proofs can be thought of as “quantitative
uniqueness proofs”. We have already encountered this in the context of linear equations. At this stage, we could begin to develop analogous results in the nonlinear
context, but we will defer this to § 5.
HINT: Consider the fixed point map F 0 (x0 )(T (x) − x) + F(x) = g. (Can you motivate this
choice?)
13
HINT: Consider |T (x) − T (y)|2 .
14
HINT: the calculation is easier if you substitute λ = εL; note that you always have 0 < ε < 1.
12
36
4.3
Maximal existence interval and blowup
For simplicity, we will assume from now on that f is locally Lipschitz in the following
sense: f ∈ C([0, ∞); RN ) and
∀M, T > 0 ∃L > 0 s.t. ∀t ∈ [0, T ], x, y ∈ BM (0) |f (t, x) − f (t, y)| ≤ L|x − y|.
(39)
In this case, Picard’s Theorem and the regularity theorem establish that the IVP
ẋ = f (t, x),
x(T0 ) = x0 ,
t > T0 ≥ 0
(40)
has a unique solution x1 ∈ C 1 ([T0 , T1 ]; RN ) for some T1 > T0 .
Suppose now that we use the final value x1 (T1 ) to pose a new IVP
ẋ = f (t, x),
x(T1 ) = x1 (T1 ),
t > T1
(41)
which again has a unique solution x2 ∈ C 1 ([T1 , T2 ]; RN ). Moreover, we can piece
these together,
x1 (t), T0 ≤ t ≤ T1 ,
x(t) :=
(42)
x2 (t), T1 < t ≤ T2 .
We can then show that, in fact, x ∈ C 1 ([0, T2 ]; RN ) and it solves the IVP (40) on
the combined interval [T0 , T2 ].
Lemma 4.9. Suppose that x1 ∈ C 1 ([T0 , T1 ]; RN ) solves (40), x2 ∈ C 1 ([T1 , T2 ]; RN )
solves (41), and x is defined through (42). Then x ∈ C 1 ([T0 , T2 ]; RN ) and x solves
(40) for t ∈ [T0 , T2 ].
Proof. Clearly, x ∈ C([0, T2 ]; RN ) ∩ C 1 ([0, T2 ] \ {T1 }; RN ) and one can readily check
that it satisfies the integral formulation (38). Hence, Proposition 4.6 gives that
x ∈ C 1 and satisfies the ODE in the strong sense.
The question we now want to investigate is how far we can go by repeating this
construction. Since the time increments that we are adding in each step might be
rapidly decreasing, we would not expenct to eventually fill out the entire future time
t ∈ [0, +∞), but it is possible that the solution to the IVP “ceases to exist” for some
T∗ < +∞, which the following example shows:
Example 4.10.
(a) Consider the IVP ẋ = −x, x(0) = 1, then the solution is x(t) = e−t . Ignoring
the fact that the Picard iteration covnerges globally for this case, Picard’s theorem
only yields existence and uniqueness in a small interval of length T < 1. (The
lipschitz constant is L = 1.) But since the solution is bounded and since L is a global
constant, one can check from the proof of Picard’s theorem that the time increments
for “piecing together” solutions are bounded below, hence we can construct the whole
trajectory this way.
(b) Consider the IVP
ẋ = −x−2 , x(0) = 1,
37
then the solution is given by x(t) = (1 + t)−1 . The “piecing together trick” might
yield the full trajectory, but it is not completely obvious since the neighbourhood
BR shrinks to zero.
A similar issue applies to ẋ = x, x(0) = 1. Since kf k∞,BR (xn ) grows to infinity for
|xn | → ∞.
(c) Consider the IVP
ẋ = xk ,
x(0) = 1,
where k ∈ N, k 6= 1. Then it is easy to see that
1
x(t) = 1 − t(k − 1) 1−k .
That is, for k > 1, the solution cannot be continued continuously beyond the blow-up
time T∗ = 1/(k − 1).
Definition 4.11. We say that x ∈ C 1 ([0, T∗ ); RN ), T∗ ∈ (0, +∞], is a maximal
solution to the IVP (28) if, for any y ∈ C 1 ([0, T ); RN ) satisfying the same IVP,
T ≥ T∗ and x = y on [0, T∗ ), then T = T∗ .
The time T∗ is then called maximal existence time or blowup time.
The next results establishes the existence and uniqueness of maximal solutions
as well as a characterisation of blow-up.
Theorem 4.12. 1. Let f satisfy the local Lipschitz condition (39), then the IVP
(28) has a unique maximal solution x ∈ C 1 ([0, T∗ ); RN ).
2. Either, T∗ = +∞, or |x(t)| → +∞ as t → T∗ .
Proof. 1. Let T∗ := sup{T > 0|∃ x ∈ C 1 ([0, T ]; RN ) solves IVP }. Let Tj ↑ T∗ ,
Tj < T∗ , then exist xj ∈ C 1 ([0, Tj ]; RN ) solutions to the IVP.
(1a) If i < j then xi (t) = xj (t) for t ∈ [0, Ti ]:
Let s := sup{t ∈ [0, Ti ]|xi (r) = xj (r) for r ∈ [0, t]}. Since xi , xj are continuous
it follows that xi (s) = xj (s) as well. If s = Ti , then we are done. If 0 ≤ s < Ti ,
then Picard’s theorem implies that there exists > 0 such that xi = xj in [s, s + ].
Hence, s was not maximal, and the only possibility is that s = Ti .
[end of proof of (1a)]
Let x(t) := xj (t) for t ∈ [Tj−1 , Tj ), then by Lemma 4.9 x ∈ C 1 ([0, T∗ ); RN ) and
solves the IVP.
Maximality: x is maximal by construction (there cannot exist T > T∗ and a
solution on [0, T )).
Uniqueness: If x, y are both maximal with maximal existence times Tx , Ty , then
x = y on [0, Tx ) ∩ [0, Ty ), by the same argument as step (1a). If Tx < Ty , then Tx
was not maximal, hence Tx = Ty .
2. Suppose, for contradiction, that |x(t)| ≤ R < ∞ and T∗ < +∞. By compactness there exists tj ↑ T∗ and x∗ ∈ RN such that x(tj ) → x∗ . We claim that, in fact,
x(t) → x∗ as t → T∗ .
To see this, we note that the local Lipschitz condition (39) implies that |ẋ(t)| =
|f (x(t))| ≤ M := kf k∞,BR for all t ∈ [0, T∗ ). Therefore, |x(t) − x(tj )| ≤ M |t − tj |.
Letting j → ∞, we obtain |x(t) − x∗ | ≤ M |t − T∗ |.
38
Next, we apply Picard’s theorem to obtain a solution y ∈ C 1 ([T∗ , T∗ + ]; RN )
of the ODE, with initial condition y(T∗ ) = x∗ . Applying Lemma (39) we obtain a
solution z ∈ C 1 ([0, T∗ + ]; RN ), z(t) = x(t), t < T∗ , z(t) = y(t), t ∈ [T∗ , T∗ + ], to
the IVP. Thus, x was not maximal, which means that our initial assumption that x
is bounded and T∗ < ∞ must be false.
Remark 4.13. The local Lipschitz assumption (39) was made for the sake of
simplicity only. It is in fact enough to require that f ∈ C([T0 , T1 ) × U ; RN ), for some
open set U ⊂ RN , and that for any compact subset V × [T0 , T2 ] ⊂ [T0 , T1 ) × U , f
satisfies the Lipschitz condition
|f (t, x) − f (t, y)| ≤ L|x − y|
for t ∈ [T0 , T2 ], x, y ∈ V,
where L may depend on V and on T2 .
We then distinguish the cases: (i) T∗ = T1 ; (ii) dist(x(t), ∂U ) → 0 as t → T∗ ; (iii)
|x(t)| → ∞ as t → T∗ .
(Consider the following exercise as an example.)
Exercise 4.10. Consider the scalar graviational IVP,
r̈(t) = −
1
,
r2
r(0) = r0 > 0,
ṙ(0) = 0.
Prove that (i) Picard’s Theorem applies, i.e. there exists T > 0 and a solution
r ∈ C 1 ([0, T ]; R), but (ii) that there cannot exist a global solution. (Note that, since
the right-hand is not locally Lipschitz, our maximal solution theorem does not apply
but we can still establish “blow-up” in finite time; however see Remark 4.13.)
4.3.1
Global existence
Definition 4.14. We say that the solution to an IVP exists globally if the maximal
solution exists on [0, +∞).
One approach to understanding whether solutions experience blow-up or exist
globally, is to impose growth conditions on f that limit the rate of growth of local
solutions, for example,
|f (t, x)| ≤ c(b(t) + |x|α ),
(43)
where b ∈ C(R), α ≥ 0.
Exercise 4.11. Suppose f satisfies (43). Under which conditions on b, α can you
deduce global existence?
Consider first the case b = 1, general α. Then consider the case α = 1, general
15
b.
Sometimes, very specific properties of an IVP can lead to global solutions.
Exercise 4.12. Let N = 1, f (t, x) = f (x) and f (0) = f (1) = 0, and f is Lipschitz
in [0, 1]. Show that, if x0 ∈ [0, 1] then there exists a global solution.
15
HINT: Use Gronwall’s lemma to construct positive examples. Consider Example 4.10 to construct negative examples.
39
Does the solution have a limit as t → +∞?
For a wide class of physical models, we would be very disappointed if solutions
did not exists globally. Indeed, if such a result were to fail then this would cast
some doubt on the validity of the model, or it might shed some light on its range of
validity (cf. gravity!).
Exercise 4.13. Let E ∈ C 2 (RN ) and assume that ∇2 E is bounded on sub-levels:
for each e > 0, let Le := {x ∈ RN |E(x) ≤ e}, then supx∈Le k∇2 E(x)k < +∞.
Moreover, let M, C ∈ RN ×N be symmetric and positive definite. Show that the
solution to the IVP
M ẍ + C ẋ = −∇E(x),
x(0) = x0 , ẋ0 = ẋ0 ,
exists globally.
4.4
16
Linear IVPs again
We now return to completing our theory of the general linear IVP (20).
Theorem 4.15. Let A ∈ C k ([0, ∞); RN ×N ), b ∈ C k ([0, ∞); RN ), and k ≥ 0, and
x0 ∈ RN . Then there exists a unique global solution x to the IVP
ẋ = A(t)x + b(t),
x(0) = x0 ,
which belongs to x ∈ C k+1 ([0, T ); RN ).
Proof. Let f (x, t) := A(t)x+b(t). For each T < ∞ we have A ∈ C([0, T ]; RN ×N ), b ∈
C([0, T ]; RN ), hence we have
a := max kA(t)kF < +∞ and b := max |b(t)| < +∞.
t∈[0,T ]
t∈[0,T ]
It is easy to see that a is a Lipschitz constant in RN , t ∈ [0, T ], i.e., if f (t, x) =
A(t)x + b(t), then
|f (t, x) − f (t, y)| ≤ a|x − y|
for t ∈ [0, T ], x, y ∈ RN .
In particular Theorem 4.12 applies. Let x be a maximal solution with maximal
existence interval [0, T∗ ), T∗ ≤ T < ∞. Following the calculation in § 3.5 (uniqueness
for linear IVPs) we multiply the ODE with x(t), then apply the Cauchy–Schwarz
and Cauchy inqualities, to obtain
d 1
|x|2
dt 2
≤ (Ax) · x + |b · x|
≤ kAkF |x|2 + 21 |b|2 + 21 |x|2
= 21 (α|x|2 + β),
where α = 2a + 1 and β = b.
16
HINT: Recall the energy-estimates from Exercise .
40
Using Gronwall’s Inequality (cf. Exercise 3.10),
ξ˙ ≤ αξ + β
⇒
ξ ≤ eαt ξ(0) + β e
αt −1
α
,
we obtain that ξ(t) is bounded in any finite interval, [0, T∗ ). This means, that blowup cannot occur, and hence the solution is global.
The regularity is easy to deduce.
The next result is the immediate generalisation of Corollary 3.4 to nonhomogeneous coefficients.
Corollary 4.16 (Homogeneous Solution Space). Let A ∈ C([0, T ); RN ×N ),
then the space
S := x ∈ C 1 ([0, T ); RN )ẋ = Ax
is a linear vector space over R with dimension N .
Exercise 4.14. Prove Corollary 4.16.
[[ removed non-examinable section ]]
5
Stability
In most applications, the data of an IVP, e.g.,
• the initial value x0 ,
• the coefficients A(t), b(t) in a linear ODE,
• or some physical parameters (mass, damping coefficient, etc) in a nonlinear
PDE
are never known exactly but only approximately. However, if the error is small, then
we would expect that the resulting solutions are close.
More generally, every model is just a model and only an approximation to reality (or to better models). We are therefore looking for a mechanism to estimate
(qualitatively or quantitatively) the accuracy of our models.
Such statements almost feel like uniqueness results (if the data is the same then
the solutions are the same), and indeed the constructive uniqueness proof in section
3.5 leads to a stablity result of the kind we are now seeking. The main technical
tool, proved by a small variation of that argument, is Gronwall’s Inequality, of which
many variants exist.
A more general motivation for the studies in this section is that all models are
models. The machinery we are developing here will allow us to assess how “close”
different models are. For example, we will be able to make the formal derivation
of simplified models rigorous and precisely quantify when they are valid (and when
they are not).
41
5.1
Gronwall’s Inequality
There are many variants of Gronwall’s inequality. A particularly simple (but often
powerful) variant is a slight generalisation of the one we introduced in Exercise 3.10:
Lemma 5.1 (Gronwall Inequality 1). Let α, β ∈ C([0, T ]), ξ ∈ C 1 ([0, T ]), and
ξ˙ ≤ αξ + β,
then
Rt
ξ(t) ≤ e
0
α(r) dr
Z
t R
t
e
ξ(0) +
s
α(r) dr
β(s) ds.
0
Proof. Multiply with the integrating factor e−
Rt
0
α(r) dr
.
Next, we introduce an integral-version that is convenient to use later on in this
module. But recall, that there are many other variants that can be useful in other
circumstances.
We motivate the Generalized Gronwall’s inequality by considering two linear
IVPs,
ẋi = Axi + bi ,
xi (0) = xi,0 , i = 1, 2, t ∈ [0, T ).
where A ∈ RN ×N , bi ∈ C([0, T ); RN ) and xi,0 ∈ RN .
Suppose that the error in the “data” is small,
|b1 − b2 | ≤ εb ,
|x1,0 − x2,0 | ≤ ε0 ,
can we then deduce that the solutions remain close? (We could also perturb A; see
Exercise 5.1.)
Let e := x2 − x1 . Subtracting the equation for x1 from the equation for x2 , we
obtain
ė = ẋ2 − ẋ1
= A(x2 − x1 ) + b2 − b1
= Ae + η,
where η := b2 (t) − b1 (t). This is called an error equation.
We integrate this equation to obtain
Z t
Z t
e(t) = e(0) +
Ae(s) ds +
η(s) ds.
0
0
Taking the norm on each side,
Z t
Z t
|e(t)| ≤ |e(0)| + η(s) ds + Ae ds
0
0
Z t
Z t
≤ |e(0)| +
|η(s)| ds +
kAk|e| ds
0
0
Z t
≤ ε0 + tεb + kAk
|e| ds
0
Z t
=: α
|e| ds + β(t),
0
42
where α := kAk and β := ε0 + tεb . This is an integral version of the inequality we
derived in Exercise 3.10. We now derive the corresponding Gronwall inequality. [[
For the exam, the formula in part (a) need not be memorised. ]]
Lemma 5.2 (Generalised Gronwall Inequality). (a) Suppose that α, β, ξ ∈
C([0, T ]; R) and
Z
t
α(s)ξ(s) ds + β(t),
ξ(t) ≤
t ∈ [0, T ],
0
and α(t) > 0 for all t, then
Z
ξ(t) ≤ β(t) +
t
α(s)β(s)e
Rt
s
α(r) dr
ds,
t ∈ [0, T ].
(44)
0
(b) If, in addition, β(s) is monotonically increasing, then
ξ(t) ≤ β(t)e
Rt
0
α(s) ds
.
(45)
Proof. [[ this proof is not examinable in the general form. Special cases can be asked.
]]
Rt
(a) Let Φ(t) := 0 αξ ds, then
Φ̇(t) = α(t)ξ(t) ≤ αΦ + αβ.
By Gronwall 1, Lemma 5.1, and since Φ(0) = 0, we obtain
Z t R
t
e s α dr α(s)β(s) ds,
Φ(t) ≤
0
and inserting back into the previous line,
Z t R
t
e s α dr α(s)β(s) ds + α(t)β(t).
α(t)ξ(t) ≤ α(t)
0
Dividing through by α completes (a).
(b) If β is monotonically increasing, then (a) gives
Z t
Rt
α(r)
dr
α(s)e s
ds .
ξ(t) ≤ β(t) 1 +
0
The integrand satisfies, α(s)e
Rt
s
α(r) dr
d
= − ds
e
Rt
s
α(r) dr
, hence integrating gives (45).
Exercise 5.1. Consider the linear IVPs
ẋi = Ai xi + b,
xi (0) = x0 ,
i = 1, 2,
on an interval [0, T ]. Suppose that supt∈[0,T ] kA1 (t) − A2 (t)kF ≤ εA . Use the generalized Gronwall inequality to derive an estimate of the form
kx1 − x2 k∞,[0,T ] ≤ CA εA .
43
and try to make the constant CA as small as you can.
[[ The following exercise is not examinable, but the concept how a negative exponent α can be used to obtain improved estimates can be required in the exam.
(However, this does not require the more complex gronwall inequality used here.) ]]
Exercise 5.2 (Gronwall with Decay*).
(a) An advantage of Lemma 5.1
over the generalized GI, Lemma 5.2 is that α may be strictly negative. To see this,
consider the scalar IVPs
ẋi (t) = axi (t),
xi (0) = xi,0 ,
i = 1, 2; where a < 0.
Derive error estimates with both variants of the Gronwall Inequality.
(b) To apply this to a vectorial problem with forcing terms, we have to work a
bit harder. Consider the IVPs
ẋi (t) = Axi (t) + bi (t),
xi (0) = xi,0 .
and suppose that A is negative definite, i.e., there exists α < 0 such that
(Ah) · h ≤ α|h|2
∀h ∈ RN .
Moreover, let |b1 (t) − b2 (t)| ≤ β(t). Mimicking the uniqueness proof in § 3.5, derive
the error inequality
d 2
ξ ≤ −αξ 2 + βξ,
dt
where |ξ(t)| = |x1 (t) − x2 (t)|. (Note that ξ 2 is differentiable.)
(c) We now need a corresponding Gronwall inequality. Prove the following result:
Lemma 5.3. Suppose that α, β, ξ ∈ C([0, T ]) with ξ 2 ∈ C 1 ([0, T ]), ξ, β ≥ 0, and
d 2
ξ
dt
≤ 2αξ 2 + βξ.
Then,
Rt
ξ(t) ≤ e
0
α(r) dr
Z
ξ(0) +
t R
t
e
s
α(r) dr
β(s) ds.
(46)
0
Rt
R t Rt
Sketch of proof. Let η(t) := e 0 α(r) dr ξ(0)+ 0 e s α(r) dr β(s) ds. Let ε > 0 be arbitrary
Rt
and define Iε := {t ∈ [0, T ]|ξ(t) ≥ η(t)+εe 0 α dr }. Let t0 := inf Iε . Since ξ(t0 ) ≥ ε > 0
it follows that ξ is differentiable in a neighbourhood of t0 . Hence the differential
inequality
for ξ implies ξ˙ ≤ αξ + β, and from this one can deduce that ξ(t) ≤
Rt
α
η(t) + εe 0 dr for t0 ≤ t < t + δ. This gives a contradiction. Hence Iε must be empty.
Since ε was arbitrary, this proves the result. 17
(d) Deduce from (b) and (c) that
Z
αt
|x1 (t) − x2 (t)| ≤ e |x1,0 − x2,0 | +
t
eα(t−s) |b1 (s) − b2 (s)| ds.
0
17
(Alternatively, if one knows that absolutely continuous functions satisfy the fundamantal
theorem of calculus, one can skip this complicated proof and just write ξ˙ ≤ αξ + β right away.)
44
5.2
Well-posedness of the general IVP
We will now consider the well-posedness of the general nonlinear IVP. We consider
this in full generality. Let T > 0, U ⊂ RN open, let f, g ∈ C([0, T ]×U ), and consider
the IVPs
ẋ(t) = f (t, x),
ẏ(t) = g(t, y),
and
(47)
x(0) = x0 ,
y(0) = y0 ,
We think of the IVP for x as the “exact” model and of the IVP for y as a
“approximate” model, i.e., y0 ≈ x0 and g ≈ f (in some sense). We can now use the
generalized Gronwall inequality to easily estimate the “model error”.
Theorem 5.4. Consider the two IVPs (47), with solutions x, y ∈ C 1 ([0, T ); U ),
and define the following constants:
• Lipschitz condition: L := supt∈[0,T ],x6=y∈U
|f (t,x)−f (t,y)|
,
|x−y|
• Model error: εM := supt∈[0,T ],x∈U |f (t, x) − g(t, x)|.
Suppose that L, εM < ∞, then
|x(t) − y(t)| ≤ eLt |x0 − y0 | + teLt εM .
Proof. We take the difference of the two IVPs in (47) and integrate to obtain
Z t
x(t) − y(t) = x0 − y0 +
f (s, x(s)) − g(s, y(s)) ds
Z0 t = x0 − y 0 +
f (s, x(s)) − f (s, y(s)) ds
Z t 0
+
f (s, y(s)) − g(s, y(s)) ds.
0
Taking the modulus on each side we estimate
Z t
f (s, x(s)) − f (s, y(s)) ds
|x(t) − y(t)| ≤ |x0 − y0 | +
0
Z t
f (s, y(s)) − g(s, y(s)) ds
+
0
Z t
≤ |x0 − y0 | + L
|x(s) − y(s)| ds + tεM .
0
Applying (45) with α = L and β(t) = |x0 − y0 | + tεM yields the result.
Lt
REMARK: Applying (44) instead of (45) would give us e L−1 εM instead of teLt εM
which is a slightly better estimate.
45
5.3
Regular perturbation theory
Theorem 5.4 gives a general mechanism how to measure the accuracy of a model,
or the dependence on parameters. A closely related question is, how to construct
simplified models when we are given a more complex equation (we have already
encountered this in § 2.1). Perturbation theory is one mechanism that helps us in
this endeavour.
Example 5.5. Let us look at a particularly simply example:
ẋε (t) = εx(t) + g,
x(0) = 0, ε ≥ 0.
(48)
Then
eεt − 1
.
ε
We can now Taylor expand this solution in ε:
xε (t) = g
xε (t) = gt + g 21 t2 ε + g 16 t3 ε2 + O(ε3 ).
Now, suppose we turn the procedure around and insert an assumed expansion
into (48): xε = x0 + εx1 + ε2 x2 + O(ε3 ), then formally
ẋ0 + εẋ1 + ε2 ẋ2 + O(ε3 ) = εx0 + ε2 x1 + O(ε3 ) + g.
x0 (0) + εx1 (0) + ε2 x2 (0) + O(ε3 ) = 0.
Equating coefficients of εk , we obtain equations for the expansion coefficients:
ẋ0 = g,
x0 (0) = 0,
ẋ1 = x0 ,
x1 (0) = 0,
ẋ2 = x1 ,
...
x2 (0) = 0.
Solving these gives x0 (t) = tg, x1 (t) = 21 t2 g and x2 (t) = 16 t3 g.
We did not just get lucky, but this is indeed a general procedure that can be
carried out for a wide variety of problems. Consider a general IVP with a parameter
ε,
ẋε = f (ε; t, xε )
xε (0) = z(ε).
(49)
where f is differentiable with respect to ε and continuous with respect to t, x and
satisfies a suitable local Lipschitz condition so that the IVP is well-posed (at least
locally). In addition we now also require that f is differentiable with respect to x.
Example 5.6 (0th Order Perturbation Theory). To gain some first intuition
about the methods used to make perturbation theory rigorous, it may be worthwhile
to first look at the 0th order only. Letting ε → 0 in (49) we simply obtain
ẋ0 = f (0; t, x),
x0 (0) = z(0).
46
For simplicity, also assume that k∂x f k∞ , k∂ε f k∞ are both finite (i.e., f is globally
Lipschitz jointly in ε ∈ [0, 1] and x ∈ RN ). In particular x, xε both have global
solutions now. Then,
Z
t
xε (t) − x(t) = f (ε; s, xε ) − f (0; s, x) ds
0
Z t
f (ε; s, xε ) − f (0; s, x) ds
≤
Z0 t
≤
L(ε + |xε − x0 |) ds.
0
Applying the generalised Gronwall inequality, we obtain
kxε − x0 k∞,[0,T ] ≤ ε LT eLT .
We only develop to first order, and hence make a general ansatz for xε = x0 +
x1 ε + O(ε2 ). Inserting this into the IVP gives
ẋ0 + εẋ1 + O(ε2 ) = f (ε; t, xε )
= f (0; t, xε ) + ε∂ε f (0; t, xε ) + O(ε2 )
= f 0; t, x0 + εx1 + O(ε2 ) + ε∂ε f 0; t, x0 + O(ε) + O(ε2 )
= f (0; t, x0 ) + ε ∂x f (0; t, x0 )x1 + ε∂ε f (0; t, x0 ) + O(ε2 ).
Expansion of the initial condition simply yields
x0 (0) + εx1 (0) + O(ε2 ) = z(0) + ε∂ε z(0) + O(ε2 ).
Equating coefficients we obtain
ẋ0 = f (0; t, x0 ),
x0 (0) = z(0)
ẋ1 = ∂x f (0; t, x0 )x1 + ∂ε f (0; t, x0 ),
x1 (0) = ∂ε z(0).
(50)
(51)
Theorem 5.7 (First-Order Perturbation Theory). Suppose that f ∈
N
2
C 2 ([0, 1]() × [0, ∞)(t) × RN
(x) ; R ) and z ∈ C ([0, 1]() ). Then there exists T ∈ R
such that unique solutions xε , x1 , x0 ∈ C 1 ([0, T ]; RN ) to (49), (51) exist in [0, T ] and
moreover
xε − (x0 + εx1 )
≤ Cε2 ,
∞,[0,T ]
where C may depend on T and on properties of f .
The final time T may be chosen to be any T < T∗ where T∗ is the blow-up time
for x0 . The estimate is then valid for all ≤ 0 (T ).
Proof. [[ This proof is examinable, but in general it will be allowed to assume that
solutions x, x exist in a time interval [0, T ) that is independent of . In special cases
it will be required to establish this assumption. It is also possible that refined proofs
47
for special cases are asked in the exam. If the multi-variate differentiation aspects
are daunting, then make sure you understand the proof for N = 1 at least. ]]
Since f is continuously differentiable with respect to x and ∂x f is bounded in
all compact subsets of [0, 1] × [0, T ] × RN , it follows from Theorem 4.12 that there
exist unique maximal solutions for xε , x0 , x1 . Let T 0 be strictly less than any of the
three corresponding blow-up times. Then there exists a compact set U ⊂ RN such
that xε (t), x0 (t), x1 (t) ∈ U for all t.
Let e := xε − x0 − εx1 , then the ODEs give
ė = f (ε; t, x0 + εx1 + e) − f (0; t, x0 ) − ε∂ε f (0; t, x0 ) − ε∂x f (0; t, x0 )x1 =: RHS
We rewrite the RHS as follows:
h
i
RHS = f (ε; t, x0 + εx1 + e) − f (ε; t, x0 + εx1 )
h
i
+ f (ε; t, x0 + εx1 ) − f (0; t, x0 ) − ε∂ε f (0; t, x0 ) − ε∂x f (0; t, x0 )x1
=: RHSLIP + RHSEXP.
For the first term we simply apply a Lipschitz estimate,
RHSLIP ≤ L|e|,
since all trajectories are contained in U and f ∈ C 1 (U ).
The second term is formally O(ε2 ) since this is how we derived the equations for
x0 , x1 . To make this precise, we simply note that, for fixed, x0 , x1 , t, and
ψ() := f (ε; t, x0 + εx1 ),
where have
RHSEXP ≤ ψ() − ψ(0) − ψ 0 (0) ≤ M1 2 ,
where M1 ≤ |ψ 00 (θ)| ≤ k∇2 f k∞,[0,1]×U < ∞. (Note that,
ψ 00 (θ) = ∂ε2 f (θ; x0 + θx1 ) + ∂ε ∂x f (θ; x0 + θx1 ) + ∂x2 f (θ; x0 + θx1 ) .
Thus, we have shown that
RHS ≤ L|e| + M1 ε2 ,
for some constants L, M that depend on bounds of the derivatives of f in [0, 1] ×
[0, T 0 ] × U . Integrating the error equation, and inserting the expansion for z, we
obtain
Z t
2
|e(t)| ≤ |e(0)| + M1 ε + L
|e| ds
0
Z t
2
≤ Mε + L
|e| ds,
0
since e(0) = z(ε) − z(0) − ε∂ε z(0) = 21 ε2 ∂ε2 z(θ3 ).
Applying Lemma 5.2 (either (a) or (b)) we obtain at |e(t)| ≤ Cε2 for all t ∈ [0, T 0 ].
To complete the theorem, we still need to show that T 0 is independent of . See
exercise below.
48
[[ removed non-examinable section ]]
Exercise 5.3 (Mild nonlinearity). Consider the IVP ẋ = Ax + f (x), where
kf k∞,RN < ∞, A ∈ RN ×N , x(0) = x0 .
(i) Derive a formal first-order perturbation expansion and explicitly compute
x0 , x1 in terms of matrix exponentials and Duhamel’s formula.
(ii) State a rigorous approximation result (either derive this directly, or employ
Theorem 5.7).
Exercise 5.4. Apply the first-order perturbation expansion to the IVP ẍ + x +
εx3 = 0, x(0) = 1, ẋ(0) = 0. State a rigorous approximation result.18
Exercise 5.5 (Pendulum 2).
Consider again the pendulum equation from
Exercise 2.6:
k
g
ϕ̈ + ϕ̇ + sin(ϕ) = 0,
m
l
ϕ(0) = ε, ϕ̇(0) = 0, compared with the linearised pendulum
ψ̈ +
g
k
ψ̇ + ψ = 0,
m
l
ψ(0) = 1, ψ̇(0) = 0.
Apply the first-order perturbation theory to this problem to prove rigorously
that
|ϕ(t) − εψ(t)| ≤ C(t)ε2 .
Derive an explicit bound on C(t). Hence, state on which time-scale you expect the
linearised solution to be accurate.
6
Numerical Simulation using Euler’s Method
In § 5.3 we have discussed a methodology for constructing simplified models in order
to obtain good or at least qualitative approximations of models that are difficult to
solve analytically. But in many cases even these simplified models can be difficult
(impossible) to solve. In other cases there are no natural approximations to make.
Numerical simulation provides a tool to study a model (a differential equation) in
a relatively straightforward fashion. Of course nothing comes for free, and indeed
there are many potential pitfalls in numerical simulation as well, some of which we
will study here.
6.1
Construction of Euler’s method and some experiments
Consider again the general explicit first order IVP (28), with solution x ∈
C 1 ([0, T ); RN ). Since x is differentiable, we can approximate x near t = 0 with
an affine function:
x(h) = x(0) + hẋ(0) + O(h2 ).
18
HINT: You need not convert this to a first-order system to carry out the expansion.
49
y
(x2 , y2 )
(x1 , y1 )
y(x)
(x0 , y0 )
x
Figure 3: Geometric construction of Euler’s method. The green lines represent
solutions with different initial values.
Inserting the fact that ẋ = f (t, x), we obtain
x(h) = x0 + hf (0, x0 ) + O(h2 ).
Suppose we define
x1 := x0 + hf (0, x0 ).
We can now take x1 as a new initial value at time t = h, and define
x2 := x1 + hf (h, x1 ).
Iterating the procecure leads to the Explicit Euler Method: given the initial
condition x0 at t = 0, and a step-size h > 0, let
xn+1 := xn + hf nh, xn ,
n = 0, 1, 2, . . . .
(52)
(We will call it simply the Euler Method.)
Remark 6.1 (Geometric Description of Euler’s Method).
scalar IVP,
y 0 (x) = f (x, y(x)),
y(0) = y0 ,
Consider a
with some (locally unique) solution y(x). Then we can can approximate the curve
(x, y(x) near (0, y0 ) by replacing it with its tangent. The process can the be iterated
as visualized in Figure 3. This is of course equivalen to the above construction.
The key question that arises: does Euler’s method in any meaningful sense provide approximations to the solution of the IVP? From its construction, we expect
that the error in Euler’s method is of the order O(h). (Not O(h2 ), since we need to
take O(1/h) steps to reach some finite time t.) Before we begin to investigate this
rigorously, we perform some simple numerical tests.
50
6
exact
h = 0.1
h = 0.2
h = 0.4
h = 0.5
5
4
3
x(t)
2
1
0
−1
−2
−3
−4
0
0.2
0.4
0.6
0.8
1
t
1.2
1.4
1.6
1.8
2
Figure 4: Euler’s method applied to ẋ = −5x. We observe that, for decreasing h we
get more and more accurate approximations. For large h the numerical approximation is even qualiatively completely wrong.
(a) A linear problem: The simplest problem is ẋ = ax, x(0) = 1, where a ∈ R.
We obtain of course x(t) = eat , and we compare this for a = −5 against
solutions of Euler’s method for h = 0.1, 0.2, 0.4, 0.5 in Figure 4. We observe
that, for decreasing h we get more and more accurate approximations. For
large h the numerical approximation is even qualitatively completely wrong.
This is something to investigate further.
(b) Nonlinear pendulum: With euler’s method in hand we can now explore solutions of the damped nonlinear pendulum (cf. Example 2.6) for large θ (though
we can also do this using phase plane analysis). In Figure 5 we show the Euler’s
method approximations to the solutions for both the linear and nonlinear pendulum equations with initial values θ(0) = 0.9π, θ̇(0) = 0, and step h = 0.004.
We can clearly distinguish the linear from the nonlinear behaviour (despite
both being periodic).
(c) N -body problem: For general N , we cannot compare against an exact solution,
so let us look at N = 2. In this case, one shows that the motion of each
of the particles acts like a one-body problem about the centre of mass of
the two particles. One can then show that the trajectories are ellipsi. (See
MA134: Geometry and Motion, central force theory) For example, if we write
x1 (t) = x(t), x2 (t) = −x(t), x(0) = (1, 0), ẋ(0) = θ̇0 (0, 1), then r(t) = r(θ) can
be written as r(θ) = ((1 − (2θ̇0 )−2 ) cos(θ) + (2θ̇0 )−2 )−1 .
We observe that the trajectory quickly deviates from the exact solution and
gives a completely wrong qualitative prediction. Decreasing the mesh size still
shows convergence, but it is clear that this method is not suitable for simulating
51
3
nonlinear
linear
2
θ(t)
1
0
−1
−2
−3
0
5
10
15
20
t
25
30
35
40
Figure 5: Euler’s method applied to the linear and nonlinear pendulum equations.
the long-term behaviour of the many-body problem.
6.1.1
Convergence for linear systems
Let us consider a linear, constant coefficient initial value problem
ẋ = Ax,
x(0) = x0 ,
where A ∈ RN ×N . Then, the exact solution is given by x(t) = eAt x0 . Euler’s method,
with step h > 0, for this problem reads
xn+1 = xn + hAxn = (I + hA)xn ,
and hence, the solution can be written as
xn = (I + hA)n x0 .
Let tn := nh, then we can write also as
xn = I +
tn A
n
n
x0 ,
and we note that the polynomial (I + tnnA )n is in fact a common approximation to
etn A , if A were scalar.
If A is diagonalisable, A = T −1 DT , where D = diag(di ) is diagonal, then we
obtain
n
xn = T −1 I + tnnD T x0
−1
tn di n
= T diag 1 + n
T x0 .
52
h = 0.05
2
1
1
0
0
−1
−1
−2
−2
−1
0
h = 0.01
2
1
−2
−2
2
−1
0
1
2
Figure 6: Euler’s method applied to the two-body problem. We observe that the
trajectory quickly deviates from the exact solution and gives a completely wrong
qualitative prediction. Decreasing the mesh size still shows convergence.
Thus, if we hold t ≡ tn = nh fixed and let h → 0, n → ∞, we obtain that
xn → T −1 diag etdi T x0 = etA x0 = x(t).
That is, Euler’s method is convergent. We will generalize this to first-order nonlinear
systems in §6.2.
Remark 6.2. More generally, one can show that, since pn (z) = (I + (z/n))n → ez
uniformly in any compact subset of C, it follows that pn (tA) → etA as n → ∞, for
any matrix A ∈ RN ×N . But we will just treat this case together with the nonlinear
case.
6.2
Convergence Result
The idea, similarly as our “error estimates” in § 5.2 and § 5.3, is to write an equation
for the error and then use a Gronwall-type argument to estimate the magnitude of
the solution.
The standard approach, which lies at the foundation of almost every convergence
proof of a numerical scheme (or indeed any other convergence proof), is to “insert”
the exact solution into the approximation scheme, and estimate the error by which
it fails to satisfy the approximation scheme.
Suppose, therefore, that x ∈ C 1 ([0, T ); RN ) is the unique solution to the IVP
(28), and for simplicity of notation let tn = hn, where h is the Euler step size, then
we define the truncation error Tn ∈ RN through the formula
⇔
x(tn+1 ) = x(tn ) + hf (tn , x(tn )) + hTn ,
x(tn+1 ) − x(tn )
= f (tn , x(tn )) + Tn
h
that is,
Tn :=
x(tn+1 ) − x(tn )
− f (tn , x(tn )).
h
53
(53)
Since the discrete trajectory (xn )n∈N satifies the Euler scheme xn+1 = xn +
hf (tn , xn ), defining the error en := x(tn ) − xn , we obtain
en+1 = en + h f (tn , x(tn )) − f (tn , xn ) + hTn .
Taking the norm on either side, applying the triangle inequality, and assuming that
f satisfies a Lipschitz condition, yields
|en+1 | ≤ (1 + hL)|en | + h|Tn |.
(54)
This is clearly reminiscent of the differential inequality from which we obtained
Gronwall’s Lemma. Indeed, we can derive a discrete variant, specialized for this
particular problem.
Lemma 6.3 (Discrete Gronwall Inequality). Let ξn ∈ R satisfy
ξn+1 ≤ αξn + βn ,
for n = 0, 1, . . . ,
then
n
ξn ≤ α ξ0 +
n−1
X
αn−1−i βi .
i=0
Proof. The result is best proved by induction. For n = 0, the result is trivial. For
n ≥ 1, we have
ξn ≤ αξn−1 + βn−1
= αα
n−1
ξ0 + α
n−2
X
αn−2−i βi + βn−1
i=0
= αn ξ0 +
n−1
X
αn−1−i βi .
i=0
Specializing the discrete Gronwall inequality to (54), and observing that (1 +
hL)n ≤ enhL = eLtn , we obtain
|en | ≤ etn L |e0 | + h
n−1
X
eL(tn −ti ) |Ti |.
i=0
Since we have x(0) = x0 , e0 = 0. Also estimating eL(tn −ti ) ≤ eLT for tn ≤ T , we
obtain
n−1
X
LT
|en | ≤ e h
|Ti |.
i=0
We are left to estimate the truncation error. Expanding to second order (note
that for this, we will need that x ∈ C 2 ([0, T ]; RN )),
x(tn+1 ) = x(tn + h) = x(tn ) + ẋ(tn )h + 21 cn h2 ,
54
where cn = (ẍi (θi ))N
i=1 , for some θi ∈ (tn , tn+1 ), and using the fact that ẋ = f (t, x),
we obtain Tn = 12 cn h. Or, estimating more crudely,
|Tn | ≤ Ch,
for tn+1 ≤ T,
where C = 12 kẍk∞,(0,T ) .
Thus, we have proven the following result:
Theorem 6.4. Let f ∈ C 1 ([0, T ]×RN ), let x ∈ C 2 ([0, T ]; RN ) be the solution to the
first order IVP (28), and let (xn )n∈N be the discrete trajectory obtained by Euler’s
method with step size h. Suppose there exists R independent of h ∈ (0, 1] such that
|x(t)|, |xn | ≤ R
for t, tn ≤ T,
then there exists C independent of h such that
x(tn ) − xn ≤ Ch
for tn = nh ≤ T.
Exercise 6.1 (*). It is possible to remove the assumption that |x(t)|, |xn | ≤ R
in Theorem 6.4. The theorem then reads as follows:
Let f ∈ C 1 ([0, T ] × RN ), let x ∈ C 2 ([0, T∗ ); RN ) be the unique maximal solution
to the first order IVP (28). Fix T < T∗ .
Then there exists h0 > 0 and C0 > 0 (both may depend on T ) such that, if
h ∈ (0, h0 ] and (xn )n∈N is the discrete trajectory obtained by Euler’s method with
step size h, then
x(tn ) − xn ≤ C0 h
for tn = nh ≤ T.
(55)
Prove this result, using the following outline (or otherwise):
1. Fix h0 , C0 > 0, to be specified later and define T := {T 0 ∈ (0, T ]| (55) holds
with T = T 0 }.
Show that, if T 0 ∈ T , h ≤ h0 then |xn − x0 | ≤ R where R = kx − x0 k∞,[0,T ] +
C0 h0 .
2. Let L := k∂f k∞,[0,T ]×BR (x0 ) and M := kf k∞,[0,T ]×BR (x0 ) .
Show that, if T 0 ∈ T , then |x(tn ) − xn )| ≤ C1 h for tn ≤ T 0 , where C1 is
independent of T and h ∈ (0, h0 ].
3. Let T 0 := max T . Choose C0 and h0 in such a way that T 0 < T would give a
contradiction.
6.3
Long-time behaviour
To conclude our analysis of the Euler method, we present a few interesting directions,
that would be covered in more details in a focused module on the numerical solution
of differential equations.
55
6.3.1
Region of Stability
We want to investigate the issue of the oscillatory and growing solutions of Euler’s
method that we observed in our first numerical experiment in § 6.1. To this end, we
consider again the linear scalar IVP ẋ = ax, x(0) = 1, with a < 0. Following the
foregoing computations, we have x(t) = eat , and
xn = (1 + ha)n .
We note that, while x(t) → 0 as t → ∞, we have xn → 0 as n → ∞ if and only if
0 < h < 1/|a|.
A numerical scheme, for which we would obtain xn → 0, independently of the
choice of h is called A-stable. This is a key concept for so-called stiff equations, which
are normally characterised by a slow evolution with fast timescales for “nearby”
trajectories. In the above example, 0 is the slow trajectory and x(t) converges rapidly
towards it. The fast timescales restrict the stepsize, even though to approximate the
solution itself we could take a much larger step. Suppose, for example, that in the
“exact” problem, we have x(0) = 0, but due to round-off or error in the data, we
have x(0) = 1. Although ẍ ≡ 0, which means we could take arbitrarily large
steps, we will get an exponentially growing solution unless we choose h sufficiently
small.
Exercise 6.2. (a) Let A ∈ RN ×N be diagonalisable with eigenvalues λ1 , . . . , λN ,
<(λi ) < 0. Show that x(t) → 0 as t → ∞, where x is the solution to ẋ = Ax, x(0) =
x0 .
(b) Apply the Euler method to this linear IVP, to obtain a discrete trajectory
xn , n = 0, 1, 2, . . . . Show that xn → 0 as n → ∞, for any initial value x0 , if and only
if |1 + hλi | < 1 for i = 1, . . . , N .
Exercise 6.3. Consider the Implicit Euler Method,
xn+1 = xn + hf h(n + 1), xn+1 .
(There is an issue associated with solving the nonlinear system, but we can ignore
this for the present exercise.)
Apply this to the linear IVP ẋ = ax, a < 0, x(0) = 1, which yields a discrete
trajectory (xn )∞
n=0 ⊂ R. Show that xn → 0 as n → ∞, without restriction on the
step size h.
6.3.2
Energy identity for Hamiltonian systems
Even if we do choose our time step “sufficiently small” according to the analysis of
the last section, we still see, in the two-body problem, that the trajectories escape
to infinity rather than remaining bounded as is suggested by conservation of energy.
To see what happens, we look at a linear scalar example (linear pendulum),
ẍ + x = 0,
x(0) = x0 ,
ẋ(0) = ẋ0 ,
which observes the energy conservation formula
d
H
dt
= 0,
where H = 12 |x|2 + 21 |ẋ|2 .
56
Euler’s method, first formulated for the system ẋ = p, ṗ = −x reads
xn+1 = xn + hpn ,
pn+1 = pn − hxn .
Multiplying the first equation with xn , the second equation with pn and adding them
up, yields
xn+1 xn + pn+1 pn = x2n + hpn xn + p2n − hxn pn
= x2n + p2n .
Adding 21 x2n+1 + 12 p2n+1 on either side, we can rewrite this as
1 2
x
2 n+1
+ 12 p2n+1 − 21 x2n − 12 p2n = 12 x2n+1 − xn+1 xn + 12 x2n + 12 p2n+1 − pn+1 pn + 12 p2n
= 12 |xn+1 − xn |2 + 21 |pn+1 − pn |2 .
Thus, if we define the Hamiltonian Hn := 12 x2n + 21 p2n , and assume that |xn+1 −
xn | = O(h), then we can write this most naturally in the form
Hn+1 − Hn
|xn+1 − xn |2 |pn+1 − pn |2
=h
+
,
h
h2
h2
or, equivalently,
n
1X
|xi+1 − xi |2 |pi+1 − pi |2
+
.
Hn = H0 + h
h
2 i=1
h2
h2
In particular, if x0 6= 0, then Hn → ∞ as n → ∞ and therefore |xn | + |pn | → ∞.
Stated more precisely, the energy grows by a factor O(h) over an O(1) time interval.
This is clearly undesirable if we want to simulate the long-term behaviour of a
Hamiltonian system. Much work has gone into developing and studying numerical
methods for ODEs that do preserve energy (in some meaningful way); see the literature on “geometric numerical integration” or “symplectic integrators”. We introduce
one particular example below.
Interestingly, for a practicioner (scientist) it is often unimportant to recover a
good approximation to a specific trajectory, but it is crucial to preserve certain statistical properties of the exact trajectory in the discrete trajectory. For this reason,
schemes that conserve energy and possibly other key quantities of interest are of
huge importance in computational science. We investingate one example in the next
two exercises.
Exercise 6.4. (a) Apply the implicit Euler method (cf. Exercise 6.3) to the linear
pendulum ẍ = x, and show that it satisfies the discrete energy identity
n
|xi+1 − xi |2 |pi+1 − pi |2
1X
Hn = H0 − h
h
+
.
2 i=1
h2
h2
Deduce that all discrete trajectories tend to zero.
(b) The Euler-A method is obtained by taking one of the
xn+1 = xn − hpn
pn+1 = pn + hxn+1 .
57
Show that there exists an “approximate Hamiltonian” H̃n = Hn + O(h) which is
conserved, i.e., H̃n+1 = H̃n . Deduce that the discrete trajectory is O(h) close to the
continuous trajectory in the following sense:
∃C s.t. min |x(t) − xn | + |ẋ(t) − pn | ≤ Ch,
t∈R
where ẍ = x, x(0) = x0 , ẋ(0) = p0 .
Exercise 6.5. Formulate explicit Euler, implicit Euler and Euler-A for the ODE
ẍ = Ax, where A ∈ RN ×N is spd, and generalise the respective energy identities. Remark 6.5. In pseudocode, we can write the Euler method and Euler-A method,
respectively, as
Euler:
Euler-A:
xtemp ← x
x ← x − hp
x ← x − hp
p ← p + hx
p ← p + hxtemp
That is, Euler-A is even more efficient to implement since it doesn’t require additional storage.
Exercise 6.6. Consider the Euler-B method for a Hamiltonian system,
pn+1 = pn − h∇E(xn )
xn+1 = xn + hpn+1 .
For the case of a quadratic energy, does this method create, dissipate, or conserve
energy?
6.4
Peano’s theorem
[[ removed non-examinable section ]]
7
Error Growth in (Damped) Hamiltonian Systems
[[ this section is not examinable. (only some brief examples were covered at the end
of the module) ]]
8
Autonomous Systems
In this section, we begin to study autonomous systems of ODEs. We assume throughout that f : RN → RN is locally Lipschitz and study solutions of
ẋ(t) = f (x),
x(0) = x0 .
58
(56)
Note that most of the explicit examples we have considered so far are of this
autonomous form.
For technical reasons it is now much more convenient to solve (56) both forwards
and backwards in time. Upon simply making the coordinate transformation t → −t,
we obtain ẋ = −f (x). Since −f (x) is also locally Lipschitz, it follows from Picard’s
Theorem that this system also has a unique solution for short time. By piecing
together the forward and backward solutions (see Lemma 4.9) we see that there
exists T > 0 such that a unique solution x ∈ C 1 ([−T, T ]; RN ) to (56) exists.
Our theory in previous sections was primarily concerned with short time behaviour of solutions to IVPs (i.e., in a finite time interval), whereas we will now
focus more on the long-time behaviour.
8.1
Maximal solutions
We begin our study of autonomous systems by establishing some further details of
their solutions.
First, Theorem 4.12 translates as follows:
Corollary 8.1. Under our standing assumptions, there exist T− , T+ ∈ (0, ∞] and
a unique x ∈ C 1 ((T− , T+ ); RN ) solving (56), with T± being maximal.
If |T± | < ∞ then |x(t)| → ∞ as t → T± .
Further, we obtain the following Lemma, which encodes the fact that the system
is autonomous.
Lemma 8.2. Let x ∈ C 1 ((T− , T+ ); RN ) be a maximal solution to ẋ = f (x), then
1. For any t0 ∈ R, y(t) := x(t − t0 ) is a maximal solution with existence interval
(T− − t0 , T+ − t0 ).
2. If z ∈ C 1 ((T−0 , T+0 ); RN ) is also a maximal solution and x(t0 ) = z(t00 ), then
T± − t0 = T±0 − t00 .
Proof. Both results follow immediately from the fact that f does not depend on t.
1. Since ẏ(t) = ẋ(t−t0 ) = f (x(t−t0 )) = y(t), y solve the ODE in (T− −t0 , T+ −t0 ).
To see that it is maximal, we we use the fact that, either, T± = ±∞ and hence
T± − t0 = ±∞; or alternatively, |x(t)| → ∞ as t → T± and hence |y(t)| → ∞ as
t → T± − t0 .
2. Let y(t) := x(t − t0 + t00 ), then y is a maximal solution with existence interval
(T− −t0 +t00 , T+ −t0 +t00 ). Moreover, y(t00 ) = x(t0 ) = z(t00 ). Since y, z are both maximal
solutions with the same ‘initial condition’ at t = t0 , it follows that y ≡ z. Hence,
they must have the same existence interval, which implies T± − t0 = T±0 − t00 .
The domain of f , RN in our case, is called phase space. The curve {x(t)|t ∈
(T− , T+ )} ⊂ RN , where x is a maximal solution, is called a trajectory or an orbit.
Note that, x(t) and x(t − t0 ) give rise to the same trajectories.
The set of all trajectories is called the phase portrait.
Theorem 8.3. Let f ∈ C(RN ; RN ) be locally Lipschitz, then for each x0 ∈ RN
there exists a unique maximal solution with x(0) = x0 and hence a unique maximal
trajectory passing through x0 .
59
Each maximal solution x ∈ C 1 ((T− , T+ ); RN ) falls into one of the following three
categories:
1. Equilibrium: ẋ(t) = 0 for some t ∈ (T− , T+ ). In this case, T± = ±∞ and
ẋ ≡ 0.
2. Periodic Orbit: ẋ(t) 6= 0 for all t and there exist t1 > 0 such that x(t1 ) = x(0).
In this case, T± = ±∞, and x is periodic with period t1 (and possibly with a
another smaller period).
3. ẋ(t) 6= 0 for all t and x(t) 6= x(s) for all t, s ∈ (T− , T+ ).
Proof. We have already established the uniqueness of maximal solutions and Lemma
8.2 establishes that all trajectories through x0 are identical.
The categorisation into the three cases is obvious. We are only left to prove the
stated properties in cases 1, 2.
1. If ẋ(t0 ) = 0 for some t0 , then f (x(t0 )) = 0 and hence x(t) := x(t0 ) is a solution.
2. If x(t1 ) = x(0), then by defining z(t) := x(t−nt1 ) for t ∈ [nt1 , (n+1)t1 ), n ∈ Z
defines a solution of (56) with existence interval R and it is therfore maximal. Since
maximal solutions are unique it follows that x = z and in particular it is periodic
with period t1 .
Example 8.4. Let f ∈ C ∞ (R2 ; R2 ) be defined by
−x2
x1
2
2
f (x) :=
+ (1 − x1 − x2 )
.
x1
x2
Then f is locally Lipschitz.
The only equilibrium is the point x = 0. For all other trajectories, let x(t) =
r(t)(cos θ(t), sin θ(t)), then r(t) = |x(t)| is differentiable since x(t) 6= 0, and we can
derive
ṙ = (1 − r2 )r, and θ̇ = 1.
We note that, if r(0) = 1, then ṙ ≡ 0, θ(t) = θ0 + t is a solution, hence, the unit
circle is a periodic trajectory.
For r(0) 6= 1 we obtain the two solutions
r(t) =
1/2
1
,
1 + e−2t
and r(t) =
1/2
1
.
1 − e−2t
The first solution tends to 1 as t → ∞ and to 0 as t → −∞, while the second solution
tends to 1 as t → ∞ and to ∞ as t → −∞. These correspond to spirals converging
to the unit circle, and we can therefore sketch the phase portrait in Figure 7.
60
Generated by CamScanner from intsig.com
Figure 7: Phase portrait for the autonomous ODE described in Example 8.4.
8.2
Remark on dynamical systems
[[ From this section, only the definition of the flow-map is required, which is generally
a useful concept. ]]
The autonomous IVP (56) is a special case of a dynamical system.
Definition: A dynamical system is a triple (G, M, Φ) where G is a semigroup
with identity element 0, M is a set (phase space), and Φ : U ⊂ G × M → M such
that
Φ(0, x) = x,
Φ(g2 , Φ(g1 , x)) = Φ(g1 + g2 , x) for (g1 , x), (g2 , x), (g1 + g2 , x) ∈ U.
In the context of autonomous ODEs, we let G := R, M := RN and define
Φ(t; x0 ) := x(t), where x is the maximal solution of (56) if t ∈ (T− , T+ ) and undefined
otherwise. U is taken to be the domain of definition. Φ is also called the flow map
or simply flow.
One may readily check that this satisfies the definition of a dynamical system.
To be precise, it is a continuous dynamical system. For discrete dynamical systems
we take G = Z or G = N.
Dynamical systems can be further studied in specialised advanced modules.
Exercise 8.1. (i) Can the trajectory
sin(t)
x(t) =
,
sin(2t)
t∈R
be the solution of a first-order autonomous system? Justify your answer.
(ii) Compute the flow map Φ(t, x) for the ODEs (a) ẋ = Ax, A ∈ RN ×N ; (b)
ẋ = x2 , x ∈ R.
(iii) Show that Φ(t, x) := et (1 + x) − 1 is a flow map and find the corresponding
autonomous ODE.
Exercise 8.2. Let f ∈ C 2 (RN ; RN ) and let Φ(t, x) be the associate flow map.
Show that Φ is Fréchet differentiable in x and compute the derivative ∂x Φ(t, x).
61
(HINT: Look at regular perturbation theory for the case when f = f (t, x) and
x0 = z().)
8.3
Stable equilibria
Lemma 8.5. Let x ∈ C 1 ((T− , T+ ); RN ) be a maximal solution to (56). If x(t) →
x+ ∈ RN as t → T+ , then T+ = +∞ and f (x+ ) = 0.
Proof. Since x+ ∈ RN it follows that x(t) is bounded as t → T+ and according to
Theorem 4.12 we have T+ = +∞.
To show that f (x+ ) = 0 we fix i ∈ {1, . . . , N } and note that, for each t < T+
there exists τi ∈ (t, t + 1) such that
xi (t + 1) − xi (t) = ẋi (τi ) = fi (x(τi )).
Since, as t → ∞, τi → ∞ and therefore x(τi ) → x+ , which implies that fi (x(τi )) →
fi (x+ ). Since xi (t + 1) − xi (t) → 0 we obtain that fi (x+ ) = 0.
Definition 8.6. A point x∗ ∈ RN is called a critical point of f (or, fixed point, or,
equilibrium point of the ODE ẋ = f (x)) if f (x∗ ) = 0. A critical point is called
• stable (in the sense of Lyapunov) if, for any δ > 0 there exists > 0 such
that, if x0 ∈ B (x∗ ) then x(t) ∈ Bδ (x∗ ) for all t > 0, where x is the maximal
solution to (56).
• asymptotically stable (or, an attractor) if it is stable and in addition there
exists > 0 such that, for all x0 ∈ B (x∗ ), x(t) → x∗ as t → ∞, where x is
the maximal solution to (56).
• exponentially stable, if there exists , α, C > 0 such that, for any x0 ∈ B (x∗ ),
and x the maximal solution to (56), we have
|x(t) − x∗ | ≤ Ce−αt |x0 − x∗ |.
(57)
• unstable, if it is not stable.
Clearly, exponential stability implies asymptotic stability and asymptotic stability
implies Lyapunov stability.
Exercise 8.3. According to our definition, is the fixed point x∗ = 0 of the system in Example 8.4, Lyapunov stable, asymptotically stable, exponentially stable,
unstable? What about the system
−x2
x1
2
2
f (x) :=
− (1 − x1 − x2 )
.
x1
x2
62
Due to the exponential nature of solutions to linear problems, exponential stability is often the easiest to verify.
Proposition 8.7. Let A ∈ RN ×N and b ∈ RN , and suppose that all eigenvalues of
A have negative real part. Then, there exists a unique critical point Ax∗ = b and it
is exponentially stable. Moreover, there exist C, α > 0 such that,
keAt kF ≤ Ce−αt .
Exercise 8.4. Prove Proposition 8.7.
19
Theorem 8.8. Suppose that f ∈ C 1 (RN ) has a fixed point x∗ and that all eigenvalues of ∂f (x∗ ) have negative real part. Then x∗ is exponentially stable.
Proof. [[ This proof will only be examined under simplifying assumptions, or by breaking it up into small pieces with some guidance. ]]
The basic idea is to apply regular perturbation theory and use that fact that
f (x) = 0 + ∂f (x∗ )(x − x∗ ) + o(|x − x∗ |). But because we only estimated errors for
finite time, this will not be quite sufficient.
Assume without loss of generality that x∗ = 0. Let A := ∂f (0), then Proposition
8.7 implies that
At e z ≤ Ce−αt |z|
∀z ∈ RN ,
(58)
for some α > 0, C ≥ 1. Since f ∈ C 1 there exists ω : [0, ∞) → [0, ∞), monotonically
increasing, such that ω(r) ↓ 0 as r ↓ 0 and |f (z) − Az| ≤ ω(|z|)|z| for all z ∈ RN .
Choose 0 > 0 such that 2ω(2C)/α ≤ 1 for all ≤ 0 .
Let x(t) be a maximal solution to (56) with |x0 | := ≤ 0 , then
ẋ(t) = f (x(t)) = Ax(t) + b(t),
where b(t) := f (x(t)) − Ax(t), and hence,
|b(t)| ≤ ω(|x(t)|)|x(t)|.
The Duhamel formula gives
Z t
At
|x(t)| = e x0 +
eA(t−s) b(s) ds
Z0 t
At A(t−s)
e
≤ e x0 +
b(s) ds
Z0 t
≤ Ce−αt +
e−α(t−s) ω(|x(s)|)|x(s)| ds.
0
19
Sketch of a possible proof: First, one changes basis, and thus reduces the problem to the
case when A is block diagonal, where each block is in Jordan normal form. Then we can suppose
(without loss of generality) that A is an N × N Jordan block. One then checks that there exist
N linearly independent solutions of ẋ = Ax (e.g., solve the IVP recursively) of the form x(t) =
(p1 (t), . . . , pN (t))e−αt where pi (t) are polynomials. From this the result can be readily derived.
63
Figure 8: Illustration of the Hartmann–Grobmann theorem: The local phase portrait
is homeomorphic to the phase portrait of the linearised equation.
Let T > 0 be maximal such that |x(t)| ≤ 2C for all t ∈ [0, T ] (such T clearly
exists). Then we can further estimate, for t ∈ [0, T ],
Z t
−αt
e−α(t−s) dt
|x(t)| ≤ Ce + 2Cω(2C)
0
2ω(2C)
≤ Ce−αt + C
α
−αt
≤ C e
+ 1 < 2C.
Thus, unless T = +∞, it cannot be maximal. We therefore obtain that x is a global
solution and that |x(t)| ≤ 2C for all t ≥ 0.
So far we have proven that
Z t
−αt
|x(t)| ≤ Ce + ω(2C)
e−α(t−s) |x(s)| ds,
0
which we can write, equivalently, as
Z
ξ(t) ≤ C + ω(2C)
t
ξ(s) ds,
0
where ξ(t) := eαt |x(t)|. Applying the Generalised Gronwall’s Inequality (45), we
obtain that
ξ(t) ≤ Cetω(2C) ,
or equivalently,
|x(t)| ≤ Cet[ω(2C)−α] |x0 |,
and we note that ω(2C) − α ≤ −ω(2C) < 0. (Moreover we note that, in the limit
→ 0, we obtain the rate e−αt .) This completes the proof of exponential stability. Remark 8.9. In Theorem 8.8 we have seen that the long-term behaviour of the
ODE ẋ = f (x) near x∗ is qualitatively the same as for the linearised equation
64
u̇ = ∂f (x∗ )u where u = x − x∗ ; in the sense that |x(t) − x∗ | ≤ Ce−αt . This was,
provided that the eigenvalues of ∂f (x∗ ) had negative real part.
We would expect this to be true much more generally, since we can always use
the fact that
|f (x∗ + u) − ∂f (x∗ )u| ≤ ω(|x|)|x|,
that is, the linear part dominates the flow and hence the phase portrait.
This intuition is made rigorous by the Hartmann–Grobmann theorem. [[ the next
result is not required for the exam ]]
Theorem 8.10 (Hartmann–Grobmann). Let f ∈ C 1 (RN ; RN ) and let Φ(t, x) be
the corresponding flow map. If x∗ = 0 is a critical point of f such that all eigenvalues
of ∂f (0) have non-zero real component, then there exists ε > 0 and a homeomorphism ϕ : RN → RN such that
ϕ(etA u) = Φ(t; ϕ(u))
for u, etA u ∈ B (0).
Exercise 8.5. Check the stability of x∗ = 0 for
−x2
x1
2
2
f (x) :=
± (1 − x1 − x2 )
.
x1
x2
using Theorem 8.8.
Exercise 8.6. 1. Consider ẋ = Ax for A ∈ RN ×N . Show that, if A has at least
one eigenvalue with positive real part, then x∗ = 0 is unstable.
2. Consider ẋ = f (x) with critical point x∗ . If all eigenvalues of ∂f (x∗ ) have
non-positive real part then this gives us no information about the stability of x∗ .
Demonstrate this by considering the following two examples:
1. ẋ = µx3 , for µ 6= 0.
2. Let µ ∈ R and
f (x) :=
−x2
x1
+
µ(x21
+
x22 )
x1
x2
.
[[ removed non-examinable section ]]
Exercise 8.7. (i) Suppose f (x∗ ) = 0 and there exists δ > 0 such that
(f (x) − f (x∗ )) · (x − x∗ ) ≤ 0
∀x ∈ Bδ (x∗ ).
(In this case, f is called monotone.) Then x∗ is stable.20
(ii) Suppose that f (x∗ ) = 0 and there exist δ, α > 0 such that
(f (x) − f (x∗ )) · (x − x∗ ) ≤ −α|x − x∗ |2
20
HINT: show that L(x) = |x − x∗ |2 is a Lyapunov function
65
∀x ∈ Bδ (x∗ ),
then x∗ is exponentially stable.
Exercise 8.8. 1. Consider the ODE ẋ = −xp , p ∈ N. Show that x∗ = 0 is
asymptotically stable if p is odd, and for p even it is unstable.
2. Turn the linear pendulum equation ẍ = −x into a first-order system. Is the
equilibrium (x∗ , ẋ∗ ) = (0, 0) stable, asymptotically stable, unstable?
3. Show that (0, 0) is an exponentially stable equilibrium of the damped nonlinear
pendulum equation ẍ + cx + sin(x) = 0, where c > 0.
Exercise 8.9. (i) Show that, if x∗ is an exponentially stable equilibrium of ẋ =
f (x), then it is also stable in the sense of Lyapunov.
(ii) Consider the autonomous system
x1 x2
,
|x|
x2
ẋ2 = x1 + x2 − x2 |x|2 − 1 .
|x|
ẋ1 = x1 − x2 − x1 |x|2 +
Show that x(t) → 0 for all trajectories with |x(0)| < 1, but that x∗ = 0 is not stable
in the sense of Lyapunov.
8.4
The method of Lyapunov
The method of Lyapunov (or, second method of Lyapunov; or direct method of
Lyapunov) is a universal tool in the analysis of equilibria, in cases where linearisation
techniques fail.
To motivate the idea, we first consider the gradient flow equation
ẋ(t) = −∇E(x),
where E ∈ C 2 (RN ; R). Then we have
d
E(x(t)) = ∇E(x(t)) · ẋ(t) = −|∇E(x(t))|2 .
dt
That is, E(x(t)) is strictly decreasing. If x∗ is an isolated local minimizer then we
would expect that starting close to x∗ the system is driven towards x∗ . (We will
make this precise later on.)
The method of Lyapunov replaces the energy with a generic functional L ∈
1
C (RN ; R). Then we have
d
L(x(t)) = ∇L(x(t)) · f (x(t)),
dt
that is, if ∇L · f ≤ 0, then L is decreasing. From this property, we can deduce
stability (or instability) of equilibria.
Theorem 8.11. Suppose that x∗ is a critical point of f , that L ∈ C 1 (RN ; R) such
that x∗ is an isolated local minimizer of L. We assume without loss of generality
that L(x∗ ) = 0.
66
1. Suppose there exists δ > 0 such that ∇L(x) · f (x) ≤ 0 for all x ∈ Bδ (x∗ ), then
x∗ is stable.
2. Suppose there exists δ > 0 such that ∇L(x) · f (x) < 0 for all x ∈ Bδ (x∗ ) \ {x∗ },
then x∗ is asymptotically stable.
3. If there exists δ > 0 such that ∇L · f > 0 in Bδ (x∗ ) \ {x∗ }, then x∗ is unstable.
Remark 8.12. In cases 1, 2 in Theorem 8.11 we call L a Lyapunov function,
respectively strict Lyapunov function, for x∗ .
Proof. Because x∗ is an isolated local minimizer there exists δ0 > 0 such that, for
K := Bδ (x∗ ) and for any 0 < δ < δ0 ,
µ := inf L(x) > 0.
x∈∂K
Moreover, let V := {x ∈ K|L(x) < µ}. Since L is continuous it immediately follows
that V is an open neighbourhood of x∗ .
1. Let δ0 be chosen sufficiently small so that ∇L · f ≤ 0 in Bδ0 (x∗ ).
Fix δ ≤ δ0 and let x(t) be a maximal solution with x0 ∈ V . We claim that T∗ =
+∞ and x(t) ∈ K for all t > 0. For contradiction, assume that there exists t1 > 0
such that x(t) ∈ K c . Since x is continuous there exists t2 < t1 such that x(t2 ) ∈
∂K. Let t2 be minimal. (Since ∂K is compact, it can be chosen that way.) Since
L(x(t2 )) = µ but L(x(t)) ≤ L(x0 ) < µ since ∇L · f ≤ 0, we have a contradiction.
Since V is open, there exists > 0 such that B (x∗ ) ⊂ V , and every trajectory
starting in B will remaind in K = Bδ . Therefore, x∗ is stable.
(REMARK: If δ > δ0 , then choose according to δ = δ0 and this will suffice.)
2. Choosing δ sufficiently small, we ensure that ∇L · f < 0 for x ∈ Bδ (x∗ ) \ {x∗ }.
Fix such a δ. Arguing as in 1, we show that if x0 ∈ V , then there exists a unique
global solution with x(t) ∈ V for all t > 0. Since V is open we can inscribe a ball
B (x∗ ) and hence to prove asymptotic stability we only need to show that x(t) → x∗
as t → ∞.
Since L(x(t)) is monotonically decreasing, it follows that L(x(t)) ↓ L0 as t → ∞,
for some L0 ≥ 0. This implies that L(x(t)) ≥ L0 for all t, hence we define
W := {x ∈ K|L0 ≤ L(x)} and M := max ∇L(x) · f (x).
x∈W
If L0 > 0 then, according to our assumption, M < 0, and hence we obtain that
Z t
L(x(t)) − L(x(0)) =
∇L(x(s)) · f (x(s)) ds ≤ tM → −∞, as t → ∞.
0
But this contradicts the fact that L(x(t)) ≥ L0 > 0. Therefore, L0 = 0; that is,
L(x(t)) → 0 as t → ∞.
Suppose now, for contradiction, that there exists a subsequence tj → ∞ such
that |x(tj ) − x∗ | ≥ > 0, then L(x(tj )) 6→ 0, which contradicts what we have just
proven. Therefore x(t) → x∗ as t → ∞.
67
3. Let δ be given by the assumption and K, V as above. We claim that, if x0 ∈ V ,
then x(t) ∈ ∂K for some t > 0. This would immediately contradict stability.
Let T > 0, maximal, such that x(t) ∈ K for all t ∈ [0, T ]. Let
M := max ∇L(x) · f (x)x ∈ K, L(x0 ) ≤ L(x) ,
then M > 0 and since L(x(t)) ≥ L(x0 ), it follows that
Z t
∇L(x(s)) · f (x(s)) ds ≥ tM.
L(x(t)) − L(x0 ) =
0
Therefore, T < +∞, which implies that x(T ) ∈ ∂K.
Remark 8.13. Lyapunov functions are sometimes obtained from physical idea,
e.g., L(x) = E(x) for gradient flows, or can sometimes simpy be “guessed”. Often
the construction of a Lyapunov function is difficult (or even impossible). A useful
technique is to make a guess with free parameters and then adjust the parameters
in order to make L a Lyapunov function.
Exercise 8.10. 1. Let f (x) = −xp , p ∈ N. Show that, if p is odd, then L(x) = x2
is a Lyapunov function for x∗ = 0. Deduce that x∗ = 0 is stable.
2. Let f (x) = −∇E(x) for E ∈ C 2 (RN ). If x∗ is an isolated local minimiser,
show that L(x) = E(x) is a Lyapunov function for x∗ . If it is also an isolated critical
point, then L = E is a strict Lyapunov function. Deduce in the first case that x∗ is
stable and in the second case that x∗ is asymptotically stable.
Show that, if ∇2 E(x∗ ) is positive definite, then x∗ is exponentially stable.
Exercise 8.11. Characterise the stability of (0, 0) of
(i)
(ii)
(iii)
ẋ = −x + y 2 ,
ẋ = −x5 − xy,
ẋ = −xy 2 + xy,
ẏ = −2yx − y 3 ,
ẏ = −y 3 + x4 ,
ẏ = −2x2 − y 3 .
[[ removed non-examinable section ]]
Remark 8.14 (First integrals). A first integral of an ODE ẋ = f (t, x) is a
d
H(t, x(t)) = 0 along solutions of the ODE. It is easily
function H(t, x) such that dt
seen that this is equivalent to ∂t H(t, x) + ∂x H(t, x)f (x) ≡ 0 in RN .
In the case of Hamiltonian systems we have seen that first integrals are useful to
construct Lyapunov functions. This is more generally the case as we will see later.
Further, first integrals prescribe hypersurfaces in phase space on which trajectories
evolve.
68
8.4.1
Basin of attraction
Let x∗ be a critical point of f : RN → RN , locally Lipschitz, then the basin of
attraction of x∗ is defined as
Σ(x∗ ) := x0 ∈ RN Φ(t; x0 ) → x∗ as t → ∞ ,
where Φ is the propagator of the ODE ẋ = f (x).
Lyapunov’s method can be used to obtain estimates on the basin of attraction:
Proposition 8.15. Suppose the conditions of Theorem 8.11, 2, hold. Fix µ > 0 and
define
Kµ to be the component connected to x∗ of the sublevel set {x ∈ RN |L(x) <
µ .
If Kµ is compact, then the set Vµ := {x ∈ Kµ |L(x) < µ} belongs to the basin of
attraction of x∗ .
Proof. Exercise.
8.5
Stability in Damped Hamiltonian Systems
Consider the damped Hamiltonian system

 ẍ + C ẋ + ∇E(x) = 0,
x(0) = x0 ,

ẋ(0) = p0 ,
⇔




ẋ
ṗ
x(0)



p(0)
=
=
=
=
p
−Cp − ∇E(x),
x0 ,
p0 ,
(59)
where E ∈ C 2 (RN ; R) and C ∈ RN ×N .
Lyapunov’s method immediately gives us the following stability result.
Corollary 8.16. If C = 0 and x∗ ∈ RN is an isolated local minimizer of E, then
it is a stable equilibrium of the ODE (59).
Proof. L(x, p) = 12 |p|2 + E(x) is a Lyaponov function for x∗ .
Suppose now that C is positive definite, i.e., hT Ch ≥ c0 |h|2 for all h ∈ RN . If we
try to apply Lyapunov’s method with L = 21 |p|2 + E(x), then we obtain
∇L(x, p) · f (x, p) = ∇x L · p + ∇p L · (−Cp − ∇E(x))
= ∇E(x) · p − p · Cp − p · ∇E(x)
= −pT Cp
≤ −c0 |p|2 ≤ 0.
But it is always false that ∇L(x, p) · f < 0 in any neighbourhood of z∗ = (x∗ , 0).
One “way out” is of course to apply the exponential stability result, Theorem 8.7:
Corollary 8.17. Let x∗ be a local minimizer of E, such that ∇2 E(x∗ ) is positive
definite. (In particular, x∗ is then an isolated critical point.) Suppose, moreover, that
C ∈ RN ×N is symmetric and positive definite.
69
Then, (x∗ , 0) is an exponentially stable equilibrium of the damped Hamiltonian
system
ẋ = p
ṗ = −Cp − ∇E(x).
Proof. Let H := ∇2 E(x∗ ).
The plan is of course to apply Theorem 8.8. To that end, we need to show that
all eigenvalues of the block matrix
0
I
A :=
−H −C
have negative real part. Let λ be an eigenvalue and (x, p) the associated eigenvector,
then
p = λx,
−Hx − Cp = λp.
If x = 0 then p = 0, which is a contradition; hence x 6= 0.
Inserting the first into the second equation, we obtain
−Hx − Cλx = λ2 x.
Multiply with x∗ (transpose and conjugate) and assume (wlog) that |x|2 = x∗ x = 1,
then
−x∗ Hx − λx∗ Cx = λ2 .
Let 2b := x∗ Cx > 0 and c := x∗ Hx > 0, then we obtain
λ2 + 2bλ + c = 0,
and one readily checks that all roots of this equation have negative real part.
The last result is restricted to local minima when ∇2 E(x∗ ) is positive definite.
This will not always be the case. We observe, however, that ∇L · f < 0, strictly,
unless p = 0. Moreover, if p = 0 and x 6= x∗ , then ∇E(x) 6= 0 and therefore the
system will not spend any time in such a state and we therefore expect that “most
of the time” ∇L · f < 0. This gives us hope that z(t) → z∗ as t → ∞.
The technical tool that we require is called La Salle’s invariance principle:
Theorem 8.18. Consider the autonomous ODE ẋ = f (x) and suppose f (x∗ ) = 0.
Let L ∈ C 1 (RN ) satisfy the following conditions:
(i) x∗ is an isolated local minimizer of L and an isolated critical point;
(ii) ∃δ > 0 such that ∇L(x) · f (x) ≤ 0 for all x ∈ Bδ (x∗ );
(iii) If x is a maximal solution contained in Bδ (x∗ ) and ∇L(x(t))·f (x(t)) ≡ 0 (i.e.,
L is constant along x), then x ≡ x∗ .
70
Then x∗ is stable and asymptotically stable
Proof. (i) and (ii) imply that x∗ is stable.
Let x(t) ∈ Bδ (x∗ ) for all t > 0. Then L(x(t)) is decreasing and hence L(x(t)) ↓ L0 .
Let ω be the limit set of x,
ω := lim x(tj )tj ↑ ∞, limit exists .
For each z ∈ ω with x(tj ) → z we have
L(z) = lim L(x(tj )) = L0 ,
That is, L is constant in ω.
Now let y0 ∈ ω and let ẏ = f (y), y(0) = y0 be a maximal solution. From
x(tj ) → y0 and Gronwall’s inequality, it follows that x(tj + t) → y(t) for all t ∈ R
and hence it follows that y(t) ∈ ω for all t. (ω is an invariant set; this is an important
lemma in dynamical systems)
Thus, we conclude that dtd L(y(t)) = 0. (iii) therefore implies that ω = {x∗ }. Corollary 8.19. Let x∗ be an isolated local minimizer of E and let C be positive
definite. Then x∗ is a stable and asymptotically stable equilibrium of the damped
Hamiltonian system (59).
Proof. Clearly, we have (i) and (ii) with L(z) = 21 |p|2 + E(x).
Let z be a maximal solution contained in Bδ and suppose ∇L(z(t)) · f (z(t)) ≡ 0
for all t, then p(t) = 0 for all t. Hence, x is constant. Therefore x(t) ≡ x0 and
p(t) ≡ 0. But this implies that ∇E(x∗ ) = 0. Since we assumed that x∗ is not only an
isolated local minimizer but also an isolated critical point, it follows that x0 = x∗ .
Thus, (iii) is satisfied as well, and we obtain the stated result.
8.6
Examples of planar autonomous systems
[[ removed non-examinable section ]]
References
[Tes12] Gerald Teschl.
Ordinary Differential Equations and Dynamical Systems, volume 140 of Graduate Studies in Mathematics.
AMS, Providence, Rhose Island, 2012.
online version:
http://www.mat.univie.ac.at/ gerald/ftp/book-ode/ode.pdf.
71