1 REVIEW OF BASIC ODES
1
1
Review of Basic ODEs
[Much of the material in this section is adapted from various editions of differential equations
texts by Edwards and Penney, and Boyce and DiPrima.]
A differential equation is an equation containing an unknown function and its derivatives
as well as independent variables. To solve a differential equation is to find a function that,
when substituted into the differential equation, produces an identity. Such a function is a
solution to the differential equation. If the unknown function is a function of a single independent variable, then the differential equation is an ordinary differential equation (ODE).
If the unknown function is a function of more than one independent variable and partial
derivatives occur in the equation, then it is a partial differential equation (PDE). These
notes will concern ODEs.
1.1
Separable First Order ODEs
The order of a differential equation is the order of the highest order derivative that appears
dy
= H(x, y) is separable if H may be written in the
in the equation. A first order ODE dx
g(x)
form H(x, y) = g(x)φ(y) = f (y) for some functions g, f , and φ. That is, H must be a
product of two functions, one of which involves only x and the other of which involves only
y. The solution
by separating the variables and (after a change of variable)
R is then found
R
integrating: f (y) dy = g(x) dx + C. Solving the resulting equation for y (if possible) will
result in an explicit solution.
= kx.
Example: dx
dt
Solution:
Z
Z
dx
= k dt
x
⇒ ln x = kt + C
⇒ x = Cekt .
Note the behavior of the arbitrary constant in this example. When we moved from the
penultimate step, we used the exponential function on each side of the equation. That
means that the right hand side becomes ekt+C = eC ekt = Cekt . It’s certainly not the case
that we’re claiming the eC = C for some particular constant C. Instead we’re saying that if
C is some arbitrary constant, then eC is some arbitrary constant as well and, assuming no
other complications are at hand, we may just as well call it C.
A solution containing one or more arbitrary constants with the property that every solution to the differential equation may be found by correctly choosing the values of the
constants is called a general solution. It is usually the case that the number of arbitrary constants in a general solution to a given differential equation equals the order of the differential
equation.
The next example demonstrates how to solve an initial value problem. An initial value
problem is a differential equation along with a set of initial conditions. That is, the value of
the unknown solution (and possibly its derivatives) is given for some value of the independent
variable. To solve an initial value problem, you first find the general solution of the differential
equation. Then you use the initial condition to find the value of the arbitrary constants.
1 REVIEW OF BASIC ODES
Example:
Solution:
dy
dx
= x2 y 2 ,
2
y(1) = −1.
Z
Z
dy
= x2 dx
2
y
1
−1
= x3 + C.
⇒
y
3
2
y(1) = −1 ⇒ C =
3
−1
1 3 2
⇒
= x +
y
3
3
−3
⇒y= 3
.
x +2
Here’s an example for you to try.
dy
Example: dx
= y4x+1 ,
y(0) = 1.
Solution?????:
1.2
First Order Linear ODEs
A differential equation is nth order if the highest order derivative that appears in the equation
is order n. A differential equation is linear if it is linear in the dependent variable and its
derivatives.
For the time being we will consider only nth order linear differential equations of the form
P0 (x)y (n) + P1 (x)y (n−1) + · · · + Pn (x)y = F (x)
(1)
where each Pi and F are continuous on some open interval I on which we seek a solution.
If P0 (x) 6= 0 on I, then we may divide by P0 (x) to obtain
y (n) + p1 (x)y (n−1) + · · · + pn (x)y = f (x).
(2)
(Note that if P0 (x) = 0 for some x ∈ I, then we can’t do the division that we did to obtain
(2). In that case we need to use more advanced techniques such as the series solutions in
section 3.)
If f (x) ≡ 0 on I, that is if
y (n) + p1 (x)y (n−1) + · · · + pn (x)y = 0,
(3)
we say the equation is homogeneous. Otherwise the equation is non-homogeneous.
In the first order case, equation (2) becomes
dy
+ P (x)y = Q(x).
dx
(4)
The solution to (4) follows from the product rule. Recall that for differentiable functions
f and g, Dx (f (x)g(x)) = f 0 (x)g(x) + f (x)g 0 (x). The strategy is to multiply (4) by an
1 REVIEW OF BASIC ODES
3
integrating factor ρ(x) and then choosing ρ(x) in such a way that the left side becomes
f 0 g + f g 0 for some functions f and g. Proceeding in this fashion
dy
dx
dy
ρ(x) dx
+ P (x)y = Q(x)
+ ρ(x)P (x)y = ρ(x)Q(x)
leads to the question: for what
function ρ(x) is it the case that Dx ρ(x) = ρ(x)P (x). The
R
answer is that ρ(x) = exp( P (x) dx). (Check it.) Making this choice of ρ and using the
product rule means that the equation is now
R
R
d
(exp( P (x) dx)y) = exp( P (x) dx)Q(x)
dx
R
R
R
⇒ exp( P (x) dx)y = exp( P (x) dx)Q(x) dx
R
R
R
⇒ y = exp(− P (x) dx) exp( P (x) dx)Q(x) dx.
The calculation just completed is the basis for the proof of the following
Theorem 1 (Existence and Uniqueness of Solutions) If P and Q are continuous on
an open interval I containing a point x0 , then, for any y0 ∈ R, the initial value problem
dy
+ P (x)y = Q(x),
dx
y(x0 ) = y0
has a unique solution on all of I.
Example: xy 0 + 3y = 2x5 , y(2) = 1.
Solution?????:
1.3
nth Order Linear ODEs
Given any constants c1 , c2 , . . . , cm ∈ R and any functions f1 , f2 , . . . , fm , the expression c1 f1 +
c2 f2 + · · · + cm fm is called a linear combination of the functions. We call a set of functions
f1 , f2 , . . . , fm linearly dependent on an open interval I if there exist constants c1 , c2 , . . . , cm ∈
R not all zero such that the linear combination c1 f1 + c2 f2 + · · · + cm fm ≡ 0 on I. A set of
functions that is not linearly dependent on I is called linearly independent on I.
The following theorems establish the framework for the solution of the nth order linear
homogeneous differential equation given in (3).
Theorem 2 (Principle of Superposition) Any linear combination of solutions to the
nth -order linear homogeneous differential equation (3) is also a solution to (3).
P
Proof: Let y1 , y2 , . . . , ym be solutions to (3), and let c1 , c2 , . . . , cm ∈ R. Let Y = m
i=1 ci yi .
1 REVIEW OF BASIC ODES
4
If we plug Y into the left hand side of (3), then we get
Y (n) + p1 (x)Y (n−1) + · · · + pn (x)Y
!(n)
!(n−1)
m
m
m
X
X
X
=
ci y i
+ p1 (x)
ci y i
+ · · · + pn (x)
ci y i
i=1
m
X
=
i=1
m
X
=
=
=
!
(n)
ci y i
!
(n−1)
ci y i
+ · · · + pn (x)
i=1
!
(n)
ci y i
i=1
m
X
+
m
X
!
(n−1)
ci p1 (x)yi
(n)
(n−1)
+ p1 (x)yi
i=1
m
X
ci y i
i=1
+ ··· +
i=1
ci (yi
i=1
m
X
+ p1 (x)
i=1
m
X
m
X
ci pn (x)yi
i=1
+ · · · + pn (x)yi )
ci 0
i=1
= 0.
The zero comes from the fact that each yi is a solution to (3). Thus Y is a solution to (3)
and the theorem is proved. Theorem 3 (Existence and Uniqueness of Solutions) Given the nth -order linear differential equation (2). Assume that each pi and f are continuous on some open interval I
containing a point a. If b0 , b1 , . . . , bn−1 ∈ R, then (2) has a unique solution on all of I such
that
y(a) = b0 , y 0 (a) = b1 , y 00 (a) = b2 , . . . , y (n−1) (a) = bn−1 .
The proof of this theorem requires some theorems from real analysis and, so, will be omitted.
Given a set of n (n − 1)-times differentiable functions f1 , f2 , . . . , fn , their Wronskian is
the n × n determinant given by
f1
f2
···
fn f10
f20
···
fn0 W =
(5)
.
..
..
..
.
.
.
(n−1) (n−1)
(n−1) f1
f2
· · · fn
The Wronskian may also be written as W (x) or W (f1 , f2 , . . . , fn ) depending on whether we
want to emphasize the value of the independent variable or the functions used to create the
Wronskian.
Theorem 4 (Wronskians of Solutions) Suppose that y1 , y2 , . . . , yn are n solutions to the
homogeneous nth -order linear equation (3) on an open interval I where each pi is continuous.
If W is the Wronskian of y1 , y2 , . . . , yn , then:
1 REVIEW OF BASIC ODES
5
1. If y1 , y2 , . . . , yn are linearly dependent on I, then W ≡ 0 on I.
2. If W (x0 ) 6= 0 for some x0 ∈ I, then y1 , y2 , . . . , yn are linearly independent on I.
Proof: (Part 1.) If y1 , y2 , . . . , yn are linearly dependent on I, then there exist c1 , c2 , . . . , cn
not all zero such that c1 y1 + c2 y2 + · · · + cn yn = 0 on I. Taking n − 1 derivatives of this
yields the system
c1 y 1 + c2 y 2 + · · · + cn y n = 0
c1 y10 + c2 y20 + · · · + cn yn0 = 0
..
..
.
.
(n−1)
(n−1)
(n−1)
c1 y1
+ c2 y 2
+ · · · + cn y n
= 0.
We can write this system as the matrix equation Y c = 0 where the determinant of Y is the
Wronskian W (y1 , . . . , yn ). Now we know that c 6= 0. Since 0 is a solution to the equation
Ax = 0 for any matrix A, it must be the case that Y is singular. That is, the determinant
of Y is zero. In other words, W (x) = 0 for all x ∈ I.
(Part 2) This is the contrapositive of Part 1. The next theorem states that if we can find n linearly independent solutions to an nth order homogeneous linear ODE, then we have, in effect, found every solution.
Theorem 5 (General Solutions) Let y1 , y2 , . . . , yn be n linearly independent solutions to
the nth -order homogeneous linear ODE (3) on an open interval I with each pi P
continuous.
If Y is any solution to (3), then there exist c1 , c2 , . . . , cn ∈ R such that Y (x) = ni=1 ci yi (x)
for each x ∈ I.
Proof: Let a ∈ I, let y1 , y2 , . . . , yn be linearly independent solutions to (3) on an open
interval I, and let Y be any solution to (3) on I. Consider the system
c1 y1 (a) + c2 y2 (a) + · · · + cn yn (a) = Y (a)
c1 y10 (a) + c2 y20 (a) + · · · + cn yn0 (a) = Y 0 (a)
..
..
.
.
(n−1)
(n−1)
(n−1)
c1 y 1
(a) + c2 y2
(a) + · · · + cn yn
(a) = Y (n−1) (a).
The Wronskian of this system is non-zero by Theorem 4, so the system
Pn has a solution
c 6= 0. Use the entries of this vector to define a function G by G(x) = i=1 ci yi (x). Then
G(a) = Y (a), G0 (a) = Y 0 (a), . . . , G(n−1) (a) = Y (n−1) (a) and both Y and G solve (3). So, by
uniqueness (Theorem 3), it must be the case that Y = G. 1.4
Homogeneous Constant Coefficient Linear ODEs
It will be convenient from time to time to use operator notation for differential equations.
We will generally use D or Dx in addition to d/dx to represent differentiation with respect
to x. The nth derivative can then be denoted by Dn .
1 REVIEW OF BASIC ODES
6
Defining more general operators will save us some ink. For example, consider the homogeneous constant coefficient linear ODE
an y (n) + an−1 y (n−1) + · · · + a1 y 0 + a0 y = 0
(6)
where each ai is a real constant. If we define the operator L to be
L = an Dn + an−1 D(n−1) + · · · + a1 D + a0 ,
then we may write (6) in the form Ly = 0—a much more concise formulation. It is reasonably straightforward to show that polynomials in the “variable” D, known as polynomial
operators, behave in the manner one would expect from polynomials while in fact being
linear operators.
One way of solving a homogeneous constant coefficient linear ODE is to use characteristic
polynomials. Given (6), the corresponding characteristic polynomial is
an rn + an−1 rn−1 + · · · + a1 r + a0 = 0.
(7)
This polynomial arises as follows. Suppose that we want to decide whether the function
y = erx is a solution to (6) for some constant r. If we plug erx into (6), we get
an (erx )(n) + an−1 (erx )(n−1) + · · · + a1 (erx )0 + a0 (erx ) = 0
⇔
an rn erx + an−1 rn−1 erx + · · · + a1 rerx + a0 erx = 0
⇔
an rn + an−1 rn−1 + · · · + a1 r + a0 = 0.
So y = erx is a solution to (6) if and only if r is a root of (7). Thus finding a solution to (6)
is equivalent to finding roots of (7). This root-finding results in three cases.
1.4.1
Distinct Real Roots
Suppose the roots of the characteristic polynomial are all real and distinct: r1 , r2 , . . . , rn .
Then the functions er1 x , er2 x , . . . , ern x are all solutions to (6). Since there are n solutions and
they are linearly independent (check it), then the general solution to (6) in this case is
y(x) = c1 er1 x + c2 er2 x + · · · + cn ern x .
1.4.2
Repeated Real Roots
Now suppose that the characteristic polynomial has only one root, r, of multiplicity n. Then
y = erx is a solution to (6), but to find the general solution, we need n − 1 more linearly
independent solutions. You can show (and should try to show) that in this case y = xk erx is a
solution to (6) for each k = 0, 1, 2, . . . , n−1 and that these solutions are linearly independent.
So in this case, the general solution is
y(x) = c1 erx + c2 xerx + c3 x2 erx + · · · + cn xn−1 erx
= (c1 + c2 x + c3 x2 + · · · + cn xn−1 )erx .
1 REVIEW OF BASIC ODES
1.4.3
7
Complex Roots
Now consider the case where (7) has a complex root. Since (7) is a polynomial with real
coefficients, then complex roots will appear in conjugate pairs in the form a ± bi.
We’ll now take a brief detour through some complex analysis though we won’t prove
any of the claims we make here. First of all, the Taylor series for ex , sin x, and cos x each
converge for all complex numbers just as they do for all real numbers. This fact and a bit
of calculation will establish Euler’s Formula: eiθ = cos θ + i sin θ.
If we forge blindly ahead, we can conclude that since a ± bi are roots of the characteristic
polynomial, then e(a+bi)x and e(a−bi)x are each solutions to (6). That means that a linear
combination of them is also a solution. That is, we can write a solution in the form
C1 e(a+bi)x + C2 e(a−bi)x =
=
=
=
C1 eax ebix + C2 eax e−bix
eax [C1 (cos bx + i sin bx) + C2 (cos(−bx) + i sin(−bx)]
eax [(C1 + C2 ) cos bx + i(C1 − C2 ) sin bx]
eax (c1 cos bx + c2 sin bx)
where c1 = C1 + C2 and c2 = i(C1 − C2 ). The conclusion is that the real-valued functions
eax cos bx and eax sin bx are two linearly independent solutions to (6) whenever a ± bi are
roots to the characteristic polynomial.
Repeated complex roots are accommodated in the same way as repeated real roots.
Example: We can illustrate what happens in the general case by this example. Suppose
the roots of the characteristic polynomial for a given constant coefficient linear homogeneous
ODE problem are 3, 3, 3, 5, 8, 2+7i, 2-7i. Then the general solution will be c1 e3x + c2 xe3x +
c3 x2 e3x + c4 e5x + c5 e8x + e2x (c6 cos(7x) + c7 sin(7x)).
1.5
Non-homogeneous Constant Coefficient Linear ODEs
Now we’ll consider the case of nth -order non-homogeneous constant coefficient linear ODEs.
That is, we’ll consider equations of the form
an y (n) + an−1 y (n−1) + · · · + a1 y 0 + a0 y = f (x)
(8)
where each ai is a real constant. If we again define the operator L to be
L = an Dn + an−1 D(n−1) + · · · + a1 D + a0 ,
then we may write (8) in the form Ly = f .
First notice that if yp is a given particular solution to Ly = f , and Y is any other solution
to Ly = f , then the function Y − yp is a solution to the homogeneous equation Ly = 0.
That’s because the linearity of the operator L implies that L(Y −yp ) = LY −Lyp = f −f = 0.
Given a particular solution yp and any other solution Y to Ly = f , we will call the function
Y − yp the complementary solution and denote it by yc . So in short, any solution Y to
Ly = f may be written in the form Y = yc + yp where yc is a solution to Ly = 0 and yp is
a given solution to Ly = f . Since we already have some techniques for solving Ly = 0, the
next few sections will focus on how to produce a single solution to Ly = f .
1 REVIEW OF BASIC ODES
1.6
8
Undetermined Coefficients
The method of undetermined coefficients is applicable to equation (8) (or, equivalently, the
equation Ly = f ) if the function f (x) is a linear combination of functions of the form
erx Pm (x) sin(kx) or erx Pm (x) cos(kx)
(9)
where Pm is a polynomial of degree m and r, k ∈ R.
The rationale for the method is this: since L is a linear combination of derivative operators, then functions f that are linear combinations of functions of the form (9) may only
be produced by L from functions that are of the form (9). That’s because derivatives of
polynomials are polynomials, derivatives of exponentials are exponentials, and derivatives of
sin or cos are either sin or cos. So a reasonable choice to make for a trial solution to (8)
is a linear combination of the terms of f and its derivatives. The only complication occurs
when some of the terms of f or its derivatives satisfy Ly = 0, the associated homogeneous
equation.
We may express the method as follows:
Case 1: If no term in f or any of its derivatives satisfies Ly = 0, then let yp be a linear
combination of all the linearly independent terms of f and its derivatives. Determine the
coefficients of yp by substituting yp into the equation Ly = f , equating the coefficients of
like terms, and solving the resulting system of equations.
Case 2: If some term in f or any of its derivatives satisfies Ly = 0, then for each term
of f of the form (9), let
yj = xs [(A0 + A1 x + · · · + Am xm )erx cos(kx)
+ ((B0 + B1 x + · · · + Bm xm )erx sin(kx)
where s is the smallest non-negative integer such that no term in yj satisfies Ly = 0. Let yp
be the sum of all the yj terms. Determine the coefficients of yp by substituting yp into the
equation Ly = f , equating the coefficients of like terms, and solving the resulting system of
equations.
Example: Solve y 00 −y 0 −6y = x2 +1. Note that yc = c1 e3x +c2 e−2x . Since no term in x2 +1
or its derivatives is a linear combination of e3x and e−2x then we can let yp = Ax2 + Bx + C.
Plugging this into the original differential equation yields 2A − (2Ax + B) − 6(Ax2 + Bx +
C) = x2 + 1. Gathering and equating like terms yields the system −6A = 1, −2A − 6B =
1
25
x2 + 18
x − 108
0, 2A − 2B − 6C = 1. Solving this system yields the particular solution yp = −1
6
−1 2
1
25
3x
−2x
which leads to the general solution Y = yc + yp = c1 e + c2 e
+ 6 x + 18 x − 108 .
00
0
3x
Example: Solve y − y − 6y = e . The complementary solution is the same as in
the previous example. Note that the function f appears in the complementary solution as
well. So we must take as our trial solution yp = Axe3x rather than just Ae3x . Substituting
this yp into the equation yields the particular solution yp = 15 xe3x and the general solution
Y = c1 e3x + c2 e−2x + 15 xe3x .
1 REVIEW OF BASIC ODES
1.7
9
Variation of Parameters
The method of variation of parameters can be used on a general nth -order linear nonhomogeneous equation of the form
y (n) + pn−1 (x)y (n−1) + · · · + p1 (x)y 0 + p0 (x)y = f (x)
if we know the complementary solution yc = c1 y1 + · · · + cn yn to the associated homogeneous
problem. The particular solution in this case is the function yp = u1 y1 + · · · + un yn where
the functions yi are the n linearly independent solutions that appear in the complementary
solution and the functions ui are unknown functions that we need to determine.
We will derive the method for finding u1 and u2 in the second order case and then state
the method for the nth -order case. Suppose that we want to solve the equation
y 00 + p(x)y 0 + q(x)y = f (x)
(10)
where p, q, and f are continuous on some open interval I. Suppose the yc = c1 y1 + c2 y2 is
the complementary solution. Let Y = u1 y1 + u2 y2 . In order to substitute Y into (10), we’ll
need to calculate Y 0 and Y 00 . Now,
Y 0 = u01 y1 + u1 y10 + u02 y2 + u2 y20
= (u1 y10 + u2 y20 ) + (u01 y1 + u02 y2 ).
Now, loosely speaking, we are trying to determine two functions u1 and u2 , but we only
have one condition, namely (10). So we are free to specify a second condition. We will do
so now. It will make the job of finding u1 and u2 easier if we can avoid involving u001 and u002 .
So we will make our second condition the following:
u01 y1 + u02 y2 = 0.
(11)
Using this condition means that
Y 0 = u1 y10 + u2 y20 .
And so it follows that
Y 00 = u01 y10 + u1 y100 + u02 y20 + u2 y200 .
Because y1 and y2 are solutions to the associated homogeneous problem, it follows that
y100 = −p(x)y10 − q(x)y1
y200 = −p(x)y20 − q(x)y2 .
Thus,
Y 00 = u01 y10 + u1 (−p(x)y10 − q(x)y1 ) + u02 y20 + u2 (−p(x)y20 − q(x)y2 )
= u01 y10 + u02 y20 − p(x)(u1 y10 + u2 y20 ) − q(x)(u1 y1 + u2 y2 )
= u01 y10 + u02 y20 − p(x)Y 0 − q(x)Y.
1 REVIEW OF BASIC ODES
10
So,
Y 00 + p(x)Y 0 + q(x)Y = u01 y10 + u02 y20 .
By (10), this implies that Y is a solution if u01 y10 + u02 y20 = f . So now we have two conditions
on u1 and u2 . In particular,
u01 y1 + u02 y2 = 0
u01 y10 + u02 y20 = f.
Think of this a system of two linear equations in the unknowns u01 and u02 . Since y1 and y2 are
linearly independent solutions to a homogeneous linear ODE, their Wronskian is non-zero.
Thus this system has a solution. If we then integrate u01 and u02 , we will have the sought
functions u1 and u2 .
In the nth -order case, we must solve the system
u01 y1 + u02 y2 + · · · + u0n yn = 0
u01 y10 + u02 y20 + · · · + u0n yn0 = 0
..
.
0 (n−1)
0 (n−1)
0 (n−1)
u1 y 1
+ u2 y 2
+ · · · + un yn
= f
for the unknown functions u01 , . . . , u0n , integrate, and form the particular solution yp = u1 y1 +
u2 y2 + · · · + un yn .
1.8
Systems of First Order ODEs
Systems of first order ODEs are interesting in their full generality, but first consider this
connection to what we have just completed. Given an nth -order ODE that can be solved for
its highest order term. We can write such an equation in the form
x(n) = F (t, x, x0 , x00 , . . . , x(n−1) ).
(12)
Now introduce the n new dependent variables x1 , x2 , . . . , xn defined by x1 = x, x2 = x0 , x3 =
x00 , . . . , xn = x(n−1) . Note that x01 = x0 = x2 , etc. That is, we now have a system of first
order equations
x01 = x2
x02 = x3
..
.
0
xn−1 = xn
x0n = f (t, x1 , x2 , . . . , xn ).
(13)
A solution to this system is a set of functions x1 , x2 , . . . , xn that satisfy the system. Note
that a set of functions x1 , x2 , . . . , xn is a solution to (13) if and only if the corresponding
function x = x1 is a solution to (12). So one way of solving (12) is to solve (13).
1 REVIEW OF BASIC ODES
11
Example: Write the third order equation x(3) + 4x00 − (x0 )3 + x = et as a system of three
first order equations. Let x1 = x, x2 = x0 , x3 = x00 . The system is then
x01 = x2
x02 = x3
x03 = −4x3 + x32 − x1 + et
Note that the original equation was non-linear, but the method still works.
Example: Write the system of two second order ODEs
x00 = 3x0 − y
y 00 = x0 + y 0 − t
as a system of four first order ODEs. Let x1 = x, x2 = x0 , y1 = y, y2 = y 0 . The solution is
then
x01
x02
y10
y20
=
=
=
=
x2
3x2 − y1
y2
x2 + y2 − t.
In general, a linear first order system is one that may be written in the form
x01 = p11 (t)x1 + p12 (t)x2 + · · · + p1n (t)xn + f1 (t)
x02 = p21 (t)x1 + p22 (t)x2 + · · · + p2n (t)xn + f2 (t)
..
..
.
.
0
xn = pn1 (t)x1 + pn2 (t)x2 + · · · + pnn (t)xn + fn (t).
(14)
If f1 = f2 = · · · = fn = 0, then we say the system is homogeneous. Otherwise it is
non-homogeneous.
We will be able to make statements about linear systems more concisely if we introduce
the concept of a matrix-valued function. A matrix-valued function is a function that accepts
a scalar as an input and produces a matrix as an output. We will denote matrix-valued
functions with the same typeface as matrices: A(t), or x(t) in the special case of a vectorvalued function. The components of a matrix-valued function are all scalar functions of t.
Continuity and differentiability of A(t) are defined component-wise, and the derivative A0 (t)
is calculated by taking the derivative of each component of A(t). Using this notation, the
system (14) becomes
x0 = P (t)x + f (t).
The theorems for systems of linear equations are essentially recapitulations of the theorems for linear equations.
Theorem 6 (Existence and Uniqueness of Solutions) Suppose that each pij and each
fi in (14) are continuous on some open interval I containing a point a. Given constants
b1 , b2 , . . . , bn , the first order linear system (14) has a unique solution on all of I satisfying
x1 (a) = b1 , . . . , xn (a) = bn .
1 REVIEW OF BASIC ODES
12
We’ll omit the proof.
Theorem 7 (Superposition for Homogeneous Linear Systems) Suppose that x1 , . . . ,
xm are m solutions to the n × n homogeneous linear system x0 = P (t)x. Then for any constants c1 , . . . , cm , x = c1 x1 + · · · + cm xm is a solution as well.
Proof: Given the n×n homogeneous linear system x0 = P (t)x with solutions x1 , . . . , xm .
Let x = c1 x1 + · · · + cm xm for some constants c1 , . . . , cm . Then
x0 =
=
=
=
c1 x01 + · · · + cm x0m
c1 P (t)x1 + · · · + cm P (t)xm
P (t)(c1 x1 + · · · + cm xm )
P (t)x.
Thus, x is a solution. We say a set of length n vector-valued functions x1 , . . . , xn is linearly dependent on an
interval I if there exist constants c1 , . . . , cn not all zero such that c1 x1 + · · · + cn xn = 0.
Otherwise the set of functions is linearly independent.
The Wronskian of a set of n length n vector-valued functions x1 , . . . , xn is the determinant
of the matrix whose columns are x1 , . . . , xn .
The proofs of the next two theorems for systems of equations are very similar to the
corresponding theorems for equations and so will be omitted.
Theorem 8 (Wronskian of Solutions) Suppose that x1 , . . . , xn are solutions to the n×n
homogeneous system x0 = P (t)x on an open interval I over which P (t) is continuous. Let
W be the Wronskian of x1 , . . . , xn . Then
1. if x1 , . . . , xn are linearly dependent on I, then W ≡ 0 on I;
2. if W (t0 ) 6= 0 for some t0 ∈ I, then x1 , . . . , xn are linearly independent on I.
Theorem 9 (General Solutions) Suppose that x1 , . . . , xn are linearly independent solutions to the n × n homogeneous system x0 = P (t)x on an open interval I over which P (t)
is continuous. If x is any solution to x0 = P (t)x, then there exist constants c1 , . . . , cn such
that x = c1 x1 + · · · + cn xn .
It follows from what we have stated so far that every homogeneous n × n system x0 =
P (t)x for which P (t) is continuous over an open interval I does in fact have n linearly
independent solutions. This may be established as follows. Consider the initial value problem
x0 = P (t)x, x(a) = [0 · · · 010 · · · 0]T in which the 1 in the initial condition occurs in the j th
slot. By Theorem 6, this initial value problem has a unique solution xj on all of I. This is true
for each j = 1, . . . , n. Now the set of vectors x1 (a), . . . , xn (a) is a linearly independent set
(they are the columns of the identity matrix), so their Wronskian is non-zero. So, according
to Theorem 8, the set of solutions x1 , . . . , xn are linearly independent on I. Actually finding
those solutions explicitly is our next job.
2 SYSTEMS OF LINEAR ODES
2
13
Systems of Linear ODEs
[Much of the material in this section is adapted from various editions of differential equations
texts by Edwards and Penney, and Boyce and DiPrima.]
In this section, we will concern ourselves with solving the n × n linear system
x0 = P (t)x + f (t)
(15)
and the associated homogeneous equation
x0 = P (t)x
(16)
as well as initial value problems based on these systems.
Note that the relationship between solutions of a non-homogeneous system and the solutions of the associated homogeneous system are the same for systems as they were for equations. That is, if we are solving the non-homogeneous system (15) and xc = c1 x1 +· · ·+cn xn
is the general solution of the associated homogeneous system (16), then the general solution
to (15) is x = xc + xp where xp is a particular solution to (15).
To check this, note that if x = xc + xp as described, then
x0 =
=
=
=
2.1
x0c + x0p
P (t)xc + P (t)xp + f (t)
P (t)(xc + xp ) + f (t)
P (t)x + f (t)
Initial Value Problems
First note that, given n linearly independent solutions to the homogeneous system (16), we
may write the general solution x(t) = c1 x1 (t) + · · · + cn xn (t) as x(t) = X(t)c where X(t)
is the matrix whose columns are the n linearly independent solutions. That is, X(t) =
[x1 (t) · · · xn (t)].
Using this notation, it is straightforward to produce the solution to an initial value
problem. Suppose we wish to solve the initial value problem x0 = P (t)x, x(a) = b. If we
have x1 , . . . , xn in hand, then solving the initial value problem reduces to solving a matrix
equation. Because x(t) = X(t)c and x(a) = b, then c = X(a)−1 b. We use the components
of c to form the particular solution to the initial value problem.
2.2
Eigenvalue Method for Constant Coefficient Systems
Suppose for the moment that the coefficient matrix in the homogeneous system (16) is a
constant matrix. That is, we want to solve the system x0 = Ax where A is a constant
matrix. Take as a trial solution x = veλt where v is an unknown constant vector. Note that
in this case x0 = λveλt . Substituting this trial solution into x0 = Ax yields λveλt = Aveλt .
If we now cancel eλt from both sides we find that for our trial solution to actually be a
solution, it must be the case that λv = Av. That is, λ must be an eigenvalue of A and v
must be an associated eigenvector.
2 SYSTEMS OF LINEAR ODES
14
If there are n distinct eigenvalues, then the corresponding eigenvectors are guaranteed
to be linearly independent. Thus the general solution to x0 = Ax is x(t) = v 1 eλ1 t +
v 2 eλ2 t + · · · + v n eλn t . If there are not n distinct eigenvalues, then there may not be a
linearly independent set of eigenvectors. In this case we would need to consider generalized
eigenvectors to complete our linearly independent set. This is complicated and somewhat
tedious, and you did this in Multivariable II, so we are not going to do it again here. Instead
we will introduce a new method based on matrix exponentials.
2.3
Matrix Exponentials
In this section we will show that, just as the solution to x0 = kx is x = Cekt , the solution to
x0 = Ax is x = ceAt . Now we just need to figure out what that all means. We begin with
a definition.
Definition: Let x1 , . . . , xn be n linearly independent solutions to the system x0 = Ax
where A is a constant matrix. The matrix Φ(t) = [x1 · · · xn ] is called a fundamental matrix
for the system.
This is essentially just new notation for something we already have, but it is fairly
standard and will let us write things in a compact manner. For example, we have the
following theorem.
Theorem 10 (Fundamental Matrix Solutions) Let Φ(t) be a fundamental matrix for
the homogeneous linear system x0 = Ax. Then the solution to the initial value problem
x0 = Ax, x(0) = x0 is x(t) = Φ(t)Φ(0)−1 x0 .
Proof: Since each column of Φ(t) satisfies x0 = Ax, then Φ0 (t) = AΦ(t). The general
solution of x0 = Ax is x = c1 x1 + · · · + cn xn where x1 , . . . , xn are the linearly independent columns of Φ(t). So this general solution may be written as x = Φ(t)c where
c = [c1 , . . . , cn ]T . To find c, note that Φ(t) has an inverse because its columns are linearly
independent. So, x0 = x(0) = Φ(0)c. Thus c = Φ(0)−1 x0 . Therefore the solution to the
initial value problem is x(t) = Φ(t)Φ(0)−1 x0 . Note that once we find Φ(t)Φ(0)−1 for a given system of differential equation, we can
solve x0 = Ax, x(0) = x0 for a sequence of different x0 ’s quickly.
From work in calculus with infinite series, we know the Taylor series for ex . It is a fact
from complex analysis that the same Taylor series holds for all complex values as well. That
is,
∞
X
zn
z2 z3
z
+
+ ··· =
e =1+z+
2!
3!
n!
n=0
for all z ∈ C. We now make the following definition.
2 SYSTEMS OF LINEAR ODES
15
Definition: Let A be an n × n matrix. Define the matrix exponential of A to be
A2 A3
A
e
= I +A+
+
+ ···
2!
3!
∞
X
An
=
n!
n=0
!
k
X
An
= lim
.
k→∞
n!
n=0
We will omit the proof of the following theorem.
Theorem 11 (Existance of the Matrix Exponential) The matrix exponential eA is defined for all n × n matrices A.
The matrix exponential has the following useful properties that are analogues of the
standard exponential function.
1. e0 = I. This follows directly from the definition.
2. If BA = AB, then eA+B = eA eB . We need the commutativity to handle the mixed
terms that arise when we multiply out (A + B)n .
3. (eA )−1 = e−A . This is a result of combining properties 1 and 2 above.
Note that property 3 implies that eA is invertible for every n × n matrix A. In particular
eA has linearly independent columns and a non-zero determinant.
If t is a scalar variable, then it follows that
2
3
t
t
eAt = I + At + A2 + A3 + · · · .
2!
3!
It is possible to show that this series converges absolutely and so may be differentiated
term-by-term. Thus,
d At
d
t2
t3
(e ) =
(I + At + A2 + A3 + · · · )
dt
dt
2!
3!
2
2t
3t
= A + A2 + A3
+ ···
2!
3!
t
t2
= A + A2 + A 3 + · · ·
1!
2!
2
2t
= A(I + At + A
+ ···)
2!
= AeAt .
Thus eAt satisfies the matrix equation X 0 = AX. In other words, each column of eAt
satisfies x0 = Ax. Because the columns of eAt are linearly independent, we find that eAt is
a fundamental matrix for the n × n constant coefficient linear system system x0 = Ax.
2 SYSTEMS OF LINEAR ODES
16
By Theorem 10, we know that the solution to the initial value problem x0 = Ax, x(0) =
x0 is x(t) = Φ(t)Φ(0)−1 x0 where Φ(t) is a fundamental matrix for the system. Since eAt
is a fundamental matrix, then that means the following theorem holds.
Theorem 12 (Matrix Exponential Solutions) The solution to the initial value problem
x0 = Ax, x(0) = x0 where A is an n × n constant matrix is x(t) = eAt x0 .
It is interesting to note that, since the solution to an initial value problem is unique, we
know that for any fundamental matrix Φ(t) for a system x0 = Ax, we have that eAt =
Φ(t)Φ(0)−1 .
2.4
Non-homogeneous Linear Systems
In this section we will show how the variation of parameters method may be used to find a
particular solution to the non-homogeneous problem
x0 = P (t)x + f (t)
(17)
for continuous matrix-valued functions P and f .
We have already established that the general solution to (17) is of the form x = xc + xp
where xc is the complementary solution to the associated homogeneous equation and xp is
a particular solution to (17). We will assume throughout this section that we have already
found xc as the linear combination of n linearly independent solutions x1 , . . . , xn and that
x1 , . . . , xn are known and fixed.
Variation of parameters proceeds in a manner similar to that in the single equation
case. That is, let Φ(t) be a fundamental matrix for the associated homogeneous problem
x0 = P (t)x with columns x1 , . . . , xn . Then xc = Φ(t)c for some constant vector c. For
the trial particular solution, let xp = Φ(t)u(t) where u(t) = [u1 (t) · · · un (t)]T is a vectorvalued function whose components are unknown functions of t. We seek to determine the
components of u(t) that lead to a particular solution xp of (17).
We now proceed by substituting our trial solution xp = Φ(t)u(t) into (17). This yields
Φ0 (t)u(t) + Φ(t)u0 (t) = P (t)Φ(t)u(t) + f (t).
Because Φ(t) is a fundamental matrix, we know that each column of Φ(t) satisfies the
equation x0 = P (t)x. That means that Φ(t) satisfies the matrix equation Φ0 (t) = P (t)Φ(t).
Using this fact we find that
Φ0 (t)u(t) + Φ(t)u0 (t) = Φ0 (t)u(t) + f (t)
⇒
Φ(t)u0 (t) = f (t)
⇒
u0 (t) = Φ(t)−1 f (t)
R
⇒
u(t) = Φ(t)−1 f (t) dt
where in the last step we integrate component-by-component. (Note that we can invert Φ(t)
because it is a fundamental matrix
and so has linearly independent columns.) Thus the
R
particular solution is xp = Φ(t) Φ(t)−1 f (t) dt.
The preceding argument is the proof of the following theorem.
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
17
Theorem 13 (Variation of Parameters) If Φ(t) is a fundamental matrix for x0 = P (t)x
and the functions P (t) and f (t) are continuous on some open interval I, then
Z
xp = Φ(t) Φ(t)−1 f (t) dt
is a particular solution to (17) on I.
Using what we know about the complementary solution, we find that the general solution
to (17) is
Z
Φ(t)−1 f (t) dt.
x(t) = Φ(t)c + Φ(t)
Suppose now that we wish to solve the initial value problem
x0 = P (t)x + f (t),
x(a) = xa
(18)
where xa is some constant vector representing the initial conditions and P (t) and f (t) are
continuous on some open interval containing the number a. The general solution we derived
contains an indefinite integral which implicitly contains an arbitrary constant of integration.
We want to pick the constant in a “good” way. That is, we will pick the constant in such a
way that xp (a) = 0. This will simplify our calculations when solving a specific problem. In
particular, choose the particular solution to be
Z t
xp = Φ(t)
Φ(s)−1 f (s) ds.
(19)
a
This guarantees that xp (a) = 0. Thus, using what we derived in the previous sections about
solutions to homogeneous initial value problems as well as the current argument, the solution
to the initial value problem (18) is
Z t
−1
x(t) = Φ(t)Φ(a) xa + Φ(t)
Φ(s)−1 f (s) ds.
a
Finally, if P (t) is actually a constant matrix A, then our work with matrix exponentials
shows that the solution to the non-homogeneous initial value problem
x0 = Ax + f (t),
is
x(t) = eAt x0 + eAt
x(0) = x0
Z
t
e−As f (s) ds.
0
3
Power Series Solutions for Linear ODEs
[The material in this section is taken largely from the article Leibniz’s Formula, Cauchy Majorants, and Linear Differential Equations by Michael Mezzino and Mark Pinsky published in
the December 1998 issue of Mathematics Magazine and from various editions of differential
equations texts by Edwards and Penney.]
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
18
We now return to the question of how to solve linear nth -order ODEs. Our focus here
will be on finding solutions to
an (t)y (n) + an−1 (t)y (n−1) + · · · + a1 (t)y 0 + a0 (t)y = f (t)
(20)
under certain conditions on the coefficient functions ai and f . The solutions we seek will be
in the form of power series. So to begin, we need to review some facts about power series.
3.1
Some Facts About Power Series
Definition: Given an infinite sequence of constants {cn }∞
n=0 and a number a, a power series
in x − a is an infinite series of the form
∞
X
cn (x − a)n .
n=0
In the special case where a = 0 we say we have a power series in x.
There is a one-to-one correspondence between a power series and its coefficients. That
is, if you know the sequence {cn }∞
n=0 , then you know the series. So a lot of our effort will be
focused on finding these coefficients.
The convergence of a power series has a very nice property given in the following theorem
from calculus.
P
n
Theorem 14 If ∞
n=0 cn (x − a) is a power series then exactly one of the following holds:
1. The series converges only for x = a and diverges everywhere else.
2. There exists a number R > 0 such that the series converges absolutely for all x in the
interval (a − R, a + R) and diverges for all x in (−∞, a − R) ∪ (a + R, ∞). (R is called
the radius of convergence of the power series.) At the points x = a − R and x = a + R,
the series may diverge, converge conditionally, or converge absolutely.
3. The series converges absolutely at every real number. In this case we say the radius of
convergence is infinite.
P
n
If a power series ∞
on an interval I,P
then it defines a function
n=0 cn (x − a) converges
P∞
n
n
on that interval. We may write f (x) = n=0 cn (x − a) and call ∞
n=0 cn (x − a) the power
series representation of f .
On the other hand, suppose we are given a function f . The Taylor series for f at a is
∞
X
f (n) (a)
n=0
n!
(x − a)n .
Now it is not automatically the case that the Taylor series for f converges to f at any
point other that x = a even if the Taylor series converges at points other that x = a.
(A theorem from calculus gives the precise conditions under which the Taylor series for a
function converges to the function.) Despite this, it is usually the case for the functions that
we see in applications that the Taylor series for f converges to f .
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
19
Definition: A function f is analytic at x = a if the Taylor series for f at a converges to
f on some open interval containing a.
For example, polynomials, rational functions, sin, cos, and ex are all analytic everywhere
that they are defined. Furthermore, sums, products, and quotients of analytic functions are
analytic wherever they are defined.
Sums and products of convergent power series converge wherever both of the constituent
power series converge. In particular, if
f (x) =
∞
X
an (x − a)n
n=0
and g(x) =
∞
X
bn (x − a)n ,
n=0
then
f (x) + g(x) =
and f (x)g(x) =
3.2
∞
X
(an + bn )(x − a)n
n=0
∞
X
n
X
n=0
i=0
!
ai bn−i
(x − a)n
Series Solutions About Ordinary Points
Power series methods for ODEs break into two classes: solutions about ordinary points and
solutions about singular points.
Definition: Given the ODE
y (n) + an−1 (t)y (n−1) + · · · + a1 (t)y 0 + a0 (t)y = f (t).
(21)
If each f and each of the coefficient functions ai are analytic at a point a, then a is called
an ordinary point. Otherwise a is called a singular point.
Note that an equation of the form (21) may be obtained from one of the form (20) by
dividing by an (t). Furthermore, if each coefficient function in (20) is analytic at a and
an (a) 6= 0, then dividing shows that a is an ordinary point of (21). Finally, note that if each
of the coefficient functions in (20) is a polynomial, then a is an ordinary point if and only if
an (a) 6= 0.
To develop the series solution technique, we will need the following result
Theorem 15 (Leibniz’s Binomial Formula) If f and g are n-times differentiable functions, then so is f g, and
n X
n
(n)
(f g) =
f (k) g (n−k) .
(22)
k
k=0
Proof: The proof proceeds by induction.
Initial step: If n = 1, the theorem holds because it is the product rule proved in calculus.
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
20
Inductive step: Suppose that
(j)
(f g)
j X
j
=
f (k) g (j−k)
k
k=0
for each j = 1, 2, . . . , n − 1.
Then
d
[(f g)(n−1) ]
(f g)(n) =
dx "
#
n−1 d X n−1
=
f (k) g (n−1−k)
(by the inductive hypothesis)
k
dx k=0
n−1 X
d
n−1
f (k) g (n−1−k)
=
k
dx
k=0
n−1 X
n−1
=
f (k+1) g (n−1−k) + f (k) g (n−k)
(by the product rule)
k
k=0
n−1 n−1 X
X
n−1
n−1
(k+1) (n−1−k)
=
f
g
+
f (k) g (n−k)
k
k
k=0
k=0
n
n−1
X n−1
X n−1 (k) (n−k)
=
f g
+
f (k) g (n−k)
k−1
k
k=1
k=0
(by shifting the index on the first sum)
n−1
n−1
(0) (n)
=
f g +
f (n) g (0)
0
n−1
n−1 X
n−1
n−1
+
+
f (k) g (n−k)
k−1
k
k=1
n−1 X
n
(0) (n)
(n) (0)
= f g +f g +
f (k) g (n−k)
k
k=1
n
X n
=
f (k) g (n−k) . k
k=0
3.2.1
First Order Problems
Consider the initial value problem
a(t)y 0 (t) + b(t)y(t) = f (t),
y(t0 ) = y0
(23)
in which y0 is a constant, a, b, and f are analytic at t0 , and a(t0 ) 6= 0 (that is, t0 is an
ordinary point). We seek solutions of the form
y(t) =
∞
X
yn
n=0
n!
(t − t0 )n
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
21
where y0 = y(t0 ), y1 = y 0 (t0 ), y2 = y 00 (t0 ), . . . , yn = y (n) (t0 ), . . .. Since a, b, and f are analytic
at t0 , they have power series representations
∞
X
an
a(t) =
b(t) =
n=0
∞
X
n=0
f (t) =
n!
bn
(t − t0 )n
n!
∞
X
fn
n=0
(t − t0 )n
n!
(t − t0 )n .
We are assuming that the differential equation is valid at t0 . That means that
a(t0 )y 0 (t0 ) + b(t0 )y(t0 ) = f (t0 ).
(24)
In other words,
a0 y1 + b0 y0 = f0 .
Since a0 = a(t0 ) 6= 0 and b0 , f0 , and y0 are known, we can solve for y1 .
Now differentiate (23) and evaluate it at t0 . This yields
a1 y1 + a0 y2 + b1 y0 + b0 y1 = f1 .
Again, everything but y2 is known and a0 6= 0, so we can solve this for y2 .
We now use Leibniz’s binomial formula to differentiate (23) n times and evaluate this
equation at t0 to produce a recursive expression for the coefficients yn . That is,
n n X
X
n
n
ak yn−k+1 +
bk yn−k = fn .
k
k
k=0
k=0
If we pull out the term involving yn+1 , we get
First Order Recursion Formula
n n X
X
n
n
a0 yn+1 +
ak yn−k+1 +
bk yn−k = fn .
k
k
k=1
(25)
k=0
Since a0 6= 0 and all the other terms are known, we can solve this for yn+1 . This gives us
a general recursive relationship for the coefficients of the power series representation of the
solution y. That is, we’ve solved the problem.
Example: Solve y 0 − 4y = 0, y(0) = 1. In this case, a0 = 1, ai = 0 for i > 0, b0 = −4,
bi = 0 for i > 0, and fi = 0 for all i. Plugging into the first order recursion yields
n
1yn+1 + 0 +
(−4)yn = 0.
0
That is, yn+1 = 4yn . That means that yn = 4yn−1 , yn−1 = 4yn−2 , etc. Combining these
shows that yn = 4n y0 . Using the given value y0 = 1 we have that yn = 4n . Therefore the
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
22
solution is
∞
X
4n
y(t) =
n=0
∞
X
=
n=0
4t
n!
tn
(4t)n
n!
= e .
3.2.2
Second Order Problems
Consider the initial value problem
a(t)y 00 (t) + b(t)y 0 (t) + c(t)y(t) = f (t),
y(t0 ) = y0 , y 0 (t0 ) = y1
(26)
in which y0 and y1 are constants, a, b, c, and f are analytic at t0 , and a(t0 ) 6= 0 (that is, t0
is an ordinary point). We seek solutions of the form
y(t) =
∞
X
yn
n=0
n!
(t − t0 )n
where y0 = y(t0 ), y1 = y 0 (t0 ), y2 = y 00 (t0 ), . . . , yn = y (n) (t0 ), . . .. Since a, b, c, and f are
analytic at t0 , they have power series representations
∞
X
an
a(t) =
b(t) =
n=0
∞
X
n=0
c(t) =
f (t) =
n!
bn
(t − t0 )n
n!
∞
X
cn
n=0
∞
X
n=0
(t − t0 )n
n!
(t − t0 )n
fn
(t − t0 )n .
n!
In a fashion similar to the first order problem we may derive a recurrence relation by using
Leibniz’s binomial formula to differentiate (26) n times and evaluating at t0 . This yields the
following.
Second Order Recursion Formula
n n X
X
n
n
a0 yn+2 +
ak yn−k+2 +
(bk yn+1−k + ck yn−k ) = fn .
(27)
k
k
k=1
k=0
We use this relationship the same way we use the first order one.
Example: Solve y 00 − ty = 0, y(0) = 0, y 0 (0) = 1. In this problem, the coefficients are
a0 = 1, c1 = −1, y1 = 1 and all other coefficients are 0. Plugging this into the recursion
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
23
n
yields yn+2 + 0 +
(−1)yn−1 = 0. That is, yn+2 = nyn−1 . The only non-zero coefficients
1
for y are then y1 = 1, y4 = 2, y7 = 2 · 5, y10 = 2 · 5 · 8, . . .. Thus
y(t) = t +
= t+
2t4 10t7
2 · 5 · · · (3n − 1)t3n+1
+
+ ··· +
+ ···
4!
7!
(3n + 1)!
∞
X
2 · 5 · · · (3n − 1)t3n+1
n=1
3.3
(3n + 1)!
Convergence of Solutions
The method of producing power series solutions that we developed in the last section does
in fact produce power series that formally solve the differential equations at hand. Unfortunately, nothing that we discussed lets us know whether those power series actually converge
anywhere other than their center points. We will rectify that oversight in this section.
We will restrict ourselves to considering only second order problems. The proof for
higher order problems is similar and only more complicated because of notation. We will
also restrict our attention to problems with leading coefficient 1. This is justified because
of the following. All the coefficient functions were assumed to be analytic about a given
center point t0 with the additional condition that the leading coefficient function a2 (t0 ) 6= 0.
Analytic functions are continuous, and a continuous function that is non-zero at a point is
also non-zero on some open interval containing that point. So we can divide the equation
by the leading coefficient function a2 (t), preserve the analytic nature of the other coefficient
functions, and not change the value of the solution. That is, under the conditions discussed,
if y is a solution to
a(t)y 00 (t) + b(t)y 0 (t) + c(t)y(t) = f (t),
then it is also a solution to
y 00 (t) +
c(t)
f (t)
b(t) 0
y (t) +
y(t) =
a(t)
a(t)
a(t)
and vice versa. Furthermore, we will assume the the coefficient functions have power series
representations centered at 0 that converge on the interval [−1, 1]. Since the coefficient
functions were originally assumed all to be analytic at t0 , a change of variable can shift the
center of each series from t0 to 0 and rescale the interval of convergence for each series so
that it contains [−1, 1]. We may now state the convergence theorem.
Theorem 16 (Convergence of Series Solutions at Ordinary Points) Suppose that the
functions b, c, and P
f are analytic at 0 and have convergent power series representations on
yn n
[−1, 1]. Let y(t) = ∞
n=0 n! t be the power series solution to
y 00 (t) + b(t)y 0 (t) + c(t)y(t) = f (t)
(28)
P
yn n
generated by the method in the last section. Then ∞
n=0 n! t converges absolutely on (−1, 1).
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
24
Proof: Our strategy will be to solve a different differential equation that we produce
by replacing the coefficient functions b, c, and f by new functions that dominate b, c, and
f and for which we can solve the differential equation. The proof proceeds in two cases:
homogeneous and non-homogeneous.
Case 1: Homogeneous. Suppose that f ≡ 0. Since
c are analytic
P∞ atcn0,n they
P∞ bbnand
n
have convergent power series representations b(t) = n=0 n! t and c(t) = n=0 n! t . By
hypothesis, these converge on [−1, 1].
Since the series both converge at t = 1, then the sequences { bn!n } and { cn!n } are both
bounded. That is, there exist B, C > 0 such that |bn!n | ≤ B and |cn!n | ≤ C for all n.
Now consider the initial value problem
Y 00 (t) −
C
B
Y 0 (t) −
Y (t) = 0,
1−t
(1 − t)2
Y (0) = ȳ0 , Y 0 (0) = ȳ1
for some constants ȳ0 and ȳ1 . Use Y (t) = (1 − t)−r as a trial solution. Substituting this into
the differential equation yields
r(r + 1)(1 − t)−r−2 −
B
r(1
1−t
− t)−r−1 −
C
(1
(1−t)2
− t)−r = 0
r(r + 1) − Br − C = 0
r2 + (1 − B)r − C = 0
√
B−1± (1−B)2 +4C
r=
2
Since B, C > 0, we are guaranteed two distinct real roots, r1 and r2 , which then produce two
linearly independent solutions, (1 − t)r1 and (1 − t)r2 . The general solution to the differential
equation is then
Y (t) = α1 (1 − t)r1 + α2 (1 − t)r2 .
The initial conditions ȳ0 and ȳ1 may be used to find the constants α1 and α2 thus producing
a particular solution. This particular solution
is analytic on (−1, 1) and so has a convergent
P∞
power series representation. Call it Y (t) = n=0 ȳn!n tn .
Using facts about geometric series, we see that
∞
∞
X
X
B
C
=
C(n + 1)tn .
=
Btn and
1 − t n=0
(1 − t)2
n=0
Define b̄n = Bn! and c̄n = C(n + 1)!. It then follows that b̄n ≥ |bn | and c̄n ≥ |cn |.
Using the recurrence technique defined in the last section, we know that
n X
n
ȳn+2 −
(b̄k ȳn+1−k + c̄k ȳn−k ) = 0.
k
k=0
Note that if ȳ0 , ȳ1 ≥ 0, then ȳn ≥ 0 for all n.
We now return to consideration of (28). In particular, we will produce two linearly
independent solutions to the homogeneous version of this equation. To produce the first
solution, let y0 = ȳ0 = 1 and y1 = ȳ1 = 0. To produce the second solution, let y0 = ȳ0 = 0
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
25
and y1 = ȳ1 = 1. In either case, we may now show that |yn | ≤ ȳn for all n by induction. The
n = 0 and n = 1 case hold by definition. As the induction hypothesis, assume that |yj | ≤ ȳj
for each j = 0, 1, 2, . . . , n + 1. Using the recursion twice we see that
n X
n
(bk yn+1−k + ck yn−k )
|yn+2 | = k
k=0
n
X
n
≤
(b̄k ȳn+1−k + c̄k ȳn−k )
k
k=0
= ȳn+2 .
Thus |yn | ≤ ȳn for each n.P
This fact, along with the comparison test forPinfinite series from
∞ yn n
ȳn n
calculus and the fact that ∞
n=0 n! t converges on (−1, 1), implies that
n=0 n! t converges
absolutely on (−1, 1). This completes the proof in the homogeneous case.
Case 2: Non-homogeneous. Since the general solution to a non-homogeneous differential
equation is a complementary solution plus a particular solution, the only thing left to do
here is find a particular solution to (28).
By an argument similar to that in the homogeneous case, there exist positive constants
B, C, F such that
|cn |
|fn |
|bn |
≤ B,
≤ C,
≤F
n!
n!
n!
for each n. Define
b̄n = Bn!, c̄n = C(n + 1)!, f¯n = (n + r + 1)(n + r)(n + r − 1) · · · (r + 2)
p
for each n, where r = ((B − 1) + (1 − B)2 − 4(C + F ))/2 is the positive solution to
r(r + 1) − Br − C = F .
P∞ ȳn n
Let Y (t) = (1 − t)−r and let ȳn be defined such that
n=0 n! t is the power series
representation of Y (t). A calculation shows that Y solves the differential equation
C
F
B
Y0−
Y =
.
2
1−t
(1 − t)
(1 − t)−r−2
P∞
n
Using the recursion shows that 0 ≤ ȳn+2 = n=0
(b̄k ȳn+1−k + c̄k ȳn−k ) + f¯n .
k
Now let y0 = ȳ0 = 1 and y1 = ȳ1 = r. Then yn ≥ 0 for all n. And as in the homogeneous
case, an induction argument shows that
∞ X
n
|yn+2 | ≤ (bk yn+1−k + ck yn−k ) + |fn |
n=0 k
∞ X
n
≤
(b̄k ȳn+1−k + c̄k ȳn−k ) + f¯n
k
Y 00 −
n=0
= ȳn+2 .
Finally, as in the homogeneous case, the comparison test completes the proof. 3 POWER SERIES SOLUTIONS FOR LINEAR ODES
3.4
26
Series Solutions about Regular Singular Points
We now turn to the case of singular points in homogeneous second order equations. We will
restrict our analysis to the case where x = 0 is a singular point. This is justified by the fact
that if x = a is a singular point, then the change of variable t = x − a shifts the singular
point to 0. Consider the equation
A(x)y 00 + B(x)y 0 + C(x)y = 0
in which A, B, C are analytic at x = 0 and x = 0 is a singular point. Divide the equation by
A(x). This yields the equation
y 00 + P (x)y 0 + Q(x)y = 0
where P = B/A and Q = C/A. It is a property of analytic functions that, at each point
x = a, a quotient of analytic functions is either analytic at a or approaches ±∞ as x → a.
Since x = 0 is a singular point, then either P or Q or both are not analytic at x = 0.
We now classify singular points. Define p(x) = xP (x) and q(x) = x2 Q(x). Using these
functions we can rewrite our equation as
y 00 +
p(x) 0 q(x)
y + 2 y=0
x
x
(29)
Definition: The singular point x = 0 is a regular singular point of (29) is both p and q
are analytic at x = 0. Otherwise it is an irregular singular point.
We will only consider regular singular points in this course. Regular singular points are
common in applications. Examples of situations that produce regular singular points at
x = 0 include p and q both polynomial and p and q rational and defined at x = 0.
3.4.1
Method of Frobenius
The simplest version of (29) occurs when p(x) = p0 and q(x) = q0 for some constants p0 , q0 .
In this case the equation becomes
x2 y 00 + xp0 y 0 + q0 y = 0.
We can show by direct substitution that the function y = xr is a solution to this equation if
and only if r is a root of
r(r − 1) + p0 r + q0 = 0.
(30)
This is called the indicial equation.
Now return to the general case. Suppose we have functions p and q, analytic at x = 0,
in (29). The p and q have power series representations
p(x) =
∞
X
pn
n=0
n!
xn ,
q(x) =
∞
X
qn
n=0
n!
xn .
Let ρ > 0 be the minimum of the radii of convergence of these two series.
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
27
Frobenius showed in the 1870’s that if r1 , r2 ∈ R are roots of (30) and r1 ≥ r2 , then there
exists a sequence {yn } such that
∞
X
yn n
r1
x
y=x
n!
n=0
converges on (−ρ, ρ) and is a solution to (29) for x > 0. This is known as a Frobenius
solution.
Furthermore, if r1 − r2 is not an integer, then there is a second, linearly independent,
Frobenius solution for x > 0 that converges on (−ρ, ρ) given by
y=x
r2
∞
X
ŷn
n=0
n!
xn .
Frobenius also showed that if r1 , r2 ∈ C/R, then there are two linearly independent
solutions of the form given above. In this course we will restrict ourselves to the case of real
roots of the indicial equation. P
Finally, note
if y = xr yn xn is a solution to the differential equation for x > 0,
P that
r
n
then y = |x|
yn x is a solution for x < 0.
3.4.2
Recursion for Regular Singular Points
We will now produce a recursion formula for the coefficients of the Frobenius solution for
x > 0 to (29) corresponding to the larger root, r of the indicial equation in the case of a
regular singular point at x = 0.
By the work of Frobenius, we know that the equation
x2 y 00 + xp(x)y 0 + q(x)y = 0
has a solution of the form
y=x
r
∞
X
yn
n=0
n!
xn .
From this it follows that
y =
y0 =
∞
X
yn
n=0
∞
X
n=1
y 00 =
n!
yn
(n + r)xn+r−1 ,
n!
∞
X
yn
n=2
xn+r ,
n!
(n + r)(n + r − 1)xn+r−2 .
Define
Y (x) =
∞
X
yn
n=0
n!
xn .
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
28
By substitution we can see that
y = xr Y,
y 0 = rxr−1 Y + xr Y 0 ,
y 00 = r(r − 1)xr−2 Y + 2rxr−1 Y 0 + xr Y 00 .
Substituting these expressions into the original differential equation yields the following.
x2 y 00 + xp(x)y 0 + q(x)y = 0
x2 (r(r − 1)xr−2 Y + 2rxr−1 Y 0 + xr Y 00 ) + xp(x)(rxr−1 Y + xr Y 0 ) + q(x)xr Y = 0
r(r − 1)xr Y + 2rxr+1 Y 0 + xr+2 Y 00 + rp(x)xr Y + xr+1 p(x)Y 0 + q(x)xr Y = 0
xr+2 Y 00 + (2rxr+1 + xr+1 p(x))Y 0 + (r(r − 1)xr + rxr p(x) + xr q(x))Y = 0
xr (x2 Y 00 + (2rx + xp(x))Y 0 + (r(r − 1) + rp(x) + q(x))Y ) = 0
x2 Y 00 + (2rx + xp(x))Y 0 + (r(r − 1) + rp(x) + q(x))Y = 0
where the final equation follows from the fact that we are not interested in the case where
x = 0.
We now take the nth derivative of this last equation using Leibniz’s binomial formula (22)
and simplify.
n n X
X
n
n
2 (k) (n−k+2)
(x ) Y
+
(2rx + xp(x))(k) Y (n−k+1)
k
k
k=0
k=0
n
X n
+
(r(r − 1) + rp(x) + q(x))(k) Y (n−k) = 0,
k
k=0
n
2 (n+2)
(n+1)
xY
+ 2nxY
+2
Y (n)
2
+(2rx + xp(x))Y (n+1) + n(2r + p(x) + xp0 (x))Y (n)
k n X
n X k
+
x(j) p(k−j) (x)Y (n−k+1)
k
j
j=0
k=2
n
X n
+
(r(r − 1) + rp(x) + q(x))(k) Y (n−k) = 0
k
k=0
k
Now we can show that j=0
x(j) p(k−j) (x)Y (n−k+1) = xp(k) (x) + kp(k−1) (x) and,
j
because r is a root of the indicial equation, r(r − 1) + rp0 + q0 = 0. Using these two facts,
we evaluate the equation at x = 0 to obtain the
Second Order Singular Recursion Formula
n n X
X
n
n
(n(n − 1) + np0 + 2nr)yn +
kpk−1 yn−k+1 +
(rpk + qk )yn−k = 0. (31)
k
k
Pk
k=2
k=1
Here are two examples to illustrate the use of this formula.
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
29
Example: Find a solution to
x2 y 00 + xy 0 + x2 y = 0
(known as Bessel’s Equation of Order 0).
First note that p(x) = 1 and q(x) = x2 . This implies that p0 = 1, q2 = 2, and all the
other pi and qi are 0. The indicial equation has a repeated
r = 0. So the Frobenius
P∞ root
yn n
0
theory only guarantees one solution of the form y = x
n=0 n! x . Let y0 be arbitrary. For
n = 1 the recursion formula (31) shows that y1 = 0. For n ≥ 2 the recursion reduces to
n(n − 1)yn + nyn + n(n − 1)yn−2 = 0.
Simplifying this yields
n2 yn + n(n − 1)yn−2 = 0.
This implies that yn = 0 for all odd n. For n ≥ 2 even we have that
n(n − 1)
yn−2
n2
n−1
= −
yn−2
n n−3
n−1
−
yn−4
= −
n
n−2
..
.
(n − 1)(n − 3) · · · 1
y0
= (−1)n/2
n(n − 2) · · · 2
n!
= (−1)n/2 n/2 2
y0
(2 ) (n/2)!
n!
= (−1)n/2 n
y0
2 ((n/2)!)2
(2m)!
= (−1)m 2m
y0
2 (m!)2
yn = −
where the last equality follows from letting m = n/2. Thus the solution is
∞
X
(−1)m y0 2m
x .
y(x) =
22m (m!)2
m=0
Example: Find two linearly independent solutions to
2x2 y 00 + 3xy 0 − (x2 + 1)y = 0.
First note that p(x) = 3/2 and q(x) = −(x2 + 1)/2. This implies that p0 = 3/2,
q0 = −1/2, q2 = −1, and all the other pi and qi are 0. The indicial equation has roots
r1 = 1/2 and r2 = −1. So the Frobenius theory guarantees two solutions, one of the form
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
P
P∞
yn n
−1
y = x1/2 ∞
x
and
the
other
of
the
form
y
=
x
n=0 n!
n=0
the recursion reduces to y1 = 0 and
yn =
30
yn n
x .
n!
For the root r1 = 1/2,
n−1
yn−2
2n + 3
for n ≥ 2. For the root r2 = −1, the recursion reduces to y1 = 0 and
yn =
n−1
yn−2
2n − 3
for n ≥ 2.
3.4.3
Reduction of Order
In the cases where the Frobenius theory does not guarantee two linearly independent Frobenius solutions, we need a way to construct the second linearly independent solution that the
general theory of second order equations guarantees exists. The method we will use is called
Reduction of Order.
Suppose we are given a second order linear differential equation of the form
y 00 + P (x)y 0 + Q(x)y = 0
with P and Q continuous on some open interval I. Suppose further that y1 is a solution to
this differential equation. We will derive a method to produce a second linearly independent
solution y2 from y1 .
Define v(x) = y2 (x)/y1 (x). Then y2 (x) = v(x)y1 (x). Substitute this function into the
differential equation. Since
y20 = v 0 y1 + vy10 , and
y200 = v 00 y1 + 2v 0 y10 + vy100 ,
the substitution yields
v 00 y1 + 2v 0 y10 + vy100 + P (x)(v 0 y1 + vy10 ) + Q(x)vy1 = 0.
Using the fact that y1 is a solution to the differential equation, shows that vy100 + P (x)vy10 +
Q(x)vy1 = 0. This implies that
v 00 y1 + (2y10 + P (x)y1 )v 0 = 0.
This is a first order equation in v 0 . Let u = v 0 . Then we have
u0 y1 + (2y10 + P (x)y1 )u = 0
y0
u0 + (2 1 + P (x))u = 0.
y1
The last equation holds if y1 (x) 6= 0 on I.
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
31
This is a first order linear equation in u that we can solve using integrating factors. Let
Z
y10
ρ = exp
2 + P (x) dx
y1
Z
= exp 2 ln |y1 | + P (x) dx
Z
2
= |y1 | exp
P (x) dx .
Using this as the integrating factor yields
u0 ρ + (2
y10
+ P (x))ρu = 0.
y1
This implies that (uρ)0 = 0, or, in other words, uρ = C for some constant C. Thus u = Cρ−1 .
Since u = v 0 we have that
v 0 = Cρ−1
=
Cy1−2
Z
exp − P (x) dx .
Integrating this shows that
Z v=
Cy1−2
Z
exp − P (x) dx
dx + K
for some constant K. Since we are simply looking for any linearly independent solution, we
can choose C and K. Let C = 1 and K = 0. Then
R
Z
exp − P (x) dx
dx.
v=
y12
And so,
Z
y2 = y1
R
exp − P (x) dx
dx.
y12
That’s the general reduction of order technique.
We’ll now apply it to the case of a differential equation with a regular singular point at
x = 0 for which the Frobenius theory only guarantees one Frobenius solution.
Suppose we have the differential equation
y 00 +
p(x) 0 q(x)
y + 2 y=0
x
x
where p and q are analytic at x = 0. Suppose that the roots of the indicial equation are r1
and r2 , that r1 ≥ r2 , and that r1 − r2 is an integer N ≥ 0. Thus r2 = r1 − N . Since we have
the roots of the indicial equation in hand, we can factor the equation.
r(r − 1) + p0 r + q0 = (r − r1 )(r − (r1 − N )).
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
32
Rearranging this, we can arrive at
− p0 − 2r1 = −1 − N,
(32)
a fact we will need in what follows.
To use reduction of order, we will let
P (x) =
p(x)
p0
p2
=
+ p1 + x + · · · .
x
x
2!
Then
e−
R
P (x) dx
2
= e−p0 ln x+p1 x+p2 /(2·2!)x +···
2
= x−p0 ep1 x+p2 /(2·2!)x +···
= x−p0 (1 + A1 x + A2 x + · · · ).
The last equality follows from the fact that the composition of analytic functions is analytic.
The first term in that final series is 1 because when x = 0 in the series in the second to last
line we get e0 which is 1.
Using the fact that
∞
X
yn n
y 1 = xr 1
x
n!
n=0
and the reduction of order formula, we get
Z −p0
x (1 + A1 x + A2 x + · · · )
P
y2 = y1
dx
yn n 2
x2r1 ( ∞
n=0 n! x )
Z
(1 + A1 x + A2 x + · · · )
dx
= y1 x−p0 −2r1
(1 + B1 x + B2 x2 + · · · )
Z
= y1 x−1−N (1 + C1 x + C2 x2 + · · · ) dx
where we pick the constant term in the B-series to be one (because it is arbitrary), the Cseries is the quotient of the A-series and the B-series, and we have used (32) on the exponent
of x.
There are now two cases to consider: N = 0 and N > 0.
If N = 0, then
Z
1
y2 = y1
+ C1 + C2 x + · · · dx
x
= y1 ln x + y1 (C1 x + C2 x2 + · · · )
!
∞
X
y
n
= y1 ln x + xr1
xn (C1 x + C2 x2 + · · · )
n!
n=0
= y1 ln x + x
r1 +1
∞
X
n=0
bn x n
3 POWER SERIES SOLUTIONS FOR LINEAR ODES
33
If N > 0, then
Z
y2 = y1
x−1−N (1 + C1 x + C2 x2 + · · · + CN xn + · · · ) dx
Z
1
CN
C1
+ N +1 + N + · · · dx
x
x
x
−N
x
C1 x−N +1
= y1 CN ln x + y1
+
+ ···
−N
−N + 1
!
∞
X
1
y
C1 x
n n
−N
r1
x x
+
+ ···
= y1 CN ln x + x
n!
−N
−N
+
1
n=0
= y1
= y1 CN ln x + x
r2
∞
X
bn x n
n=0
where the last line follows because r2 = r1 − N .
Example: Bessel’s Equation of Order 0,
x2 y 00 + xy 0 + x2 y = 0,
has one Frobenius solution
∞
X
(−1)m 2m
y1 =
x ,
22m (m!)2
m=0
x > 0.
Use this and the preceding argument to find a second linearly independent solution.
Solution: We showed on page 29 that the indicial equation has one repeated root r1 =
r2 = 0 and that the one guaranteed Frobenius solution is the y1 given above. Since N = 0,
the reduction of order method guarantees that there is a second linearly independent solution
of the form
∞
∞
X
X
y2 = y1 ln x + x0+1
bn xn = y1 ln x +
bn xn+1 .
n=0
n=0
We will determine the coefficients bn of this solution by direct substitution. First note that
∞
y20
=
y10
y1 X
bn (n + 1)xn ,
+
ln x +
x
n=0
=
y100
y0
y 0 x − y1 X
ln x + 1 + 1 2
+
bn (n + 1)nxn−1 .
x
x
n=1
∞
y200
Substituting these expressions into Bessel’s equation yields
x2 y100
ln x +
2y10 x
− y1 +
∞
X
bn (n + 1)nxn+1
(33)
n=1
+ xy10 ln x + y1 +
∞
X
bn (n + 1)xn+1
(34)
n=0
+ x2 y1 ln x +
∞
X
n=0
bn xn+3 = 0.
(35)
4 NUMERICAL METHODS FOR ODES
34
Now we know that y1 is a solution to Bessel’s equation, so, in particular,
x2 y100 ln x + xy10 ln x + x2 y1 ln x = 0.
This along with some simplification reduces (35) to
2y10 x
+
∞
X
bn (n + 1)nx
n+1
n=1
+
∞
X
bn (n + 1)x
n=0
n+1
+
∞
X
bn xn+3 = 0.
n=0
We can calculate y10 from the definition of y1 and substitute it in. This, along with some
reindexing of series yields
2x
∞
∞
X
X
(−1)m 2m 2m−1
2
2
x
+b
x+2b
x
+2b
x
+
(bn (n+1)nxn+1 +bn (n+1)xn+1 +bn−2 xn+1 ) = 0,
0
1
1
2m (m!)2
2
m=1
n=2
which after some simplification yields
2
b0 x + 4b1 x +
∞
X
n=2
x
n+1
∞
X
(−1)m+1 4m 2m
(bn (n + 1) + bn−2 ) =
x .
22m (m!)2
m=1
2
We will now equate corresponding coefficients on each side of the equation to determine each
bn . First, we have that b0 = 0 and b1 = 1/4. Next, since the right hand side contains only
even powers, bn (n + 1)2 + bn−2 = 0 for all even n. Because b0 = 0, this reduces to saying
that bn = 0 for all even n. If n is odd, we can relate n to m by n = 2m − 1. In this case, the
recursion becomes
(−1)m+1 4m
,
b2m−1 (2m)2 + b2m−3 = 2m
2 (m!)2
or
b2m−3
(−1)m+1 4m
−
,
m ≥ 2.
b2m−1 = 2m
2
2
2 (m!) (2m)
(2m)2
Thus we have a recursion for the coefficients of the solution y2 .
4
Numerical Methods for ODEs
[Much of the material in this section is adapted from various editions of differential equations
texts by Edwards and Penney and numerical analysis texts by Burden and Faires.]
In this section we will develop three methods for the solution of first order ordinary
differential equations and three methods for systems of first order ODEs. The methods that
we will consider are Euler’s method, the improved Euler’s method, and the Runge-Kutta
method. In terms of practical applicability, the Runge-Kutta method is most useful, but
developing the Euler and improved Euler methods will make it easier to understand what is
happening with the Runge-Kutta method.
4 NUMERICAL METHODS FOR ODES
4.1
35
Euler’s Method for First Order Equations
We return once again to the problem of solving the first order initial value problem
dy
= f (x, y),
y(a) = y0 .
(36)
dx
We will assume throughout this section on numerical methods that the initial value problem
does indeed have a unique solution, y(x).
Euler’s method approximates the true solution y(x) on a set of grid points {x0 , x1 , . . .}
where x0 = a and xi − xi−1 = h for each i = 1, 2, . . .. The constant h is called the step size.
Thus we have that xi = a + ih for each i = 0, 1, 2, . . ..
The true value of the solution at the grid points is given by y(x0 ), y(x1 ), y(x2 ), . . .. The
approximate solutions generated by Euler’s method will be designated by y0 , y1 , y2 , . . . where
yi ≈ y(xi ) for each i = 0, 1, 2, . . .. Euler’s method produces these approximations in a
step-by-step, or sequential, fashion.
Euler’s method is a sequence of linear approximations. The algorithm proceeds as follows:
dy
= f (x, y) and y(a) = y0 . Let x0 = a. The slope
Pick a step size h. We are given dx
of a solution curve passing through the point (x0 , y0 ) is given by f (x0 , y0 ). So y(x1 ) ≈
y0 + hf (x0 , y0 ). Let y1 = y0 + hf (x0 , y0 ). This produces the point (x1 , y1 ). Now repeat the
process. Stated concisely, the algorithm is
yn+1 = yn + hf (xn , yn ),
n = 1, 2, . . . .
Example: Given
dy
= xy 2 ,
dx
y(1) = −2.
Approximate y(2).
Solution: We will apply Euler’s method with h = 0.25. (Note that in this problem, we
can calculate the true solution analytically. It is y = −2/x2 . So y(2) = −0.5. We will use
this to analyze the performance of the approximation method.) The (truncated) results of
Euler’s method are displayed below.
Error
n xn
yn
y(xn ) y(xn ) − yn
0
1
−2
−2
0
1 1.25
−1
−1.28
−.28
2 1.5 −.6875 −.8888
−.2013
3 1.75 −.5102 −.6530
−.1428
4
2 −.3963
−.5
−.1037
% Error
100|y(xn )−yn |
|y(xn )|
0
21.87
22.65
21.86
20.74
Cutting step size will, if we use exact arithmetic, reduce error.
dy
Theorem 17 (Error in Euler’s Method) Suppose that dx
= f (x, y), y(a) = y0 has a
2
unique solution on the interval [a, b] and that y ∈ C [a, b] (the set of functions with two
continuous derivatives on [a, b]). Then there exists a constant C such that if y1 , y2 , . . . , yk
are the Euler approximations to y(x1 ), y(x2 ), . . . , y(xk ) for x1 , x2 , . . . , xk ∈ [a, b] with step
size h, then
|yn − y(xn )| ≤ Ch, n = 1, 2, . . . , k.
4 NUMERICAL METHODS FOR ODES
36
We omit the proof.
Example: Continuing the example above, we can repeatedly use Euler’s method with
decreasing step size to illustrate the theorem. Recall that y(2) = −0.5 is the true value.
Step size Approx to y(2)
1/4
−.3963
1/8
−.4511
1/16
−.4760
1/100
−.4962
1/200
−.4981
Error %Error |Error|/h
−.1037 20.74
.4148
−.0489
9.78
.3912
−.0240
4.80
.3840
−.0038
.38
−.0019
.38
So in this example, it appears that C ≈ .38.
In reality, the accumulation of roundoff error and the length of computation prevent us
from taking h arbitrarily small. We tend to reduce the step size until the approximation stabilizes, but not until the computation takes too long or the approximation becomes unstable
due to roundoff. This process is easier and more useful when we move to the more accurate
Runge-Kutta method.
If we try to apply a numerical method to a problem without a solution, it will usually
still produce output. That is why it is important to determine ahead of time whether a given
problem actually has a solution before applying a numerical technique to approximate that
solution. Here is an example of what happens when we attempt to apply Euler’s method to
a problem that has an asymptote.
Example:
−4
dy
= −y 2 ,
y(0) =
dx
3
Use Euler’s method to approximate y(1).
Solution: Repeatedly using Euler’s method to approximate y(1) with decreasing step size
produces the following output.
Step size Approx to y(2)
1/4
−8.6610
1/8
−24.8084
1/16
−414.9263
1/100
Overflow
The true solution is y(x) = 4/(4x − 3) which has an asymptote at x = 3/4. Thus there
is no solution on the interval [0, 1]. The behavior of Euler’s method—increasing order of
magnitude for each halving of step size—is one indication that something is going wrong.
4.2
Improved Euler Method
The Improved Euler Method is an example of a predictor-corrector method. We apply it
to the same first order ordinary differential equations as Euler’s method. The algorithm
proceeds as follows. The set up is the same as with Euler’s method. Given a point (xn , yn ),
we seek to produce the next point (xn+1 , yn+1 ) in our sequence of approximate solutions. We
begin by taking an Euler step. That is, we do a linear approximation at the point (xn , yn )
4 NUMERICAL METHODS FOR ODES
37
and approximate the solution at xn+1 . Instead of calling this yn+1 (which is what we do in
Euler’s method), we call it un+1 . We then calculate the slope at the point (xn+1 , un+1 ). We
average this slope and the slope from the original linear approximation and it is this average
slope that we use to produce the next solution point (xn+1 , yn+1 ). In short,
• Given (xn , yn ) and
dy
dx
= f (x, y).
• Let k1 = f (xn , yn ).
• Let un+1 = yn + hk1 .
• Let k2 = f (xn+1 , un+1 ).
• Let k = 12 (k1 + k2 ).
• Let yn+1 = yn + hk.
The improved Euler method is more accurate than the Euler method as explained in the
following theorem.
dy
= f (x, y), y(a) =
Theorem 18 (Error in Improved Euler’s Method) Suppose that dx
3
y0 has a unique solution on the interval [a, b] and that y ∈ C [a, b]. Then there exists a constant C such that if y1 , y2 , . . . , yk are the improved Euler approximations to y(x1 ), y(x2 ), . . .,
y(xk ) for x1 , x2 , . . . , xk ∈ [a, b] with step size h, then
|yn − y(xn )| ≤ Ch2 , n = 1, 2, . . . , k.
We omit the proof.
The fact that the error bound here is order h2 rather that order h as it is in Euler’s
method means that when we cut our step size in half, we expect a 4-fold increase in accuracy
rather than the 2-fold increase we expect with Euler.
4.3
Runge-Kutta Method