INTRODUCTION
Numerical analysis involves the study, development and analysis of algorithms for obtaining
numerical solutions to various mathematical problems. Frequently numerical analysis is called the
mathematics of scientific computing. Numerical analysis is the development and study of procedures
for solving problems with computer. The art and science of preparing and solving scientific and engineering
problems have undergone considerable changes due to the available digital computing systems.
Digital computers are the principal means of calculation in numerical analysis and consequently
it is very important to understand how they operate. A computer has a finite word length and so only a
fixed number of digits are stored and used during computation.
1.
Errors
Even in storing an exact decimal number in its converted form in the computer memory, an
error is introduced. This error is machine dependent. Also at the end of computation of aparticular
problem, the final result in the computer should be converted into a form understandable to the user.
Therefore an additional error is committed at this stage too. This error is called local round off error.
Thus we define
Error = True Value – Computed Value
In order to determine the accuracy of an approximate solution, errors are measured in different
ways.
Definition 2 : Absolute error = | error |
Definition 3 : Relative error =
Error
True Value
Definition 4 : Round Off Error : is the quantity R which must be added to the finite representation
of a computed number in order to make it the true representation of that number.
When a number N is written in floating point form with t digits, say, in base 10 as,
N = ( 0 ⋅ d1d 2d 3....d t ) 10e , d1 ≠ 0
1
We say that the number N has t significant digits. For example, 0.3 agrees with
significant digit. The round off error for this representation will be
1
to one
3
1
− 0.3 .
3
All the errors defined above are machine errors and can be minimized by using computing aids
of higher precision.
Methematically, in numerical analysis we usually come across two types of errors.
(i)
Inherent Errors
It is that quantity of error which is present in the statement of the problem itself, before finding
its solution. It arises due to the simplified assumptions made in the mathematical modelling of a problem.
It can also arise when the data is obtained from certain physical measurements of the parameters of the
problem.
(ii)
Truncation Errors
These are errors caused by using approximate formulae in computations. e.g. when a function
f ( x ) is evaluated from an infinite series, if we use only first few terms of the series to compute value of
function f ( x ) , we get an approximate answer. Here, the error is due to truncating the series.
Suppose f ( x ) = cos x . Then
f (x) = 1 −
x2 x4 x6
x 2n
+ − ..... + (−1) n
+ ....
2! 4! 6!
(2n)!
If we retain the first n terms, the truncation error (TE) is
TE = ( −1) n+1 ⋅
x 2n+ 2
x 2n+ 4
x 2n + 6
+ ( −1)n+ 2
+ ( −1)n+ 3
......
(2n + 2)!
(2n + 4)!
(2n + 6)!
The study of this type of error is associated with the problem of convergence. Some special
terminology is used to describe the rapidity with which a sequence converges.
Big O and Little o Notation :
Let { xn } and {α n} be two different sequences.
The equation xn = o (α n ) (we say xn is “little oh” of α n )
xn
=0
n→∞ α
n
lim
2
To avoid division by zero, we say that
xn = o (α n ) if xn ≤ ε n α n and ε n → 0 as n → ∞ .
We write xn = o (α n ) (we say xn is “big oh” of α n ) if there is a constant C and number n0
such that xn ≤ C α n when n ≥ n0 .
These two notations give a coarse method of comparing two sequences. They are frequently
used when both sequences converge to 0. If xn → 0 , α n → 0 and xn = o (α n ) , then xn converges
to 0 at least as rapidly as α n does. If xn → 0 , α n → 0 and xn = o (α n ) then xn converges to 0
more rapidly than α n does.
Definition : The truncation error is the quantity T which must be added to the true representation of
the quantity in order that the result is exactly equal to the quantity we are seeking to generate.
2.
Stability in Numerical Analysis
A number of mathematical problems have solutions that are quite sensitive to small computational
errors, for example round off error. To deal with this phenomenon, we introduce the concept of stability.
A numerical method for solving mathematical problem is considered stable if the sensitivity of the
numerical answer to the data is no greater than in the original mathematical problem.
A numerical method is said to be stable if the effect of any single fixed round off error is
bounded.
3.
Problem Solving Using Computers
In order to solve a given problem using computer the major steps involved are -
(i)
Choosing an appropriate numerical method
(ii)
Designing an algorithm
(iii)
Programming
(iv)
Computer Execution.
In Unit1 to 4 we discuss various numerical methods (and their analysis) for solving transcendental
and polynomial equations, system of linear equations, differential equations, numerical methods available
to interpolate and approximate functions, integration and evaluation of eigen values and eigen vectors
of symmetric matrices.
❖❖❖
3
UNIT - I
TRANSCENDENTAL AND POLYNOMIAL EQUATIONS
Introduction :
One of the basic problems in science and engineering is the computation of roots of an equation
f ( x ) = 0 . The equation f ( x ) = 0 is called algebraic or polynomial equation if it is purely a polynomial
in x. It is called a transcendental equation if f ( x ) contains trignometric, exponential or logarithmic
functions.
Definition 1.0.2 :
A number ξ is a solution of f ( x ) = 0 if f (ξ ) = 0 . Such ξ is called root or zero of f ( x ) = 0 .
Geometrically, a root of f ( x ) = 0 is the value of x at which the graph of y = f ( x ) intersects
the x-axis.
Definition 1.0.2 :
If we can write f ( x ) = ( x − ξ ) m g ( x ) where g ( x ) is bounded and g (ξ ) ≠ 0 then ξ is
called a multiple root of multiplicity m. In this case f ( ξ ) = f ' ( ξ ) = f "(ξ ) = .... = f (m −1) ( ξ ) = 0 .
and f
m
( ξ ) ≠ 0 for m = 1 the root ξ is called simple root.
The following are the basic properties of polynomial equation.
(i)
Every polynomial equation of nth degree, where n is positive integer has exactly n roots.
(ii)
Complex roots occur in pairs i.e. if a + ib is a root of f ( x ) = 0 , so is a – ib.
(iii)
If x = a is a root of f ( x ) = 0 , a polynomial of degree n then f ( x ) = ( x −a ) g ( x ) where
g ( x ) is a polynomial of degree (n – 1).
(iv)
Descartes Rule of Signs :
The number of positive roots of a polynomial equation f ( x ) = 0 with real coefficients cannot
exceed the number of changes in sign of the coefficients in the polynomial f ( x ) = 0 . Similarly, the
number of negative roots of f ( x ) = 0 cannot exceed the number of changes in the sign of the coefficients
4
of f (− x) = 0 . For example, consider f ( x ) = x3 − 3 x2 + 4 x − 5 = 0 . The coefficients of this equation
are (1, –3, 4, –5). As there are three changes in sign, the given equation will have at the most three
positive roots.
(v)
Intermediate Value Property :
If f ( x ) is a real valued continuous function in the closed interval a ≤ x ≤ b then a function
takes each value between f (a ) and f (b) . In particular if f (a ) and f (b) have opposite signs, then
the graph of a function y = f ( x ) crosses the x-axis at least once. i.e. f ( x ) = 0 has at least one root
between a and b.
i.e.
f (ξ ) = 0 , a < ξ < b
There are generally two types of methods used to find roots of f ( x ) = 0 .
(i)
Direct Methods :
These methods give the exact value of the roots in a finite number of steps. Further the methods
give all the roots at the same time. These methods require no knowledge of the initial approximation of
a root of the equation f ( x ) = 0 .
e.g. solutions of polynomial equation are known for polynomials of degree upto cubic. i.e.
for
a1
a0 x + a1 = 0 , x = − a
0
for
(ii)
a0 x + a1 x + a2 = 0 , x =
2
−a1 ± a12 − 4a0a2
2a0
Interrative Methods
These methods are based on the idea of successive approximations i.e. starting with one or
more initial approximations to the root, we obtain a sequence of approximate solutions which converges
to a root of given equation. In the next section we describe some numerical methods for the solution of
equation f ( x ) = 0 .
5
1.2
Bisection Method
This method is due to Bolzano.
Step 1 : Choose x0 and x1 such that f ( x0 ) f ( x1 ) < 0 suppose x0 < x1 .
(By intermediate value principle root lies between x0 and x1 )
Define I 0 = ( x0 , x1 ) .
Step 2 : The desired root is approximately defined by x2 =
x0 + x1
.
2
If f ( x2 ) = 0 then x2 is the desired root of f ( x ) = 0 .
If f ( x2 ) ≠ 0 , calculate f ( x0 ) f ( x2 ) .
If f ( x0 ) f ( x2 ) < 0 then define I1 = ( x0 , x2 ) .
Otherwise define I1 = ( x2 , x1 ) and f ( x2 ) f ( x1 ) < 0 .
Step 3 : Define I 0 = I1 and go to step 1. Thus at each iteration we either find the desired root to the
required accuracy or narrow the length of interval to half of the length of interval at previous step. This
process is continued to determine a smaller and smaller interval within which the desired root lies. If he
permissible error is ε , then the approximate no. of iterations (n) required may be determined from the
relation
x1 − x0
≤ε
2n
Note : The no. of iterations required to achieve required accuracy depends upon the initial interval I 0 .
If the length of I 0 is sufficiently small we will reach at the solution in less no. of iterations.
EXAMPLES ....................................................................................................
1.2.1 : Find a real root of the equation, f ( x ) = x3 − x1 − 1 = 0
Answer :
Step 1 : Since f (1) = −1 < 0 and f (2) = 5 > 0 , the root lies between 1 and 2.
i.e. I 0 = ( x0 , x1 ) = (1,2) .
Step 2 : x2 =
x0 + x1
= 1.5 , f ( x2 ) = f (1.5 ) =
2
6
1.3 Iteration Methods Based on First Degree Equation
Although the bisection method is easy to compute it is not very efficient. For most functions we
can improve the speed at which the root is approach through different schemes. Almost every functions
can be approximated by a straight line over a small interval. We begin from a value that is near to a
root. This initial value can be obtained by looking at the graph of a function or from few iterations of
bisection method.
Iteration methods are obtained by approximating f ( x ) by a polynomial of degree one in the
neighbourhood of root. Thus
f ( x ) = a0 x + a1 = 0 ⇒ x = −
a1
, a ≠0
a0 0
...... (1.3.1.)
The parameters a0 and a1 are to be determined by prescribing two approximate conditions
on f ( x ) and / or its derivatives.
1.3.1 Secant Method
Suppose xk −1 and xk are two approximations to the root, then we determine a0 and a1 by
using linear approximation.
f ( xk ) = a0 xk + a1
f ( xk −1 ) = a0 xk −1 + a1
On solving above two equations simultaneously for a0 and a1 we get
a0 =
f ( xk ) − f ( xk −1 )
xk − xk −1
and a1 =
xk f ( xk −1 ) − xk −1 f ( xk )
xk − xk −1
from equation (1.3.1) we get, the next approximate root
xk +1 =
xk −1 f ( xk ) − xk f ( xk −1 )
f ( xk ) − f ( xk −1 )
which may be written as
xk +1 = xk −
xk − xk −1
f k , k = 1, 2, 3, ....
f k − f k −1
where f k = f ( xk ) and f k −1 = f ( xk −1 ) .
This is called the Secant or the Chord Method.
7
.... (1.3.2)
Geometrically, in this method we choose two points on the curve and plot the line passing
through these two points. The point of intersection of the straight line with the x-axis is the next
approximation to the root (Fig. 1.1).
x0
x1
x3
x2
Fig. 1.1
1.3.2 Regula Falsi Method
This is the oldest method for finding the real root of an equation f ( x ) = 0 and closely resembles
bisection method. This method is also called as method of false position. In this method we choose two
points x0 and x1 such that f ( x0 ) and f ( x1) are of opposite signs. Since the graph of y = f ( x )
crosses the x-axis between these two points, a root must lie in between these points. Now the equation
of the Chord joining the two points ( x0 , f ( x 0 ) ) , ( x1, f ( x1 ) ) is
f ( x1 ) − f ( x0 )
y − f ( x0 ) =
( x − x0 )
x1 − x0
The point of intersection of the chord with the x-axis is given by putting y = 0. Thus we get,
− f ( x0 ) =
f ( x1 ) − f ( x0 )
x1 − x0
( x − x0 )
On solving above equation for x we obtain
x = x0 −
f ( x0 )
f ( x1 ) − f ( x0 )
( x1 − x0 )
8
Hence the second approximation to the root of f ( x ) = 0 is given by
x2 = x0 −
f ( x0 )
f ( x1 ) − f ( x0 )
( x1 − x0 )
.... (1.3.2.1)
If now f ( x2 ) and f ( x0 ) are of opposite signs then the root lies between x0 and x2 and we
replace x1 by x2 in (1.3.2.1) and obtain the next approximation. Otherwise we replace x0 by x2 and
generate next approximation. The procedure is repeated till the root is obtained to the desired accuracy.
The repeated application of this procedure generates a sequence.
( x0 , f ( x 0 ) )
x3
x2
x1
x0
( x1, f ( x1 ) )
( x2 , f ( x2 ) )
First iteration
Suppose the approximate solution after (k – 1) iterations is denoted by xk . Then the sequence
{xk } approaches to the root ξ
as k → ∞ i.e. f ( ξ ) = 0 .
1.3.3 Newton Raahson Method
This method is generally used to improve the result obtained by one of the previous methods.
Firstly we derive this method by using linear approximation. Suppose xk is a point in the neighbourhood
of the root of f ( x ) = 0 . If we approximate f ( x ) by a polynomial of degree one we get
f ( x ) = a0 x + a1 .
∴ f ( xk ) = a0 xk + a1
and
f ' ( xk ) = a0
... (1.3.3.1)
9
where a prime denotes differentiation with respect to x. On solving for as and a1 we get
a0 = f ' ( xk ) and a1 = f ( xk ) − f ' ( xk ) xk
From equation (1.3.1) we get,
x=−
f ( xk ) − f ' ( xk ) xk
f ' ( xk )
= xk −
f ( xk )
f ' ( xk )
Thus we get the next approximate root as
xk +1 = xk −
f ( xk )
f ' ( xk ) , k = 0, 1, 2, 3, ...
... (13.3.2)
This method is called the Newton Raphson method. The method (1.3.3.2) may also be obtained
directly from Secant method (1.3.2.2) by taking the limit xk −1 → xk . In the limiting process i.e. when
xk −1 → xk the chord passing through the points ( xk , f ( xk ) ) and ( xk −1 , f ( xk −1 ) ) converges to the
tangent at point ( xk , f ( xk ) ) . Thus in this case the problem of finding a root of equation f ( x ) = 0 is
equivalent to finding the point of inteersection of the tangent to the curve y = f ( x ) at the point
( xk , f ( xk ) ) with the x-axis. The Newton Raphson method requires two values f ( xk ) and f ' ( xk ) .
The method is applicable only when f ' ( xk ) ≠ 0 i.e. root is a simple root.
The method can also be derived by using Taylor series representation. Let x0 be an approximate
root of f ( x ) = 0 and let x1 = x0 + h be the correct root so that f ( x1 ) = 0 . Expanding
f ( x1 ) = f ( x0 + h ) by Taylor series about x0 , we obtain
f ( x1 ) = f ( x0 + h ) = f ( x0 ) + hf ' ( x0 ) +
h2
f " ( x0 ) + .... = 0
x
Neglecting the second and higher order derivatives we have
f ( x0 ) + hf ' ( x0 ) = 0
i.e.
h=−
f ( x0 )
f ' ( x0 )
10
Therefore
x1 = x0 + h = x0 −
f ( x0 )
f ' ( x0 )
is the better approximation than x0 .
Successive approximations are given by x2 , x3 , ... where
xn+1 = xn −
f ( xn )
f ' ( xn ) , n = 0, 1, 2, 3, ...
This is same as the formula (1.3.3.2).
1.4
Rate of Convergence
All the methods described in section 1.3 are iterative methods and repeatative application of
these methods generate a sequence of approximate solutions. Convergence of this sequence is an
important subject that we will discuss now.
1.4.1
Orders of Convergence
Some special terminology is used todescribe the rapidity with which a sequence convergences.
Let { xn } be a sequence of real numbers tending to a limit x*. We say tha the rate of convergence is at
least linear if there is a constant c < 1 and an integer N such that
xn+1 − x * ≤ C xn − x *
(n ≥ N )
... (1.4.1.1)
The convergence is atleast quadratri C if there are a constant C (not necessary less tnhan 1)
and an integer N such that
xn+1 − x * ≤ C xn − x *
2
(n ≥ N )
... (1.4.1.2)
In general if there are positive constants C, largest α and an integer N such that
xn+1 − x * ≤ C xn − x *
α
(n ≥ N )
... (1.4.1.3)
We say that the rate of convergence is atleast α . The constant C is called the asymptotic error
constant.
If xk is an approximate root of f ( x ) = 0 and ξ is a solution of equation f ( x ) = 0 then
ε k = xk − ξ is the error in the solution. If the sequence ε k → 0 as k → ∞ , we say that iterative
methods discussed in section 1.3 are convergent. We assume that ξ is a simple root of f ( x ) = 0 so
that f ' ( ξ ) ≠ 0 .
11
1.4.2
Rate of convergence of Secant Method
Suppose ξ is a simple root of f ( x ) = 0 i.e. f ( ξ ) = 0 .
ε k = xk − ξ is an error. On substituting xk = ξ + ε k in (13.2.2) we get,
ε k +1 = ε k −
( ε k − ε k −1 ) f ( ξ + ε k )
f (ξ + ε k ) − f (ξ + ε k −1 )
Expanding f ( ξ + ε k ) and f ( ξ + ε k −1 ) in Taylor series about the root ξ and observing that
f ( ξ ) = 0 we get,
ε k +1 = ε k −
( ε k − ε k −1 ) ε k f ' (ξ ) +
εk 2
f "(ξ ) + ...
2!
1
( ε k − ε k −1 ) f ' (ξ ) + (ε k 2 − ε k −12 ) f " (ξ ) + ...
2
εk2
ε
−
ε
ε
f
'
(
ξ
)
+
f " ( ξ ) + ...
( k k −1 ) k
2!
= εk −
1
( ε k − ε k −1 ) f ' ( ξ ) + ( ε k + ε k −1 ) f "( ξ ) + ...
2
−1
1
ε k 2 f "( ξ )
f " (ξ )
= ε k − ε k +
+ ... 1 + ( ε k + ε k −1 )
+ ...
2!
f
'
(
ξ
)
2
f
'
(
ξ
)
1
ε k 2 f "( ξ )
f "( ξ )
= ε k − ε k +
+ ... 1 − ( ε k + ε k −1 )
+ ...
2! f ' (ξ )
f '( ξ )
2
1
= 1 + x + x 2 + x3 + ...
Q
1− x
f " (ξ ) ε k 2 f "( ξ )
1
= ε k − εk − ε k ( ε k + ε k −1 )
+
+ ...
2
f ' ( ξ ) 2! f ' ( ξ )
2
1 2
f "( ξ )
− ε k (ε k + ε k −1 )
+
...
2
f ' (ξ )
=
1
f "( ξ )
ε k ε k −1
+ 0 ( ε k 2ε k −1 + ε k 3 )
2
f ' (ξ )
12
Thus we write ε k +1 = C ⋅ ε k ε k −1 where C = 1 f "( ξ ) and we ignore higher powers of ε k .
2 f ' (ξ )
The equation
ε k +1 = C ⋅ ε k ε k −1
... (1.4.2.1)
is called error equation.
To determine the order of converence discussed in section 1.4.1, we have to determine the
number α such that ε k +1 = A ⋅ ε kα where A and α are to be determined. If we replace k by k –1 we
get,
ε k = Aε k −1α
ε k −1 = A
i.e.
1
−1
αε α
k
Substituting the values of ε k −1 and ε k +1 in equation (1.4.2.1) we get,
Aε kα = C ⋅ ε k A
ε k α = CA
1
−1
αε α
k
−1− 1
1+ 1
αε
α
k
Comparing the powers of ε k on both sides of above equation we get,
α =1 +
1
α
which is quadratic equation in α and we get α =
α=
1
(1 ± 5 ) . The highest value of
2
1
(1 + 5 ) and we find that the rate of convergence of secant method is α = 1.618 and
2
A=C
1.4.3
α
1+α
.
Rate of Convergence of Regula Falsi Method
If the function f ( x ) in the equation f ( x ) = 0 is convex in the interval ( x0 , x1 ) that contains
the root, then one of the points x0 or x1 is always fixed and the other point varies with k. If the point
x0 is fixed, then the function f ( x ) is approximated by the straight line passing through the points
( x0 , f ( x 0 ) ) and ( xk , f ( xk ) ) , k = 1, 2, 3,....
13
Suppose ξ is simple root of f ( x ) = 0 i.e. f ( ξ ) = 0 and ε k = xk − ξ is an error in
approximate solution xk . Since the point ( x0 , f ( x 0 ) ) is fixed we can write
xk +1 = x0 −
f ( x0 )( xk − x0 )
f ( xk ) − f ( x0 )
∴ξ + ε k +1 = ξ + ε 0 −
f ( ξ + ε 0 ) ξ + ε k − (ξ + ε 0 )
f (ξ + ε k ) − f (ξ + ε 0 )
ε 02
ε
f
'
(
ξ
)
+
f " (ξ ) + ... ( ε k − ε 0 )
0
2!
∴ε k +1 = ε 0 −
2
εk
ε 02
ε
f
'
ξ
+
f
"
ξ
+
...
−
ε
f
'
ξ
+
f " ( ξ ) + ...
(
)
(
)
(
)
k
0
2!
2!
( ε k − ε 0 ) f ' (ξ ) ε 0 +
= ε0 −
( ε k − ε 0 ) f ' (ξ ) 1 +
ε 02 f "( ξ )
+ ...
2! f ' ( ξ )
ε k + ε 0 f "( ξ )
+ ...
2! f ' (ξ )
ε + ε 0 f "( ξ )
ε 0 2 f "(ξ )
= ε 0 − ε 0 +
+ ... 1 − k
+ ...
2! f ' (ξ )
2! f ' ( ξ )
=
ε 0 ( ε k + ε 0 ) f " ( ξ ) ε 02 f " (ξ )
−
+ ...
2!
f ' ( ξ ) 2! f ' ( ξ )
=
1 f "( ξ )
ε 0ε k + 0 ( ε 0ε k 2 , ε k 3 )
2 f ' (ξ )
Since ε 0 = x0 − ξ is the error in the first approximation and is independent of k, we can write
C=
1 f "( ξ )
ε0
2 f ' (ξ )
and we get error equation
ε k +1 = Cε k
Here C is asymptotic error constant and by equation (1.4.1.2), we observe that the Regular
Falsi method has at least linear rate of convergence.
14
1.4.4
Rate of Convergence of Newton Raphson Method
Suppose ξ is a simple root of f ( x ) = 0 i.e. f ( ξ ) = 0 but f ' ( ξ ) ≠ 0 . Suppose ε k = xk − ξ
is an error in the approximate solution xk . On substituting xk = ξ + ε k in equation (1.3.3.2) and
expanding f ( ξ + ε k ) and f ' ( ξ + ε k ) in Taylor series about the point ξ and using the fact that
f ( ξ ) = 0 and f ' ( ξ ) ≠ 0 , we obtain
ε k2
ε
f
'
(
ξ
)
+
f "( ξ ) + ....
k
2!
ξ + ε k +1 = ( ξ + ε k ) −
2
ε
f ' ( ξ ) + ε k f "( ξ ) + k f '"( ξ ) + ....
2!
−1
ε k 2 f " (ξ )
f "(ξ )
ε k +1 = ε k − ε k +
+ .... 1 + ε k
+ ....
2! f ' ( ξ )
f ' (ξ )
ε k 2 f "( ξ )
f "(ξ )
= ε k − ε k +
+ .... 1 − ε k
+ ....
2! f ' (ξ )
f ' (ξ )
ε 2 f "( ξ )
f "( ξ ) ε k 3 f "(ξ )
= εk −εk + k
+ .... + ε k 2
+
2! f ' ( ξ )
f ' (ξ ) 2! f ' (ξ )
=
2
ε k 2 f " (ξ )
+ 0 (ε k 3 )
2! f ' ( ξ )
On neglecting terms containing ε k 3 and higher powers of ε k we get an error equation
1 f "( ξ )
ε k +1 = Cε k 2 where C = 2 f ' (ξ )
Thus by equation (1.4.1.2) we obsrve that Newton Raphson method has second order
convergence.
Note : If the root ξ of f ( x ) = 0 is a root of multiplicity two or more then the rate of convergence for
Newton Raphson method is one.
15
1.5
Interation Methods
To describe this method for finding the roots of equation
f (x) = 0
... (1.5.1)
We rewrite this equation in the form
x = φ (x )
... (1.5.2)
Let x0 be an approximate value of the desired root ξ . Substituting it for x on the right hand
side of equation (1.5.2) we obtain the first approximation
x1 = φ ( x0 )
The successive approximations are then given by
x2 = φ ( x1 )
x3 = φ ( x2 )
-----------
xn = φ ( xn−1 )
Thus we get a sequence of approximate solutions { xk } . The convergence of this sequence
depends on the suitable choice of function φ ( x ) and initial approximation x0 . The function φ ( x ) is
called an iteration function. If the function φ is continuous and the sequence { xn } converges to x*
then
xn+1 = φ ( xn )
⇒ x* = lim xn+1 = Lt φ ( xn ) = φ ( Lt xn ) = φ ( x *)
n →∞
n →∞
n →∞
Thus x* is a root of equation (1.5.2) if the iteration function φ is continuous function.
The following theorem gives a necessary and sufficient condition for the convergence of
sequence { xn } .
Theorem : If φ ( x ) is a continuous function in some interval [ a, b] that contains the root and
φ ' ( x ) ≤ C < 1 in this interval, then for every choice of x0 ∈ [ a, b] , the sequence { xk } determined
from
xk +1 = φ ( xk ) , k = 0, 1, 2, 3, ....
16
Converges to the root ξ of x = φ ( x ) .
Proof : Since ξ is a root of equation x = φ ( x ) ,
ξ = φ (ξ )
... (1.5.3)
xk +1 = φ ( xk )
... (1.5.4)
From equation (1.5.3) and (1.5.4) we get,
ξ − xk +1 = φ ( ξ ) − φ ( xk ) , k = 0, 1, 2, 3, ...
Using the mean value theorem, we get
ξ − xk +1 = φ ' (θ k )(ξ − xk )
θ k ∈ L (ξ , xk )
where L (ξ , xk ) represents a line segment joining ξ and xk .
Similarly, we obtain
ξ − xk = φ ' (θ k −1 )(ξ − xk −1 )
θ k −1 ∈ L (ξ , xk −1 )
ξ − xk −1 = φ ' (θ k −2 )(ξ − x k− 2 )
θ k − 2 ∈ L (ξ , xk − 2 )
-------------------------------
ξ − x1 = φ ' (θ 0 )(ξ − x0 )
θ 0 ∈ L (ξ , x0 )
Since each θ i ∈ [ a, b] , φ ' (θ i ) ≤ C < 1 and ξ − xk +1 ≤ C k +1 ( ξ − x0 )
Thus,
ε k +1 ≤ C k +1 ε 0
Since C < 1, the right hand side of above inequality goes to zero as k becomes large and it
follows that the sequence of approximations { xk } . Converges to the root ξ if C < 1.
Note : The root obtained by this method is unique. Suppose ξ1 and ξ 2 are two distinct roots of
equation (1.5.2). i.e. ξ1 = φ ( ξ1 ) and ξ 2 = φ ( ξ2 ) .
Then we get
ξ1 − ξ2 = φ (ξ 1) − φ ( ξ 2 )
= φ ' ( θ ) ( ξ1 − ξ 2 )
∴ (ξ1 − ξ 2 ) [1 − φ ' ( θ )] = 0
17
But,
φ ' (θ ) ≤ C < 1
∴1 − φ ' ( θ ) ≠ 0
∴ξ1 −ξ 2 = 0
and therefore ξ1 = ξ 2 , hence the root is unique.
In general the speed of convergence depends on the value of C; the smaller the value of C, the
faster would be the convergence. Therefore, the speed of convergence dependent upon the choice of
φ ( x ) . There are many ways of rewriting f ( x ) = 0 in the form x = φ ( x ) . For example the equation
f ( x ) = x 3 + x 2 − 1 = 0 can be expressed as
x = (1 + x )
−1
x = (1 − x 3 )
1
x = (1 − x 2 )
1
2
= φ1 ( x )
(say)
2
= φ2 ( x)
(say)
3
= φ3 ( x)
(say)
x = 1 + x − x 2 − x3 = φ4 ( x )
(say)
We have to choose that function φi ( x ) for which φi ' ( x ) < 1 . Since f (0) = − 1 and f (1) = 1,
we know that root lies between 0 and 1.
for x ∈ [ 0,1]
φ4 ' ( x ) = 1 − 2 x − 3x 2 </ 1
φ2 ' ( x ) =
=
−1
1(
1 − x 3 ) 2 ( − 3x 2 )
2
−1
3 2(
x 1 − x3 ) 2 → ∞
2
as x → 1
Observe that the functions φ2 , φ3 , φ4 are not the expected choices of iteration function, as
φi '⋅ ( x ) </ 1 for i = 2, 3, 4 in the interval [ 0, 1 ].
If we choose φ ( x ) = (1 + x )−
and
φ '( x) =
2
1
−3
then φ ' ( x ) = − (1 + x ) 2
2
3
1
(1 + x )− 2 < 1
2
max φ ' ( x ) =
0 ≤ x≤1
1
1
2 8
∀x ∈ [ 0,1]
= 0.17678 < 1
and the iteration method xk +1 = φ ( xk ) converges to the root as φ ' ( x ) < 1 .
18
1.6
Polynomial Equations
Polynomial functions are of special importance. They are everywhere continuous, they are
smooth, their derivatives are also continuous and smooth and they are readily evaluated. Descarte’s
rule of signs predict the number of positive roots. Polynomials are particularly well adapted to computers
because the only mathematical operations they require for evaluation are addition, subtraction and
multiplication.
To determine the roots of polynomial equation it is necessary to have the following information
(i)
The exact number of real and complex roots along with their multiplicity.
(ii)
The interval in which each real root lies.
By fundamental theorem of algebra we know that a polynomial of degree n has exactly n
roots. Decarte’s rule of signs gives only upper limit of no. of + ve and – ve real roots. This rule does not
give the exact number of positive and negative real roots. The exact number of real roots of a polynomial
can be found by Sturms theorem.
Let f ( x ) be a polynomial of degree n. Let f1 ( x ) represent its first order derivative. The
remainder of f ( x ) divided by f1 ( x ) taken with the reverse sign is denoted by f 2 ( x ) . Let f 3 ( x )
denotes the remainder of f1 ( x ) divided by f 2 ( x ) with the reverse sign. Continue this process till we
arrive at a constant. We thus obtain a sequence of functions
f ( x ), f1( x), f 2 ( x ),...., f k ( x )
This sequence is called Sturm sequence.
Sturm Theorem : The number of real roots of the equation f ( x ) = 0 on [ a, b] equals the difference
between the number of changes of sign in the Sturm sequence at x = a and x = b, provided that
f (a ) ≠ 0 , f (b) ≠ 0 .
Since a polynomial of degree n has exactly n roots, the number of complex roots equals ( n number of real roots), where a real root of multiplicity r is counted r times.
If x 1, x 2, x 3, ...., x n are real roots of f ( x ) then
f ( x ) = a0 ( x − x1 )( x − x2 ) ( x − x3 ) ... ( x − xn )
Complex roots occur in pair. If x 1, x 2 are complex roots then ( x − x1 )(x − x 2 ) is a polynomial
of degree two with real co-efficients and in this case
f ( x ) = a0 ( x 2 + px + q ) ( x − x3 ) ... ( x − xn )
19
Thus it is obvious that the methods of finding roots of polynomial equation should include the
determination of either linear factor ( x − p ) or quadratic factor x 2 + px + q .
In this section two methods are presented. Birge Vieta method is used to determine the linear
factor ( x − p ) whereas Bairstow method is used to determine the quadratic factor x 2 + px + q .
1.6.1
Birge-Vieta Method
This method is used to determine real root of polynomial equation
Pn ( x ) = a0 x n + a1x n−1 + a2 x n−1 + ..... + an−1x + a n = 0
... (1.6.1.1)
If P is a root of polynomial Pn ( x ) then ( x − p ) is factor of polynomial Pn ( x ) . Suppose p is an
approximate root of Pn ( x ) . If we divide Pn ( x ) by a factor ( x − p ) then we get a quotient Qn−1 a
polynomial of degree (n – 1) and a remainder. The remainder R depends on choice of p. i.e. if we
change the value of p, R will get change. Birge Vieta method gives a procedure to make R zero.
Suppose p is an approximate root of Pn ( x ) .
Pn ( x ) = ( x − p ) Qn −1 (x ) + R
... (1.6.1.2)
where Qn−1 (x ) = b 0x n−1 + b1x n−2 + b3 x n−3 + .... + bn−2 x + bn−1
.... (1.6.1.3)
a polynomial of degree (n – 1) and R is remainder.
The coefficients in polynomial Qn−1 i.e. bi and remainder R are functions of p. Birge Vieta
method gives a procedure to determine p such that R( p) = 0 .
Pn ( p ) = ( p − p ) Qn −1 ( p) + R ( p ) = R( p)
... (1.6.1.4)
R ( p) = 0
Equation (1.6.1.4) is the equation in variable p and any iterative method discussed in section
1.3 can be used to determine the root p. Application of Newton Raphson method discussed in section
1.3.3 for equation (1.6.1.4) gives
pk +1 = pk −
Pn ( pk )
Pn ' ( pk )
... (1.6.1.5)
For a polynomial equation, the computation of Pn and Pn ' can be obtained with the help of
synthetic division. On comparing the coefficients of like powers of x on both sides of equation (1.6.1.2)
and using equations (1.6.1.1) and (1.6.1.3) we get,
a0 x n + a1 x n −1 + a2 xn −2 + ..... + an −1 x + an = ( x − p ) ( b0 x n −1 + b1 x n −2 + ..... + bn −1 ) + R
20
Thus
a0 = b0 ⇒ b0 = a0
a1 = b1 − pp0 ⇒ b1 = a1 + pb0
a2 = b2 − pp1 ⇒ b2 = a2 + pb1
M
ak = bk − pbk −1 ⇒ bk = ak + pbk −1
M
an = R − pbn−1 ⇒ R = an + pbn−1
In general
bk = ak + pbk −1 , k = 1, 2, 3, .... ,n
... (1.6.1.6)
with b0 = a0 and bn = R .
From equation (1.6.1.4) we have
Pn ( p ) = R = bn
... (1.6.1.7)
To determine Pn '( p ) , differentiate (1.6.1.6) with respected p.
dbk
db
= bk −1 + p k −1
dp
dp
... (1.6.1.8)
Equation (1.6.1.8) can be written as
Ck −1 = bk −1 + pCk − 2
where
dbk
= Ck −1 , k = 1, 2, 3, ...., n
dp
Above equation can also be represented by
Ck = bk + pCk −1 , k = 1, 2, , n – 1
and
C0 =
... (1.6.1.9)
db1 d
=
( a + pb0 ) = b0
dp dp 1
(Since b0 = a0 and a0 is independent of p, differentiation of b0 with respect to p is zero).
21
On differentiating (1.6.1.7) with respect to p we get
dPn ( p ) dR dbn
=
=
= Cn−1
dp
dp dp
... (1.6.1.10)
By substituting the values of Pn ( p ) and Pn '( p ) from equation (1.6.1.7) and (1.6.1.10) in
equation (1.6.1.5) we get
pk +1 = pk −
bn
, k = 0, 1, 2, ....
Cn−1
.... (1.6.1.11)
where p0 is initial approximation of factor ( x − p ) .
The method (1.6.1.11) is called the Birge Vieta Method.
The calculations of the coefficients bk and Ck are carried out by using synthetic division.
p
p
a0
a0
C0
a1
a2
a3
...
an −2
an−1
an
pb0
pb1
pb2
...
pbn−3
pbn−2
pbn−1
b1
b2
b3
...
bn− 2
bn−1
bn = R = Pn ( p )
pC0
pC1
pC2
...
pCn−3
pCn− 2
C1
C2
C3
...
Cn−2
Cn−1 =
dR dPn ( p)
=
dp
dp
Note : If the polynomial Pn ( x ) do not contain the term x k , write ak = 0 .
Once p is calculate with desired accuracy then repeat the procedure for Qn−1 ( x) to determine
second factor of Pn ( x ) . The continuous application of Birge-Vieta method produces all real roots.
1.6.2
Bairstow Method
This method is used to extract quadratic factor from polynomial Pn ( x ) , which may give a pair
of complex roots or pair of real roots. If we divide the polynomial Pn ( x ) defined by equation (1.6.1.1)
by the quadratic factor x 2 + px + q then the quotient is a polynomial of degree (n – 2) and a remainder
is a polynomial of degree one .
Thus
Pn ( x ) = ( x 2 + px + q ) Qn−1(x ) + Rx + S
where Qn− 2 (x ) = b0 x n−2 + b1 x n−3 + b2 xn−4 + ... + bn−3 x + bn− 2
22
... (1.6.2.1)
.... (1.6.2.2)
The coefficient b0 , b1 , b2 , ... bn−3 , bn− 2 , R and S are functions of p and q. x 2 + px + q is
a factor of Pn ( x ) , if
R ( p , q ) = S ( p, q ) = 0
.... (1.6.2.3)
Suppose ( p0 , q0 ) is an initial approximation for equation (1.6.2.3) and ( p0 + ∆p, q0 + ∆q )
is the true solution of equation (1.6.2.3). Then
R ( p0 + ∆p , q0 + ∆q ) = R ( p0 , q0 ) +
∂R
∂R
∆p +
∆q = 0
∂p
∂q
S ( p0 + ∆p , q0 + ∆q ) = S ( p0 , q0 ) +
∂S
∂S
∆p +
∆q = 0
∂p
∂q
On solving above equations simultaneously for ∆p and ∆q we get
∆p = −
R S − RS p
RSq − SRq ,
∆q = − p
Rp S q − Rq S p
Rp S q − Rq S p
... (1.6.2.4)
where R p , Rq , S p , Sq are partial derivatives of R and S with respect to p and q respectively
evaluated at ( p0 , q0 ) . Functions R and S are also evaluated at ( p0 , q0 ) .
Thus to determine the true solution it is necessary to calculate ∆p and ∆q . The increaments
∆p and ∆q are known in terms of R, S, R p , Rq , S p , Sq evaluated at initial approximation ( p0 , q0 ) .
These functions R, S and their partial derivatives are obtained by comparing the coefficients of equal
powers of x in equation (1.6.2.1). From equation (1.6.1.1) and (1.6.2.1) we get
x n : a0 = b0 ⇒ b0 = a0
x n−1 : a1 = b1 + pb0 ⇒ b1 = a1 − pb0
x n− 2 : a2 = b2 + pb1 + qb0 ⇒ b2 = a2 − pb1 − b0
M
M
x n− k : ak = bk + pbk −1 + qbk− 2 ⇒ bk = ak − pbk −1 − qbk −2
M
M
x : an−1 = R + pbn−2 + qbn−3 ⇒ R = an−1 − pbn−2 − qbn−3
x 0 : an = S + qbn− 2 ⇒ S = an − qbn−2
23
In general we write
bk = ak − pbk −1 − qbk −2 , k = 1, 2, 3, ..., n
.... (1.6.2.5)
where b0 = a0 and b−1 = 0 .
From last two equations we get
R = bn−1 and S = bn + pbn−1
... (1.6.2.6)
The partial derivatives R p , Rq , S p , Sq can be determined by differentiating (1.6.2.5) with
respect to p and q. From equation (1.6.2.5) we get,
−
∂bk
∂b
∂b
∂b0 ∂b−1
= bk −1 + p k −1 + q k − 2 ;
=
=0
∂p
∂p
∂p
∂p
∂p
.... (1.6.2.7)
−
∂bk
∂b
∂b
∂b0 ∂b−1
= p k −1 + bk − 2 + q k −2 ;
=
=0
∂q
∂q
∂q
∂q
∂q
.... (1.6.2.8)
Substitution
∂bk
= −Ck −1 , k = 1, 2, 3, ..., n converts equation (1.6.2.7) into
∂p
Ck −1 = bk −1 − pCk −2 − qCk −3
If we write Ck − 2 = −
.... (1.6.2.9)
∂bk
then equation (1.6.2.8) becomes
∂q
Ck −2 = bk −2 − pCk −3 − qCk −4
.... (1.6.2.10)
From equation (1.6.2.9) and (1.6.2.10) we get a recurrence relation
Ck = bk − pCk −1 − qCk −2 , k = 1, 2, 3, ...., n –1
where C−1 = 0 and C0 = −
∂b1
∂
= − ( a1 − pb0 ) = b0
∂p
∂p
(Since a1 and b0 = a0 are independent of p and q)
From equation (1.6.2.6), (1.6.2.7) we get
Rp =
∂R ∂bn−1
=
= −Cn−2
∂p
∂p
Sp =
∂bn
∂b
+ bn−1 + p n−1 = bn−1 −Cn−1 − pCn−2
∂p
∂p
From equation (1.6.2.6) and (16.2.8) we get
24
Rq =
∂R ∂bn−1
=
= −Cn−3
∂q
∂q
Sq =
∂S ∂bn
∂b
=
+ p n−1 = −Cn− 2 − pCn−3
∂q ∂q
∂q
On substituting the above values of R, S, R p , Rq , S p , Sq in equation (1.6.2.4) and using
equation (1.6.2.6) we get,
∆p = −
=−
∆q = −
and
=−
bn−1 ( − Cn− 2 − pCn −3 ) − ( bn + pbn−1 )( −Cn −3 )
( −Cn −2 )( −C n −2 − pCn−3 ) − ( −Cn−3 )( b n−1 − Cn−1 − pCn− 2 )
bnCn− 3 − bn−1Cn− 2
− Cn− 3 ( Cn− 1 − bn− 1)
Cn2− 2
( −Cn−2 )( bn + pbn−1 ) − ( bn−1 )( bn−1 − Cn−1 − pCn− 2 )
( −Cn −2 )( −C n −2 − pCn−3 ) − ( −Cn−3 )( b n−1 − Cn−1 − pCn− 2 )
bn−1 ( C n−1 − bn−1 ) − bn Cn− 2
Cn2− 2 − Cn −3 ( Cn −1 − bn − 1)
The improved values of p and q are now
p1 = p0 + ∆p and q1 = q0 + ∆q
Repeat the procedure by replacing the initial approximation ( p0 , q0 ) by ( p1 , q1 ) , till we get
the required accuracy of p and q.
For computing bk ' s and Ck ' s we use the following representation
−p
a0
−q
−p
b0
−q
C0
a1
a2
...
an −2
an−1
an
− pb0
− pb1
...
− pbn−3
− pbn−2
− pbn−1
− qb0
...
− qbn−4
− qbn−3
− qbn−2
b1
b2
...
bn− 2
bn−1
bn
− pC0
− pC1
...
− pCn−3
− pCn− 2
− qC0
...
− qCn− 4
− qCn−3
C2
...
Cn−2
Cn−1
C1
25
When p and q are obtained to the desired accuracy the polynomial Qn− 2 ( x ) =
Pn ( x )
x + px + q
2
is
obtained from the synthetic division procedure. The next quadratic factor of Qn−2 (and hence of
Pn ( x ) ) is obtained by applying Bairstow method to Qn− 2 ( x) .
❖❖❖
26
UNIT - II
SYSTEM OF LINEAR ALGEBRAIC EQUATIONS AND
EIGEN VALUE PROBLEMS
2.1
Introduction :
System of linear equations arise in a large number of areas, both directly in modeling physical
situations and indirectly in the numerical solution of other mathematical models. These applications
occur in virtually all areas of the physical, biological and social sciences. Linear systems are involved in
optimization theory, numerical solutions of boundry value problems, partial differential equations, integral
equations and numerous other problems.
The present chapter deals with simultaneous linear algebraic equations which can be represented
generally as,
a11x1 + a12 x2 + a13 x3 + .... + a1n xn = b1
a21x1 + a22 x2 + a23 x3 + .... + a2n xn = b2
... (2.1.1)
M
an1 x1 + a n2 x2 + an3 x3 + .... + ann xn = bn
where aij (i, j = 1, 2, ...., n) are the known coefficients, bi (i = 1, 2, ..., n) are the known
values and xi (i = 1, 2, ..., n) are the unknowns to be determined.
In the matrix notation, the above system of simultaneous linear algebraic equations can be
written as
Ax = b
.... (2.2.2)
where A is square matrix of order n, x is column vector with elements xi , i = 1, 2, ..., n and b
is column vector with elements bi , i = 1, 2, ..., n.
2.2
Iteration Methods
Many linear systems are too large to be solved by dirct methods based on Gauss elimination
or matrix inversion. For these systems, iteration methods are often the only possible method of solution,
as well as being faster than elimination in many cases. In this section we discuss two iterative methods.
viz. Jacobi iteration method and Gauss Seidel iteration method.
27
A general linear iterative method for the solution of the system of equations Ax = b may be
defined in the form
X
( k +1 )
=HX
( k)
+ c , k = 0, 1, 2, .....
.... (2.2.1)
where X ( k +1) and X ( k ) are the approximations for X at the (k + 1)th and k th iterations
respectively. H is called the iteration matrix and c is column vector. In the limiting case X ( k ) converges
to the exact solution
−1
X=A b
... (2.2.2)
When the system of equatons can be ordered so that each diagonal entry of the coefficient
matrix is larger in magnitude than the sum of the magnitudes of the other coefficients in that row - such
a system is called diagonally dominant. For such system the iteration will converge for any
starting values. Formally we say that an nxn matrix A is diagonally dominant if and only if for each
i = 1, 2, 3, ..., n
n
aii > ∑ aij
j =1
j≠i
i = 1, 2, 3, ..., n
For iterative methods we rearrange the system of equations so that the diagonal entries of the
coefficient matrix A become diagonally dominant. If not, we rearrange the system of equations in such
a way that the diagonal entries of matrix A are non-zero and possibly large in magnitude. Such a
rearrangement is called pivoting.
1.
Partial Pivoting
In the first state, the first column is searched for the largest element in magnitude and brought
as the first diagonal element by interchanging the first equation with the equation having the largest
element in magnitude. In the second stage, the second column is searched for the largest element in
magnitude among the (n – 1) elements leaving the first element, and this element is brought as the
second diagonal entry ( a22 ) by an interchange of the second equation with the equation having the
largest element in magnitude. This procedure is continued until we arrive at the last equation.
Example 1 : Consider the system of equations
2 x + 2 y + z + 2u = 7
x − 2y −u = 2
3x − y − 2 z − u = 3
x − 2u = 0
28
Ans. :
1
2 x 7
2 2
1 −2 0 −1 y 2
=
3 −1 −2 −1 z 3
1 0 0 −2 u 0
max.{ 2 , 1 , 3 , 1 } = 3 appears in third equation. Therefore interchange first and third
equation
3x − y − 2 z − u = 3
x − 2y −u = 2
2 x + 2 y + z + 2u = 7
x − 2u = 0
3 −1 −2
1 −2 0
2 2
1
1 0 0
−1 x 3
−1 y 2
=
−2 z 7
−2 u 0
Consider second column excluding ( a12 ) entry..
max { −2 , 2 , 0} = 2 we can keep second equation as it is, since the −2 also gives the maximum
value.
Consider third column excluding first two entries and calculate max { 1 , 0} = 1 . There is no
need to interchange third and fourth equation and partial pivoting is complete.
The rearrangement of system is generally carries out on the augmented matrix [ A, b].
Complete Pivoting :
In this procedure we search the matrix A for the largest element in magnitude and bring it as the
first pivot. This requires not only an interchange of equations but also an interchange of position of the
variables.
29
2.2.1
Jacobi Iteration Method
This method is an iteration method and is used to determine the solution of system of linear
equations. In the system Ax = b , we assume that the quantities aii are non-zero and sufficiently large.
This can be done by partial or complete pivoting. The system of equations (2.2.1) may be written as
a11x1 = − ( a12 x2 + a13 x3 + .... + a1 n xn ) + b1
a22 x2 = − ( a21x1 + a23 x3 + .... + a2 n xn ) + b2
.... (2.2.1.1)
M
ann xn = − ( an 1x1 + an 2 x2 + .... + an(n−1) xn−1 ) + bn
From equation (2.2.1.1) we have an iteration method
x1(k +1) = −
1
1
a12 x2(k ) + a13x3(k ) + .... + a1n xn(k ) ) +
b1
(
a11
a11
x2(k +1) = −
1
1
a21x1( k ) + a23 x3(k ) + .... + a2n xn(k ) ) +
b2
(
a22
a22
.... (2.2.1.2)
M
xn(k +1) = −
(
)
1
1
an1x1( k ) + an 2 x(2k ) + .... + an (n−1) x(nk−)1 +
bn
ann
ann
Initially we can assume that x1(0) = b1 , x2(0) = b2 ...xn(0) = bn .
Since we replace the complete vector x (k ) in the right side of (2.2.1.2) at the end of each
iteration, this method is called the method of simultaneous displacement.
In the matrix form equation (2.2.1.1) can be written as
Dx = − ( L + U ) x + b
where L and U are respectively lower and upper traingular matries with zero diagonal entries,
D is diagonal matrix such that A = L + D + U.
The matrix form of equation (2.2.1.1) is used to write an iteration method in the form,
x ( k +1) = − D−1 ( L +U ) x (k ) + D−1b , k = 0, 1, 2, 3, ...
x (0) = b is the initial approximation.
Alternatively equation (2.2.1.3) can be written as
x ( k +1) = x (k ) − x ( k ) − D−1 ( L +U ) x (k ) + D−1b
30
... (2.2.1.3)
= x (k ) − D −1 ( D + L +U ) x (k ) + D−1b
= x (k ) − D −1 [ Ax ( k ) − b ]
= x (k ) + D −1 [b − Ax (k ) ]
Define V ( k ) = D−1r ( k ) = x (k +1) − x ( k ) is the error in the approximation and r (k ) = b − Ax ( k )
is the residual vector.
We solve DV (k ) = r ( k ) for V (k ) and find
x ( k +1) = x (k ) + V
(k)
.... (2.2.1.4)
These equations describe the Jacobi iteration method in an error format. Thus to solve the
system of equations by Jacobi iteration method in an error form we have the following procedure
r (k ) = b − Ax ( k )
V
(k)
= D−1 r ( k )
x ( k +1) = x (k ) + V
2.2.2
.... (2.2.1.5)
(k)
Gauss Seidel Iteration Method
We know that every matrix A can be uniquely represented as the sum of lower and upper
traingular matrix with zero diagonal entries and a diagonal matrix. The system of equations Ax = b
can be represented by
( L + D + U ) x= b
∴ Dx = − ( L + U ) x + b
∴ Dx = − Lx −Ux + b
From above equation we have an iteration method
Dx ( k +1) = − Lx (k +1) − Ux (k ) + b , k = 0, 1, 2, 3, ....
... (2.2.2.1)
Initially, we assume that x1(0) = b1 .
Method (2.2.2.1) is called Gauss Seidel iteration method. In the explicit form equation (2.2.2.1)
can be written as
( L + D ) x (k +1) = −Ux (k ) + b
−1
−1
x ( k +1) = − ( L + D ) Ux (k ) + ( L + D ) b
31
−1
−1
= x (k ) − x (k ) − ( L + D ) Ux ( k ) + ( L + D ) b
= x (k ) − ( L + D )
−1
[ ( L + D +U ) x ( k ) ] + ( L + D ) −1 b
= x (k ) − ( L + D)
−1
[ Ax (k ) − b ]
−1
x ( k +1) = x (k ) + ( L + D )
[b − Ax (k ) ]
Thus to solve the system of equations by Gauss Seidel iteration method in an error form we
have the following procedure.
r (k ) = b − Ax ( k )
Solve ( L + D ) V ( k ) = r (k ) for V (k ) by forward substitution
x ( k +1) = x (k ) + V
(k)
,
k = 0, 1, ..., n
.... (2.2.2.2)
System (2.2.2.2) describe the Gauss Seidel method in an error form.
2.2.3
Convergence Analysis of Iterative Methods
Iterative methods are methods of successive approximations. Convergence of iterative methods
is studied through error analysis. To discuss the convergence of iterative method.
x (k +1) = Hx (k ) + c , k = 0, 1, 2,3, ....
... (2.2.3.1)
where x ( k +1) and x (k ) are the approximations for x at the (k + 1)th and k th iterations
respectively, we study the behaviour of the difference between exact solution x and an approximation
x (k ) .
The exact solution of iterative method will satisfy
x = Hx + c
.... (2.2.3.2)
Sibtracting (2.2.3.1) from (2.2.3.2) and substituting ε − (k ) = x (k +1) − x we get
ε (k +1) = Hε ( k ) , k = 0, 1, 2, ....
... (2.2.3.3)
Repeatative application of (2.2.3.3) for ε ( k ) , k = 1, 2, 3, .... gives
ε (k +1) = H k ε (0) , k = 0, 1, 2, 3, ....
For Jacobi iterative method H = −D −1 ( L + U ) and c = D−1b whereas for Gauss Seidel
iterative methods H = ( L + D ) −1 U and c = ( L + D ) −1 b . For both the methods iteration matrix H
remains constant for each iteration.
32
If the error sequence {ε (k +1) } converges to zero as k → ∞ , we say that the iterativ method
converges. To study the convergence of error sequence we use the following theorems.
Theorem 2.1 : Let A be a squar matrix. Then
lim A = 0 if A < 1 or iff 9 ( A) < 1 .
m→∞
m
Before proving this theorem we explain the notations and definitions used in the statement of
the theorem.
Definition : Matrix Norm
The matrix norm || A || is a non-negative number which satisfies the properties
(i)
A ≥ 0 if A ≠ 0 and O = 0 where O is zero matrix.
(ii)
cA = c A for arbitrary complex number c.
(iii)
A+ B ≤ A + B
(iv)
AB ≤ A B
The most commonly used norms are
(i)
Euclidean norm or frobenius norm
n
F ( A) = ∑ aij
ij =1
(ii)
1
22
n
where A = aij i , j =1
Maximum norm
n
A = A ∞ = max ∑ aik
i
k =1
(maximum absolute row sum)
n
A 1 = max ∑ aik
k
(iii)
i=1
(maximum absolute column sum)
Hilbert norm or Spectral norm
The largest eigen value in modulus of a matrix A is called the spectral radius of the matrix A and
is denoted by 9 ( A) . The spectral radius is defined only for square matrices.
A
2
T
= λ where λ = 9 ( A * A ) and A* = ( A) ,
A is the complex conjugate of A.
33
Proof of Theorem 2.1 :
If A < 1 then by definition of norm of matrix.
Am ≤ A
(Q AB ≤ A B )
m
and since norm is a continuous function,
lim Am = lim A m ≤ lim A
m→∞
m→∞
m
m→∞
=0
(Q A < 1)
For simplicity, we assume that all the eigen values of the matrix A are distinct. Then there exist
a similarity transformation S such that
A = S − 1DS
where D is diagonal matrix having eigen values of A on the diagonal. Therefore
Am = ( S −1DS )( S −1DS ) ...( S −1DS )( m times )
= S −1 D ( SS −1 ) D ... ( SS −1 ) DS
= S −1D mS
λ1m
m
where D = 0
K
0
0K0
0 0K0
K
K
m
0 0 Kλ n
0
0
λ2m
K
0
(
)
lim Am = S −1 lim D m S = 0 iff all the eigen values satisfy λi < 1 , i.e. 9 ( A) < 1 .
m→∞
m→∞
Theorem 2.2 : The infinite series + A + A2 + A3 + .... converges iff lim Am = 0 . The series converges
m→∞
to ( I − A )−1 .
Proof : From the definition of convergent series we have
lim A = 0 .
m
m→∞
m
A = 0 then by theorem 2.1 we have
Suppose mlim
→∞
9 ( A) < 1
34
Since the magnitude of largest eigen value of matrix A is strictly less than 1, I − A ≠ 0 and
therefore ( I − A )−1 exists.
We know the identity
( I + A + A2 + A3 + .... + Am ) ( I − A) = I − Am+1
∴ ( I + A + A2 + A3 + .... + Am ) = ( I − Am+1 ) ( I − A )
−1
m
A = 0 (by theorem 2.1), we have
Since mlim
→∞
I + A + A2 + A3 + .... + Am = ( I − A )
−1
Theorem 2.3 : No eigen value of matrix A exceeds the norm of a matrix A. i.e. A ≥ 9 ( A)
Proof : For eigen value λ of a matrix A we have
Ax = λ x
where x is a non-zero eigen vector corresponding to eigen value λ .
∴ λ x = λ x = Ax ≤ A x
( x ≠ 0)
Thus we have λ x ≤ A x
i.e.
λ ≤ A
i.e.
9 ( A) ≤ A
(where λ is any eigen value)
Theorem 2.4 : The iteration method of the form
x (k +1) = Hx (k ) + c
for the system of equation Ax = b converges to the exact solution for any initial vector x (0) if H < 1.
Proof : We take initial vector x (0) = 0 . Then the repeated application of iteration method gives
x (1) = c
x (2) = Hx (1) + c = Hc + c = ( H + I ) c
x (3) = Hx (2) + c = H ( H + I ) c + c = ( H 2 + H + I ) c
M
x ( k +1) = ( H k + H k −1 + .... + H 2 + H + I ) c
35
lim x (k +1) = lim ( H k + H k −1 + H k − 2 + .... + H 2 + H + I ) c
k →∞
k →∞
−1
= (I − H ) c
Thus
(if H < 1 )
−1
lim x (k +1) = ( I − H ) c
k →∞
In case of Jacobi iteration method we have
H = −D −1 ( L + D ) and c = D−1b
−1
∴ ( I − H ) c = [ I + D−1 ( L + U ) ] D−1b
−1
−1
= [ D−1 D + D−1 ( L + U )] D −1b
−1
= [ D−1 ( D + L + U )] D −1b
( D −1 )
−1
= ( D + L +U )
−1
D −1b
= A−1b
=x
Thus for Jacobi iteration method we have
−1
lim x (k +1) = ( I − H ) c = x
k →∞
In case of Gauss Seidel iteration method we have
−1
−1
H = − ( L + D ) U and c = ( L + D ) b
−1
−1
−1
−1
∴ ( I − H ) c = I + ( L + D ) U ( L + D ) b
−1
−1
−1
−1
= ( L + D ) ( L + D ) + ( L + D ) U ( L + D ) b
−1
= ( L + D ) ( L + D + U )
−1
−1
( L + D )−1 b
−1
−1
= A−1 ( L + D ) ( L + D ) b
= A−1b = x
Thus for Gauss Seidel iteration method we have
−1
lim x (k +1) = ( I − H ) c = A−1b = x
k →∞
36
(Q ( AB )−1 = B −1 A− 1 )
From theorem 2.3 we observe that Jacobi iterative method converges if
9 ( H J ) = 9 ( − D −1 ( L + U ) ) = 9 ( D −1 ( L + U ) ) < 1
and Gauss Seidel iterative method converges if
9 ( H G ) = 9 ( − ( L + D )−1 U ) = 9 ( ( L + D )− 1U ) < 1
Theorem 2.5 : A necessary and sufficient condition for convergence of an iterative method
x (k +1) = Hx (k ) + c is that the eigen values of the iteration matrix H satisfy λi ( H ) < 1 , i = 1, 2, ..., n
where λi ( H ) are eigen values of matrix H.
Proof : Suppose λ1 , λ2 , λ3 ,..., λn are the eigen values of matrix H and x1 , x2 , x3 , ..., xn be the
corresponding independent eigen vectors. ε (0) is an n-vector. We write
ε (0) = c1x1 + c2 x2 + .... + cn xn
∴ H k ε (0) = c1H k x1 +c2 H k x2 + .... + cn H k xn
= c1λ1kx 1 + c2λ2k x2 + .... + cnλnk xn
n
lim ε ( k ) = H k ε (0) = lim ∑ ci λi k xi
k →∞
k →∞
i=1
k
ε (k ) → 0 iff λi → 0 as k → ∞ .
i.e.
ε (k ) → 0 iff λi ( H ) < 1 .
Definition 2.2 : The rate of convergence of an iterative method is given by
v = − log10 [ 9 ( H )]
where 9 ( H ) is the spectral radius of matrix H.
Theorem 2.6 : If A is a strictly diagonally dominant matrix, then the Jacobi iteration scheme converges
for any initial starting vector.
Proof : The Jacobi iteration scheme is given by
x ( k +1) = − D−1 ( L +U ) x (k ) + D−1b
= − D−1 ( A −D ) x (k ) + D−1b
= ( I − D −1 A) x (k ) + D−1b
37
The iteration scheme will be convergent if I − D −1 A < 1 .
Using absolute row sum norm we have
1
I − D −1A = max 1 −
aii
n
j =1
j ≠i
∑ aij
n
Since A is strictly diagonal dominant,
∑ aij
j =1
j ≠i
< aii for all i = i, 1, 2, 3, ..., n and therefore
I − D −1 A < 1 and therefor the Jacobi iteration scheme converges for any initial starting vector..
Theorem 2.7 : If A is strictly diagonally dominant matrix, then the Gauss-Seidel iteration scheme
converges for any initial starting vector.
Proof : The Gauss Seidel iteration scheme is given by
−1
−1
x ( k +1) = − ( D + L ) Ux (k ) + ( D + L ) b
= −( D + L)
−1
[ A − ( D + L ) ] x (k ) + ( D + L ) −1 b
= I − ( D + L )
−1
−1
A x (k ) + ( D + L ) b
The iteration scheme will be convergent if
9 ( I − ( D + L ) −1 A ) < 1
Let λ be an eigen value of I − ( D + L ) −1 A .
−1
I − ( D + L ) A x = λ x
( D + L ) −1 ( D + L ) − ( D + L ) −1 A x = λ x
( D + L ) −1 [ D + L − A] x = λ x
(Q A = L +D +U )
∴−Ux = λ ( D + L ) x
i.e.
−
i.e.
−
n
∑
j =i +1
n
∑
j =i +1
i
aij x j = λ ∑ aij x j
1≤ i ≤ n
j =1
i −1
aij x j = λ aii xi + λ ∑ aij x j
j=i
38
i −1
i.e.
λ aii xi = −λ ∑ aij x j −
j =1
n
∑ aij x j
j =i +1
i−1
n
∑ aij x j +
λ aii xi ≤ λ
∑
j =i
j =i +1
aij x j
.... (2.2.3.1)
Since x is an eigen vector, x ≠ 0 . We assume that x = 1 .
Choose an index i such that xi = 1 and x j ≤ 1 , ∀j ≠ i .
From equation (2.2.3.4) we get
λ aii ≤ λ
i −1
n
∑ aij +
∑
j =i
i −1
∴ λ aii −∑ aij ≤
j =i
j =i +1
aij
n
∑
j =i +1
aij
n
∴λ ≤
∑
aij
j =i +1
i −1
aii − ∑ aij
j =i
n
Since matrix A is diagonally dominant aii > ∑ aij .
j=i
j≠i
i −1
∴ aii > ∑ aij +
or
n
∑
j =i
j = i+1
i −1
n
∴ aii − ∑ aij >
j =i
∑
j =i +1
aij
aij
n
∴
∑
aij
j = i+1
i −1
aii − ∑ aij
< 1 and therefore λ < 1 .
j =i
39
But λ is any eigen value of I − ( D + L ) −1 A .
∴9 ( I − ( D + L )−1 A) < 1
Note : The rate of convergence of Gauss Seidel scheme is twice that of the Jacobi scheme. It may
happen that for the system Ax = b , 9 ( H J ) < 1 but 9 ( H G ) > 1 . Similarly it is possible to have the
system of equation Ax = b for 9 ( H G ) < 1 which but 9 ( H J ) > 1 . For these systems matrix A is not
diagonally dominant.
2.3
Matrix Factorization Method
The system of equation Ax = b can be directly solved in the following cases.
Case (i) : A = D
The system of equations is
aii xi = bi ⇒ xi =
bi
aii
for i = 1, 2, 3, ..., n ; aii ≠ 0 ∀i
Case (ii) : A = L (Lower Traingular Matrix)
The system of equations is of the form
a11x1 = b1
a21x1 + a22 x2 = b2
a31x1 + a32 x2 + a33 x3 = b3
M
an1 x1 + a n2 x2 + an3 x3 + ....... + ann xn = bn
Solving first equation we get x1 . If we substitute this value of x 1 in second equation we get x2
and so on. Since unknowns are determined by forward substitution, the method is called forward
substitution method.
Case (iii) : A = U (Upper Traingular Matrix)
In this case system of equation is of the form
a11x1 + a12 x2 + a13 x3 + ....... + a1n xn = b1
a22 x2 + a23 x3 + ....... + a2n xn = b2
40
a33x3 + ....... + a3n xn = b3
M
a( n −1)(n −1) xn −1 + a(n −1)n xn = bn −1
ann xn = bn
From last equation we get xn from second last equation on substituting the value of xn we get
xn−1 and so on.
Since the unknown are determined from back substitution, this method is called the back
substitution method.
Thus the equation Ax = b is directly solvable if the matrix A can be transformed into one of
the three cases discussed above.
2.3.1
Triagulization Method
This method is also known as the decomposition method or factorization method. In this
method, the coefficient matrix A of the system of equations Ax = b is decomposed or factorized into
the product of a lower traingular matrix L and an upper traingular matrix U. We write the matrix A =
LU.
0
l11 0
l
21 l 22 0
where, L = l 31 l 32 l 33
K K K
l n1 l n2 l n3
K
K
0
u11 u12 u13
0 u
0
22 u23
K 0 and U = 0
0 u33
K K
K K K
0
K l nn
0
0
K u1n
K u2n
K u3n
K K
K unn
Then the system of equation Ax = b becomes
LUx = b
We write above equation as the following two systems of equations
Ux = z
.... (2.3.1)
Lz = b
From Case (ii) we determine z and the form Case (iii) we solve Ux = z to calculate x .
41
2.3.1 (a) Doolittle’s Method
In this method we write A = LU where the diagonal elements of matrix L are 1. We write
a11
a
21
a31
M
an1
a12
a22
a32
an2
a13 K a1n 1
0
0
a23 K a2n l 21 1
0
u33 K a3n = l 31 l 32 1
K K K
an3 K ann l n1 l n2 l n3
K
K
0 u11 u12 u13
0 0 u 22 u23
K 0 0
0 u33
K K K K K
K 1 0
0
0
K u1n
K u2n
K u3n
K K
K unn
n ( n + 1)
n ( n + 1)
− n+
= n 2 unknowns comparing left hand side
These are n2 equations in
2
2
with right hand side product. We get componentwise equations.
a11 = u11 , a12 = u12 , a13 = u13 , ...., a1n = u1n
a21 = l 21u11 , a22 = l 21u12 + u22 , ..., a2 n = l 21u1n + u2 n
-------------------------------------------------------By using forward substitution we calculate
u11 , u12 , u13 , ...., u1n , l 21 , u22 , u23 , ...., u2n , .........
Once the matrices L and U are known the solution is obtained by representing the system
Ax = b in the form of equation (2.3.1)
2.3.1 (b) Crout’s Method
In this method we write A = LU where the diagonal elements of matrix U are all 1. We write
a11
a
21
a31
M
an1
a12
a22
a32
an2
a13 K a1n l11
a23 K a2n l 21
a33 K a3n = l 31
K
an3 K ann l n1
0
l 22
0
0
l 32 l 33
K K
l n2 l n3
K
K
0 1 u12 u13
0 0 1 u23
K 0 0
0
1
K K K K K
K l nn 0
0
0
K u1n
K u2n
K u3n
K K
K 1
Again these are n2 equations in n2 unknowns. Equating the componentwise elements of l.h.s.
and r.h.s. we get
42
a11 = l11 , a12 = l11u12 , a13 = l 11u13 , ...., a1n = l11u1n
a21 = l 21 , a22 = l 21u12 + l 22 , a23 = l 21u13 + l 22u23 , ...., a2 n = l 21u 1n + l 22u 2 n
------- ------ ------ -----By using forward substitution we get,
l11 = a11 , u12 =
a12
a
a
, u13 = 13 , ...., u1n = 1n
l11
l11
l11
l 21 = a21 , l 22 = a22 − l 21u12 , ...., u2n = a2 n − l 21u1n
Once the matrices L and U are determined the system of equations Ax = b is represented in
the form of equation (2.3.1) and system of equations (2.3.1) is solved from Case (iii) and from
Case (i) respectively.
2.4
Eigen Values and Eigen Vectors
In section 2.2 and 2.3 iterative methods for linear system of equations are discussed. Consider
the system of equation
Ax = λ x
...... (2.4.1)
Equation (2.4.1) is called eigen value problem. The eigen values of A are given by the roots of
the characteristics equation
det ( A − λ I ) = 0
..... (2.4.2)
If A is square matrix of order n, equation (2.4.2) gives a polynomial equation of degree n. The
roots of this polynomial equation are called eigen values and may be determined by the methods given
in Unit 1. Once the roots λi of polynomial (2.4.2) are known then a non-zero vector xi such that
Axi = λi xi
.... (2.4.3)
is called the eigen vector or characteristic vector corresponding to λi . On multiplying equation (2.4.3)
by a constant c we get
Acxi = λi cxi ⇒ Ay = λi y
where y = cxi i.e. y is also a characteristic vector of A corresponding to eigen value λi . This shows
that an eigen vector is determined only to within an arbitrary multiplicative constant. On premultiplying
equation (2.4.1) (m – 1) times by A we obtain
Am x = λ Am x = λ Am−1 ( Ax ) = λ Am−1λ x = λ 2 Am−1 x ...
= λm x
43
Am x = λ m x
for m = 1, 2, 3, 4, .......
..... (2.4.4)
Equation (2.4.4) shows that λ m is an eigen value of Am if λ is an eigen value of A.
Since det ( AT − λ I ) = det ( A − λ I ) , A and AT have the same eigen values. If ui is an eigen
vector corresponding to the eigen value λ , then Aui = λi ui . Premultiplication by uiT gives
u iTAui = λi ui T ui and we get
λi =
u iTAui
u iTui
If ui is an eigen vector of a matrix A then λi is the eigen value of matrix A. Thus given eigen
value λ we find eigen vector by solving Ax = λ x and given eigen vector x we find the corresponding
u TAu
x T Ax
eigen vector as T . For arbitrary u , the ratio T
is called the Rayleigh quotient.
uu
x x
Let A and B be two square matrices of same order. If a non-singular matrix S can be determined
such that
B = S −1 AS ,
.... (2.4.5)
then the matrices A and B are said to be similar and the matrix S is called similarly matrix and
the transformation is called similarly transformation. From equation (2.4.5) we write
A = SBS −1
If λi is an eigen value of A and ui is the corresponding eigen vector then
Aui = λi ui
S − 1Aui = λ S −1ui
Put ui = Svi then S −1ASvi = λ S −1Svi = λvi .
i.e. Bvi = λ vi . But then eigen values of A and B are same and given eigen vectors ui of matrix
A, S −1ui are the eigen vectors of the matrix B. A similarity transformation, where S is the matrix of
eigen vectors reduces a matrix A to its diagonal form. The eigen values of A are the diagonal elements.
If eigen vectors of A are linearly independent then S–1 exists and the matrix A is said to be diagonalizable.
44
2.4.1
Bounds on Eigen Values
The bounds on the eigen values of the matrix A are given by the theorems by Gerschgorin and
Brauer.
Theorem (Gerschgorin)
The largest eigne value in modulus of a square matrix A cannot exceed the largest sum of the
moduli of the elements along any row or any column.
Proof : Let λi we an eigen values of A and xi be the corresponding eigen vector. Suppose
xiT = [ xi1, xi 2 ,..., xin ] . Since λi is an eigen value of A and xi is corresponding eigen vector,,
Axi = λi xi
a11
a
21
L
an1
a12
a22
L
an2
a13 L a1n xi1
a23 L a2n xi 2
= λi
L L L M
an3 L ann xin
xi1
x
i2
M
xin
xir . Select the kth equation and divide it by xik . The kth equation is
Let xik = max
r
ak 1xi1 + ak 2 xi 2 + ....... + akk xik + ..... + akn xin = λi xik
xi1
x
x
x
+ ak 2 i 2 + .... + a kk ik + .... + a kn in
xik
xik
xik
xik
Then
λi = ak1
Since
xik = max xir ,
r
and
λi ≤ a k1 + a k2 + ak 3 + ..... + a kk + ..... + akn
xir
≤ 1 , r = 1, 2, ...., n
xik
Thus if λ is an eigen value then
n
λ ≤ ∑ aki
for some k.
i =1
n
∴ λ ≤ max ∑ aki
k
i =1
Thus each eigen value and therefore the largest eigen value in modulus of a square matrix A
cannot exceed the largest sum of the moduli of the elements along any row.
45
Since A and AT have same eigen values, the theorem is also true for columns. (Repeat the
procedure for AT)
Theorem (Brauer) : Let Pk be the sum of the moduli of the elements along the kth row excluding the
diagonal element akk of a square matrix A. Every eigen value of A lies inside or on the boundary of at
least one of the circles λ − akk = Pk , k = 1, 2, 3, ...., n.
Proof : Let λi be an eigen value of A and xi be the corresponding eigenvector. Suppose
xiT = [ xi1, xi 2 ,..., xin ]
Then Axi = λi xi can be written as
a11xi1 + a12 xi2 + a13 xi 3 + ..... + a1n xin = λi xi1
a21xi1 + a22 xi2 + a23 xi3 + ..... + a2n xin = λi xi 2
M
an1 xi1 + an2 xi 2 + an3 xi 3 + ..... + ann xin = λi xin
xir . Select kth equation from above n equations. The kth equation is
Let xik = max
r
ak 1xi1 + ak 2 xi 2 + ak 3xi 3 + ..... + akk xik + ..... + akn xin = λi xik
Divide above equation by xik and rearrange the terms.
λi − akk = ak1
x
x
xi1
x
x
+ ak 2 i 2 + .... + ak ( k −1) i( k −1) + ak (k +1) i (k +1) + ..... + akn in
xik
xik
xik
xik
xik
x
xir , ir ≤ 1 , ∀1 ≤ r ≤ n .
Since xik = max
xik
r
and
λi − akk ≤ a k1 + ak 2 + ..... + ak ( k +1) + ak (k −1) + ..... + akn
Thus
λi − akk ≤ Pk
Therefore all the eigenvalues of A lie inside or on the union of the above circles.
Since A and AT have same eigenvalues, theorem holds for column sum also i.e.
λi − akk ≤ a1k + a2k + ..... + a(k −1)k + a(k +1)k + ..... + ank , k = 1, 2, 3, ...., n.
46
The bounds obtained for rows and columns are independent. Hence all the eigen values of A
must lie in the intersection of these bounds. These circles are called the Gerschgorin circles and the
bounds are called the Gerschgorin bounds.
Example 2.4.1 : Estimate the eigenvalues of the matrix
1 2 −1
A = 1 1 1
1 3 −1
using the Gerschgorin bounds and Brauer theorem.
Answer : By Gerschgorin theorem corresponding to row we have
λ ≤ max {1 + 2 + −1 , 1 + 1 + 1 , 1 + 3 + −1 }
≤ max {4,3,5} = 5
i.e.
λ ≤5
Similarly by considering column sum we get,
λ ≤ max {1 + 1 + 1 , 2 + 1 + 3 , − 1 + 1 + −1 }
≤ max {3,6,3} = 6
By Brauer’s theorem every eigenvalue of A lies inside or on the boundary of atleast one of the
circles λ − akk ≤ Ak .
Corresponding to rows we have
∴ λ − 1 ≤ 3 , λ −1 ≤ 2 , λ +1 ≤ 4
Corresponding to columns we have
λ −1 ≤ 2 , λ −1 ≤ 5 , λ +1 ≤ 2
The union corresponding to row sum gives
{ λ −1 ≤ 3 U λ − 1 ≤ 2 U λ + 1 ≤ 4} ={ λ −1 ≤ 3 U λ + 1 ≤ 4}
The union corresponding to column sum gives
{ λ −1 ≤ 2 U λ − 1 ≤ 5 U λ + 1 ≤ 2} ={ λ − 1 ≤ 5 U λ + 1 ≤ 2}
Thus the required region is given by
{ λ ≤ 5} I { λ ≤ 6} I { λ − 1 ≤ 3 U λ + 1 ≤ 4} I { λ − 1 ≤ 5, λ + 1 ≤ 2}
47
Example 2.4.2 : Estimate the eigen value region of the matrix
3 2 2
A = 2 5 2
2 2 3
By Gerschgorin theorem λ ≤ max {7,9,9} = 9
Corresponding to column we have λ ≤ max {7,9,7} = 9
Since matrix A is symmetric all eigenvalues are real and λ ≤ 9 gives the interval [ –9, 9]. Thus
all eigenvalues lie in the interval [ –9, 9].
By Brauer theorem the eigenvalues lie in the region
{ λ − 3 ≤ 4 U λ − 5 ≤ 4 U λ − 3 ≤ 4}
corresponding to row sum and corresponding to coulmn sum we have
{ λ − 3 ≤ 4 U λ − 5 ≤ 4 U λ − 3 ≤ 4}
Thus we have the region
{ λ − 3 ≤ 4 U λ − 5 ≤ 4 U λ − 3 ≤ 4}
= {−4 ≤ λ − 3 ≤ 4 U −4 ≤ λ − 5 ≤ 4}
{−1 ≤ λ ≤ 7 U 1 ≤ λ ≤9 } = [ −1,9]
Thus all roots lie in the interval [ −1,9] .
2.5
Jacobi Method for Symmetric Matrices
For real symmetric matrix all eigenvalues are real and there exist a real orthogonal matrix S
such that S −1AS is a diagonal matrix D. The diagonal entries of D are all eigenvalues of matrix A. The
diagonalization is achieved by applying series of orthogonal transformations.
Computational Procedure
{
}
Let aik = max aij : i ≠ j , i, j = 1,2,3,..., n . Consider 2 × 2 matrix formed by the
intersection of ith & k th row and ith & k th column. Then we get a matrix
a
A1 = ii
aik
aik
akk
( aik = aki since A is symmetric matrix)
48
cos θ
Choose S1* = sin θ
− sin θ
* −1
cos θ and find θ such that S1 A1S1 * is diagonal matrix.
cos θ
S1*−1 A1S1* =
− sin θ
sin θ aii
cos θ aik
aik cos θ
akk sin θ
2
2
aii cos θ + aik sin2θ + akk sin θ
=
( a − a ) sin2θ + a cos2θ
kk
ii
ik
2
− sin θ
cos θ
sin2θ
+ aik cos2θ
2
2
2
aii sin θ + akk cos θ − aik sin2θ
( akk − aii )
Now we choose θ such that the off diagonal entry of matrix S1*−1 A1S1* becomes zero so that
S1*−1 A1S1* becomes a diagonal matrix. Thus we choose θ such that
( akk − aii )
tan2θ =
sin2θ
+ aik cos2θ = 0
2
−2aik
2 aik
=
akk −a ii a ii −a kk
This equation produces four values of θ . We choose θ between −
θ=
=
1
2 aik
tan −1
2
aii − akk
π
4
=−
π
π
and
and we get,
4
4
if aii ≠ akk
if aii = akk , aik > 0
π
4
if aii = akk , aik < 0
With this choice of θ construct n × n orthogonal matrix S1 as follows. Write cosθ , − sinθ ,
sinθ , cosθ at ( i , i ) , ( i , k ) , ( k , i ) , ( k , k ) positions of matrix S1 respectively. Write remaining
diagonal entries to be 1 and rest of the offdiagonal entries 0. Thus we get a matrix S1 as
49
ith
jth
0
0
L
0
1
0
1
0
L
0
i th L cos θ 0 − sinθ L
S1 =
0
L
1
L
0
0
j th L sin θ L cos θ
L
L
L
L
L
0
0
L
L
1
Define B1 = S1−1 AS1
In B1 , ( i, k ) and ( k , i ) entries are zero. Repeat procedure for B1 . We get S2 . Define
B2 = S2−1B1S 2 = S 2−1S1−1 AS1S2
After making r transformations we get,
Br = S r −1Sr−−11....S 2−1S1−1 AS1 S2 S3....Sr
= ( S1S2S3 ....S r )
−1
A (S 1S 2S 3....S r )
= S −1 AS
where S = S1 S2 S3....Sr
As r → ∞ , Br approaches a diagonal matrix with the eigenvalues on the leading diagonal.
This procedure is called Jacobi method.
The convergence to a diagonal matrix takes place even if the maximum of offdiagonal elements
are not selected and we make any offdiagonal entry zero. This modification is called the special cyclic
Jacobi method. In this method there is no search for maximum offdiagonal entry.
2.6
Householder’s Method for Symmetric Matrices
In Jacobi method symmetric matrix is converted into a diagonal matrix through similarity
transformation. In this method a symmetric matrix A is reduced to the tridiagonal form by orthogonal
transformations. The orthogonal transformations are of the form
P = I − 2 wwT
.... (2.6.1)
where w is a column vector such that wT w = 1 .
50
Observe that P is symmetric and orthogonal.
PT = ( I − 2 wwT ) = I T − 2 ( wwT
T
PT P = ( I − 2wwT
)T
= I − 2wwT = P
)( I − 2wwT )
= I − 2wwT − 2wwT + 4 wwT wwT = I
(Q wT w = 1)
At the first transformation we find xi ' s such that we get zero in the position (1, 3), (1, 4),.....,
(1, n) and zero in the corresponding positions in the first column i.e. (3, 1), (4, 1), (5, 1) .... (n, 1). Thus
one transformation P2−1AP2 = A2 , bring (n – 2) zeros in the first row and first column. In the second
transformation P3−1A2 P3 , we get ( n – 3) zeros in the second column and second row namely (2, 4), (2,
5), (2, 6) .... (2, n) and (4, 2), (5, 2) ..... (n, 2) positions. The final matrix is tridiagonal. The
tridiagonalization is completed with exactly (n – 2) Householder transformation.
The matrix Pr is constructed as follows.
The vector wr is constructed with the first (r–1) components as zeros.
wr T = [ 0,0,0,....0, xr , xr +1,...., xn ]
Since wr T wr = 1 , xr 2 + xr +12 + xr +12 + .... + xn2 = 1
with this choice of wr , Pr = I − wr wr T .
Let us illustrate this procedure for 3 × 3 and 4 × 4 matrics.
a11
A = a12
a13
a12
a22
a23
a13
a23
a33
w2T = [ 0, w2 , w3 ] , w2T w2 = 1 ⇒ w22 + w32 = 1
P2 = I − w2 w2T
1 0 0
o
= 0 1 0 − 2 w2 ( 0, w2 , w3 )
0 0 1
w3
0− 0
0 −0
1− 0
= 0 − 0 1 − 2w22 −2 w2w3
0 − 0 −2w w 1 − 2 w 2
2 3
3
51
P2T P2 = I therefore P2 is orthogonal and P2T = P2−1 .
a11
AP2 = a12
a13
a
11
= a12
a13
∴ P2T AP2
a12
a22
a23
0
0
a13 1
a23 0 1 − 2 w22 −2w2 w3
a33 0 −2 w2w3 1 − 2 w32
a12 (1 − 2w2 2 ) − 2a13w 2 w3
a22 (1 − 2w2
) − 2a23w2 w3
a23 (1 − 2w2 2 ) − 2a33w2 w3
0
1
= 0 1 − 2w22
0 −2w w
2 3
2
−2 w2w3 a12 + a13 (1 − 2 w32 )
2
−2 w2w3 a22 + a23 (1 − 2 w3 )
−2w2w3 a23 + a33 (1 − 2 w32 )
−2 w2w3
1 − w32
0
a
a12 (1 − 2 w22 ) − 2a13w2w3
11
2
= a12 a22 (1 − 2 w2 ) − 2a23w2w3
a13 a23 (1 − 2 w2 2 ) − 2a33w2w3
−2w2w3 a12 + a13 (1 − 2 w32 )
2
−2w2w3 a22 + a23 (1 − 2w3 )
−2 w2 w3 a23 + a33 (1 − 2 w32 )
P2T AP2 is tridiagonal if (1, 3) entry of the matrix P2T AP2 is zero. But (1, 3) entry of
P2T AP2 is
−2 w2 w3 a12 + a13 (1 − 2w3 2 ) = 0
a13 − 2w3 (a 12 w2 + a13 w3 ) = 0
i.e.
a13 − 2w3 r = 0
where r = a12 w2 + a13 w3
'
(1, 2) entry of P2T AP2 denoted by a12
is
'
a12
= a12 (1 − 2w2 2 ) − a13w2w3
= a12 − 2w2 r
' 2
∴ a12
= ( a12 − 2 w2r ) + (a13 − 2w3 r )
2
2
[ a13 − 2w3r = 0]
= a12 2 + a132 + 4r 2 ( w2 2 + w3 2 ) − 4r (w 2 a12 + w3a13 )
= a122 + a132 + 4r2 − 4r2
52
' 2
a12
= ( a12 − 2w2r ) + (a13 − 2w3 r )
2
Thus
2
= ( a12 − 2 w2 r ) + 0
2
' 2
a12
= a122 + a132
'
a12
= ± a122 + a132 = a12 − 2w2r = ± S
Therefore,
(say)
Now we have two equations
a13 − 2rw3 = 0
... (i)
a12 − 2rw2 = ± S
... (ii)
Multiply equation (i) by w3 and (ii) by w2 and add.
Since a12w2 + a13w3 = r , we have
r − 2 r = ± Sw2 i.e. r = ±Sw2
Now from equation (ii) we have
a12 ± 2 Sw22 = ± S
Thus
w22 = ±
w22 =
±S − a12
2S
1
1 a12
1m
, w3 = m
2
S
2 2w
2
a13
a122 + a132
In computing w3 from w2 , we choose w2 as large as possible.
We demonstrate the reduction in the following example.
Example 2.6.1 : Reduce the matrix
1 3 4
A = 3 2 −1
4 −1 1
to tridiagonal form.
Answer : Here
a122 + a132 = 9 + 16 = ±5 .
w22 =
1 a12 1 3
1m
= 1+
( w2 is max. for S = 5 since a12 is +ve choose S +ve)
2
S 2 5
53
∴ w22 =
w3 =
4
5
i.e. w2 =
a13
2 w2 a122 + a132
=
2
5
4
1
=
2
5
2⋅
⋅5
5
2 1
∴ w2T = 0,
,
5 5
and
P2 = I − 2 w2 w2T
0
1 0
3
4
= 0 −
−
5
5
4 3
0 −
5 5
0
1 0
1 0
1 3 4
3
4
3
A2 = P2T AP2 = 0 −
− 3 2 −1 0 −
5
5
5
4 −1 1
4 3
4
0 −
0 −
5 5
5
0 1 −5
0
1 0
3
4
2
11
= 0 −
−
3 −
−
5
5
5
5
4 3
1
7
0 −
4 −
5 5
5
5
1 −5 0
2 1
A2 = −5 +
5 5
1 13
0
5
5
To illustrate the procedure for 4 × 4 matrix let
a11
a
A = 12
a13
a14
a12
a22
a13
a23
a23
a24
a33
a34
a14
a24
a34
a44
54
0
4
−
5
3
5
Since the transformations PrT APr are orthogonal, the sum of squares of the elements in any
row are invariant. We will use the fact that sum of squares of elements in any row of matrix A is same
as the sum of squares of elements in corresponding row of matrix PrT APr .
Choose w2T = [ 0, x2 , x3 , x4 ] and w2T w2 = x 22 + x 32 + x 42 = 1 .
At the first transformation we find x2 , x3 , x4 such that we get zero in the position (1, 3),
(1, 4) and (3, 1), (4, 1).
In the matrix P2 first row is a unit vector and therefore the position (1, 3), (1, 4) have zero
entry if the corresponding elements in AP2 are zero. The first row of AP2 is given by the following
product.
a11
a12
a13
a14
a11
a12
a13
a22
a23
a23
a33
a24
a34
0
0
0
a14 1
2
−2 x2 x3 −2 x2 x4
a24 0 1 − 2 x2
a34 0 −2 x2 x3 1 − 2x3 2 −2 x3 x4
2
a44 0 −2 x x
−2 x3 x4 1 − 2 x4
2 4
a12 (1 − 2 x2 2 ) − 2a13 x2 x3
− 2a14 x2 x4 − 2a12 x2 x3 + a13 (1 − 2 x3 2 ) − 2 a14 x3 x4
−2a12 x2 x4 − 2a13 x3 x4 + a14 (1 − 2 x42 )
= [ a11a12
− 2 p1x2 ,
a13 − 2 p1x3 ,
a14 − 2 p1x4 ]
where p1 = a12 x2 + a13 x3 + a14 x4
Now we need to find x2 , x3 , x4 such that a13 − 2 p1x3 = 0 and a14 − 2 p1x4 = 0 . So that
(1, 3) and (1, 4) position of P2 AP2 will become zero. Since the sum of the squares of the elements in
any row is invariant underthe orthogonal transformation we have,
a112 + a12 2 + a13 2 + a14 2 = a11 2 + (a12 − 2 p1x2 ) + ( a13 − 2 p1 x3 ) + (a 14 − 2 p1x4 )
2
2
= a11 2 + (a12 − 2 p1x2 ) + 0 + 0
2
∴ a12 − 2 p1x2 = ± a122 + a132 + a14 2 = ±S1 (say)
Thus we have three equations
a13 − 2 p1x3 = 0
.... (2.6.1)
55
2
a14 − 2 p1x4 = 0
.... (2.6.2)
a12 − 2 p1x2 = ± S1
... (2.6.3)
Since matrix A is given matrix, S1 is known quantity. Multiply (2.6.3) by x2 , (2.6.1) by x3
and (2.6.2) by x4 and add. We get,
p1 − 2 p1 ( x2 2 + x3 2 + x4 2 ) = ± S1 x2
But x22 + x32 + x42 = 1 and therefore
p1 = m S1x2
Now if we put this value of p1 in equation (2.6.3) then equation becomes a quadratic equation
in x2 and can be solved for x2 .
a12 − 2 ( m S1 x2 ) x2 = ± S1
x22 =
1 a12
1m
2
S1
.... (2.6.4)
From equation (2.6.1) and (2.6.2) we get
x3 =
a13
a13
a
a14
=
x4 = 14 =
and
2 p1 m2S1 x2
2 p1 m2 S1x2
.... (2.6.5)
From equation (2.6.4) we observe that x2 and therefore p1 posses two values. Since x3 and
x4 contains x2 in the denominator, we choose the large root by x2 . This is done by taking suitable
sign in equation (2.6.4).
x22 =
x3 =
1 a12 sign ( a12 )
1+
2
S1
a13 sign ( a12 )
a14 sign ( a12 )
, x4 =
2S1 x2
2 S1x2
where sign ( a12 ) is sign function which takes value – 1 if a12 < 0 and value 1 if a12 > 0 .
Thus the transformation P2 AP2 produces zero value in (1, 3), (1, 4) and therefore (3, 1),
(4, 1) positions. One more transformation discussed for 3 × 3 matrix produces zeros in (2, 4) and
(4, 2) positions. The resulting matrix will be a tridiagonal matrix.
56
Example 2.6.21 : Use the Householder’s method to reduce the give matrix A into the tridiagonal
form.
4 −1 −2 2
−1 4 −1 −2
A=
−2 −1 4 −1
2 −2 −1 4
Answer :
First iteration : Let w2 = [ 0, x2 , x3, x 4 ]
T
2
2
2
S1 = ( −1) + ( −2 ) + (2 ) = 3
x22 =
x3 =
1 ( −1) ( −1) 2
1+
= 3
2
3
( − 2 ) ( − 1)
2(3) 2
=
3
∴ x2 =
2
3
1
2 ( −1)
1
=−
6 , x4 =
6
2(3) 2
3
0
0
1
2
−2 x2 x3
0 1 − 2 x2
P2 =
2
0 −2 x2 x3 1 − 2x3
0 −2 x x
−2 x3 x4
2 4
0
1 0
0
1
2
0 − 3 − 3
−2 x2 x4
=
2 2
−2 x3 x4 0 −
3 3
1 − 2 x42
2
1
0
3
3
0
2
3
1
3
2
3
A2 = P2 AP2
0
1 0
0 − 1 − 2
3
3
=
2 2
0 −
3 3
2
1
0
3
3
0
0
1 0
2 4 −1 −2 2
1
2
0 −
−
3 −1 4 −1 −2
3
3
1
2 2
−2 −1 4 −1 0 −
3
3 3
2 −2 −1 4
2
2
1
0
3
3
3
57
0
2
3
1
3
2
3
0
1 0
0 − 1 − 2
3
3
=
2 2
0 −
3 3
2
1
0
3
3
0
2 4 3 0
3 −1 −2 −4
1
−2 −3 3
3
2 4 2
2
3
0
1
0
1
0
0
4 3
16 2
1
3
3
3
3
=
2 16
1
−
0
3
3
3
1
1 4
−
0
3
3 3
Second Iteration : P3 = [ 0,0, x3 , x4 ] ; x32 + x42 = 1
T
2
2
2
5 x 2 = 1 1 m 3 = 1 1 + 2
2 1
S1 = + =
; 3
2
5 2
5
3 3
3
3
x32 =
1 5+2
5+2
5−2
2
2
=
and x4 = 1 − x3 = 1 −
2
5
2 5
2 5
2
Suppose x32 = a then x4 =
1
0
P3 = 0
0
1
and we have
20a
1
0
0
1
0 1 − 2a −
5
1
1
0 −
1−
10a
5
0
0
0
A3 = P3 A2 P3
58
0
0 1
0 4 3
1 0
0 3 16 2
3
3
3
1
0 1 − 2a −
2 16
1 0
−
5 0
3
3
3
1
1
1
1 4 0
0 −
1−
−
10a 0
5
3
3 3
1
0
= 0
0
1
0
= 0
0
0
0
4 3
0
16
0 3
3
1
0 1 − 2a −
2
5 0
3
1
1
0 −
1−
1
10a 0
5
3
0
1
3
4
16
3
3
=
0 − 5
3 5
0
0
2.7
0
1
0
0
0
5
−
3 5
16
3
9
5
0
2 5 ( 1− 2 a ) −1
3 5
16 (
1
1 − 2a ) +
3
3 5
1
4
− (1 − 2a ) −
3
3 5
0
0
1
0 1 − 2a −
5
1
1
0 −
1−
10a
5
0
1
0
0
2
1
1
−
+ 1−
3 5 3 10 a
16 1
1
−
− 1 −
2 5 3 10a
1
4
1
+ 1−
3 5 3 10 a
0
0
0
9
5
12
5
Power Method
In section 2.2.3 we have seen that convergence of iterative method depends upon the spectral
radius of iteration matrix H. Spectral radius of a matrix is the largest eigen value in modulus. Therefore,
it is necessary to calculate the largest eigen value (in magnitude). Power method is normally used to
determine the largest eigen value (in magnitude) of the given matrix (spectral radius of a matrix).
Suppose we want to determine the largest eigen value of a square matrix A of order n. Let λ1 ,
λ2 , λ3 , ...., λn be the distinct eigen values of matrix A arranged in decreasing order (in magnitude).
λ1 > λ2 > λ3 > ..... > λn
.... (2.7.1)
and v1 , v2 , v3 , ..., vn be the corresponding linearly independent eigen vectors.
59
Avi = λi vi , i = 1, 2, 3, ....., n
..... (2.7.2)
Since eigen vectors are linearly independent, any eigen vector v in the eigen space, spanned
by the eigen vectors v1 , v2 , v3 , ..., vn , can be written as
v = c1v1 + c2v2 + ..... + cnvn
.... (2.7.3)
[Since v1 , v2 , v3 , ..., vn are linearly independent vectors, this set is a basis of eigen space]
Premultiplying equation (2.7.3) by A we get,
Av = A (c1v1 + c2v2 + ..... + cnvn )
= c1 Av 1 + c2 Av 2 + c3 Av3 + ..... + cn Avn
From equation (2.7.2) we have
Av = c1λ1 v1 + c2 λ2 v2 + c3λ3 v3 + ..... + cn λn vn
λ
λ
λ
= λ1 c1v1 + 2 c2v2 + 3 c3v3 + ..... + n cnvn
λ1
λ1
λ1
λ
λ
λ
A ( Av ) = A λ1 c1v1+ 2 c2v2 + 3 c3v3 + ..... + n cnvn
λ1
λ1
λ1
λ
λ
λ
= λ1 c1 Av1 + 2 c2 Av2 + 3 c3 Av3 + ..... + n cn Avn
λ1
λ1
λ1
λ
= λ1 c1λ1 v1 + 2
λ1
λ3
λn
c2 λ2 v2 + λ c3λ3 v3 + ..... + λ
1
1
cn λn vn
(Q Avi = λi vi )
2
2
2
λ
λ
λ
= λ12 c1v1 + 2 c2 v2 + 3 c3v3 + ..... + n cnvn
λ1
λ1
λ1
Repeatative premultiplication of A gives,
= λ1k
k
Av
A
k +1
v
k
k
k
λ2
λ3
λn
c1v1 + c2v2 + c3v3 + ..... + cnvn
λ1
λ1
λ1
λ
= λ1k +1 c1v1 + 2
λ1
k +1
λ
c2v2 + 3
λ1
60
k +1
k +1
λ
c3v3 + ..... + n
λ1
.... (2.7.4)
cnvn .... (2.7.5)
As k becomes very large, right hand side of equation (2.7.4) and (2.7.5) will be dominated by
λ1k c1v1 and λ1k +1c1v1 respectively..
k
λ
[Since λ1 > λi , i = 2, 3, 4, ... n, i → 0 as k → ∞ ]
λ1
Thus
Ak v ≅ λ1k c1v1
and
Ak +1v ≅ λ1k +1c1v1
Now the eigen value λ1 is obtained as the ratio of the corresponding components of Ak +1v
and Ak v .
λ1 ≅
λ1k +1c1 ( v1 )r
λ1k c 1( v1) r
and
( Ak +1v )r
k →∞ ( Ak v )
r
λ1 = Lt
, r = 1, 2, 3, ..., n
where the suffix r denotes the rth component of the vector.
The iteration is stoped when the magnitudes of the differences of the ratios are less than the
given error tolerance.
In order to keep roundoff error in control, we normalize the vector before premultiplying by A.
For computation purpose we follow the following procedure.
Let v0 be a non-zero arbitrary initial vector. (we choose v0 in such a way that v0T v1 ≠ 0 )
Define yk +1 = Avk
Suppose mk +1 is the largest element in magnitude of yk +1 ,
yk +1
Define vk +1 = m
k +1
Calculate
( yk +1 ) r
( vk ) r , r = 1, 2, 3, ..., n
If all the ratios are less than the given error tolerance
i.e. if
( yk +1 ) r ( yk +1 ) s
−
( vk )r ( vk ) s
<ε
∀r , s = 1,2,..., n
61
( yk +1 ) r
Then ( v )
k r
→ λ1 as k → ∞ .
The vector vk +1 is the required eigen vector..
The initial vector v0 is usually chosen as a vector with all components equal to unity if no
suitable approximation is available.
❖❖❖
62
UNIT - III
INTERPOLATION, DIFFERENTIATION AND
INTEGRATION
Interpolation is used to approximate given function by a polynomial or it is used to fit a polynomial
when the data is given in tabular form. There are two main uses of interpolation. The first use is the
reconstruction of function when it is not given explicity and second use is to replace a function f ( x ) by
an interpolating polynomial P( x) so that many common operations such as determination of roots,
differentiation, integration etc. which are required to perform on f ( x ) may be performed using P( x) .
In this unit we first discuss the methods of constructing the interpolating polynomial P( x) to a
given function. We determine the deviation of the given function f ( x ) from the approximating polynomial
P( x) by estimating truncation error bounds. We discuss numerical methods of differentiation and
integration of a given function f ( x ) .
3.1
Interpolation
Definition 3.1.1 : Interpolating Polynomial
A polynomial P( x) is called an interpolating polynomial if the value of P( x) and/or its certain
order derivatives coincide with those of f ( x ) and/or its derivatives at one or more tabular points.
3.1.1 Lagrange Interpolation
Linear Interpolation :
We want to determine a polynomial of degree one denoted by
P1 ( x ) = a1 x + a0
where a0 and a1 are arbitrary constants.
Satisfying p1 ( x0 ) = f ( x0 ) and p1 ( x1 ) = f ( x1 ) .
i.e. we want to interpolate a function by a polynomial of degree one.
Since
p1 ( x0 ) = f ( x0 ) , f ( x0 ) = a1 x0 + a0
and
p1 ( x1 ) = f ( x1 ) ⇒ f ( x1 ) = a1 x1 + a0
63
Above equations are two linear equations in two unknowns a0 and a1 . Simultaneous evaluation
of these two equations gives
a1 =
Thus
f ( x0 ) − f ( x1 )
x0 − x1
f ( x0 ) − f ( x1 )
and a0 = f ( x0 ) − x0
x0 − x1
f ( x0 ) − f ( x1 ) f ( x0 ) − f ( x1 )
p1 = f ( x0 ) − x0
+
x
x
−
x
x
−
x
0
1
0
1
∴ p1 ( x ) =
( x − x1 ) f x + ( x − x0 ) f x
( )
( )
( x0 − x1 ) 0 ( x1 − x0 ) 1
.... (3.1)
In other words we want to determine a polynomial p1 ( x ) = a1 x + a0 which satisfies
f ( x0 ) = a1 x0 + a0
f ( x1 ) = a1 x1 + a0
Above equations are three equations in two unknowns and therefore they have to be linearly
dependent.
p1 ( x ) x 1
f ( x0 ) x0 1 = 0
f ( x1 ) x1 1
∴ p1 ( x ) [ x0 − x1 ] − x f ( x0 ) − f ( x1 ) + 1 x1 f ( x0 ) − x0 f ( x1 ) = 0
∴ p1 ( x ) =
( x − x1 ) f x + ( x − x0 ) f x
( )
( )
( x0 − x1 ) 0 ( x1 − x0 ) 1
= l 0 ( x ) f ( x0 ) + l1 ( x ) f ( x1 )
where l 0 ( x ) =
..... (3.1.1.1)
x − x1
x − x0
and l1 ( x ) =
x0 − x1
x1 − x0
The functions l 0 ( x ) and l1 ( x ) are called Lagrange fundamental polynomials. These
polynomials satisfy
l 0 ( x ) + l1 ( x ) = 1 , l i ( x j ) = δ ij , i, j = 0, 1
In general, if we have (n + 1) distinct points a ≤ x0 < x1 < x2...... < xn ≤ b of [a, b] and a
value of the function f ( x ) is known at these points, we can determine the polynomial
64
pn ( x ) = a0 + a1 x + a2 x 2 + .... + an x n
which satisfies pn ( xi ) = f ( xi ) , i = 0, 1, 2, ....., n.
In other words we have,
pn ( x ) = a0 +a1x + a2 x 2 + a3 x3 + .... + an x n
.... (1)
f ( x0 ) = pn ( x0 ) = a0 + a1x0 + a2 x02 + a3 x03 + .... + an x0n
.... (2)
f ( x1 ) = pn ( x1 ) = a0 + a1x1 + a2 x12 + a3 x13 + .... + an x1n
.... (3)
M
M
f ( xn ) = pn ( xn ) = a0 + a1xn + a2 xn 2 + a3xn3 + .... + an xnn
... (n +2)
Equation (2) to ( n + 1) are n + 1 equations in (n + 1) unknowns a0 , a1 a 2 , ..., a n . Equations
(2) to (n + 2) has unique solution if the corresponding coefficient matrix is non-singular i.e.
1
1
1
M
1
x0
x1
x2
xn
x0 2 L x0n a0
f ( x0 )
x12 L x1n a1
f
x
(
)
1
x22 L x2n a2 =
M
M
L
f ( xn )
xn 2 L xn n an
..... (3.2.1.2)
has a unique solution if the Vandermonde’s determinant
1 x0
1 x1
v ( x0 , x1 , x2 ,..., xn ) = 1 x2
M
1 xn
x02
x12
x22
xn2
x03 L x0n
x13 L x1n
x23 L x2n ≠ 0
xn3 L xnn
If we determine a0 , a1 a 2 , ..., a n from system (3.2.1.2) then the system (1), (2), ... (n + 2)
will be linearly dependent and therefore its determinant should be zero.
65
i.e.
pn ( x ) 1
x
x2
f ( x0 ) 1
x0
f ( x1 ) 1
x0 2 L
x1
x1
x2
x22 L x2n
f ( xn ) 1 xn
xn 2 L xn n
f ( x2 ) 1
M
2
L
L
xn
x0n
n
x1
=0
..... (3.2.13)
To evaluate the l.h.s. of equation (3.2.1.3) we use the following property of the determinants.
Let
1 x0
1 x1
1 x2
v ( x0 , x1 , x2 ,..., xn−1x ) =
M
1 xn−1
1 x
x02
x03
L
x12
x13
L
x22
x23
L
xn−12
x2
x0n
x1n
x2n
xn−1n
x n
xn−13 L
x3
1 x0
1 x1
= ( x − x0 ) ( x − x1 )(x − x2 ) .... ( x − xn−1 ) 1 x2
M
1 xn−1
L
x0 2
x12
x2 2
L
L
L
L
xn−12 L
x0n−1
x1n−1
x2n−1
xn−1n−1
= ( x − x0 ) ( x − x1 )( x − x2 ) ....( x − xn ) ∨ ( x0 , x1 , x2 ,...., xn−1 )
To evaluate determinant in equation (3.2.1.3) we expand the determinant with respect to first
column. The evaluation of determinant in equation (3.2.1.3) gives
1
1
pn ( x ) 1
M
1
x0
x1
x2
xn
1 x
x02 L x0n
x12 L x1n
1 x1
2
n − f (x )
x2 L x2
x2
0 1
L
L L
1 xn
xn2 L xnn
66
x2
x12
L
L
x2 L
L L
2
xn 2 L
xn
x1n
x2n + ....
L
xnn
1
x
x0
1
n +1
... + ( −1) 1
x1
L L
1 xn−1
L
x2
x02
x12
L
L
L
L
xn−12 L
xn
x0n
x1n = 0
L
xn−1n
∴ pn ( x ) v ( x0 , x1 , x2 ,...., xn ) − f ( x0 )v ( x, x1 , x2 , x3,..., xn ) + f ( x1 )v ( x, x0 , x2 , x3 ,..., xn )
n +1
.... + ( −1) v ( x , x0, x1 , x2 ,...., xn−1 ) = 0
∴ pn ( x ) =
v ( x , x1 , x2 ,...., xn )
v ( x0 , x1 , x2 ,...., xn )
+
=
+
v ( x , x0 , x2 , x3 ,...., xn )
f ( x0 ) −
v ( x , x0, x1 , x3 , x4 ...., xn )
v ( x0 , x1 , x2 ,...., xn )
v ( x , x1 , x2 , x3 ,...., xn )
v ( x2 , x0 , x1 , x3,...., xn )
n
f ( x2 ) + ..... + ( −1)
f ( x0 ) +
v ( x0 , x1 , x2 ,...., xn )
v ( x , x0 , x1 , x3 ,...., xn )
v ( x0 , x1 , x2 ,...., xn )
f ( x1 )
v (x , x0, x1,...., xn−1 )
v ( x0 , x1 , x2 ,...., xn )
v ( x , x0 , x2 , x3 ,...., xn )
v ( x1, x0 , x 2 ,...., xn )
f ( x2 ) + .... +
f ( xn )
f ( x1 )
v ( x , x0, x1 ,...., xn−1 )
v ( xn , x 0, x1 , x2 ,...., xn−1 )
f ( xn ) ... (3.2.1.3)
[If we interchange any row of determinant, the value of determinant changes its sign]
From the elimentary properties of determinant observe that
v ( x, x1 , x2 , x3 ,...., xn ) = ( x − x1 )(x − x2 ) ...... ( x − xn ) v ( x1 , x2 ,..., xn )
and
Thus
v ( x0 , x1 , x2 , x3 ,...., xn ) = ( x0 − x1 )( x0 − x2 ) ......( x0 − xn ) v ( x1, x2 ,..., xn )
v ( x , x1 , x3,...., xn )
( x − x1 )( x − x2 ) ( x − x3 ) .... ( x − xn )
= l 0 ( x ) (say)
v ( x0 , x1 , x2 ,...., xn ) ( x0 − x1 )( x0 − x2 )( x0 − x3 ) ....( x0 − xn )
=
v ( x , x0 , x2 , x3 ,...., xn )
( x − x0 ) ( x − x2 ) ( x − x3 ) .... ( x − xn )
Similarly, v ( x , x , x , x ,...., x ) = ( x − x ) ( x − x ) ( x − x ) ....( x − x ) = l1 ( x ) (say)
1 0 2 3
n
1
0
1
2
1
3
1
n
In general,
v ( x , x0 , x1,....x i −1 , xi+1,...., xn )
( x − x0 ) ( x − x1 ) .... ( x − xi −1 )( x − xi +1 ) .... ( x − xn )
v ( xi , x0, x1 , x2 ,...., xi−1, xi +1,...., xn ) ( xi − x0 )(x i − x1 ) .....( xi − xi −1 )( xi − xi+1 ) ...... ( xi − xn )
=
= li ( x )
67
(say)
Now equation (3.2.1.3) becomes
pn ( x ) = l 0 ( x) f ( x0 ) + l i ( x ) f ( xi ) + .... + l n ( x ) f ( xn )
n
= ∑ l i ( x) f ( xi )
i=0
n
pn ( x ) = ∑ l i ( x) f ( xi ) where
i =0
∏ ( x − xk )
l i ( x) = k ≠ i
∏ ( xi − xk )
is called Lagrarge interpolating polynomial.
k ≠i
3.1.2
Newtons Interpolating Polynomial
In the last section we have seen that if we have (n + 1) distinct points
a ≤ x0 < x1 < x2..... < xn ≤ b and the value of a function f ( x ) at these points then we can fit a
polynomial pn ( x ) of degree n such that
pn ( xi ) = f ( xi ) , i = 0, 1, 2, ..., n.
Now represent this polynomial pn ( x ) in form
pn ( x ) = a0 + a1 ( x − x0 ) + a2 ( x − x0 ) ( x − x1 ) + .... + an ( x − x0 ) ( x − x1 ) .... ( x − xn−1 )
We want to calculate a0 , a1 , a 2 , ..., a n in such a way that
pn ( xi ) = f ( xi ) , i = 0, 1, 2, ..., n.
f ( x0 ) = pn ( x0 ) = a0
f ( x1 ) = pn ( x1 ) = a0 + a1 ( x1 − x0 )
f ( x2 ) = pn ( x2 ) = a0 + a1 ( x2 − x0 ) + a2 ( x2 − x0 ) ( x2 − x1 )
M
f ( xn ) = pn ( xn ) = a0 + a1 ( xn − x0 ) + a2 ( xn − x0 )( xn − x1 ) + ....
+ an ( xn − x0 )( xn − x1 ) .... ( xn − xn−1 )
68
By forward substitution we get, a0 , a1 , a 2 , ..., a n as follows,
a0 = f ( x0 )
∴ a2 =
a1 =
f ( x1 ) − a0 f ( x1 ) − f ( x0 )
=
= f [ x0 , x1] (say)
x1 − x0
x1 − x0
a2 =
f ( x2 )
f ( x0 )
( x2 − x0 ) f ( x1 ) f ( x0 )
−
−
−
( x2 − x0 ) ( x2 − x1 ) ( x2 − x0 ) ( x2 − x1 ) ( x2 − x0 ) ( x2 − x1 ) x1 − x0 x1 − x 0
f ( x2 )
f ( x1 )
−1
1
−
+ f ( x0 )
+
( x2 − x0 ) ( x2 − x1 ) ( x1 − x0 ) ( x2 − x1 )
( x2 − x0 ) ( x2 − x1 ) ( x2 − x1 ) ( x1 − x 0 )
=
f ( x0 )
f ( x2 )
f ( x1 )
+
+
( x2 − x0 ) ( x2 − x1 ) ( x1 − x0 ) ( x1 − x2 ) ( x0 − x1 )( x0 − x2 )
= f [ x0 , x1 , x 2 ] (say)
In general by induction we can prove that
ai =
f ( x0 )
( x0 − x1 )( x0 − x2 ) .... ( x0 − xi )
+
+
f ( x1 )
+ ....
( x1 − x0 ) ( x1 − x2 ) .... ( x1 − xi )
f ( xi )
( xi − x0 )( xi − x1 ) .... ( xi − xi −1 )
= f [ x0 , x1,...., xi ]
(say)
Thus we have
pn ( x ) = f ( x0 ) + f [ x0 , x1 ] (x − x0 ) + f [ x0 , x1 , x2 ] ( x − x0 ) ( x − x1 ) + .....
... + f [ x0 , x1 , x2 ,..., xn ] ( x − x0 ) ( x − x1 ) ..... ( x − xn−1 )
The polynomial pn ( x ) is called Newton’s divided difference interpolating polynomial.
The coefficients a1 = f [ x0 , x1 ] , a2 = f [ x0 , x1, x2 ] , ......, an = f [ x0 , x1 , x2 ,...., xn ] are
called Newton’s divided differences.
69
3.1.2.1
(i)
Properties of Newton’s Divided Differences
f [ x0 , x1] =
f ( x1 ) − f ( x0 )
=
x1 − x0
f ( x0 ) − f ( x1 )
x0 − x1
= f [ x1 , x0 ]
f [ x0 , x1 ] = f [ x1, x0 ] (symmetry)
(ii)
(iii)
f [ x0 , x1] =
f ( x1 ) − f ( x0 )
x1 − x0
=
f ( x1 ) f ( x0 )
−
x1 − x0 x1 − x0
=
f ( x0 )
f ( x1 )
+
x1 − x0 ( x0 − x1 )
f [ x0 , x1, x2 ] =
f ( x0 )
( x0 − x1 )( x0 − x2 )
+
f ( x1 )
f ( x2 )
+
( x1 − x0 ) ( x1 − x2 ) ( x2 − x0 ) ( x2 − x1 )
=
1
( x2 − x 0 ) f ( x1 ) +
f ( x) +
( x2 − x0 )
( x1 − x0 ) ( x1 − x2 )
=
f ( x0 )
f ( x2 )
1
1
1
+ f ( x1 )
−
+
−
( x2 − x0 ) x0 − x1
x1 − x 2 x1 − x0 ( x2 − x1 )
=
f ( x1 ) − f ( x0 ) f ( x2 ) − f ( x1 )
1
−
+
x1 − x0
x2 − x1
( x2 − x0 )
=
1
− f [ x0 , x1 ] + f [ x1, x2 ]
( x2 − x0 )
=
f [ x1, x2 ] − f [ x0 , x1]
( x2 − x0 )
In general,
f [ x0 , x1 , x2 ,..., xk −1, xk ] =
f [ x1, x2 ,..., x k ] − f [ x0 , x1 , x2 ,..., xk −1 ]
xk − x0
Thus Newton’s divided differences are calculated as follows,
f [ x0 , x1] =
f ( x1 ) − f ( x0 )
x1 − x0
70
, k = 3, 4, ...., n
f [ x0 , x1 , x2 ] =
f [ x1, x2 ] − f [ x0 , x1 ]
f [ x0 , x1 , x2 , x3 ] =
x2 − x0
f [ x1, x2 , x3 ] − f [ x0 , x1 , x 2 ]
x3 − x0
.........
The Newton’s divided differences may be calculaed with the help of following table.
x0
x1
x2
x3
x4
(iv)
f ( x0 )
⇒
f [ x0 , x1]
⇒
f [ x1, x2 ]
⇒
f [ x2 , x3 ]
⇒
f [ x3 , x4 ]
f ( x1 )
f ( x2 )
f ( x3 )
f ( x4 )
⇒ f [ x0 , x1, x2 ]
⇒ f [ x1 , x2 , x3 ]
⇒ f [ x2 , x3 , x4 ]
⇒ f [ x0 , x1 , x2 , x3 ]
⇒f [ x0 , x1 , x2 , x3, x4 ]
⇒ f [ x1 , x2 , x3 , x4 ]
Newtons divided differences are symmjetries in all the variables
f [ x0 , x1 ] = f [ x1, x0 ]
f [ x0 , x1, x2 ] = f [ x1, x0, x2 ] = f [ x2 , x1 , x0 ]
etc.
f [ x0 , x1 , x2 , x3 ] = f [ x1, x0 , x2 , x3 ] = f [ x0 , x1 , x3, x2 ] etc.
3.1.3 Uniqueness of Interpolating Polynomials
In section 3.1.1 we have seen how to calculate Lagrange’s interpolating polynomial pn ( x ) if
the values of function f ( x ) are known at (n + 1) nodes x0 , x1 , x2 , ...., xn . In section 3.1.2 we have
studied the evaluation of Newton’s divided interpolating polynomial if the values of function f ( x ) are
known at (n + 1) points x0 , x1 , x2 , ...., xn . We have seen that both the polynomials are polynomials
of degree n. In this section we prove that the polynomials obtained by two different methods are same.
71
Suppose p ( x) is an interpolating polynomial of the given function f ( x ) satisfying
f ( xi ) = p ( xi ) , i = 0, 1, 2, ...., n and p * ( x ) is another interpolating polynomial of same degree n
satisfying f ( xi ) = p * ( xi ) , i = 0, 1, 2, ...., n. We show that p ( x ) = p * ( x ) .
Define Q ( x ) = p ( x ) − p * ( x )
Since p ( x ) and p * ( x ) are polynomials of degree n. Q ( x ) is a polynomial of degree at
most n.
Q ( xi ) = p ( xi ) − p * ( xi ) = f ( xi ) − f ( xi ) = 0 , i = 0, 1, 2, ...., n
Thus Q ( x ) is a polynomial of degree less than or equal to n has (n + 1) distinct roots x0 , x1 ,
x2 , ...., xn . But a polynomial of degree n has exactly n roots (real or complex). Therefore Q ( x ) has
at the most n roots. But Q ( x ) has (n + 1) distinct roots x0 , x1 , x2 , ...., xn . This implies that
Q ( x) ≡ 0 .
Therefore Q ( x ) = p ( x ) − p * ( x ) ≡ 0 i.e. p ( x ) = p * ( x )
Thus, the interpolating polynomials obtained in two different ways may be different in form, but
are identical.
3.1.4 Truncation Error Bounds
In the last section we have seen that interpolating polynomial of degree n is unique Linear
interpolation gives a polynomial p1 ( x ) of degree one. The polynomial p1 ( x ) coincides with the
function f ( x ) at x0 and x1 . It deviates from f ( x ) at all other points in the interval ( x0 , x1 ) . This
deviation is called the truncation error and is written as
E1 ( f ; x ) = f ( x ) − p1 ( x )
The expression for E1 ( f ; x ) is derived by using Rolle’s theorem.
Rolle’s Theorem : If g ( x ) is a continuous function on some interval [ a, b] and differentiable on
(a, b) and if g ( a ) = g ( b ) = 0 , then there is at least one point ξ inside (a, b) for which g ' ( ξ ) = 0 .
72
Error Bound for Linear Interpolation
Suppose p1 ( x ) is a linear interpolating polynomial for the function f ( x ) . The polynomial
p1 ( x ) coincides with f ( x ) at x0 and x1 . Define a function g ( t ) as
g ( t ) = f ( t ) − p1 ( t ) − f ( x ) − p1 ( x )
( t − x0 ) ( t − x1 )
( x − x0 ) ( x − x1 )
where x ∈ ( x0 , x1 ) is a fixed point. Now function g (t ) is continuous and differentiable.
g ( x0 ) = f ( x0 ) − p1 ( x0 ) = 0 and g ( x1 ) = f ( x1 ) − p1 ( x1 ) = 0
at t = x , g ( x ) = f ( x ) − p1 ( x1 ) − f ( x ) − p1 ( x ) = 0
Thus in the interval ( x0 , x ) , g ( x0 ) = g ( x ) = 0 and by Rolle’s theorem ∃ at least one point
ξ , inside ( x0 , x ) for which g ' ( ξ1 ) = 0 .
Similarly in the interval ( x , x1 ) , g ( x ) = g ( x1 ) = 0 and by Rolle’s theorem ∃ at least one
point ξ 2 inside ( x , x1 ) for which g ' ( ξ2 ) = 0 .
Thus x0 < ξ1 < x < ξ 2 < x1 and g ' ( ξ1 ) = g ' ( ξ2 ) = 0 .
Now on applying Rolle’s theorem for g ' ( t ) on the interval ( ξ1 , ξ 2 ) we get ξ ∈ ( ξ1 , ξ 2 ) such
that g " (ξ ) = 0 . Since x0 < ξ1 < x < ξ 2 < x1 , ξ ∈ ( x0 , x1 ) .
Now differentiating g (t ) twice with respect to t we obtain,
g ' ( t ) = f '( t ) − p1 ' ( t ) − f ( x ) − p1 ( x )
( t − x1 ) + ( t − x0 )
( x − x0 ) ( x − x1 )
g " ( t ) = f "( t) − p1 "( t ) − f ( x ) − p1 ( x )
2
( x − x0 ) ( x − x1 )
2 f ( x ) − p1 ( x )
= f "( t ) −
( x − x0 ) ( x − x1 )
[Q p1 ( t ) is polynomial of degree 1 p1 " ( t ) = 0 ]
2 f ( x ) − p1 ( x )
g " (ξ ) = 0 ⇒ f " ( ξ ) =
( x − x0 ) ( x − x1 )
73
Thus
f "( ξ )
f ( x ) − p1 ( x ) = ( x − x0 ) ( x − x1 )
2
Therefore, the truncation error in linear interpolation is given by,
E1 ( f ; x ) = f ( x ) − p1 ( x ) =
If f " ( x ) ≤ M 2
1
( x − x0 ) ( x − x1 ) f "( ξ )
2
∀x ∈[ x0 , x1 ] then,
f ( x ) − p1 ( x ) =
≤
1
( x − x0 ) ( x − x1 ) f "( ξ )
2
1
Max ( x − x0 ) ( x − x1 ) M 2
2 x0 ≤ x≤ x1
Let w ( x ) = ( x − x0 ) ( x − x1 ) then w ' ( x ) = ( x − x0 )+ ( x − x1 )
and w ' ( x ) = 0 ⇒ x =
x=
x0 + x1
. Hence maximum value of
2
( x − x0 ) ( x − x1 )
is attained at
x0 + x1
.
2
Max ( x − x0 ) ( x − x1 )
x
x +x
x + x
(x − x )
= 0 1 − x0 0 1 − x1 = 1 0
2
2
4
2
1 ( x1 − x0 )
f ( x ) − p1 ( x ) ≤
M2
2
4
2
Thus
1 ( x1 − x0 )
Thus bound for truncation error E1 ( f ; x ) is
M2 .
2
4
2
Thus the truncation error
1 ( x1 − x0 )
E1 ( f ; x ) ≤
2
4
2
Max f " ( x )
x∈(x 0 ,x 1 )
74
..... (3.1.4.1)
Truncation Error for higher order interpolating polynomial
Interpolating polynomial pn ( x ) of degree n coincides with (n + 1) times differentiable function
f ( x ) at { x0 , x1 , x2 ,...., xn } (n + 1) points. The truncation error for this polynomial approximation is
E n ( f ; x ) = f ( x ) − pn ( x )
For x ∈ ( a, b ) and x ≠ xi , i = 0, 1, 2, ....m define
g ( t ) = f ( t ) − pn ( t ) − f ( x ) − pn ( x )
( t − x0 ) ( t − x1 ) ..... ( t − xn )
( x − x0 ) ( x − x1 ) ..... ( x − xn )
.... (3.1.4.2)
Observe that g ( xi ) = 0 , ∀ i = 0, 1, 2, ..., n.
In the interval
( x0 , x1 ) , g ( x0 ) = g ( xi ) = 0 . Function
g is continuous and differentiable.
Therefore by Rolle’s theorem ∃ξ1 ∈ ( x0 , x1 ) such that g ' ( ξ1 ) = 0 .
Similarly in the interval ( x1 , x2 ) ∃ξ2 such that g ' ( ξ2 ) = 0 . In general ∃ξi ∈ ( xi −1, xi ) such
that g ' ( ξi ) = 0 .
Now g ' ( ξ1 ) = g ' ( ξ2 ) = 0 . g ' is continuous and differentiable therefore by Rolle’s theorem
∃η1 ∈ (ξ1 , ξ2 ) such that g " (η1 ) = 0 .
∃η2 ∈ (ξ 2 , ξ3 ) such that g " (η 2 ) = 0 .
In general ∃ηi (ξi ,ξi +1 ) such that g " (ηi ) = 0 , i = 1, 2, ...., (n – 1).
Thus repeated application of Rolle’s theorem for g (t ) , g ' ( t ) , g " ( t ) , ..., g (n ) (t ) gives
ξ ∈ ( a, b ) such that g (n+1) (ξ ) = 0 .
Differentiating function g defined in equation (3.1.4.2), ( n + 1) times with respect to t, we get,
g (n+1) ( t ) = f ( n+1) ( t ) − pn ( n+1) ( t ) −
( n + 1) ! f ( x ) − pn ( x )
( x − x0 ) ( x − x1 ) ..... ( x − xn )
Since pn is a polynomial of degree (n) pn (n+1) ( t ) = 0 .
g (n+1) (ξ ) = 0 ⇒ f ( n+1) ( ξ ) −
( n + 1) ! f ( x ) − pn ( x )
( x − x0 ) ( x − x1 ) ..... ( x − xn )
75
=0
and
f ( x ) − pn ( x ) =
( x − x0 ) ( x − x1 ) ..... ( x − xn )
( n + 1) !
f
(n +1)
(ξ )
Thus the truncation error in intempolating polynomial is given by
E n ( f ; x ) = f ( x ) − pn ( x ) =
3.2
( x − x0 ) ( x − x1 ) .....( x − xn )
( n + 1)!
f (n+1) ( ξ )
Finite Difference Operators
Let the tabular points x0 , x1 , x2 ,...., xn be equally spaced i.e. x1 − x0 = h , x2 − x1 = h ,....,
xn − xn−1 = h .
In general xi = x0 + ih , i = 1, 2, 3, ...., n.
Define the following operators.
(i)
Forward differene operator ∆f ( xi ) = f ( xi +1 ) − f ( xi ) .
(ii)
Backward difference operator ∇f ( xi ) = f ( xi ) − f ( xi −1 ) .
(iii)
h
Central difference operator δ f ( xi ) = f xi + −
2
(iv)
Average operator µ f ( xi ) =
(v)
Shift operator Ef ( xi ) = f ( xi + h ) .
h
f xi − .
2
1
h
h
f xi − + f xi + .
2
2
2
3.2.1 Relations between finite difference operators
(i)
∆f ( xi ) = f ( xi +1 ) − f ( xi ) = ∇f ( xi+1 ) ⇒ ∆fi = ∇f i+1
where f ( xi ) is denoted by f i .
(ii)
h h
h h
∆f ( xi ) = f ( xi + h ) − f ( xi ) = f xi + + − f xi + −
2 2
2 2
h
= δ f xi +
2
1
∴∆f ( x0 + ih ) = δ f x0 + i + h ⇒ ∆fi = δ f 1
i+
2
2
76
(iii)
∆f ( xi ) = f ( xi +1 ) − f ( xi ) ⇒ f ( xi + h ) − f ( xi ) = Ef ( xi ) − f ( xi )
= ( E − 1) f ( xi )
Thus ∆f ( xi ) = (E − 1) f ( xi ) ⇒ ∆ ≡ E − 1 .
(iv)
∇f ( xi ) = f ( xi ) − f ( xi −1 )
Since Ef ( xi ) = f ( xi +1 ) ⇒ E − 1 f ( xi+1 ) = f ( xi )
Thus ∇f ( xi ) = f ( xi ) − E −1 f ( xi )
= (1 − E − 1 ) f ( xi )
Therefore ∇ ≡ 1 − E −1
(v)
E1 f ( xi ) = f ( xi + h )
∴E
1
2
1
−1
1
f ( xi ) = f xi + h and E 2 f ( xi ) = f xi − h
2
2
h
h
But f xi + − f xi − = δ f ( xi )
2
2
∴E
(vi)
1
2
f ( xi ) − E
2
f ( xi ) = δ f ( xi ) ⇒ E
µ f ( xi ) =
1
h
h
f xi + + f xi −
2
2
2
=
1 12
−1
E f ( xi ) + E 2 f ( xi )
2
⇒µ=
(vii)
−1
(
1 12
−1
E +E 2
2
1
2
−E
−1
2
=δ
)
E Ef ( xi ) = Ef ( xi + h ) = f ( xi + h + h ) = f ( xi + 2h )
In general E n f ( xi ) = f ( xi + nh )
∆ ∆f ( xi ) = ∆ f ( xi + h ) − f ( xi )
= ∆f ( xi + h ) − ∆f ( xi )
= f ( xi + h + h ) − f ( xi + h ) − f ( xi + h ) − f ( xi )
77
= f ( xi + 2h ) − 2 f ( xi + h ) + f ( xi )
∆f ( xi ) = f ( xi + h ) − f ( xi )
= ( E − 1) f ( xi )
n
n
k n
⇒ ∆ n f ( xi ) = ( E − 1) f ( xi ) = ∑ ( −1) f ( xi +n −k )
k
k =0
n
n
k n
∆ n f ( xi ) = (1 − E −1 ) f ( xi ) = ∑ ( −1) f ( xi −k )
k
k =0
(
Similarly δ n f ( xi ) = E
1
2
−E
− 12
)
n
f ( xi )
(
1
k n
= ∑ ( −1) E 2
k
) ( E − 12 )
n− k
k
f ( xi )
n− k
k
−
k n
= ∑ ( −1) E 2 E 2 f ( xi )
k
n− k
k n
= ∑ ( −1) E 2
k
k n
= ∑ ( −1)
k
=
k
n−k
f xi − h +
h
2
2
k n
∑ ( −1) k f
k=0
n
k
f xi − h
2
n
n ( )k n
x
+
−
k
h = ∑ −1 f i + n − k
i
2
k=0
k 2
3.2.2 Relations Between Differences and Derivative
In section 3.2.1 we have derived relations between different finite difference operators. If the
relation between derivative and any finite difference operator is obtained then the relation between
derivative and any finite difference operator is known.
Suppose f is smooth function
∆f ( xi ) = f ( xi + h ) − f ( xi )
= f ( xi ) + hf ' ( xi ) +
h2
f "( xi + θ h ) − f ( xi )
2!
78
h2
= hf ' ( xi ) +
f " ( xi + θ h )
2!
Therefore,
Thus
f ' ( xi ) =
1
h
∆f ( xi ) − f "( xi + θ h )
h
2
=
1
∆f ( xi ) + 0( h)
h
Df ( xi ) =
∆
f ( xi ) + 0( h)
h
Since finite difference operators are defined for discrete equidistant points and derivative is
defined for containuous functions, exact relation between derivative and differences cannot be obtained
and we get an error.
3.3
Numerical Differenciation
In this section we discuss methods for approximating the derivatives of a given function.
Numerical differentiation methods are developed by using one of the following techniques.
(i)
Methods based on interpolation
In this method the function values or table values are used to approximate a function by a
polynomial (by using Lagranges interpolation or Newton’s divided difference formula) and this polynomial
is differentiated to get the derivative of a function.
(ii)
Methods Based on Finite Difference Operators
In this method derivative is obtained by considerng Newton’s forward difference operator or
Newton’s backward difference operator or using shift operator. Since there is relation between every
pair of finite difference operators, derivatives are obtained by using any finite difference operator.
(ii)
Methods Based on Undetermined Coefficients
In this method function is written in the form of linear combination of values of a function at
some points and the coefficients of this linear combination are obtained by using Taylor series expansion.
Now we discuss each of this methods in detail.
79
Methods Based on Interpolation :
Given the values of function f ( x ) at a set of points x0 , x1 , x2 ,...., xn , obtain an interpolation
polynomial pn ( x ) by using any interpolation technique (Lagrenge interpolation or Newton’s divided
difference formula). Thus
f ( x ) = pn ( x ) + E n ( f ; x )
where En ( f ; x ) is an error in the approximation.
In section 3.1.4 we have seen that
En ( f ; x ) =
( x − x0 ) ( x − x1 ) ..... ( x − xn )
( n + 1)!
f (n+1) ( ξ )
f ' ( x ) = p n ' ( x ) + En ' ( f ; x )
In general the values of pn( r ) ( xk ) gives the approximate values at the point xk .
The quantityEn(r ) ( f ; x ) = f (r ) ( x ) − pn (r ) ( x ) is called the error of approximation in the rth
order derivative at any point x.
Example : The following data for the function f ( x ) = x4 is given.
x
f ( x)
0.2
0.3
0.4
0.0016
0.0081
0.0256
Find f ' ( 0.4 ) and f "( 0.4 ) using quadratic interpolation. Compare the results with exact
solution obtain the bound on the truncation error.
Answer : Using Lagrange interpolation we have
p2 ( x ) =
=
( x − x1 )( x − x2 ) f x + ( x − x0 ) ( x − x2 ) f x + ( x − x0 ) ( x − x1 ) f x
( )
( )
( )
( x0 − x1 )( x0 − x2 ) 0 ( x1 − x0 ) ( x1 − x2 ) 1 ( x2 − x0 ) ( x2 − x1 ) 2
( x − 0.3) ( x − 0.4 )
(
)(
)
( 0.0016 ) + x − 0.2 x − 0.4 ( 0.008)
( 0.2 − 0.3) ( 0.2 − 0.4 )
( 0.3 − 0.2) ( 0.3 − 0.4)
+
80
( x − 0.2 )( x − 0.3)
( 0.0256 )
( 0.4 − 0.2 )( 0.4 − 0.3 )
= ( x 2 − 0.7 x + 0.12 )
( 0.0081)
0.0016 ( 2
+ x − 0.6 x + 0.08 )
0.02
−0.01
+ ( x 2 − 0.5 x + 0.06 )
( 0.0256)
0.02
= ( 0.08) ( x 2 − 0.7 x + 0.12 ) − 0.81( x 2 − 0.6 x + 0.08) + (1.28 ) ( x 2 − 0.5x + 0.06 )
= 0.55x 2 − 0.21x + 0.0216
f ' ( 0.4 ) ≅ p2 ' ( 0.4) = ( 0.55) ( 2 ) ( 0.4 ) − 0.21
= 0.440 − 0.21 = 0.230
f " ( 0.4 ) ≅ p2 " ( 0.4) = ( 0.55) ( 2 ) = 1.10
Thus the approximate solutions are f ' ( 0.4) ≅ 0.23 , f "( 0.4 ) = 1.1 .
The exact solutions are
3
f ' ( 0.4 ) = 4 ( 0.4 ) = 0.256
f " ( 0.4 ) = 4 ( 3) x2 = 1.92
f ' ( x ) = 4 x 3 , f " ( x ) = 12 x 2 , f '"( x ) = 24 x
∴ M 3 = max
0.2 ≤ x ≤0.4
f '" ( x ) = 24 ( 0.4 ) = 9.6
( 0.1) 2
h2
(
)
E2 ' 0.4 ≤
M3 =
9.6 = ( 0.01) ( 3.2 ) = 0.032
3
3
E2 " ( 0.4 ) ≤ hM 3 = ( 0.1) ( 9.6 ) = 0.96
Methods Based on Finite Difference Operators
In this section we derive relation between derivative and finite difference operators.
Ef ( x ) = f ( x + h )
= f ( x ) + hf ' ( x ) +
h2
f " ( x ) + ......
2!
= f ( x ) + hDf ( x ) +
h2 2
D f ( x) + ......
2!
81
d
D = dx
h2 D2
= 1 + hD +
+ ...... f ( x )
2!
= ehD f ( x )
Thus
E = e hD
∴ hD = log E
∆2 ∆ 3
(
)
= log 1 + ∆ = ∆ −
+
− ....
2
2
1
1
= − log (1 − ∇ ) = ∆ + ∇ 2 + ∇ 3 + ....
2
3
2
∆2 ∆3
h D = ∆ −
+
+ ....
2
3
2
2
2
1
1
= ∇ + ∇ 2 + ∇ 3 + ....
2
3
Keeping only first term in each of the above series we have
f ' ( xk ) ≅
≅
≅
f ( xk +1 ) − f ( xk )
Forward differences
h
f ( xk ) − f ( xk −1 )
Backward differenes
h
f ( xk +1 ) − f ( xk −1 )
Central differences
2h
Similarly,
f " ( xk ) ≅
≅
≅
f ( xk + 2 ) − 2 f ( xk +1 ) + f ( x )
h2
f ( xk ) − 2 f ( x k−1 ) + f ( xk −2 )
h2
f ( xk +1 ) − 2 f ( xk ) + f ( xk −1 )
h2
Forward differences
Backward differences
Central differences
First two expressions for both the representations are of first order whereas the third
representation is of second order.
82
Methods Based on Undetermined Coeficients
In this method we write f (r ) ( x ) as a linear combination of the value of f ( x ) at an arbitrary
chosen set of tabular points. Determine the coefficients in linear combination by using Taylor series
expansions of function at some point and by equating the equal powers of derivatives.
For example, assume that the tabular points are equispaced with step length h. Write
h r f ( r ) ( xk ) =
v
∑ a p f ( xk + p )
p =− v
The local truncation error is defined as
E (r ) ( xk ) =
v
1 r ( r)
h
f
x
−
a p f ( xk + p )
(
)
k
∑
r
h
p =−v
The coefficients a p are determined by requiring the method to be of particular order..
Example : A differentiation rule of the form
f ' ( x0 ) = α 0 f ( x 0 ) +α1 f ( x1 ) + α2 f ( x2 )
where xk = x 0 + kh , is given. Find the values of α 0 , α1 , α 2 so that the rule is exact for
f ∈ p2 . Find the error term.
Answer :
f ' ( x0 ) = α 0 f ( x 0 ) +α1 f ( x1 ) + α2 f ( x2 )
= α 0 f ( x0 ) + α1 f ( x0 + h ) + α 2 f ( x0 + 2h )
h2
h3
= α 0 f ( x 0 ) + α1 f ( x0 ) + hf ' ( x0 ) +
f " ( x 0 ) + f "' ( x0 ) + ....
2!
3!
(2h) 2
(2h) 3
+α 2 f ( x0 ) + (2h ) f ' ( x0 ) +
f " ( x 0) +
f "' ( x0 ) + ....
2!
3!
= ( α 0 + α 1 + α 2 ) f ( x0 ) + α1h + α 2 ( 2h ) f ' ( x 0 )
h2
h3
+ [α1 + 4α 2 ] f " ( x0 ) + α1 + 23α 2 f "' ( x0 ) + ......
2!
3!
83
On equating the coefficients of equal powers of derivatives we have,
f ( x0 ) : α0 + α1 +α 2 = 0
f ' ( x0 ) : (α1 + 2α 2 ) h = 1
f "( x0 ) :α1 + 4α 2 = 0
f "' ( x0 ) :α1 + 8α 2 = 0....
Since above system of equations contain three arbitrary values α 0 , α1 , α 2 we can consider
only first three equations to obtain the values of α 0 , α1 , α 2 . From first three equations we have the
following
α1 + 4α 2 = 0 ⇒ α1 = −4α 2
(α1 + 2α 2 ) h = 1⇒ ( −4α 2 + 2α 2 ) h = 1 ⇒ α 2 =
Thus
α1 =
−1
2h
2
−1
, α2 =
and α 0 +α 1 +α 2 = 0 gives
h
2h
2 1 −3
α 0 = −α1 − α 2 = − +
=
h 2 h 2h
The leading term in the error expression is
(α1 + 8α 2 )
h3
h3
2 8
f '"( ξ ) = − f '"(ξ )
3!
h 2h
3!
−4 h
=
f '"( ξ )
2h 6
3
=−
Thus the error term E = −
h2
f '" (ξ )
3
h2
f '" (ξ )
3
Since the error term contains third derivative, the method is exact for the functions whose third
derivative is zero. i.e. method is exact for f ∈ p2 .
84
3.4
Numerical Integration
The general problem of numerical integration is to find the numerical value of the integral
b
I = ∫ w ( x) f ( x ) dx
a
We assume that w ( x ) and f ( x ) are Riemann integrable functions on [a, b]. w ( x ) > 0 defined
on [a, b] is called weight function. The integral I is written as a finite linear combination of values of
f ( x ) in the form
b
I = ∫ w ( x) f ( x ) dx ≅
a
n
∑ λk f ( xk )
k=0
..... (3.4.1)
xk ∈ [ a, b] are called nodes and are distributed on the interval [a, b] with xk −1 < xk , k = 1, 2,
3, ....n. The coefficients λk , k = 1, 2, 3, ...n are called weighs of integration rule or quadrature formula
(3.4.1).
The error is given by
b
n
a
k =0
Rn = ∫ w ( x) f (x ) dx − ∑ λk f ( xk )
Definition : An integration method (3.4.1) is said to be of order p, if it produces exact results
( Rn = 0) for all polynomials of degree less than or equal to p.
Methods Based on Interpolation
Given the (n + 1) nodal values and the corresponding values of f ( xk ) , the Lagrange
interpolating polynomial is given by
n
f ( x ) = ∑ l k ( x ) f ( xk ) + π ( x )
k=0
π ( x)
where l k ( x ) = ( x − x ) π ' ( x
k
k
)
f (n+1) ( ξ )
( n + 1)! ; x0 < ξ < xn
and π ( x ) = ( x − x0 ) ( x − x1 ) ......( x − xn )
From equation (3.4.1) we get,
b
I = ∫ w ( x) f ( x ) dx
a
85
b
(
)
n
π ( x ) f n+1 (ξ )
= ∫ w ( x ) ∑ l k ( x ) f ( xk ) +
dx
(
)
n
+
1
!
k
=
0
a
(
)
b
b
f n+1 ( ξ )
(
)
(
)
= ∑ ∫ w x l k x dx f ( xk ) +
π ( x ) w ( x ) dx
∫
(
)
n
+
1
!
k=0
a
a
n
=
n
∑ λk f ( xk ) + Rn
..... (3.4.2)
k=0
b
b
1
( n +1)
(ξ ) dx
where λk = ∫ w ( x ) l k ( x ) dx and error Rn = ( n + 1) ! ∫ π ( x ) w( x) f
a
a
b
1
( n +1)
(ξ ) dx
The error Rn = ( n + 1) ! ∫ π ( x ) w( x) f
a
( n +1)
(η ) b ( ) ( )
=
w x π x dx for some η ∈ ( a, b ) .
( n + 1) ! ∫
f
a
Rn ≤
f
( n +1)
(η )
( n + 1) !
b
∫ w ( x ) π ( x ) dx
a
The error term can also be determined by
Rn =
b
C
(
)
f n+1 (η )
( n + 1) !
where C = ∫ w ( x ) x
a
n +1
n
dx − ∑ λk xk
n +1
k =0
C is called error constant. If C is zero for f ( x ) = x n +1 then we take the next term f ( x ) = xn + 2
and naturally the error will be zero for all polynomials of degree (n + 1).
Thus our aim is to determine the weights λk and nodal pionts xk such that the error term Rn
is minimum. For simplicity we assume that all the nodal points are equispaced and end points are nodal
points.
The approximate value of the integral is written as
b
n
a
k=0
I = ∫ w ( x) f ( x ) dx = ∑ λk f ( xk )
86
Newton Cotes Methods
In this method we assume that w ( x ) ≡ 1 and the nodes are equispaced with x0 = a , xn = b
b −a
. The weights λk are called cotes numbers. We calculate the weights λk by using
n
Lagrange interpolation. From equation (3.4.2) we know that
and h =
b
n
a
k=0
I = ∫ w ( x) f (x ) dx = ∑ λk f ( x k ) + Rn
( n +1 )
(η ) ( ) ( )
λk = ∫ w ( x ) l k ( x ) dx and Rn =
π x w x dx .
( n + 1) ! ∫
b
f
a
b
a
b
λk = ∫ w ( x ) l k ( x ) dx
a
Now
lk ( x) =
( x − x0 ) ( x − x1 ) ..... ( x − xk −1 )( x − xk +1 ) ..... ( x − xn )
( xk − x0 )( xk − x1 ) .... ( xk − xk −1 )( xk − xk +1 ) ..... ( xk − xn )
Since all nodes are equispaced, xi = x0 + ih and
lk ( x) =
=
( x − x0 ) ( x − x1 ) ..... ( x − xk −1 )( x − xk +1 ) .....( x − xn )
( kh ) ( k − 1) h ( k − 2 ) h.... h ( −h ) ( −2h ) ..... ( −( n − k ) h )
( x − x0 ) ( x − x1 ) ..... ( x − xk −1 )( x − xk +1 ) .....( x − xn )
k ! hk ( −1) n− k ( n − k )! hn− k
Substitute x = x0 + sh then x − xi = x0 + sh − ( x0 + ih ) = ( s − i ) h .
lk ( x) =
=
sh ( s − 1) h( s − 2) h....( s − k + 1) h( s − k − 1)h ....( s − n )h
k !( n − k )!( −1) n−k h n
s ( s − 1)( s − 2)....( s − k + 1)( s − k −1)....( s − n) h n
k !( n − k )!( −1) n− kh n
Since w ( x ) = 1 and x = x0 + sh , dx = hds .
x = a = x0 ⇒ s = 0 and x = b = xn = x 0 + nh = x0 + sh ⇒ s = n .
b
xn
a
x0
∴ λk = ∫ w ( x ) l k ( x ) dx = ∫ l k ( x ) dx
87
n
=∫
s ( s − 1)( s − 2)....( s − k + 1)( s − k − 1)....( s − n )
k !( n − k )!( −1) n− k
0
hds
..... (3.4.3)
π ( x ) = ( x − x0 ) ( x − x1 ) .....( x − xn )
= sh ( s −1)h ( s − 2)h .....( s − n ) h
π ( x ) = h n +1 s ( s − 1)( s − 2)( s − 3).....( s − n) where x = x0 + sh
In equation (3.4.2) we have
f
Rn =
( n +1 )
(η ) b ( ) ( )
∫ π x w x dx
( n + 1) !
f
=
( n +1)
a
(η ) n
∫h
( n + 1) !
n +1
s ( s − 1)( s − 2)( s − 3).....( s − n ) ⋅ hds
0
(
)
h n+ 2 f n+1 (η )
=
∫ s (s − 1)(s − 2).....(s − n ) ⋅ ds
( n + 1) !
0
n
... (3.4.4)
Thus from equation (3.4.3), (3.4.4) and (3.4.2) we have
b
n
a
k=0
I = ∫ f ( x ) dx = ∑ λk f ( xk ) + Rn
n
where λk = ∫
s ( s − 1)( s − 2)....( s − k + 1)( s − k − 1)....( s − n) hds
k !( n − k )!( −1) n− k
0
(
and
)
hn+ 2 f n+1 (η )
Rn =
s( s −1)( s − 2).....( s − n) ds
( n + 1) ! ∫
0
n
..... (3.4.5)
From equation (3.4.5) for different values of n we get different numerical methods of integration.
Case 1 : n = 1 : Trapezoidal Rule
b
∫ f ( x ) dx = λ0 f ( x0 ) + λ1 f ( x1 )
a
= λ0 f ( a ) + λ1 f ( b )
88
( −1)1−0
( s − 1) 2
λ0 =
h ∫ ( s − 1) ds = −h
0!(1 − 0)! 0
2
1
(−1)1−1
s2
λ1 =
h sds = h ⋅
1!(1 −1)! ∫0
2
1
1
=
0
1
=
0
h
2
h
2
b−a
= ( b − a ) and we get
n
Since n = 1, h =
b
∫ f (x ) dx =
(b − a )
2
a
b
∫ f (x ) dx =
f (a ) +
(b − a )
a
2
(b − a )
2
f (b ) =
(b − a )
2
[ f (a ) + f ( b)]
[ f ( a ) + f ( b )]
..... (3.4.6)
equation (3.4.6) is called Trapezoidal rule.
From equation (3.4.5) we get error in Trapezoide rule as
1
h3
R1 = ∫ s( s −1) dx ⋅ f " ( ξ )
2! 0
1
s3 s2
− f "( ξ )
3 2 0
h3
=
2
h3 1 1
= − f "( ξ )
2 3 2
=−
h3
f " (ξ )
12
Alternatively R1 =
where ξ ∈ ( a, b )
C
f " (η ) since trapezoidal rule is exact for a polynomial of degree one we
2!
calculate C for f ( x ) = x2 .
b
C = ∫ x dx −
2
(b − a )
2
a
x3
=
3
b
−
a
(b − a )
2
[ f ( a ) + f ( b )]
[a 2 + b 2 ]
89
=
b3 − a3 ( b − a) ( a2 + b2 )
−
3
2
b 2 + ab + a2 a2 + b 2
(
)
= b−a
−
3
2
=
(b − a) 2
[ 2b + 2ab + 2 a2 − 3a2 − 3b2 ]
6
=−
(b − a)3
6
The error
(b − a )
C
R1 = f " (η ) = −
f " (η )
2!
12
3
where η ∈ ( a, b )
Thus the trapezoidal rule for numerical integration is
b
∫ f (x ) dx =
(b − a )
a
2
[ f (a ) + f ( b)]
and the error in the formula is
R1 = −
( b − a )3
12
f "(η )
To determine the error bound we calculate maximum absolute value of R1 by evaluating values
of f " (η ) .
Case 2 : n = 2 : Simpson’s Rule
Here h =
b −a
b −a a +b
=
. x0 = a , x1 = a +
, x2 = b .
2
2
2
From equation (3.4.5) we have
b
∫ f (x ) dx = λ0 f ( x0 )+ λ1 f ( x1)+ λ 2 f ( x2 )
a
λ0 =
( −1)2− 0
0!(2 − 0)!
2
h ∫ ( s − 1)(s − 2)ds
0
90
2
h
( s − 2)2 ( s − 2)3
= ( s − 1)
−
2
2
6 0
h
4 (−2)3
= − ( −1) −
2
2
6
h
4
= − −2 +
2
3
=
h
3
( −1) 2−1
λ1 =
h s ( s − 2) ds
1!(2 −1)! ∫0
2
2
s3 2s2
= −h −
3
2 0
8
= −h − 4
3
=
4h
3
( −1) 2−2
h s ( s − 1)ds
2!(2 − 2)! ∫0
2
λ2 =
2
h s3 s2
= −
2 3 2 0
=
h 8 4
−
2 3 2
=
h
3
Thus we have
b
∫ f (x ) dx =
a
(b − a )
f (a ) + 4 f
6
a +b
+ f ( b)
2
91
1
rd rule. Since three observations are used
3
to derive the formula, the formula is exact for all polynomials upto order two. The error may occur for
Which is called the Simpson’s rule or Simpson’s
f ( x ) = x3 . Let us calculate C for f ( x ) = x3 .
3
(b − a) 3
a +b
3
C = ∫ x dx −
a + 4
+b
6
2
a
b
3
=
(b − a ) 3
1( 4
3
b − a4 ) −
2a + ( a + b ) + 2b 3
4
12
=
b4 − a4 ( b − a )
[ 2a 3 + a 3 + 2a 2b + 3ab2 + b3 + 2b 3 ]
−
4
12
=
b4 − a4 ( b − a ) 3
[ a + a 2b + ab2 + b 3 ]
−
4
4
=
b4 − a4 ( b − a )
2
2
−
( a + b ) ( a + b )
4
4
=0
This shows that method is exact for polynomials of degree upto three. Let use caulculate C for
f ( x ) = x4 .
4
(b − a ) 4
a +b
4
C = ∫ x dx −
a + 4
+b
6
2
a
b
4
1 ( 5 5 ) (b − a ) 4 1 (
4
b −a −
a + a + b) + b4
5
6
4
=
C=
=
(b − a)
1( 5
[ 4a 4 + a4 + 4a 3b + 6 a2b 2 + 4 ab3 + b4 + 4b 4 ]
b − a5 ) −
5
24
(b − a)
24 ( b 4 + ab3 + a 2b2 + a3b + a 4 ) − 5 ( 5a 4 + 4a3b + 6a 2b2 + 4ab3 + 5b4 )
120
=
(b − a)
[ −a 4 − b4 + 4ab3 − 6a 2b 2 + 4a 3b ]
120
=−
(b − a) (b − a)4
120
92
=−
(b − a)5
120
Then error in Simpson’s
1
rd rule is
3
( b − a ) 5 (iv)
R =−
f (η )
4!(120 )
Since b − a = 2h the error term
( 2h )5 ( iv )
R =−
f (η )
24 (120 )
=−
32h5
( )
f iv (η )
( 24 ) (120 )
=−
h5
( )
f iv (η )
( 3) ( 30 )
h5 (iv )
=−
f (η )
90
where η ∈ ( a, b )
From above two cases we observe that for large value of n we get better approximation.
However for large n ( n ≥ 8, n ≠ 9 ) some of λk ' s become negative and therefore higher order Newton
Cotes formulas are not commonly used.
Conclusions :
In this unit we have seen how to approximate a function or a given tabular values of function by
a polynomial. If (n + 1) observations or values of function are known we can fit a unique polynomial of
degree n. Lagrange interpolation method and Newtons divided difference formula generate the same
polynomial.
Finite difference operators and polynomial interpolation is used to calculate the derivative of a
given function. Method of undetermined coefficients can also be used to find out the derivatives of a
given function.
Methods of numerical integration are developed by approximating a function by a polynomial
and on integrating this approximating polynomials. It is shown that trapezoidal method is first order
method whereas Simpson’s one third rule is of order three.
❖❖❖
93
UNIT - IV
NUMERICAL SOLUTION OF
DIFFERENTIAL EQUATION
Euler’s Method :
Consider the interval [a, b] and the initial value problem x ' = f ( t , x ) with x ( a ) = x0 (initial
point).
Divide interval [a, b] into equal sub-interval and select mesh point
t k = a + hk , k = 0, 1, ..... N and h =
b −a
N
Suppose x , x ' , x " are all continuous. The Euler Method iteration formula is ξ 0 = x0 .
ξ i+1 = ξ i + hf [ ti , ξ i ]
Example 1 : Solve the initial value problem x ' =
t−x
, on the interval [0, 3], x ( 0) = 1 .
2
Solution : Let h = 0.5, ξ 0 = x ( 0 ) = 1 .
ξ1 = ξ 0 + hf [ t 0 , ξ0 ]
0 −1
= 1 + ( 0.5 )
2
This for t0 = 0 , x1 = 0.5
= 0.75
0.5 − 0.75
ξ 2 = ξ1 + hf [ t1 , ξ1 ] = 0.75 + 0.5
= 0.6875
2
for t 1 = 0.5, x 2 = 1
1 − 0.6875
ξ 3 = 0.6875 + 0.5
= 0.765625
2
for t 2 = 1, x 3 = 1.5
1.5 − 0.765625
ξ 4 = 0.765625 + 0.5
= 0.9492187
2
for t 3 = 1.5, x 4 = 2
2 − 0.9492187
ξ 5 = 0.9492187 + 0.5
= 1.2119141
2
for t 4 = 2, x 5 = 2.5
94
2.5 − 1.2119141
ξ 6 = 1.2119141 + 0.5
= 1.5339355
2
Now,
x' =
e
t
2
for t 5 = 2.5, x 6 = 3
t x
x t
− ⇒ x '+ =
multiply by e t 2 we have,
2 2
2 2
(
)
1
t t
t
x
t t
x '+ = e 2 ⇒ e 2 x = e 2
2
2
2
integrate
t
t t
∫ ( e 2 x ) dt = ∫ e 2 dt
1
2
⇒e
t
t
2x
t
t e 2
1e 2
=
−∫
dt + C
2 1
2 1
2
2
= te
t
2
− 2e
t
−t
by initial condition 1 =0.2 + C ⇒ C = 3
∴ x = t − 2 + Ce
−t
x = t − 2 + 3e
2
2
+ C = ( t − 2 )e
t
2
+C
2
Example 2 : Solve x ' = −2 + x 2 with x ( 0) = 1 , with h = 0.2, 0.1 and 0.05 on [0, 1].
Solution : Let h = 0.2
i = 0, 1, 2, 3, 4, X 0 = 1
X i +1 = Xi − 2ht i X i 2
For
i = 0, t0 = 0 , X 0 = 1 .
X ( 0.2 ) ≈ X1 = X 0 − 2ht0 x02 = 1
For
i = 1, t1 = 0.2 , X1 = 1 .
⇒ X 2 = 1 −2 ( 0.2 ) ( 0.2 ) (1) = 0.92
2
For
i = 2, t 2 = 0.4 , X 2 = 0.92 .
⇒ X 3 = 0.92 − 2 ( 0.2 ) ( 0.4 ) ( 0.92 ) = 0.78458
2
For
i = 3, t3 = 0.6 , X 3 = 0.78458 .
⇒ X 4 = 0.63684
X 5 = 0.50706
95
Now for,
h = 0.1
u0 = 1, u1 = 1, u2 = 0.98, u3 = 0.94158, u4 = 0.88839
u5 = 0.82525, u6 = 0.75715, u7 = 0.68835, u8 = 0.62202
u9 = 0.56011, u10 = 0.50364.
Order of Euler’s Method
Lemma :
Let a > 0 , b ≥ 0 and let xn , n = 0, 1, ...... be a sequence of non-negative numbers satisfying
the inequality
xn +1 ≤ (1 + a ) xn + b
Then
xn ≤ e na x0 + b
...... (1)
ena − 1
a
....... (2)
Proof : We have x1 ≤ (1 + a ) x0 + b
x2 ≤ (1 + a ) x1 + b ≤ ( a + 1) (1 + a ) x0 + b + b = (1 + a ) x0 + (1 + a ) b + b
2
x3 ≤ (1 + a ) x2 + b ≤ ( a + 1) (1 + a ) x0 + (1 + a ) b + b + b = (1 + a ) x0 + (1 + a ) b + (1 + a ) b + b
2
3
2
Continuing in this way we see that,
n −1
xn ≤ (1 + a ) x0 + b ∑ (1 + a ) = (1 + a ) x0 + b
n
j
n
j=0
(1 + a )n − 1
a
From the finite geometric series formula.
From the Maclaurin expansion of e a , we have 1 + a < e a ,
For a > 0 and thus that (1 + a ) n < ena . Substituting this into (3) gives
xn ≤ e na x0 + b
ena − 1
a
Hence the result.
96
........ (3)
Theorem : Let X be the solution to
X ' = f (t , x )
....... (1)
and suppose that C = X ( t ) ≤ d for t ∈ ( t 0 , tk ) .
Let ξ i , i = 0,...., N be the Euler Method approximation to x ( t i ) .
Where ti = t0 + ih with h =
tk − t0
.
N
If X " ( t ) ≤ M for some constant M and all t ∈ ( t 0 , tk ) and the function f in (1) is Lipschitz
continuous with constant L in the reactangle
R = {( t , x ) | t 0 ≤ t ≤ t k ; c ≤ x ≤ d }
Then
x ( ti ) − ξi ≤
Mh ( L (ti −t0 ) )
e
−1
2L
...... (2)
Proof : Let Ei = X ( ti ) − ξi be the error made by Euler’s Method at the ith step. From Taylor’ss
theorem applied to X ( t ) and (1) we have
Ei +1 = X ( ti+1 ) − ξi +1
1
= X ( t i ) + hX ' ( t i ) + h2 X " ( Ei ) − ξ i + hf ( ti , ξi )
2
1
= Ei + h X ' ( t i ) − f ( ti , ξi ) + h2 X "(τ i )
2
1
= Ei + h f ( ti ) x ( ti ) − f ( ti , ξi ) + h2 X " (τ i )
2
......... (3)
For some τ i ∈ ( t i , ti +1 ) since X " is bounded by M and f is Linchitz continuous.
Ei +1 ≤ Ei + hL X ( ti ) − ξi +
Mh 2
Mh 2
= Ei (1 + hL ) +
2
2
Thus from above lemma
Ei ≤
Mh 2 eihL − 1 Mh ( ihL )
=
e −1
2
hL
2L
Since E0 = 0 . The result now following by putting ih = ti − t0 .
97
...... (4)
δi .
Theorem : Let δ = 0max
≤i ≤ N
Then under the same hypothesis as above Theorem
hM δ ( L( ti −t0 ) )
L t −t
X ( t i ) − ξi ≤
+
−1 + δ0 e ( i 0 )
e
2 L hL
Let ξ 0 = x0 + δ0 , ξ i+1 = ξ i + hf ( ti , ξi ) + δ i+1 .
Runge-Kutta Method
One Step Method
y0 = y ( x0 )
yi +1 = yi + hφ ( xi yi , h )
The function φ predicts the direction that the solution will take for the point ( xi , yi ) if φ
satisfying the condition
lim φ ( xi , yi ; h ) = f ( x, y )
h→ 0
Then we get the improved solution and the method is said to be consistent.
Modified Euler Method
Instead of using YEuler as the approximation to Y ( xi ) we use it to locate another point near
the trajections of y ( xi ) take the Estimated slope φ to be the average of slope of
( xi +1, yEuler ) .
y0 = x0
M 1 = f ( x i , yi )
M 2 = f ( xi + h, yi + hM1 )
Then
yi +1 = yi + h
M1 + M 2
2
This is modified Euler method.
98
( xi , yi )
and
Example : Solve y ' =
x− y
, [0, 3], y0 = y ( 0 ) = 1 , h = 0.5.
2
Solution : Let x = 0, M1 = −0.5 , M 2 = f ( xi+ h , yi + hM 1 ) = −0.125
M + M2
y1 = y 0 + h 1
= 0.84375
2
x = 0.5, M1 = −0.171875 , M 2 = 0.1210937 .
y2 = 0.8310546
if x = 1, M1 = 0.0844727, M 2 = 0.3133845,then y3 = 0.9305114
if x = 1.5, M 1 = 0.2847443, M 2 = 0.4635582, then y4 = 1.117887
if x = 2, M1 = 0.4412062, M 2 = 0.5809048, then y5 = 1.3731148
if x = 2.5, M 1 = 0.5634426, M 2 = 0.6725819, then y6 = 1.6821209
Mid Point Method
Using Euler’s method approximate the solution at the mid point of xi , xi +1 and take estimated
slope φ to be f at that point
y0 = x0
h
M
M 2 = f xi + , yi + h 1
2
2
M 1 = f ( x i , yi )
∴ yi+1 = yi + hM 2
Note that Local truncation error for one step method defined as
Ei ( h ) = y ( xi +1 ) − y ( xi ) − hφ ( xi , yi , h )
Question : Find the order of local truncation error for the modified Euler Method.
Answer : Assume that f has continuous third order partial derivatives in a rectangle containing the
solution y ( x) .
From Taylor’s theorem.
y ( xi +1 ) = y ( xi + h ) = y ( xi ) + hy ' ( xi ) +
h2
h3
y " ( x i ) + y "' ( x i ) + 0 ( h 4 )
2!
3!
99
h
h2
y ( xi +1 ) − y ( xi ) = h y' ( xi ) + y "( xi ) +
y "' ( xi )
2
6
..... (1)
Differentiating the differential equation y ' = f ( x , y ) we obtain
y" =
y "' =
∂f ( x , y ) ∂f ( x , y )
∂f ( x, y )
∂f ( x , y )
+
y' =
+ f ( x, y )
∂x
∂y
∂x
∂y
∂ 2 f ( x, y )
∂x
2
+
+
=
∂ 2 f ( x, y )
∂x∂y
y '+ f ( x, y )
∂ ∂
∂
∂
f ( x, y ) +
f ( x, y ) ⋅
f ( x, y )
∂x ∂y
∂x
∂y
∂
∂2
∂
f ( x, y )
f ( x, y ) f ( x , y ) + f ( x , y ) 2 f ( x , y ) ⋅ f ( x, y )
∂y
∂y
∂y
∂2
∂2
∂
∂
f
(
x
,
y
)
+
2
f
(
x
,
y
)
f ( x, y ) ⋅
f ( x, y )
f ( x, y ) +
2
∂x
∂y
∂x
∂x∂y
2
∂2
2 ∂
+ 2 f ( x, y ) f ( x, y ) + f ( x, y ) f ( x, y )
∂y
∂y
y ' ( xi ) = f ( xi , yi ) = fi
∴ (1) ⇒ yi +1 − yi = hfi +
h2
h3
f i,x x + 2 f i x, y fi + f i ,yy f i 2
f
+
f
f
+
(
i ,x
i, y i )
2!
3!
+ f i, y fi , x + f i, y 2 fi + 0 ( h 4 )
φ ( xi , yi ; h ) =
=
.......... (2)
M1 + M 2 1
= f ( xi , yi ) + f ( xi + h, yi + hM 1 )
2
2
1
f ( xi , yi ) + f ( xi + h, yi + hf ( xi , yi ) )
2
Taylor’s Theorem for two variables
f ( x, y , h ) = f ( x , y ) + h
+
∂
∂
f ( x , y) + k
f (x, y)
∂x
∂y
2
1 2 ∂2
∂2
2 ∂
h
f
(
x
,
y
)
+
2
hk
f
(
x
,
y
)
+
k
f ( x, y )
2
2
2 ∂x
∂x∂y
∂y
100
Apply to the slope estimate φ for modified Euler method we obtained.
φ ( xi , yi ; h ) =
(
)
1
1
fi + f i + hf i ,x + hfi fi , y + h 2 f i ,x x + 2 L 2 f i ,x y f i + h 2 f i , yy fi 2 + 0 ( h 3 )
2
2
= fi +
1
h
h2
1
h2
f i, x + f i, y f i +
fi ,xx + h 2 f i x, y f i +
f i , yy fi 2
2
2
4
2
4
hφ ( xi , yi , h ) = hf i +
h2
h2
h3
h3
h3
fi , x +
fi , y fi +
f i,x x +
f i,x y f i +
f i,y y f i 2
2
2
4
2
4
........ (3)
........ (4)
Subtract equation (4) from (2) we get,
1
1
1
yi +1 − yi − hφ ( xi , yi ; h ) = h3 − fi , xx − f i,x y f i −
f i,y y fi 2
12
6
10
+
∈i ( h ) ≤ kh3
1
1
f i , y f i, x + f i , y 2 fi + 0 ( h4 ) − 0 ( h3 )
6
6
k is constant.
Provided 2nd partial derivative of f is continuous.
Therefore, the local direction error of Euler modified method of order 3 in h.
Theorem : Consider the one step method
yi +1 = yi + hφ ( xi , yi h )
Let φ be Lipschitz continuous with consistent L in the variable x in the reactangle R.
R = {( x , y ) : x0 ≤ x ≤ x k ; c ≤ y ≤ d } where c ≤ y ( xi ) and ξ i ≤ d for the space point
xi = 0, 1, ..., N then
y ( xi ) − ξi ≤
∈ ( L ) ( L ( xi − x0 ) )
e
−1
hL
Proof : ξ i is approximation to yi .
yi +1 − ξi +1 = yi +1 − ξi + hφ ( xi , ξi ; h )
Add and subtract hφ ( xi , yi ; h ) + yi
∴ yi +1 − ξi +1 = yi +1 − hφ ( xi , yi ; h ) + yi + hφ ( xi , yi ; h ) + yi − ξi + hφ ( xi , ξi ; h )
101
= ∈i ( h ) + yi − ξi + h φ ( xi , yi ; h ) − φ ( xi , ξi ; h )
≤ ∈i ( h ) + yi − ξi + h φ ( xi , yi ; h ) − φ ( xi , ξi ; h )
≤ ∈i ( h ) + yi − ξ i + hL yi − ξi
= ∈ ( h ) + (1 + hL ) yi − ξi
From the assumption of Lipschitz continuity for φ and by the lemma above
e na −1
na
(
)
x
≤
1
+
a
x
+
b
⇒
x
≤
e
x
+
b
n+1
n
n
0
a
We have, y ( xi ) − ξi ≤
∈ ( h ) ( L ( xi − x0 ) )
e
−1
hL
Q y0 = ξ 0
Question : Show that the global truncation error made by the modified Euler Method is O (h 2 ) .
Answer : We seen that the local truncation error is order three.
This follows from the above theorem, once we have established that
φ ( xi , yi ; h ) =
1
f ( x , y ) + f ( x+ h, x + hf ( x, y ) )
2
is Lipschitz continuous. Assume that f itself is Lipschitz continuous with constant Lf. we have
1
φ ( x , y1; h ) − φ ( x, y2 ; h ) = f ( x , y1 ) + f ( x + h, y1 + hf ( x, y ) )
2
−
1
f ( x, y2 ) + f ( x + h, y2 + hf ( x, y2 ) )
2
≤
1
1
f ( x, y1 ) − f ( x, y2 ) + f ( x + h, y1 + hf ( x , y1 ) ) − f ( x + h, y2 + hf ( x, y2 ) )
2
2
≤
1
1
L f y1 − y 2 + L f y 1 + hf ( x, y1 ) − y2 + hf ( x, y2 )
2
2
≤
1
1
1
L f y1 − y 2 + ( L f y1 − y 2 ) + L f h f ( x , y1 ) − f ( x, y2 )
2
2
2
1
≤ L f y1 − y2 + L2 f h y1 − y2
2
1
≤ L f + hL2 f ( y1 − y 2 )
2
102
1 2
Thus φ is Lipschitz continuous with constant Lφ = L f + hL f .
2
Note that, similarly we can show that the midpoint method can have a global truncation error of order
two.
Example : y ' = −ty , [0, 0.15], h = 0.05 with y ( 0 ) = 1
Solve by Euler modified method.
Solution : Euler Method : ξ i+1 = ξ i + hf ( t , ξi )
ξ1 = 1 + 0.05 ( 0×1) = 1
for → 0.05
ξ 2 = 1 + 0.05 ( −0.05 × 1) = 0.9975
for → 0.1
ξ 3 = 0.9975 + 0.05 ( −0.1 × 0.9975 ) = 0.9925125
for → 0.15
Euler Modified yi +1 = yi + h
M1 + M 2
2
(0, 0.05, 0.1, 0.15)
M 1 = f ( x i , yi )
M 2 = f ( xi + h, yi + hM1 )
M1 = 0
M 2 = −0.05
y1 = 1 + 0.05
Mid point
ξ0 =1 → 0
( 0 + ( −0.05) ) = 0.99375
y2 = 0.9923205
2
yi +1 = yi + hM 2
h
hM 1
M 2 = f xi + , yi +
2
2
y1 = 1 + 0.15 ( −0.075) = 0.98875
Example : Solve y ' = −ty .
Solution :
y'
= −t ⇒
y
( log y ) ' = −t
intigrating both side we have
⇒ log y = −
t2
⇒ y = e− 2
t2
+c
2
∫ log ( y ) ' dy = −∫ tdt
t2
⇒ y = e− 2 +c
∴ y = 0.988813
103
1 = ec ⇒ c = 0
Exercise :
1)
Solve y' = – 2xy2, if y (0) = 1, h = 0.2, 0.1, 0.05 on [0, 1]
2)
Solve y' = x 2 + y2, if y (0) = 0, h = 0.5 on [0, 2].
Runge Kutta Methods
The most general Runge-Kutta Method involving two slope calculations is
M 1 = f ( x i , yi )
y0 = x0
M 2 = f ( xi + α h, yi + h β M1 )
yi +1 = yi + h ( w1M 1 + w2 M 2 ) where α ∈ [ 0,1] .
Note : 1. This gives the modified Euler Method when α = β = 1 and w1 = w2 =
method when α = β =
1
and the midpoint
2
1
and w1 = 0 , w2 = 1 .
2
2. Not every choice of α and β will lead to a method that has order three local truncation error,,
∂f
and so forth we have from Taylor’s
’s
∂x
Theorem by sequence of computation (Similar to that used above).
however indeed with yi = y ( xi ) , f i = f ( xi , yi ) , f i ,x =
∈i ( h ) = yi +1 − yi − h w1 f i + w2 f ( xi + α h, yi + β hf i )
= yi ' h +
1
yi " h 2 − w1h f i + w2h ( f i + f i, xα h + f i, y β hfi ) + 0 ( h3 )
2
1
1
= (1 − w1 − w2 ) hf i + − α w2 fi , x h2 + − β w2 f i , y fi h 2 + 0 ( h 3 )
2
2
Thus since f ( x , y ) is arbitrary for ∈i to be 0 ( h 3 ) we must have
w1 + w2 = 1
⇒ β =α
1
2
1
w2 =
2α
α w2 =
β w2 =
1
2
w1 = 1 −
1
2α
It can be shown that no choice of α can lead to an order of local truncation error greater than
three. Put
1
1
h
yi +1 = yi + h 1 −
M 2 = yi +
M1 +
( 2α − 1) M1 + M 2
2α
2α
2α
Note : Every Runge-Kutta Method should reduce to a quadrature formula when f ( x , y ) is independent
of y with w’s as weights and α ’s as abscissas.
104
If α =
1
we get
2
h
hM 1
M 1 = f ( xi , yi ) and M 2 = f xi + , yi +
2
2
yi +1 = yi + hM 2
Which is the Euler method with spacing
h
, i.e. is midpoint quadrature rule when f ( x , y ) is
2
independent of y.
For α = 1 we get
yi +1 = yi +
h
[M1 + M 2 ]
2
M 1 = f ( xi , yi ) , M 2 = f ( xi + h, yi + hM1 )
Which is Eular modified method.
Which reduces to the trapezoidal rule when f ( x , y ) independent of y.
Notethatthe general form of a Runge-Rutta Method involving n slope calculations is
y0 = x0 , M 1 = f ( xi , yi ) , M 2 = f ( xi + α 2h, yi + β 2hM 1 )
n −1
M
=
f
x
+
α
h
,
y
+
h
β
M
n
i
n
i
∑
nj
j
.....
j =1
n
yi +1 = yi + h∑ w j M j
j =1
2.
Thus a three slope Runge-Kutta Method has the form
y0 = x0 , M 1 = f ( xi , yi ) , M 2 = f ( xi + α 2h, yi + β 21M 1h )
M 3 = f ( xi + α 3h, yi + β31M 1h + β32M 2 h )
yi +1 = yi + h ( w1m1 + w2m2 + w3m3 )
By expanding the local truncation error in a manner similar to that used in above
w1 + w2 + w3 = 1
β21w2 + ( β31 + β32 ) w3 =
1
2
α 2 β21w2 + α 3 ( β31 + β32 ) w3 =
1
1
1
, α 2 β31w3 = , α 2 w2 + α 3w3 =
3
6
2
105
1 2
1
1 1
1
1
1
2
α 2 w2 + α 32w3 = , β 212w2 + ( β 31 + β32 ) w3 = , β21β31w3 =
2
2
6 2
2
6
6
⇒ β21 = α 2 , β31 + β32 = α3
⇒ w1 + w2 + w3 = 1
β21w2 + ( β31 + β32 ) w3 =
1
2
β212 w2 + ( β31 + β32 ) w3 =
2
∈i ( h ) = yi +1 − yi −h
1
1
, β21β31w3 =
2
6
{(w1 fi + w2 f ( xi ) + α 2 h, yi + β21hfi )
+ w3 f ( xi + α3 h, yi + β 31hf i + β 32 M 2 h )}
= hyi '+
{
h2
yi "− hw1 f i + w2h ( fi + f i, xα 2h + fi , y β21hf i ) +
2
(
+ w3h f i + f i , xα 3h + fi , y ( β31hf i + β 32hf ( xi + α 2 h, yi + β21hf i ) )
= hf i +
)}
h2
f i, x + f i , y f i ) − hw1 f i − w2 hf i − w2h 2 f i ,xα 2 − w2 h2 β21 fi fi , y −
(
2
− w3hfi − w3h 2α 3 f i, x − f i, y β31hf i + β32 h ( f i + f i ,xα 2h + f i, y β21hf i )
1
1
= (1 −w1 − w2 − w3 ) hfi + − w2α 2 − w3α 3 h2 f i, x + h 2 − w2 β21 − w3 ( β31 + β32 ) f i , y fi +
2
2
h3 1
h3 1
+ − α 2β 21w2 − α3 ( β31 + β 32 ) w3 fi ,x y f i + −α 22 w2 − α 32 w3 fi ,x x +
2 3
2 3
+
h3 1
h3 1
2
2
2
−
β
w
−
β
+
β
w
f
f
+
− α 2 β31w3 f i, x f i, y +
21 2 ( 31
32 )
3 i , yy i
2 3
2 6
1
+ h3 − β21β 31w3 f i , y 2 fi
6
106
α2
β21
α3
β31
β32
w1
w2
1
2
1
2
1
–1
2
1
6
4
6
w3
... (1)
1
6
α2 =
1
3
3
2
3
4
= β 21 , α 3 = , β31 = 0 , β32 = , w1 = , w2 = , w3 =
2
4
4
9
9
9
... (2)
α2 =
1
2
1
3
= β 21 , α 3 = = β 32 , β31 = 0 , w1 = , w2 = 0 , w3 =
3
3
4
4
... (3)
Runge Kutta Method for Four Slopes
Order four method
M 1 = f ( x i , yi ) ,
y0 = x0 ,
h
hM 2
M 3 = f xi + + yi +
2
2
yi +1 = yi +
h
hM 1
M 2 = f xi + , yi +
2
2
M 4 = f ( xi + h, yi + hM 3 )
h
( M 1 + 2 M 2 + 2M 3 + M 4 )
6
Define
M 1 = f ( x i , yi )
M 2 = f ( xi + α 2h, yi + β 21hM 1 )
M 3 = f ( xi + α 3h, yi + β31hM 1 + β32hM 2 )
... (1)
M 4 = f ( xi + α 4h, yi + β 41hM 1 + β42 hM 2 + β43hM 3 )
yi +1 = yi + h ( w1M 1 + w2M 2 + w3M 3 + w4 M4 )
... (2)
where the ----- α 2 , α 3 , α 4 , β21 , β31 , β32 , β41 , β42 , β43 , and w1 , w2 , w3 , w4 are
chosen to make yi +1 closer to y ( xi +1 ) .
107
Expanding as before (and matches coefficients of powers of h) we obtain the following system
of equation
α 2 = β 21 , α 3 = β31 + β32 , α 4 = β41 + β42 + β43
w1 + w2 + w3 + w4 = 1 , w2α 2 + w3α 3 + w4α 4 =
1
2
1
1
w2α 22 + w3α 3 2 + w4α 4 2 = , w3α 2 β32 + w4 (α 4 β42 + α 3 β 43 ) =
3
6
w2α 23 + w3α3 3 + w4α 4 3 =
1
1
2
2
2
, w3α 2 β32 + w4 (α2 β 42 + α3 β43 ) =
4
12
w3α 2α3 β32 + w4 ( α 2β 42 + α3 β43 ) α 4 =
1
1
, w4α 2 β32 β43 =
8
24
The equations of the above form occur in all Runge-Kutta Methods. We have 11 equations in
13 unknowns.
The method (2) will correspond to Simpson’s rule of integration.
If α 2 = α 3 and w2 = w3 the solution of equations above is given by
α2 = α3 =
1
2
w2 = w3 =
α4 = 1
1
3
w1 = w4 =
1
6
β41 = 0 = β 42 , β43 = 1
Thus the equation in (1) and (2) gives above.
To solve by ---- choice α 2 =
α2
β21
α3
β31
β32
α4
β41
β42
β43
w1
w2
w3
1
, β =0.
2 31
w4
108
1
2
1
2
1
2
0
1
2
1
0
0
1
2
6
2
6
2
6
1
6
α2 =
1
2
1
= β 21 , α 3 = , β31 = − , β32 = 1 , α 4 = 1 , β41 = 1 , β42 = −1 , β43 = 1 ,
3
3
3
w1 =
1
3
= w4 , w2 = = w3
8
8
Example : Solve
y' =
x− y
2
[0, 3]
h = 0.5
y0 = 1
Answer :
Case 1 h = 0.5
if x = 0
M1 = 0.5
M 2 = −0.3125
M 3 = −0.3359375
M 4 = −0.1660156
y1 = 1 + 0.08333 ( −0.5 − 2 × 0.3125 − 2 × 0.3359 − 0.1660156 )
= 0.8364258
if x = 0.5
M1 = 0.1682119
M 2 = −0.0221862
M 3 = −0.0404396 M 4 = 0.091897
y2 = 0.8196285
if x = 1
M1 = 0.0901857
M 2 = 0.2039125
M 3 = 0.1896966
M 4 = 0.2927615
y3 = 0.9171423
if x = 1.5
M1 = 0.2914288
M 2 = 0.3800002
M 3 = 0.3689288
M 4 = 0.44919
M 2 = 0.5171388
M 3 = 0.508516
M 4 = 0.571029
M 2 = 0.623943
M 3 = 0.6172283
M 4 = 0.6659141
y4 = 1.1036826
if x = 2
M1 = 0.4481587
y5 = 1.3595575
if x = 2.5
M1 = 0.5702212
y6 = 1.6694308
Case 2 : h = 1 y0 = 1
if x = 0
M1 = −0.5
M 2 = −0.125
M 3 = −0.21875 M 4 = 0.10975
M 2 = 0.3173828
M 3 = 0.260498 M 4 = 0.4595947
y1 = 0.8203125
if x = 1
M1 = 0.0898439
y2 = 1.1045125
109
M1 = 0.4477437
if x = 2
M 2 = 0.5858078
M 3 = 0.5512918 M 4 = 0.61209
y3 = 1.670186
Case3 : h = 1.5 y0 = 1
M1 = −0.5
if x = 0
M 2 = −0.0625
M 3 = −0.1015625 M 4 = 0.3261718
y1 = 0.8745117
if x = 1.5
M1 = 0.3127441
M 2 = 0.5704651 M 3 = 0.4738197
M 4 = 0.7073793
M 2 = 0.625
M 4 = 1.328125
y2 = 1.621685
Case 4 : h = 3 y0 = 1
M1 = −0.5
if x = 0
M 3 = −0.21875
y1 = 1.8203125
Exercise :
1)
y ' = 5 ( x − 1) y , y (0) = 5, [0, 2], h = 0.5, 0.2, 0.1
2)
y ' = −2xy 2 , y (0) = 1, [0, 1], h = 0.2, 0.1, 0.05
3)
y ' = x2 + y2 , y (0) = 0, [0, 2], h = 0.5
Example :
h = 0.25, y0 = 1
y1 = 0.8974915
y2 = 0.8364037
y3 = 0.8128696
( y4 ' ) y5 = 0.9121021
( y5 ' ) y6 = 1.1036408
( y6 ' ) y7 = 1.3595168
( y7 ' ) y8 = 1.6693928
y4 = 0.8195840
x=0
M1 = −0.5
M 2 = −0.40625
M 3 = −0.4121084
M 4 = −0.3134863
x = 0.25
M1 = −0.375
M 2 = −0.2378082
M 3 = −0.2463827
M 4 = −0.1679479
Find, x = 0.5
x = 0.75
x=1
x = 1.25
110
x = 1.5
x = 1.75
x=2
x = 2.25, and
x = 2.5
Systems of Differential Equations
dx
= f (t ,x , y )
dt
x ( t 0 ) = x0
... (1)
dy
= g (t , x, y )
dt
y ( t 0 ) = y0
This can be written as,
x ' ( t ) = f (t , x ( t ) , y ( t ) )
x ( t 0 ) = x0
y ' ( t ) = g (t , x ( t ) , y ( t ) )
y ( t 0 ) = y0
dx
= x + 2y
dt
x ( 0) = 6
dy
= 3x + 2y
dt
y ( 0) = 4
Example :
Solution to the I.V.P is
x ( t ) = 4 e4t + 2 e−t
y ( t ) = 6 e4 t − 2 e− t
The Runge-Kutta formulas of order 4 are
xk +1 = xk +
4
( f1 + 2 f2 + 2 f3 + f4 )
6
yk +1 = yk +
4
( g1 + 2 g2 + 2 g3 + g 4 )
6
111
where
f1 = f ( tk , xk , yk )
g1 = g ( t k , xk , yk )
h
h
h
f 2 = f t k + , xk + f1, yk + g1
2
2
2
h
h
h
g 2 = g t k + , xk + f1, yk + g1
2
2
2
h
h
h
f 3 = f t k + , x k + f 2 , yk + g 2
2
2
2
h
h
h
g 3 = g t k + , x k + f 2 , yk + g 2
2
2
2
f 4 = f ( t k + h, xk + hf 3 , yk + hg 3 )
g 4 = g ( t k + h, xk + hf 3 , yk + hg 3 )
Example : Solve
x ' = x+ 2 y
x ( 0) = 6
y ' = 3x + 2 y
y ( 0) = 4
[0, 0.2]
h = 0.02
Solution :
f1 = f ( 0,6,4 ) = 6 + 2 × 4 = 14
x0 +
g1 = g ( 0,8,4) = 26
h
f1 = 6.14
2
y0 +
f 2 = f ( 0.01,6.14,4.26 ) = 14.66
x0 +
h
g1 = 4.26
2
g 2 = g ( 0.01,6.14,4.26 ) = 26.94
h
f 2 = 6.1466
2
y0 +
h
f 2 = 4.2694
2
f 3 = f ( 0.01,6.1466,4.2694 ) = 14.6854
g 3 = g ( 0.01,6.1466,4.2694) = 26.9786
x0 + hf 3 = 6.293708
y0 + hg3 = 4.539572
f 4 = f ( 0.02,6.293708,4.539572 ) = 15.372852
g 4 = g ( 0.02,6.293708,4.539572 ) = 27.96028
x1 = 6 +
0.02
(14 + 2 ×14.66 + 2 ×14.6854 + 15.372852 ) = 6.29354551
6
y1 = 4 +
0.02
( 26 + 2 × 26.94 + 2 × 26.9786 + 27.960268 ) = 4.5393249
6
112
k
tk
f1
f2
f3
f4
g1
g2
g3
g4
0
0
--
--
--
--
--
--
--
--
6
4
1
0.02
14
14.66 14.6854 15.372852
26
26.94 26.9786 27.960268
6.2935
4.8393
2
0.04
15.3721
6.6150
5.11948
3
0.06
6.968525
5.74396
4
0.08
7.35474
6.41653
5
0.1
7.7769
7.1412
6
0.12
8.2381
7.9226
7
0.14
8.7414
8.7653
8
0.16
9.290209
9.6745
9
0.18
9.888271
10.6560
10
0.2
10.53962 11.715780
27.9591
xk
yk
Exercise :
1)
Solve the system x ' = 2 x + 3 y , y ' = 2 x + y with initial condition x ( 0) = −2.7 , y ( 0) = 2.8
over the interval [0, 1] use h = 0.05.
2)
Solve the system x ' = 3 x − y , y ' = 4 x − y with initial condition x ( 0) = 0.2 , y ( 0) = 0.5
over the interval [0, 2] use h = 0.05.
3)
Solve the system x ' = x − 4 y , y ' = x + y with initial condition x ( 0 ) = 2 , y ( 0 ) = 3
over the interval [0, 2] use h = 0.05.
4)
Solve the system x ' = y − 4 x , y ' = x + y with initial condition x ( 0) = 1 , y ( 0 ) = 1
over the interval [0, 1.2] use h = 0.05.
❖❖❖
113
© Copyright 2026 Paperzz