Contents
Problem 1
1.1 Problem definition
1.2 Solution of the problem
1.2.1 Linear interpolation
1.2.2 Method of least squares interpolation
1.2.3 Lagrange interpolating polynomial
1.2.4 Cubic spline interpolation
1.3 Results and discussion
1.3.1 Lagrange polynomial
Problem 2
2.1 Problem definition
2.2 Problem solution
2.2.1 Rectangular method
2.2.2 Trapezoidal rule
2.2.3 Simpson's rule
2.2.4 Gauss-Legendre method and Gauss-Chebyshev method
Problem 3
3.1 Problem definition
3.2 Problem solution
Problem 4
4.1 Problem definition
4.2 Problem solution
References
Problem 1
1.1 Problem definition
For the following data set, please discuss the possibility of obtaining a
reasonable interpolated value at x  2.3 , x  5.5 , and 7.2 via at least 4 different
interpolation formulas you are have learned in this semester.
x   0 1 2 3 4 5 6 7 8 9 10
y   0 0.12 0.112 0.231 0.183 0.121 1.235 1.560 1.603 1.812 1.9
1.2 Solution of the problem
Interpolation is a method of constructing new data points within the range of
a discrete set of known data points.
In engineering and science one often has a number of data points, as
obtained by sampling or experimentation, and tries to construct a function which
closely fits those data points. This is called curve fitting or regression analysis.
Interpolation is a specific case of curve fitting, in which the function must go
exactly through the data points.
First we have to plot data points, such plot provides better picture for
analysis than data arrays
Following four interpolation methods will be discussed in order to solve the
problem:

Linear interpolation

Method of least squares interpolation

Lagrange interpolating polynomial
Fig 1. Initial data points

Cubic spline interpolation
1.2.1 Linear interpolation
One of the simplest methods is linear interpolation (sometimes known as
lerp). Generally, linear interpolation tales two data points, say  xi , yi  and  xi 1 , yi 1  ,
and the interpolant is given by:
y  yi 
yi 1  yi
  x  xi  at the point  x, y 
xi 1  xi
Linear interpolation is quick and easy, but it is not very precise/ Another
disadvantage is that the interpolant is not differentiable at the point xi .
1.2.2 Method of least squares interpolation
The method of least squares is an alternative to interpolation for fitting a
function to a set of points. Unlike interpolation, it does not require the fitted
function to intersect each point. The method of least squares is probably best
known for its use in statistical regression, but it is used in many contexts unrelated
to statistics.
Fig 2. Plot of the data with linear interpolation superimposed
Generally, if we have n data points, there is exactly one polynomial of
degree at most n  1 going through all the data points. The interpolation error is
proportional to the distance between the data points to the power n. Furthermore,
the interpolant is a polynomial and thus infinitely differentiable. So, we see that
polynomial interpolation solves all the problems of linear interpolation.
However, polynomial interpolation also has some disadvantages. Calculating
the interpolating polynomial is computationaly expensive compared to linear
interpolation. Furthermore, polynomial interpolation may not be so exact after all,
especially at the end points. These disadvantages can be avoided by using spline
interpolation.
Example of construction of polynomial by least square method
Data is given by the table:
 x1 y1 


x2 y2 

D
 ... ... 


 xM y M 
Polynomial is given by the model:
M
f  x    w j  x j 1
j 1
In order to find the optimal parameters w  w1 ,..., w3
T
the following
substitution is being executed:
xi0  ai1 , xi1  ai 2 , …, xiM 1  aiM
 a11 ... a1M
Then: A   ... ... ...
a
 M 1 ... aMM





The error function:
M
S    f  w, xi   yi   Aw  y  min
2
2
i 1
It is necessary to find parameters w , which provide minimums to function
S:
S  Aw  y   Aw  y 
2
T
 Aw  y   yT y  yT Aw  wT AT y  wT AT Aw
S  yT y  2 yT Aw  wT AT Aw
S
 2 AT y  2 AT Aw  0
w
AT Aw  AT y  w   AT A
1
 A y
T
It should be noted that the matrix A must be nonsingular matrix.
For the given data points matrix B  AT A become singular, and it makes
impossible to construct polynomial with N  1 order, where N - number of data
points, so we will use N  9 polynomial
Fig 3. Plot of the data with polynomial interpolation superimposed
Because the polynomial is forced to intercept every point, it weaves up and
down.
1.2.3 Lagrange interpolating polynomial
The Lagrange interpolating polynomial is the polynomial P  x  of degree
  n  1 that passes through the n points
x , y
n
n
 f  xn   and is given by:
n
P  x    Pj  x  ,
j 1
Where
n
Pj  x   yi 
k 1
k j
x  xk
x j  xk
x , y
1
1
 f  x1   ,
x , y
2
2
 f  x2   , …,
Written explicitly
 x  x2  x  x3  ...  x  xn 
 x  x1  x  x3  ...  x  xn 
y1 
y  ...
 x1  x2  x1  x3  ...  x1  xn 
 x2  x1  x2  x3  ...  x2  xn  2
 x  x1  x  x3  ...  x  xn1 
... 
y
 xn  x1  xn  x2  ...  xn  xn1  n
P  x 
When constructing interpolating polynomials, there is a tradeoff between
having a better fit and having a smooth well-behaved fitting function. The more
data points that are used in the interpolation, the higher the degree of the resulting
polynomial, and therefore the greater oscillation it will exhibit between the data
points. Therefore, a high-degree interpolation may be a poor predictor of the
function between points, although the accuracy at the data points will be "perfect."
Fig 4. Plot of the data with Lagrange interpolating polynomial
interpolation superimposed
One can see, that Lagrange polynomial has a lot of oscillations due to the
high order if polynomial.
1.2.4 Cubic spline interpolation
Remember that linear interpolation uses a linear function for each of
intervals  xi , xi 1  . Spline interpolation uses low-degree polynomials in each of the
intervals, and chooses the polynomial pieces such that they fit smoothly together.
The resulting function is called a spline. For instance, the natural cubic spline is
piecewise cubic and twice continuously differentiable. Furthermore, its second
derivative is zero at the end points.
Like polynomial interpolation, spline interpolation incurs a smaller error
than linear interpolation and the interpolant is smoother. However, the interpolant
is easier to evaluate than the high-degree polynomials used in polynomial
interpolation. It also does not suffer from Runge's phenomenon.
Fig 5. Plot of the data with Lagrange interpolating polynomial
interpolation superimposed
It should be noted that cubic spline curve looks like metal ruler fixed in the
nodal points, one can see that such interpolation method could not be used for
modeling sudden data points jumps.
1.3 Results and discussion
The following results were obtained by employing described interpolation
methods for the points x1  2.3 ; x2  5.5 ; x3  7.2 :
Linear
Least squares
Lagrange
Cubic
Root mean
interpolation
interpolation
polynomial
spline
square
x1  2.3
0.148
0.209
0.015
0.14
0.146
x2  5.5
0.678
0.664
0.612
0.641
0.649
x3  7.2
1.569
1.649
1.479
1.562
1.566
Table 1. Results of interpolation by different methods in the given
points.
In order to determine the best interpolation method for the current case
should be constructed the table of deviation between interpolation results and root
mean square, if number of interpolations methods increases then value of RMS
become closer to the true value.
Linear
Least squares
Lagrange
Cubic spline
interpolation
interpolation
polynomial
x1  2.3
1.37%
43.15%
89.73%
4.11%
x2  5.5
4.47%
2.31%
5.7%
1.23%
x3  7.2
0.19%
5.3%
5.56%
0.26%
Average deviation from the RMS
2.01%
16.92%
33.66%
1.867%
Table 2. Table of Average deviation between average deviation and
interpolation results.
One can see that cubic spline interpolation gives the best results among
discussed methods, but it should be noted that sometimes cubic spline gives wrong
interpolation, especially near the sudden function change. Also good interpolation
results are provided by Linear interpolation method, but actually this method gives
average values on each segment between values on it boundaries.
Problem 2
2.1 Problem definition
For the above mentioned data set, if you are asked to give the integration of
y
between two ends a  0 and b  10 ? Please discuss the possibility accuracies of
all the numerical integration formulas you have learned in this semester.
2.2 Problem solution
In numerical analysis, numerical integration constitutes a broad family of
algorithms for calculating the numerical value of a definite integral.
There are several reasons for carrying out numerical integration. The
integrand f  x  may be known only at certain points, such as obtained by
sampling. Some embedded systems and other computer applications may need
numerical integration for this reason.
A formula for the integrand may be known, but it may be difficult or
impossible to find an antiderivative which is an elementary function. An example
of such an integrand is f  x   exp   x 2  , the antiderivative of which cannot be
written in elementary form.
It may be possible to find an antiderivative symbolically, but it may be
easier to compute a numerical approximation than to compute the antiderivative.
That may be the case if the antiderivative is given as an infinite series or product,
or if its evaluation requires a special function which is not available.
The following methods were described in this semester:

Rectangular method

Trapezoidal rule

Simpson's rule

Gauss-Legendre method

Gauss-Chebyshev method
2.2.1 Rectangular method
The most straightforward way to approximate the area under a curve is to
divide up the interval along the x-axis between x  a and x  b into a number of
smaller intervals, each of the same length. For example, if we divide the interval
into N subintervals, then the width of each one will be given by:
h
ba
N
The approximate area under the curve is then simply the sum of the areas of
all the rectangles formed by our subintervals:
N
I rect  h  f i
i 1
The summary approximation error for N intervals with width h is less than
or equal to
 rect 
h2  b  a 
24
max f   x 
Thus it is impossible to calculate maximum of the derivative function, we
can estimate integration error like value:
 rect
h2
12
 b  a   10  0   0.417
24
24
2.2.2 Trapezoidal rule
Trapezoidal rule is a way to approximately calculate the definite integral.
The trapezium rule works by approximating the region under the graph of the
function f  x  by a trapezium and calculating its area. It follows that
b

f  x  dx   b  a 
f  a   f b 
a
2
To calculate this integral more accurately, one first splits the interval of
integration  a, b into n smaller subintervals, and then applies the trapezium rule on
each of them. One obtains the composite trapezium rule:
b

f  x  dx 
a
b  a  f  a   f  b  N 1
 f

N 
2
k 1
b  a 

a  k

N 

The summary approximation error for N intervals with width h is less than
or equal to:
 trap 
ba 2
h  max f   x    trap
180
ba 2
h  0.056
180
2.2.3 Simpson's rule
Simpson's rule is a method for numerical integration, the numerical
approximation of definite integrals. Specifically, it is the following approximation:
b
 f  x  dx 
a
ba

ab
f a  4 f 
  f b 

6 
 2 

If the interval of integration  a, b is in some sense "small", then Simpson's
rule will provide an adequate approximation to the exact integral. By small, what
we really mean is that the function being integrated is relatively smooth over the
interval  a, b . For such a function, a smooth quadratic interpolant like the one used
in Simpson's rule will give good results.
However, it is often the case that the function we are trying to integrate is
not smooth over the interval. Typically, this means that either the function is highly
oscillatory, or it lacks derivatives at certain points. In these cases, Simpson's rule
may give very poor results. One common way of handling this problem is by
breaking up the interval  a, b into a number of small subintervals. Simpson's rule
is then applied to each subinterval, with the results being summed to produce an
approximation for the integral over the entire interval. This sort of approach is
termed the composite Simpson's rule.
Suppose that the interval  a, b is split up in N subintervals, with n an even
number. Then, the composite Simpson's rule is given by
b

a
N 1
N

2
2
h
f  x  dx   f  x0   2  f  x2 j   4 f  x2 j 1   f  xN 
3

j 1
j 1

The error committed by the composite Simpson's rule is bounded (in
absolute value) by
 sim 
h4
 b  a  max f  4     sim
180
h4
14
 b  a    10  0   0.056
180
180
2.2.4 Gauss-Legendre method and Gauss-Chebyshev method
Since function values are given in fixed points then just two points GaussLegendre method can be applied. If f is continuous on  1,1 , then
 1 

 f  x  dx  G  f   f  3   f 
1
2
1
1 

3
The Gauss-Legendre rule G2  f  G2( f ) has degree of precision n  3 . If
f  C
4
 1,1 , then
1
 1 
 1 
 f 
  E2  f  ,
3
 3
 f  x  dx  f 
1
where
E2  f  
f  4  c 
135
It should be noted that even in case of two points method we have to
calculate values of the function in related to x0 
1
3
, x1 
1
3
, this values could be
evaluated by linear interpolation (because it is necessary to avoid oscillations), so
estimation of integration error become very complicated process, but this error will
be less or equal to trapezoidal rule.
Mechanism of Gauss-Chebyshev method is almost the same like described
above, and integration error will be almost the same, so there is no reason to use
such methods for the current problem.
Problem 3
3.1 Problem definition
It is well known that the third order Runge-Kutta method is of the following
form
yn 1  yn 
1
 K1  4 K 2  K3  , K1  hf  xn , yn 
6
K 
h
h
h



K 2  hf  xn  , yn  1   hf  xn  , yn  f  xn , yn  
2
2 
2
2




K3  hf  xn  h, yn  K1  2K 2   hf  xn  h, yn  hf  xn , yn   2hf

h
h


 xn  , yn  f  xn , yn   
2
2


Suppose that you are asked to derived a new third order Runge-Kutta
method in the following from
yn 1  yn 
1
 K1  4 K 2  K3  , K1  hf  xn , yn 
6
h 1   
K 1    

K2  hf  xn 
, yn  1

2
2


K3  hf  xn  h, yn  1    K1  2 1    K2 
Find determine the unknowns  ,  ,  and  so that your scheme is a third
order Runge-Kutta method.
3.2 Problem solution
Generally Runge-Kutta method looks like:
q
y  x  h   y  x    Pi K i  h   z  h  ,
i 1
where coefficients K i could be calculated by the scheme:
K1  hf  xn , yn 
K2  hf  xn  2 h, yn  21 K1  h  
K3  hf  xn  3 h, yn  31 K1  h   32 K2  h  
The error function:
  h  y  x  h  z  h
Coefficients
i ,
i ,
Pi
must
be
found
to
satisfy
conditions
   0      0  ...    s   0   0 , consequently we can suppose that for each order of
Runge-Kutta scheme those coefficients are determined uniquely, it means that
there are no two different third order schemes with different coefficients. Now it is
necessary to prove statement.
For s  3 , q  3 :
  h   y  x  h   y  x   P1 K1  h   P2 K 2  h   P3 K 3  h  
 
 
 y  x  h   y  x   Phf
 x, y   P2 hf x, y  P3 hf x, y
1
x  x  2 h ; y  y  21hf  x, y 
x  x   3 h ; y  y  31hf  x, y   32 hf  x   2 h, y   21 hf  x, y 
y   f  x, y  ; y  f x  x, y   f y  x, y   f  x, y  ; y  f xx  2 f xy f  f y f x  f yy f 2  f y 2 f
 
 
   h   y   x  h   P1 f  x, y   P2 f x, y  P3 f x, y  O  h 
 
 
   h   y   x  h   2 P2  2 f x x, y   21 f y x, y  f  x, y   


 
 


  2 f x x, y 
 



  f   O  h 
2 P3  3 f x x, y  31 f  x, y   32 f x, y   32 h


 y 


f
x
,
y

f
x
,
y



21
y

  


 
 
   h   y   x  h   3P2
 
2
2
f x, y  3P3 2 f x, y  O  h 
h2
h
 
   0  y  P1 f  P2 f  P3 f  0  1  P1  P2  P3  0
   0  1  2P22  2P33  f x  1  2P2 21  2P3 31  2P3 32  f y f  0
1  2P2 2  2P33  0 ; 1  2P2  21  2P3 31  2P3  32  0
 32 f xx   31  32  ff y f xy 3

 2 32 2 f y f x   32  21 ff y 2 

   0   y   x   3P2  2 2 f xx  2 2  21 ff xy   212 f yy f 2   3P3    31   32   3 ff xy 

   31  32   3 ff x f yy 

2
2
   31  32  f f y f yy









1  3P2 2 2  3P3 32  0 ; 1  6P3 32  21  0 ; 1  6P3 32 2  0
Thus we have system of equations:
1  2 P2 2  2 P3 3  0
1  2 P   2 P   2 P   0
2 21
3 31
3 32


2
2
1  3P2 2  3P3 3  0
1  6 P    0
3 32 21

1  6 P3 32 2  0
Some of coefficients are already predefined:
P1  1 ;
6
P2  2 ;
3
1
P3  1 ;  2 
;  3  1;
6
2
32  2 1   
2 1  
1
2
2
1 2  
  2   1  0   1        0
3  2 
6
3
3
 21 
1 
;
2
31   1    ;
2 1  
1 2
1 3 
  3  1  0    0
3  2 
6
2
1
1 

1  6   2 1    
 0

6
2
      0   0
1
1
1  6   2 1    
 0

6
2
2 1  
1
1
1 2  
  2   1     2   2  1     0    0
3  2 
6
6
Obtained results show that Runge-Kutta scheme for every order is unique.
Problem 4
4.1 Problem definition
Discuss the stability problem of solving the ordinary equation y    y ,
y  0   1 via the Euler explicit scheme
  10 , say
yn1  yn   yn  x . If
  10    x  , how to apply your stability restriction?
4.2 Problem solution
The Euler method is 1st order accurate. Given scheme could be rewritten in
form of:
yn1  1    x  yn
If 1    has a magnitude greater than one then yn will tend to grow with
increasing n and may eventually dominate over the required solution. Hence the
Euler method is stable only if 1     1 or:
1  1    x   1  2    x  0
For the case   10 mentioned above inequality looks like:
2  10  x  0  0  x  0.2
Last result shows that integration step mast be less or equal to 0.2 .
For the case   10    x  , for the iteration method coefficient looks like
  10    xn 
2    xn   10   x  0  0  x 
2
10    xn 
As step x is positive value of the function   x  must be less then 10 . There
are two ways to define the best value of step x , the firs one is to define maximum
value of function   x  on the integration area, another way is to use different xn
for each value   xn  , where xn  a  n  x . So integration step is strongly depends
on value of   x  .
References
1. J. C. Butcher, Numerical methods for ordinary differential equations, ISBN
0471967580
2. George E. Forsythe, Michael A. Malcolm, and Cleve B. Moler. Computer
Methods for Mathematical Computations. Englewood Cliffs, NJ: Prentice-Hall,
1977. (See Chapter 6.)
3. Ernst Hairer, Syvert Paul Nørsett, and Gerhard Wanner. Solving ordinary
differential equations I: Nonstiff problems, second edition. Berlin: Springer
Verlag, 1993. ISBN 3-540-56670-8.
4. William H. Press, Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling.
Numerical Recipes in C. Cambridge, UK: Cambridge University Press, 1988. (See
Sections 16.1 and 16.2.)
5. Kendall E. Atkinson. An Introduction to Numerical Analysis. John Wiley &
Sons - 1989
6. F. Cellier, E. Kofman. Continuous System Simulation. Springer Verlag, 2006.
ISBN 0-387-26102-8.
7. Gaussian Quadrature Rule of Integration - Notes, PPT, Matlab, Mathematica,
Maple, Mathcad at Holistic Numerical Methods Institute
8. Burden, Richard L.; J. Douglas Faires (2000). Numerical Analysis (7th Ed. ed.).
Brooks/Cole. ISBN 0-534-38216-9.