EE1 MATHEMATICS
NUMERICAL INTEGRATION
G.A. Pavliotis
Department of Mathematics
Imperial College London
1. Trapezium rule.
2. Simpson’s rule.
3. Richardson extrapolation.
Numerical integration
• Most mathematical models in science and engineering involve
differential equations.
• The solution of a differential equation can be expressed in
terms of an integral. For example
Z
dy
= f (x) ⇒ y(x) = f (x) dx.
dx
• For most integrals no representation in terms of elementary
functions is possible, and numerical approximation becomes
necessary.
• We will study methods for approximating definite integrals of
the form
Z b
I=
f (t) dt.
(1)
a
Numerical integration
• The basic idea behind the numerical integration methods that
we will develop is to approximate the function f (t) locally by a
simple function (in particular, a polynomial) for which we can
calculate the integral exactly.
• We also want to estimate the error.
• Approximate the function f (t) locally by a linear function (first
degree polynomial) ⇒ Trapezium rule.
• Approximate the function locally by a quadratic function
(second degree polynomial) ⇒ Simpson’s rule.
• Combine local approximations to carry out ”error-balancing”
⇒ Richardson’s extrapolation.
Trapezium rule
Consider a stip of width h (which we will eventually take to be
small). We are given the value of f (t) at the end points 0 and f (h).
We approximate the function f (t) in the interval [0, h] by a straight
line joining the points (0, f (0)) and (h, f (h)):
f (h) − f (0)
f (t) ≈ f (0) + t
.
h
(2)
We can now approximate the intergral of f (t) from 0 t0 h:
Z h
Z h
f (h) − f (0)
dt
f (t) dt ≈
f (0) + t
h
0
0
h2 f (h) − f (0)
h
h
= hf (0) +
= hf (0) + f (h) − f (0)
2
h
2
2
h
=
(f (0) + f (h)) .
2
Trapezium rule
The numerical integration method based on the approximation
Z h
h
f (t) dt ≈ (f (0) + f (h))
2
0
is called the trapezium rule. This is because, when f (t) > 0, we
are approximating the area under f (t) by the area of a trapezium.
• Consider the function f (x) = x cos(x) + 1 in the interval [0, 1].
• The linear approximation (2) of this function is
f (x) ≈ f (0) + x(f (1) − f (0)) = 1 + 0.543x.
• The trapezium rule gives:
Z 1
1
(x cos(x) + 1) dx ≈ (f (0) + f (1)) = 1.2715.
2
0
• The correct value is 1.3818.
• The approximation is not very good: h = 1 is too large!
Trapezium rule
2
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Trapezium rule
The trapezium rule is a good approximation for an integral
provided that h is sufficiently small: Let T (h) denote the trapezium
approximation:
h
T (h) := (f (0) + f (h)).
2
We can write
Z
h
f (t) dt = T (h) + ǫT (h),
(3)
0
where ǫT (h) is the error of the trapezium approximation:
EXACT = APPROXIMATION + ERROR.
We want to estimate the error ǫT (h).
Result: The local error of the trapezium rule is
1 3 ′′
ǫT (h) = − h f (ξ),
12
ξ ∈ (0, h).
(4)
Trapezium rule
To prove this result, we use (something similar to) the Taylor series
expansion with a remainder:
f (h) − f (0) 1 ′′
+ f (ξ)t(t − h),
f (t) = f (0) + t
h
2
for some ξ ∈ (0, h). Consequently:
Z h
1 ′′
ǫT (h) =
f (ξ)t(t − h) dt
0 2
3
1 ′′
t
t3 h
=
f (ξ)
−
2
3
2
0
1
= − f ′′ (ξ)h3 .
12
Trapezium rule
The trapezium rule works well when the interval of integration is
small, h ≪ 1. What should we do if we want to calculate an
integral over an interval of arbitrary length?:
Z T
I=
f (t) dt.
0
T
Idea: Break the interval [0, T ] into N intervals of width h = N
and
use the trapezium rule at each small interval ⇒ approximate f (t)
locally by linear functions.
Suppose we are given the values of f (t) at the points a and b. The
equation of the straight line that passes through the points a and b
is:
bf (a) − af (b) f (b) − f (a)
f (t) ≈
+
t, t ∈ [a, b].
(5)
b−a
b−a
Trapezium rule
The trapezium rule for the integral of f (t) from a to b gives:
Z b
Z b
bf (a) − af (b) f (b) − f (a)
+
t dt
f (t) dt ≈
b
−
a
b
−
a
a
a
bf (a) − af (b) b f (b) − f (a) t2 b
=
t +
b−a
b
−
a
2
a
a
bf (a) − af (b)
f (b) − f (a) 1 2
=
(b − a) +
(b − a2 )
b−a
b−a
2
1
= bf (a) − af (b) + (f (b) − f (a))(b + a)
2
b−a
=
(f (a) + f (b)).
2
Let h = b − a. The error is
1 ′′
ǫT (h) = − f (ξ)h3 ,
12
some ξ ∈ (a, b).
Trapezium rule
Let a = nh, b = (n + 1)h:
Z nh
h
h
f (t) dt ≈ (f ((n − 1)h) + f (nh)) := (fn−1 + fn ).
2
2
(n−1)h
Consequently:
Z T
Z
f (t) dt =
0
=
=
h
f (t) dt +
0
Z
2h
f (t) dt +
h
Z
3h
f (t) dt + . . .
2h
Z
Nh
f (t) dt
(N −1)h
h
h
h
h
(f0 + f1 ) + (f1 + f2 ) + (f2 + f3 ) + . . . (fN −1 + fN )
2
2
2
2
h
(f0 + 2f1 + 2f2 + · · · + fN ) .
2
This is sometimes called the composite trapezium rule.
Result The global error of the trapezium rule is
ET (h) = −
T ′′
f (ξ)h2 = O(h2 ).
12
Trapezium rule
• Example: f (x) = x cos(x) + 1,
• Choose h =
1
4
x = [0, 1].
and use (5) in the intervals
1
1 1
1 3
3
0,
,
,
,
,
,
,1 .
4
4 2
2 4
4
• Consequently:
f (x) ≈
=
f (x) ≈
=
f (x) ≈
=
f (x) ≈
=
f (0) + 4(f (1/4) − f (0))x
1 + 0.9689x,
x ∈ [0, 1/4],
2f (1/4) − f (1/2) + 4(f (1/2) − f (1/4))x
1.0457 + 0.7863x,
x ∈ [1/4, 1/2],
3f (1/2) − 2f (3/4) + 4(f (3/4) − f (1/2))x
1.2188 + 0.4399x,
x ∈ [1/4, 1/2],
4f (3/4) − 3f (1) + 4(f (1) − f (3/4))x
1.5742 − 0.0339x,
x ∈ [1/4, 1/2],
Trapezium rule
2
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Trapezium rule
• Example: f (x) = x cos(x) + 1,
x = [0, 1].
• We apply the trapezium rule to calculate the intrgral of f (x)
from 0 to 1 with h = 14
Z 1
h
f (0) + 2f (1/4) + 2f (1/2) + 2f (3/4) + f (1)
I :=
f (x) dx ≈
2
0
1
1.0000 + 2 × 1.2422 + 2 × 1.4388
8
+2 × 1.5488 + 1.5403
= 1.3750.
• The correct value is I = 1.3818 ⇒ our error is 0.5%.
Simpson’s rule
Simpson’s rule is based on the local approximation of a function
f (t) by a quadratic function.
Consider a stip of width 2h. We are given the value of f (t) at the
end points 0 and f (2h) and at the midpoint f (h). We approximate
the function f (t) in the interval [0, 2h] by a quadratic function that
passes through the points
(0, f (0)),
(h, f (h)),
(2h, f (2h)) :
(6)
f (t) ≈ At2 + Bt + C =: P2 (t)
The quadratic function P2 (t) passes through the 3 points in (6).
This leads to a system of 3 equations for the 3 unknowns A, B, C:
0A + 0B + C
= f (0),
h2 A + hB + C
= f (h),
4h2 A + 2hB + C
= f (2h).
Simpson’s rule
We can solve this system of equations by studying the augmented
system
0
0
1
f (0)
h2
h
1
f (h)
4h2
2h 1
f (2h)
R3 ↔ R1
R2 → R2 − R3 ,
4h2
2h 0
f (2h) − f (0)
h2
h
0
f (h) − f (0)
0
0
1
f (0)
0
f (2h)
h2
h
1
f (h)
0
0
1
f (0)
R1 → R1 − R3
0
R1 →
h2
h 0
R1 − 2R2
0
0
0 f (2h) − 2f (h) + f (0)
f (h) − f (0)
1
R2 → R2 /h
0 (f (2h) − 2f (h) + f (0))/(2h2 )
h 1
0
(f (h) − f (0))/h
0
1
f (0)
0
2h 1
2h2
R1 → R1 /(2h2 ),
1
4h2
R2 → R2 − hR1
f (0)
Simpson’s rule
1 0 0
(f (2h) − 2f (h) + f (0))/(2h2 ))
0 1 0
(4f (h) − f (2h) − 3f (0))/(2h)
0 0 1
f (0)
Consequently:
A = (f (2h) − 2f (h) + f (0))/(2h2 )),
B = (4f (h) − f (2h) − 3f (0))/(2h),
C = f (0)
and
f (t) ≈ P2 (t)
2
1
1
=
f2 − 2f1 + f0 t +
4f1 − f2 − 3f0 t + f0 , (7)
2
2h
2h
for t ∈ [0, 2h]. We have used the notation
fj := f (jh), j = 0, 1, 2, . . . .
Simpson’s rule
We use the quadratic approximation to calculate the integral of
f (t) from 0 to 2h:
Z 2h
Z 2h
I=
f (t) dt ≈
P2 (t) dt
0
0
=
=
=
=
=
Z
2h
0
At2 + Bt + C dt
8 3
Ah + 2Bh2 + 2Ch
3
8 1 3
f
−
2f
+
f
h
2
1
0
3 2h2
1 +2
4f1 − f2 − 3f0 h2 + 2f0 h
2h
4
8
4
f2 − f1 + f0 + 4f1 − f2 − 3f0 + 2f0 h
3
3
3
1
4
1 f2 + f1 + f0 h.
3
3
3
Simpson’s rule
The numerical integration method based on the approximation
Z 2h
h
f (t) dt ≈ (f2 + 4f1 + f0 )
(8)
3
0
is called the Simpson’s rule. It is one of the most commonly used
methods for numerical integration.
• Consider the function f (x) = x cos(x) + 1 in the interval [0, 1].
• Let h = 12 . The quadratic approximation (2) of this function is
f (x) ≈ f (0) + x(4f (1/2) − f (1) − 3f (0)) + 2(f (1) − 2f (1/2) + f (0))
= 1 + 1.2149x − 0.6746x2 .
• Simpson’s rule gives:
Z 1
1
(x cos(x) + 1) dx ≈ (f (0) + 4f (1/2) + f (1)) = 1.3826.
6
0
• The correct value is 1.3818. The error is 0.06%!
Simpson’s rule
Simpson’s rule is a good approximation for an integral provided
that h is sufficiently small: Let S(h) denote Simpson’s
approximation:
h
S(h) := (f2 + 4f1 + f0 ) .
3
We can write
Z
2h
f (t) dt = S(h) + ǫS (h),
(9)
0
where ǫS (h) is the error of the Simpson approximation:
EXACT = APPROXIMATION + ERROR.
We want to estimate the error ǫS (h).
Result: The local error of Simpson’s rule is
ǫS (h) = −
1 5 (4)
h f (ξ),
90
ξ ∈ (0, h).
Simpson’s rule is exact for all polynomials of degree up to 3.
(10)
Simpson’s rule
Suppose we are given the values of f (t) at the points a and b as well
as the midpoint a+b
2 . The quadratic polynomial that that passes
through the points (a, f (a)), ((a + b)/2, f ((a + b)/2)), (b, f (b)) is:
f (t)
≈
−4f ( a+b
2 ) + 2f (a) + 2f (b) 2
t
2
(a − b)
−f (b)(3a + b) + 4f ( a+b
2 )(a + b) − f (a)(3b + a)
+
t
2
(a − b)
f (a)b(b + a) + f (b)a(a + b) − 4f ( a+b
2 )ba
+
.
(a − b)2
(11)
Simpson’s rule
We can obtain this formula by obtaining a linear system of
equations for the unknowns A, B, C in the quadratic
approximation:
f (t) = At2 + Bt + C.
The resulting system of equations is:
Aa2 + Ba + C
2
a+b
a+b
A
+B
+C
2
2
Ab2 + Bb + C
= f (a)
a+b
= f(
)
2
= f (b).
Simpson’s rule
Simpson’s rule for the integral of the function f (t) between a and b
gives:
Z b
b−a
a+b
f (t) dt ≈
f (a) + 4f
+ f (b) .
(12)
6
2
a
The local approximation error is
(b − a)5 (4)
f (ξ),
ǫS (h) = −
2880
for some ξ ∈ [a, b].
In the case a = 0, b = 2h the formula above reduces to
1 5 (4)
ǫS (h) = − h f (ξ),
90
ξ ∈ (0, h).
Simpson’s rule is exact for polynomials of degree up to 3. This is
because of the choice of the midpoint (a + b)/2, f ((a + b)/2).
Composite Simpson’s rule
Simpson’s rule works well when the interval of integration is small,
b − a ≪ 1. What should we do if we want to calculate an integral
over an interval of arbitrary length?:
Z T
I=
f (t) dt.
0
T
Idea: Break the interval [0, T ] into N intervals of width h = N
and
use the Simpson’s rule at each interval [(n − 1)h, (n + 1)h] ⇒
approximate f (t) locally by quadratic functions using the values of
the function at the points (n − 1)h, nh, (n + 1)h.
Composite Simpson’s rule
Let a = (n − 1)h, b = (n + 1)h:
Z (n+1)h
h
h
f (t) dt ≈ (f ((n−1)h)+4f (nh)+f ((n+1)h)) =: (fn−1 +4fn +fn+1 ).
3
3
(n−1)h
Consequently:
Z T
Z
f (t) dt =
0
2h
f (t) dt +
0
Z
4h
2h
f (t) dt +
Z
6h
f (t) dt + . . .
4h
Z
Nh
f (t) dt
(N −2)h
h
h
h
=
(f0 + 4f1 + f2 ) + (f2 + 4f3 + f4 ) + (f4 + 4f5 + f6 )
3
3
3
h
+ . . . (fN −2 + 4fN −1 + fN )
3
h
=
f0 + 4f1 + 2f2 + 4f3 + 2f4 + 4f5 + 2f6
3
+ · · · + 2f N − 2 + 4fN −1 + fN .
Composite Simpson’s rule
The composite Simpson’s rule is
Z T
h
f0 + 4f1 + 2f2 + 4f3 + 2f4 + 4f5 + 2f6
f (t) dt ≈
3
0
+ · · · + 2f N − 2 + 4fN −1 + fN .
Result The global error of Simpson’s rule is
T (4)
ES (h) = −
f (ξ)h4 = O(h4 ).
180
In particular:
Z
T
(4) T 4
f (t) dt − S(h) ≤
h max f (ξ).
0
180 ξ∈[0,T ]
Simpson’s composite rule is a very efficient method for numerical
integration, provided that f (t) is sufficiently regular in the interval
[0, T ].
Composite Simpson’s rule
• Example: Evaluate the integral
Z 8
I=
cos(sin(x)) dx = 6.089177900,
0
• using Simpson’s rule. Let us start with N = 4 ⇒ h = 8/N = 2:
Z 8
2
cos(sin(x)) dx =
(f (0) + 4f (2) + 2f (4) + 4f (6) + f (8))
3
0
= 6.203576586.
• With N = 8 ⇒ h = 8/N = 1:
Z 8
1
(f (0) + 4f (1) + 2f (2) + 4f (3) + 2f (4)
cos(sin(x)) dx =
3
0
+4f (5) + 2f (6) + 4f (7) + f (8))
= 6.081660794.
Composite Simpson’s rule
• The global approximation error of Simpson’s rule is bounded
from above by
Z
T
(4) T 4
f (t) dt − S(h) ≤
h max f (ξ)
0
180 ξ∈[0,T ]
≤
2 4
h := ES,theor (h),
9
• since T = 8 and maxξ∈[0,T ] |f (4) (ξ)| = 5.
h
Sh
ES (h)
ES,theor (h)
2
6.203576586
0.11439868592260
3.5556
1
6.08166079436034
0.00751710563967
0.2222
0.5
6.08904668518002
1.312 × 10−4
0.01389
0.25
6.08917016039564
7.7396 × 10−6
8.68056 × 10−4
Composite Simpson’s rule
• The actual error is much smaller than the theoretical error.
• The scaling of the error with h is the one predicted by the
theory:
ES (h) = Ch4 .
Composite Simpson’s rule
2
10
error
theor. error
0
10
−2
10
−4
10
−6
10
−8
10
−2
10
−1
10
0
10
1
10
Figure 2: Theoretical and actual error of Simpson’s rule for the
R8
integral 0 cos(sin(x)) dx.