Numerical Differentiation
and Integration
Adaptive Integration
 Adaptively
adjust interval size based on
function behavior or specified accuracy
smaller h
larger h
Numerical Methods © Wen-Chieh Lin
2
Adaptive Scheme
 Iteratively
subdivide the interval until the
specified accuracy is achieved
 Comparing
the magnitude of difference in previous
and current iterations
 Apply
Richardson extrapolation to further
improve accuracy; for an n-th order accurate
algorithm
more accurate less accurate
Better estimate more accurate 
n
2 1
Numerical Methods © Wen-Chieh Lin
3
Example: Adaptive Integration

http://www.cse.uiuc.edu/iem/integration/adaptivq/
Numerical Methods © Wen-Chieh Lin
4
Gaussian Quadrature

Also based on polynomial interpolation, but nodes
and weights are chosen to maximize degree of
resulting rule (or interpolant)
 x-values are not predetermined

For an n-term rule,
 there are 2n parameters  polynomial of degree
2n-1 can be obtained
 Only n function evaluations need be computed!

Node and weight parameters are determined using
method of undetermined coefficients
Numerical Methods © Wen-Chieh Lin
5
Example: Two term Gaussian quadrature
 Approximate
the integral by
1
f (t ) af (t ) bf (t )
1
1
 Apply
f (t ) t
3
f (t ) t
2
2
method of undetermined coefficients
1
3
3
3
t
dt

0

at

bt
1
2

1
f (t ) t
f (t ) 1
2
2
2
t
dt


at

bt
1
2


1
3
1
2
1
t dt 0 at
1
1
bt2
a b 1
t2 t1  1 3 0.5773
1
dt 2 a b

f (t ) f (0.5773) f (0.5773)

1
1

1
Numerical Methods © Wen-Chieh Lin
6
Change of interval
 If
the integral interval is [a, b], we can
transform it to [-1, 1]
(b a )t b a
x
2
b a
dx 
dt
2
(b a )t b a
f ( x )dx f (
)dt

a

1
2
b
1
Numerical Methods © Wen-Chieh Lin
7

2
0
Example: I sin xdx
(b a )t b a ( 2)t ( 2)
x

2
2
b a

dx 
dt 
2
4

2
0
1
t 
sin(
)dt
sin xdx 4 

1
4
1
f (t ) f (0.5773) f (0.5773)

1

 
sin(0.10566) sin(0.39434)
0.99847
4
3
Error 1.53 10
Numerical Methods © Wen-Chieh Lin
8
Gaussian Quadrature: n-term rule
 Generalization
to n-term rule
n
f (t ) w f (t )
1
1
i
1
i
i
 Determining
parameters by solving a set of 2n
nonlinear equations
0,
k 1,3,5,...,2n 1;


k
k
w1t1  wn tn  2
, k 0,2,4,...,2n 2.

k 1
Numerical Methods © Wen-Chieh Lin
9
Legendre Polynomial
 The
parameters of Gaussian quadrature can be
computed using Legendre polynomials
(n 1) Ln 1 ( x ) (2n 1) xLn ( x ) nLn 1 ( x ) 0
with L0 ( x ) 1, L1 ( x ) x
 Roots of the n-th degree Legendre polynomial
are the ti’
s for the system of equations
0,
k 1,3,5,...,2n 1;


k
k
w1t1  wn tn  2
, k 0,2,4,...,2n 2.

k 1
Numerical Methods © Wen-Chieh Lin
10
Computing weights in Gaussian quadrature
 Roots
of Legendre polynomial can be obtained
using root finding algorithms in Chap 1
3xL1 ( x ) L0 ( x ) 3 2 1
L2 ( x ) 
 x 
2
2
2
5 x 3x
L3 ( x ) 
2
3
 Replacing ti’
s by
the roots, the set of equations
for computing parameters becomes a system of
linear equations; solving it for weights
0,
k 1,3,5,...,2n 1;


k
k
w1t1  wn tn  2
, k 0,2,4,...,2n 2.

k 1
Numerical Methods © Wen-Chieh Lin
11
Values for Gaussian Quadrature
Numerical Methods © Wen-Chieh Lin
12
Multiple Integrals
 Extend
Newton-Cotes quadrature formula to
multi-dimensions
 Integrate on multiple dimensions one by one
 hold
all the other dimensions constant while
integrating with respect to y (or vice versa)
 For example,
d
d
b




f
(
x
,
y
)
dA

f
(
x
,
y
)
dy
dx

f
(
x
,
y
)
dx
dy








A
a c
c a


b
Numerical Methods © Wen-Chieh Lin
13
Example: Double Integral
0.6 3.0
Evaluating integral f ( x, y )dxdy
0.2 1.5
 Use trapezoidal rule in the x-direction

h
y 0.2 f ( x,0.2)dx  ( f1 2 f 2 2 f 3 f 4 )
1.5
2
0.5
 (0.990 2 1.568 2 2.520 4.090) 3.3140
2
3.0
Numerical Methods © Wen-Chieh Lin
14
Example (cont.)
0.6 3.0
Evaluating integral f ( x, y )dxdy
0.2 1.5
 Use trapezoidal rule in the x-direction

h
y 0.3 f ( x,0.3)dx  ( f1 2 f 2 2 f 3 f 4 )
1.5
2
0.5
 (1.524 2 2.384 2 3.800 6.136) 5.0070
2
3.0
Numerical Methods © Wen-Chieh Lin
15
Example (cont.)
0.6 3.0
Evaluating integral f ( x, y )dxdy
0.2 1.5
 Similarly,
y 0.4
I x 6.6522

y 0.5
I x 8.2368
y 0.6
I x 9.7435
 Apply Simpson’
s 1/3 rule in the y-direction
h
f ( y )dy  ( f1 4 f 2 2 f 3 4 f 4 f 5 )

0.2
3
0.6
0.1
 (3.3140 4 5.0070 2 6.6522 4 8.2368 9.7435)
3
Numerical Methods © Wen-Chieh Lin
16
Multiple Integrals—General Form
 Approximate
function by polynomial of
multiple variables up to degree s (=α+β+γ)
n
n
n
f ( x, y, z )dxdydz a a a x
1
1
1
1 
1 1

i 1 j 1 k 1
i
  
j k i
j k
y z
Generalization to n-dimensional integral is easy
 n summations
Numerical Methods © Wen-Chieh Lin
17
Monte Carlo Method

A generally viable approach in high dimensions

Can handle integral over an irregular-shape
domain

Function is sampled at n points distributed
randomly in domain of integration, and mean
function values is multiplied by area (or volume,
etc.) of domain to obtain estimate for integral

Often require millions of function evaluations
Numerical Methods © Wen-Chieh Lin
18
Example: Monte Carlo Method
 1-D
Monte Carlo
http://www.cse.uiuc.edu/iem/integration/mntcurve/
 2-D
Monte Carlo
http://www.cse.uiuc.edu/iem/integration/mntcirc/
f ( x, y )dxdy



1, x 2 y 2 1
f ( x, y ) 
2
2
0
,
x

y
1

Numerical Methods © Wen-Chieh Lin
19
When should I use Monte Carlo?
 Monte
Carlo method is not competitive for
dimensions one or two, but beauty of method
is that its convergence rate is independent of
number of dimensions
 For
example, one million points in six
dimensions amounts to only ten points per
dimension, which is much better than any
conventional quadrature rule would require for
same level of accuracy
Numerical Methods © Wen-Chieh Lin
20