Function Approximation
David Florian Hoyle
UCSC
[11/14] November 2014
Objective
Obtain an approximation for f (x ) by another function b
f (x )
Two cases:
f (x ) is known in all its domain, but it is very expensive to calculate
it.
f (x ) is known only in a finite set of points: Interpolation.
Outline
1
Approximation theory
1
2
3
4
2
Weierstrass approximation theorem
Minimax approximation
Orthogonal polynomials and least squares
Near minimax apoproximation
Interpolation
1
2
3
4
5
6
The interpolation problem
Different representations for the interpolating polynomial
The error term
Minimizing the error term with Chebyshev nodes
Discrete least squares again
Piececwise polynomial interpolation: splines
Approximation methods
We want to approximate f (x ) : [a, b ] → R by a linear combination
of polynomials
n
f (x ) ≈
∑ cj ϕ j ( x )
j =1
where
x ∈ [a, b ]
n : Degree of interpolation
cj : Basis coefficients
ϕj (x ) : Basis functions which are polynomials of degree ≤ n.
Approximation methods: Introduction
When trying to approximate f (x ) : C ⊂ R → R by
f (x ) ≈ ∑nj=1 cj ϕj (x )
We need to rely on a concept of distance between f (x ) and
∑nj=1 cj ϕj (x ) at the points we choose to make the approximation.
We are dealing with normed vector spaces of functions which can be
finite or infinite dimensional.
The space of continuos functions in Rn , the space of infinite
sequences l p , the space of measurable functions Lp and so on
Approximation methods: Introduction
If we define an inner product in this spaces then we have the
induced norm L2 :

 21
Zb
kf (x )kL2 =  f (x )2 w (x ) dx 
a
it is usefull for least squares approximation.
Approximation theory, is based on uniform convergence of f (x ) to
∑nj=1 cj ϕj (x ) . In this case, we use the supreme norm
kf (x )k∞ = max |f (x )|
x ∈[a,b ]
Approximation methods: Local vs Global
Local approximations are based on the Taylor approximation
theorem.
Global approximations are based on the Weierstrass approximation
theorem.
Approximation methods: Weierstrass approximation
theorem
Theorem
Let f (x ) be a continuous function on [a, b ] (i.e., f ∈ C [a, b ]) , then for
all e > 0,there exists a sequence of polynomials pn (x ) of degree ≤ n
that converges uniformly to f (x ) on [a, b ]. That is, ∀e > 0, ∃N ∈ N
and polynomials pn (x ) such that
∀n ≥ N, ∀x ∈ [a, b ] then kf (x ) − pn (x )k∞ ≤ ε
Where k·k∞ is the sup norm or L∞ norm:
kf (x ) − pn (x )k∞ = max |f (x ) − pn (x )|
x ∈[a,b ]
In other words,
lim pn (x ) = f (x )
n−→∞
for all x ∈ [a, b ]
Approximation methods: Weierstrass approximation
theorem
Another version
Theorem
if f ∈ C [a, b ],then for all e > 0 there exists a polynomial p (x ) such that
kf (x ) − pn (x )k∞ ≤ ε ∀x ∈ [a, b ]
This theorem tell us that any continuous function on a compact
interval can be approximated arbitrarily well by polynomials of any
degree.
Approximation methods: Weierstrass approximation
theorem
A constructive proof of the theorem is based on the Bernstein
polynomials defined on [0, 1]
n
k
n k
n −k
pn (x ) = ∑ f
x (1 − x )
n
k
k =0
such that
lim pn (x ) = f (x )
n−→∞
for all x ∈ [0, 1]
uniformly.
Weierstrass theorem is conceptually valuable but it is not
practical. From a computational point perspective it is not efficient
to work with Bernstein polynomials.
Bernstein polynomials converge very slowly!
Approximation methods: Minimax approximation
Also called the Best polynomial approximation
We want to find the polynomial pn (x ) of degree ≤ n that best
approximates a function f (x ) uniformly with f (x ) ∈ C [a, b ] :
For this, we are going to look for the infimun of the distance
between f and all possible degree ≤ n polynomial approximations
Recall that the uniform error term (i.e., using the L∞ norm) is
kf (x ) − pn (x )k∞ = max |f (x ) − pn (x )|
x ∈[a,b ]
Define dn (x ) as the infimun of the distance between f and all
possible pn (x ) approximations of f
dn (f ) = inf kf − pn k∞
pn
= inf max |f (x ) − pn (x )|
pn
x ∈[a,b ]
Approximation methods: Minimax approximation
Let pn∗ (x ) be the polynomial for which the infimum is obtained
dn (f ) = kf − pn∗ k∞
It can be shown that pn∗ exists, it is unique and it is characterized
by a property called the equioscillation property.
Algorithms to compute pn∗ are difficult/complex/not efficients.
Example: Remez algorithm.
Standard solution: Chebyshev least squares polynomial
approximation closely approximate the minmax or best
polynomial.
This strategy is called: Near minmax approximation.
Approximation methods: Minimax approximation
Existence and uniqueness of the minimax polynomial:
Theorem
Let f ∈ C [a, b ]. Then for any n ∈ N there exists a unique pn∗ (x ) that
minimizes kf − pn k∞ among all polynomials of degree ≤ n.
Sadly, the proof is not constructive so we have to rely on an
algorithm that help us compute pn∗ (x ) .
Approximation methods: Minimax approximation
But we can characterize the error generated by the minmax
polynomial, pn∗ (x ) in terms of its oscillation property:
Theorem
Let f ∈ C [a, b ]. The polynomial pn∗ (x ) is the minimax polynomial of
degree n that approximates f (x ) in [a, b ], if and only if, the error
∗
f (x ) − pn∗ (x ) , assumes the values +
− kf − pn k∞ with an alternating
change of sign in at least n + 2 points in [a, b ]. That is, in
a ≤ x0 < x1 < ... < xn+1 ≤ b we have the following
f (x ) − pn∗ (x ) = (−1)j kf − pn∗ k∞
for j = 0, 1, 2, ...n + 1
Approximation methods: Minimax approximation
The theorem give us the desired shape of the error when we want
L∞ approximation (the best approximation).
For example it says that the maximum error of a cubic
approximation should be achieved at least five times and that the
sign of the error should alternate between these points.
Approximation methods: Minimax approximation
Example 1: Consider the function f (x ) = x 2 in the interval [0, 2].
Find the minimax linear approximation (n = 1) of f (x ) .
Consider the approximating polynomial p1 (x ) = ax + b and the
errors
e (x ) = x 2 − ax − b Notice that the function e (x ) = x 2 − ax − b has a maximun when
2x − a = 0, that is when x = 2a .
x=
a
2
belongs to the interval [0, 2] when 0 ≤ a ≤ 4.
2
2
2
2
Evaluating e 2a = 2a − a 2a − b = a4 − a2 − b = − a4 − b
According to the oscillation property we need two more extremun
points:
Let’s take x = 0 and x = 2. Thus e (2) = 4 − 2a − b and
e (0) = −b.
Approximation methods: Minimax approximation
Example 1 (cont.):
According to the oscillation property we must have
h (0) = −h ( 2a ) = h (2) which implies that
h (0) = h (2)
−b = 4 − 2a − b
a=2
and
h (0) = −h
a
2
a2
−b =
+b
4
but we already know that a = 2 thus
b=−
1
2
Approximation methods: Minimax approximation
Example 1 (cont.):
Then , the approximating polynomial is
p1∗ (x ) = 2x −
1
2
and the maximun error
max x 2 − 2x −
[0,2]
1
1 =
2
2
If you try to find the quadratic minmax approximation to
f (x ) = x 2 , you will notice that things start to get complicated.
There is no general characterization-based algorithm to
compute the minimax polynomial approximation.
The Remez algorithm is based on known optimal
approximation for certain f (x ) .
Approximation methods: Minimax approximation
Example 1 (cont.):Oscillating property of the error for a minmax
approximation e (x ) = x 2 − 2x − 21 for x ∈ [0, 2]
v
Approximation methods: Least squares
Some basic concepts:
Weighting function: w (x ) on [a, b ] is any function that is positive
and has a finite integral over [a, b ],that is
Zb
w (x ) dx < ∞
a
Inner product relative to w (x ): Let f , g ∈ C[a, b ] then the inner
product is
hf , g i =
Zb
f (x ) g (x ) w (x ) dx
a
The inner product induces a norm given by:

Zb
 21
kf (x )kL2 =  f (x )2 w (x ) dx 
a
Approximation methods: Least squares
Assume a general family of polynomials
consititute a basis for C (X ) .
n
ϕj (x ) j =0 that
Assume that f (x ) is defined on [a, b ]. We want to approximate f by
a linear combination of polynomials: pn (x ) = ∑nj=0 cj ϕj (x )
Define the error as
E (x ) = f (x ) − pn (x )
The least squares approximation problem is to find the
polynomial of degree ≤ n that is closest to f (x ) in the L2 −norm
among all the polynomials of degree ≤ n.

Zb
 21
min kE (x )kL2 = min  E (x )2 w (x ) dx 
fb
fb
a
Approximation methods: Least squares
Notice that minimizing the L2 −distance between f and pn (x )
(i.e., kE (x )kL2 ) is equivalent to minimizing the square of the
L2 −distance, given by kE (x )k2L2 :
min kE (x )k2L2 = min
fb
fb
= min
fb
Zb
E (x )2 w (x ) dx
a
Zb
(f (x ) − pn (x ))2 w (x ) dx
a
The above problem is the continuos least square problem
Approximation methods: Least squares
Since pn (x ) = ∑nj=0 cj ϕj (x ) then solve the following problem
Zb
n
minn
w ( x ) f ( x ) − ∑ cj ϕ j ( x )
{cj }j =0 a
j =0
!2
dx
FOC’s w.r.t ck :
Zb
2 w (x ) f (x ) −
a
n
∑ cj ϕ j ( x )
j =0
!
ϕk (x ) dx = 0
∀k = 0, 1, 2, ...n
Approximation methods: Least squares
The normal equations are given by
n
Zb
j =0
a
∑ cj
ϕk (x ) ϕj (x ) w (x ) dx =
Zb
f (x ) ϕk (x ) w (x ) dx
a
Or in terms of inner product
n
∑ cj
j =0
ϕk , ϕj = hf , ϕk i
∀k = 0, 1, 2, ...n
∀k = 0, 1, 2, ...n
Approximation methods: Least squares
More explicitely, the normal equations are a system of n linear
equations in n unknowns:
c0 h ϕ0 , ϕ0 i + c1 h ϕ0 , ϕ1 i + c2 h ϕ0 , ϕ1 i + ... + cn h ϕ0 , ϕn i = hf , ϕ0 i
c0 h ϕ1 , ϕ0 i + c1 h ϕ1 , ϕ1 i + c2 h ϕ1 , ϕ2 i + ... + cn h ϕ1 , ϕn i = hf , ϕ1 i
...
c0 h ϕn , ϕ0 i + c1 h ϕn , ϕ1 i + c2 h ϕn , ϕ1 i + ... + cn h ϕn , ϕn i = hf , ϕn i
Approximation methods: Least squares
In matrix form

h ϕ0 , ϕ0 i h ϕ0 , ϕ1 i
 h ϕ1 , ϕ0 i h ϕ1 , ϕ1 i


...
...
h ϕn , ϕ0 i h ϕn , ϕ1 i
...
...
...
...

c0
h ϕ0 , ϕn i
 c1
h ϕ1 , ϕn i 

  ...
...
cn
h ϕn , ϕn i
In more compact form
Hc = b



hf , ϕ0 i
  hf , ϕ1 i 
=

 

...
hf , ϕn i
Approximation methods: Least squares
Candidates for ϕj (x ) :
Monomials constitute a basis for C (X ) :
1, x, x 2 , x 3 , ..., x n
Orthogonal polynomials relative to w (x ):
Zb
a
ϕk (x ) ϕj (x ) w (x ) dx = 0
for k 6= j
Approximation methods: Least squares
If we use Monomials the approximation for f (x ) can be written as
n
pn (x ) =
∑ cj x j
j =0
The least squares problem is
Zb
minn
{ cj } j = 0 a
n
f (x ) −
∑ cj x
!2
j
dx
j =0
The FOC w.r.t ck :
Zb
−2 f (x ) x k dx + 2
a
Zb
a
n
∑ cj x j
j =0
!
x k dx = 0
∀k = 0, 1, 2, ...n
Approximation methods: Least squares
FOC’s are expressed as
Zb
Where
n
∑ cj x
!
j +k
dx =
Zb
f (x ) x k dx
Zb Zb
∑nj=0 cj x j +k dx = ∑nj=0 cj x j +k dx and
a
a
Zb
∀k = 0, 1, 2, ...n
a
j =0
a
j +k +1 b
j +k +1
j +k +1
x j +k dx = jx+k +1 = b j +k−+a 1
thus,
a
a
n
∑
j =0
b j +k +1 − aj +k +1
j +k +1
!
cj =
Zb
a
f (x ) x k dx
∀k = 0, 1, 2, ...n
Approximation methods: Least squares
The
is a linear system of equations with n + 1 unknowks
above
n
( cj j =0 ) and n + 1 equations.
If we assume that [a, b] =[0, 1] then the system of equations to
n
determine each of the cj j =0 becomes
n
∑
j =0
1
j +k +1
cj =
Z1
0
f (x ) x k dx
∀k = 0, 1, 2, ...n
Approximation methods: Least squares
Explicitely the system is given by
1
1
1
1
c0 +
c1 +
c3 + ... +
cn
k +1
k +2
k +3
k +n+1
=
Z1
0
f (x ) x k dx
∀k = 0, 1, 2, ...n
Approximation methods: Least squares
for k = 0
c0 +
Z1
1
1
1
c1 +
c3 + ... +
cn = f (x ) dx
2
3
n+1
0
for k = 1
Z1
1
1
1
1
c0 +
c1 +
c3 + ... +
cn = f (x ) xdx
2
3
4
n+2
0
for k = 2
Z1
1
1
1
1
c0 +
c1 +
c3 + ... +
cn = f (x ) x 2 dx
3
4
5
n+3
0
for k = n
Z1
1
1
1
1
f (x ) x n dx
n+1 c0 + n+2 c1 + n+3 c3 + ... + n+n+1 cn =
0
Approximation methods: Least squares
In matrix form, the system canbe written as


1
 1
 2
 1
 3

 ...
1
n +1
1
2
1
3
1
4
1
3
1
4
1
5
...
...
1
n +2
1
n +3
...
...
...
...
...
1
n +1
1
n +2
1
n +3
...
1
n +n +1












 

c0


c1 
 

c2 
=
 
...  


cn






Z1
Hc = b
H is a HILBERT MATRIX
f (x ) dx



0


Z1

f (x ) xdx 


0


Z1

f (x ) x 2 dx 


0


...


Z1

f (x ) x n dx 
0
or

Approximation methods: Least squares
Problems with linear systems of equations with Hilbert Matrices:
Hc = b
H is an ill-conditioned matrix: Increasing rounding errors as we
increase n.
Condition number increases as n increases.
Approximation methods: Least squares
n
If the polynomial family ϕj (x ) j =0 is orthogonal, that is
h ϕn , ϕm i = 0 for n 6= m then the system becomes


 
0
0 ...
0
c0
h ϕ0 , ϕ0 i
hf , ϕ0 i

  c1   h f , ϕ 1 i
0
0
h ϕ1 , ϕ1 i 0 ...


=

  ...  
...
...
... ...
...
...
0
0
0 ... h ϕn , ϕn i
cn
hf , ϕn i
In this case, the coefficients are given by
ck =
hf , ϕk i
h ϕk , ϕk i
Thus
n
pn (x ) =
∀k = 0, 1, 2, ...n
hf , ϕk i
ϕ (x )
ϕk , ϕk i j
h
j =0
∑




Approximation methods: Least squares
Computations for finding the coefficients {ck }nk =0 are easy to
perform when using a family of orthogonal polynomials to
approximate a function.
n
A family of orthogonal polynomials ϕj (x ) j =0 have always a
recursive representation which make computations even faster.
Orthogonal polynomials: Most common families
There are many families of orthogonal polynomials that are basis for
function spaces:
Chebyshev: For x ∈ [−1, 1]
Tn (x ) = cos (n arccos x )
Legendre: For x ∈ [−1, 1]
Pn ( x ) =
n
(−1)n d n 2
[
1
−
x
]
2n n! dx n
Laguerre: For x ∈ [0, ∞]
Ln ( x ) =
ex d n
x n e −x
n! dx n
Hermite: For x ∈ [−∞, ∞]
Hn (x ) = (−1)n e x
2
d n −x 2 e
dx n
Most of this polynomials come from the soultion of ”important”
difference equations.
Orthogonal polynomials: Most common families
go to inneficient matlab script!
orthogonal families.m
Orthogonal polynomials: Recursive representation
Theorem
Let { ϕn (x )}n∞=0 be an orthogonal family of monic polynomials on [a, b ]
relative to an inner product h·, ·i ,
hx ϕ (x ), ϕ (x )i
ϕ0 (x ) = 1, ϕ1 (x ) = x − h ϕ 0(x ), ϕ 0(x )i . Then { ϕn (x )}n∞=0 satisfies the
0
0
recursive scheme
ϕn+1 (x ) = (x − δn ) ϕn (x ) − γn ϕn−1 (x )
where
δn =
hx ϕn , ϕn i
h ϕn , ϕn i
and
γn =
h ϕn , ϕn i
h ϕ n −1 , ϕ n −1 i
The above theorem is the so colled Gram-Schimdt process to find an
orthogonal family of monic polynomials.
Orthogonal polynomials: Recursive representation
A Chebyshev polynomial is given by
Ti (x ) = cos (i arccos x )
for i : 0, 1, 2, ..., n
and it is defined for x ∈ [−1, 1].
Chebyshev polynomials have the follwoing recursive
representation:
T0 (x ) = 1
T1 (x ) = x
Ti +1 (x ) = 2xTi (x ) − Ti −1 (x )
for i = 2, 3, ..., n
Orthogonal polynomials: Recursive representation
Recursive representation of Chebyshev polinomials: Ti (x ) can
be wrriten as
Ti (x ) = cos (i θ )
for i : 1, 2, ..., n
where θ = arccos x.
Using trigonometric identities we have
Ti +1 (x ) = cos ((i + 1) θ ) = cos (i θ ) cos (θ ) − sin (i θ ) sin (θ )
Ti −1 (x ) = cos ((i − 1) θ ) = cos (i θ ) cos (θ ) + sin (i θ ) sin (θ )
thus
Ti +1 (x ) = cos (i θ ) cos (θ ) − Ti −1 (x ) + cos (i θ ) cos (θ )
= 2 (cos (i θ ) cos (θ )) − Ti −1 (x )
Notice that cos (θ ) = cos (arccos x ) =⇒ cos (θ ) = x and that
cos (i θ ) = Ti (x ) . Therefore
Ti +1 (x ) = 2xTi (x ) − Ti −1 (x )
for i = 2, 3, ..., n
Orthogonal polynomials: Recursive representation
Recursive representation of Chebyshev polinomials (cont.):
The first five Chebyshev polynomials are
T0 (x ) = 1
T1 (x ) = x
T2 (x ) = 2xT1 (x ) − T0 (x ) = 2x 2 − 1
T3 (x ) = 2xT2 (x ) − T1 (x ) = 2x 2x 2 − 1 − x
= 4x 3 − 3x
T4 (x ) = 2xT3 (x ) − T2 (x ) = 2x 4x 3 − 3x − 2x 2 − 1
= 8x 4 − 8x 2 + 1
Chebyshev polynomials are polynomials with leading coefficient of
2n−1
Orthogonal polynomials: Recursive representation
Recursive representation of Chebyshev monic polinomials
(cont.):
The monic Chebyshev polynomial is defined as
en (x ) = cos (n arccos x )
T
2n−1
for n > 1
Using the general recursive formula for orthogonal monic polynomials
we obtain the recursion shceme for the monic Chebyshev polynomial
e0 (x ) = 0
T
e1 (x ) = x
T
e2 (x ) = x T
e0 (x )
e1 (x ) − 1 T
T
2
and
en+1 (x ) = x T
en (x ) − 1 T
en−1 (x ) for n ≥ 2
T
4
Orthogonal polynomials: Recursive representation
Going from the monic recursive representation to the original
Chebyshev representation:
2n e
1 2n−2 e
2n−1 e
T
x
−
Tn−1 (x ) for n ≥ 2
T
x
=
x
(
)
(
)
n
n
+
1
2n
4 2n−2
2n − 1
1 1
1
1
Tn+1 (x ) = x n−1 Tn (x ) −
Tn−1 (x )
2n
4 2n−2
2
1 2n
2n
Tn−1 (x )
Tn+1 (x ) = x n−1 Tn (x ) −
4 2n−2
2
1
= 2xTn (x ) − 22 Tn−1 (x )
4
= 2xTn (x ) − Tn−1 (x )
Orthogonal polynomials
The weigthing function for a Chebyshev polynomial must satisfy
the orthogonality condition
Z1
Ti (x ) Tj (x ) w (x ) dx =
−1
0 for i 6= j
Cj for i = j
where Cj is a constant
Integrating by substitution and solving for w (x ) yields
w (x ) = √
1
1 − x2
Orthogonal polynomials
Orthogonality property of Chebyshev polynomials:

Z1
0 for i 6= j

π
for i = j 6= 0
Ti (x ) Tj (x ) w (x ) dx =
 2
π
for
i =j =0
−1
Orthogonal polynomials: Chebyshev least squares
Approximate f ∈ C([−1, 1]) with Chebyshev polinomials.
Coefficients of the continuous least squares are given by
ck =
h f , Tk i
hTk , Tk i
Z1
=
∀k = 0, 1, 2, ...n
f (√
x ) Tk ( x )
dx
1−x 2
−1
Z1
Tk ( x ) 2
√
dx
1−x 2
−1
where Tk (x ) = cos (k arccos x ) or it is given by its recursive version.
Notice that the denominator has two cases: For k = 0 →
Z1
−1
T0 ( x ) 2
√
dx
1−x 2
= π and for k = 1, 2, ..., n →
Z1
−1
Tk ( x ) 2
√
dx
1−x 2
=
π
2.
Orthogonal polynomials: Chebyshev least squares
Therefore
1
c0 =
π
Z1
−1
and
2
ck =
π
Z1
−1
f (x )
√
dx
1 − x2
f (x ) Tk (x )
√
dx
1 − x2
∀k = 1, 2, ...n
The Chebyshev polynomial approximation is
n
Tn (x ) = c0 +
∑ ck Tk (x )
k =1
Orthogonal polynomials: Chebyshev least squares
Near minimax property of Chebyshev least squares:
Theorem
Let pn∗ (x ) be the minmax polynomial approximation to f ∈ C [−1, 1]. If
Tn∗ (x ) is the nth degree Chebyshev least square approximation to
f ∈ C [−1, 1] then
4
∗
∗
kf − pn k∞ ≤ kf − Tn k∞ ≤ 4 + 2 ln n kf − pn∗ k∞
π
and
lim kf − Tn∗ k∞ = 0
n→∞
uniformly
Chebyshev least squares approximation is nearly the same as the
minmax polynomial approximation and that as n → ∞, then Tn∗
converges to f uniformly.
Orthogonal polynomials: Chebyshev least squares
Chebyshev least squares (cont.):EXAMPLE
Approximate f (x ) = exp (x ) for x ∈ [−1, 1]
First order polynomial approximation of exp (x ) is
exp (x ) ≈ c0 T0 (x ) + c1 T1 (x )
where
T0 (x ) = cos ((0) (arccos x )) = 1
T1 (x ) = cos (arccos x ) = x
and
c0 =
1
π
Z1
−1
c1 =
2
π
Z1
−1
exp (x )
√
dx = 1.2661
1 − x2
exp (x ) x
√
dx = 1.1303
1 − x2
Orthogonal polynomials: Chebyshev least squares
Chebyshev least squares (cont.):EXAMPLE
Second order polynomial approximation of exp (x ) is
exp (x ) ≈ c0 T0 (x ) + c1 T1 (x ) + c2 T2 (x )
where c0 , c1 , T0 and T1 are the same as before and
T2 (x ) = cos (2 arccos x )
andc
c2 =
2
π
Z1
−1
exp (x ) T2 (x )
√
dx = 0.2715
1 − x2
Orthogonal polynomials: Chebyshev least squares
Chebyshev least squares (cont.):EXAMPLE
go to inneficient matlab script!
cheby least squares.m
Type CC for each marix of coefficients....
Interpolation: Basics
Usually we don’t have the value of f (x ) for all its domain.
We only have the value of f (x ) at some finite set of points:
(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xn , f (xn ))
Interplation nodes or points: x0 , x1 , ..., xn
Interpolation problem: Find the degree ≤ n polynomial pn (x )
that passes through these points:
f (xi ) = pn (xi )
∀i : 0, ...n
Interpolation: Basics
Existence and uniqueness of the interpolating polynomial
Theorem
If x0 , ..., xn are distinct, then for any f (x0 ) , ..., f (xn ) there existis a
unique polynomial pn (xi ) of degree ≤ n such that the interpolation
conditions
f (xi ) = pn (xi ) ∀i : 0s, ...n
are satisfied.
Linear interpolation
The simplest case is linear interpolation (i.e., n = 1) with two
data points
(x0 , f (x0 )) , (x1 , f (x1 ))
The interpolation conditions are:
f (x0 ) = p1 (x0 )
= a0 + a1 x0
f (x1 ) = p1 (x1 )
= a0 + a1 x1
Linear interpolation
Solving the above system yields
f ( x1 ) − f ( x0 )
a0 = f (x0 ) −
x0
x1 − x0
a1 =
f ( x1 ) − f ( x0 )
x1 − x0
Thus, the interpolating polynomial is
f ( x1 ) − f ( x0 )
f (x1 ) − f (x0 )
p1 (x ) = f (x0 ) −
x0 +
x
x1 − x0
x1 − x0
Linear interpolation
Notice that the interpolating polynomial can be written as
Power form
f (x1 ) − f (x0 )
f (x1 ) − f (x0 )
p1 (x ) = f (x0 ) −
x0 +
x
x1 − x0
x1 − x0
Newton form
p1 (x ) = f (x0 ) +
f (x1 ) − f (x0 )
x1 − x0
(x − x0 )
Lagrange form
p1 (x ) =
x − x1
x0 − x1
f (x0 ) +
x − x0
x1 − x0
f (x1 )
We have the same interpolating polynomial p1 (x ) written in two
different forms.
Quadratic interpolation
If we assume n = 2 and three data points
(x0 , f (x0 )) , (x1 , f (x1 )) , (x2 , f (x2 ))
Quadratic interpolation
The interpolation conditions are
f (x0 ) = p2 (x0 ) = a0 + a1 x0 + a2 x02
f (x1 ) = p2 (x1 ) = a0 + a1 x1 + a2 x12
f (x2 ) = p2 (x2 ) = a0 + a1 x2 + a2 x22
Quadratic interpolation
In matrix form the interpolation conditions are



 
1 x0 x02
a0
f ( x0 )
 1 x1 x 2   a1  =  f (x1 ) 
1
f ( x2 )
a2
1 x2 x22
or im more compact form
Va = b
Notice that V is a Vandermonde matrix which is ill-conditioned.
The condition number of V is large so it is better to compute the a0 s
by using another form of writing the interpolating polynomial.
Quadratic interpolation
But we can still do it by hand since this is a 3by3 matrix!
Solving the above system yields

x1 x2
−x0 x2


−x1 )(x0 −x2 )
(x0 −x1 )(x1 −x2 )
a0
 (x0 −(
x1 +x2 )
x0 +x2
 a1  = 
 (x0 −x1 )(x0 −x2 ) (x0 −x1 )(x1 −x2 )
1
−1
a2
(x0 −x1 )(x0 −x2 )
(x0 −x1 )(x1 −x2 )
x0 x1
(x0 −x2 )(x1 −x2 )
−(x0 +x1 )
(x0 −x2 )(x1 −x2 )
1
(x0 −x2 )(x1 −x2 )



 f ( x0 )
  f ( x1 ) 

f ( x2 )
Quadratic interpolation
Or by using the Matlab symbolic toolbox
>> syms a b c
>> A = [1 a a^2; 1 b b^2; 1 c c^2];
>> inv(A)
ans =
[ (b*c)/((a - b)*(a - c)), -(a*c)/((a - b)*(b - c)),
(a*b)/((a - c)*(b - c))]
[ -(b + c)/((a - b)*(a - c)), (a + c)/((a - b)*(b - c)),
-(a + b)/((a - c)*(b - c))]
[ 1/((a - b)*(a - c)), -1/((a - b)*(b - c)), 1/((a - c)*(b
- c))]
Quadratic interpolation
Solving the above system yields the coefficients:
x1 x2
− x0 x2
a0 =
f ( x0 ) +
f ( x1 )
(x0 − x1 ) (x0 − x2 )
( x0 − x1 ) ( x1 − x2 )
x0 x1
+
f (x2 )
( x0 − x2 ) ( x1 − x2 )
− ( x1 + x2 )
x0 + x2
a1 =
f ( x0 ) +
f ( x1 )
(x0 − x1 ) (x0 − x2 )
( x0 − x1 ) ( x1 − x2 )
− ( x0 + x1 )
f (x2 )
+
( x0 − x2 ) ( x1 − x2 )
1
−1
f ( x0 ) +
f ( x1 )
( x0 − x1 ) ( x0 − x2 )
(x0 − x1 ) (x1 − x2 )
1
+
f ( x2 )
(x0 − x2 ) (x1 − x2 )
a2 =
Quadratic interpolation
The approximating second order polynomial in ”power” form is
p2 (x ) = a0 + a1 x + a2 x 2
where a0 , a1 and a2 are defined above.
Notice that p2 (x ) is a linear combination of n + 1 = 3 monomials
each of degree 0, 1, 2.
Quadratic interpolation
After ”some” algebra we can write p2 (x ) in different forms:
Lagrange form
( x − x1 ) ( x − x2 )
( x − x0 ) ( x − x2 )
+ f ( x1 )
( x0 − x1 ) ( x0 − x2 )
( x1 − x0 ) ( x1 − x2 )
(x − x0 ) (x − x1 )
+ f ( x2 )
( x2 − x0 ) ( x2 − x1 )
p2 (x ) = f (x0 )
The above is a linear combination of n + 1 = 3 polynomials of
degree n = 2. The coefficients are the interpolated values
f (x0 ) , f (x1 ) and f (x2 ) .
Quadratic interpolation
By doing ”some” algebra we can write p2 (x ) in different forms:
Newton form
f ( x1 ) − f ( x0 )
( x − x0 )
x1 − x0

f (x2 )−f (x1 )
f (x1 )−f (x0 )
−
(x2 −x1 )
(x1 −x0 )
 ( x − x0 ) ( x − x1 )
+
( x2 − x0 )
p 2 ( x ) = f ( x0 ) +

The above is a linear combination of n + 1 = 3 polynomials each of
degree 0, 1, 2. The coefficients are the called divided differences.
Interpolation: The general case
The interpolation conditions when we have n + 1 data points:
{(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xn , f (xn ))}
∀i : 0, ...n
f (xi ) = pn (xi )
pn (xi ) written in ”power” form is
n
pn (xi ) =
∑ aj x j
j =0
Interpolation: The general case
The interpolation conditions can be written as
n
f ( xi ) =
∑ aj xi
j
∀i : 0, ...n
j =0
or
f (x0 ) = a0 + a1 x0 + ... + an x0n
f (x1 ) = a0 + a1 x1 + ... + an x1n
...
f (xn ) = a0 + a1 xn + ... + an xnn
Interpolation: The general case
In matrix form

1 x0
 1 x1

 ... ...
1 xn

a0
... x0n
 a1
... x1n 

... ...   ...
an
... xnn

f ( x0 )
  f ( x1 ) 

=
  ... 
f (xn )


The matrix to be inverted is a Vandermonde matrix: Also an
ill-conditioned matrix.
Interpolation: The general case
We can also generalize the lagrange form of the interpolating
polynomial:
pn (x ) = f (x0 ) ln,0 (x ) + f (x1 ) ln,1 (x ) + ... + f (xn ) ln,n (x )
n
where ln,j (x ) j =0 are a family of n + 1 polynomials of degree n
given by
(x − x0 ) ... x − xj −1 x − xj +1 ... (x − xn )
∀0 ≤ j ≤ n
ln,j (x ) =
xj − x0 ... xj − xj −1 xj − xj +1 ... xj − xn
More compactly
n
pn ( x ) =
∑f
j =0
xj ln,j (x )
Interpolation: The general case
For j = 0
ln,0 (x ) =
(x − x1 ) ... (x − xn )
=
(x0 − x1 ) ... (x0 − xn )
n
x − xj
∏ x0 − xj
j =0
j 6 =0
For j = 1
ln,1 (x ) =
(x − x0 ) (x − x2 ) ... (x − xn )
=
(x1 − x0 ) (x1 − x2 ) ... (x1 − xn )
n
x − xj
∏ x1 − xj
j =0
j 6 =1
For j = n
ln,n (x ) =
(x − x0 ) (x − x2 ) ... (x − xn−1 )
=
(xn − x0 ) (xn − x2 ) ... (xn − xn−1 )
n
x − xj
∏ x2 − xj
j =0
j 6 =2
Interpolation: The general case
For any ∀0 ≤ j ≤ n
n
ln,j (x ) =
x − xj
∏ xi − xj
j =0
j 6 =i
The lagrange form of the interpolating polynomial is
n
pn ( x ) =
∑f
xj ln,j (x )
j =0
with ln,j (x ) defined above.
It turns out that computing the lagrange polynomial is more efficient
than solving the vandermonde matrix!
Interpolation: The general case
We can also generalize the newton form of the interpolating
polynomial
pn (x ) = c0 + c1 (x − x0 ) + c2 (x − x0 ) (x − x1 ) ... + cn (x − x0 ) (x − x1 ) ...
where the coefficients c0 , c1 , ..., cn are called the divided difference
and are denoted by
c 0 = d ( x0 )
c 1 = d ( x1 , x0 )
c 2 = d ( x2 , x1 , x0 )
...
cn = d (xn , ..., x1 , x0 )
Interpolation: The general case
The divided differences are defined as
d ( x0 ) = f ( x0 )
d ( x1 , x0 ) =
d ( x2 , x1 , x0 ) =
=
f ( x1 ) − f ( x0 )
x1 − x0
d (x2 , x1 ) − d (x1 , x0 )
x2 − x0
f (x2 )−f (x1 )
f (x1 )−f (x0 )
−
x2 −x1
x1 −x2
x2 − x0
Interpolation: The general case
The divided differences are defined as (Cont.)
d ( x3 , x2 , x1 , x0 ) =
d (x3 , x2 , x1 ) − d (x2 , x1 , x0 )
x3 − x0
!
f (x )−f (x )
f (x )−f (x )
3
2
x3 −x2
=
d (xn , ..., x1 , x0 ) =
−
x3 −x2
2
1
x2 −x1
−
f (x2 )−f (x1 )
x2 −x1
!
f (x1 )−f (x0 )
−
x −x
x2 −x0
x3 − x0
d (xn , ..., x1 , x0 ) − d (xn−1 , ..., x1 , x0 )
xn − x0
1
2
Interpolation: The general case
The generalization of the Newton form of the interpolating
polynomial is
pn (x ) = d (x0 ) + d (x1 , x0 ) (x − x0 ) + d (x2 , x1 , x0 ) (x − x0 ) (x − x1 ) + ...
+ d (xn , ..., x1 , x0 ) (x − x0 ) (x − x1 ) ... (x − xn−1 )
or
n
p n ( x ) = d ( x0 ) +
∑d
j =1
xj , ..., x1 , x0
j −1
∏ ( x − xk )
k =0
Interpolation: The interpolation error
Theorem
Let f (x ) ∈ Cn+1 [a, b ]. Let pn (x ) a polynomial of degree ≤ n such that
it interpolates f (x ) at the n + 1 distinct nodes {x0 , x1 , ..., xn } . Then
∀x ∈ [a, b ], there exists a ξ n ∈ (a, b ) such that
f (x ) − pn (x ) =
n
1
f ( n + 1 ) ( ξ n ) ∏ ( x − xk )
(n + 1) !
k =0
Fact
The error term for the nth Taylor approximation around the point x0 is
f (n +1) ( ξ n )
( x − x0 ) n + 1
(n + 1) !
Interpolation: The interpolation error
Notice that applying the supremum norm to the interpolation error
yields
n
1
(n +1)
( ξ n ) ∏ ( x − xk ) kf (x ) − pn (x )k∞ ≤
f
∞
(n + 1) !
k =0
∞
or
max |f (x ) − pn (x )|
x ∈[a,b ]
1
≤
(n + 1) !
(n +1)
max f
(ξ n )
ξ n ∈[a,b ]
!
n
max ∏ (x − xk )
x ∈[a,b ] k =0
The R.H.S is an upper bound for the interpolation error.
Interpolation: The interpolation error
We would like to have
!)
(
n
1
(n +1)
lim
max f
max ∏ (x − xk )
=0
(ξ n )
n→∞ (n + 1)! ξ n ∈[a,b ]
x ∈[a,b ] k =0
thus
lim (f (x ) − pn (x )) = 0
n→∞
But nothing guarantees convergence (neither point or uniform
convergence).
Interpolation: The interpolation error
The maximun error depends on the interpolation
nodes
!
n
{x0 , x1 , ..., xn } through the term maxx ∈[a,b ] ∏ (x − xk ) .
k =0
Notice that no other term depends on the interpolating nodes once
we look for the maximum error.
We can choose the nodes in order to minimize the interpolation
error:
)
(
n
max ∏ (x − xk )
min
{x0 ,...,xn } x ∈[a,b ] k =0
Interpolation: Choosing the nodes
What happens if we work with uniformly spaced nodes?, that is with
nodes such that
i −1
xi = a +
(b − a) for i : 1, .., n
n−1
Recall that:
We want to interpolate a function f (x ) : [a, b ] → R
n is the degree of the interpolating polynomial: pn (x )
The interpolation conditions are
f (xi ) = pn (xi ) ...for i : 0, .., n
so, if n = 10, we need n + 1 = 11 data points.
Interpolation: Choosing the nodes
Runge’s example: Let f (x ) = 1+1x 2 defined on the interval [−5, 5].
Find the lagrange polnomial approximation for n = 11.
go to inneficient matlab script!
runge example lagrange interpolation UN.m
Play increasing the degree of interpolation and see that there is no
convergence.
The graph replicates figure 6.6 of Judd’s book.
Interpolation: Choosing the nodes
But we can choose the nodes in order to obtain the smallest value for
n
n
∏ (x − xk ) = max ∏ (x − xk )
k =0
x ∈[a,b ] k =0
∞
Chebyshev polynomials one more time: Recall the monic
Chebyshev polynomial is
ej (x ) = cos (j arccos x )
T
2j −1
with x ∈ [−1, 1] and j = 1, 2, ..., n
en (x ) are given by the soltution to
Then, the zeros of T
en (x ) = 0
T
cos (n arccos x ) = 0
cos (nθ ) = 0
where θ = arccos x. Thus θ ∈ [0, π ].
Interpolation: Choosing the nodes
Zeros occur when cos (nθ ) = 0, that is
2k − 1
nθ =
π for k = 1, 2, ..n
2
Notice that
2k − 1
cos
π
2
π = cos kπ −
2 π
π
= cos (kπ ) cos
− sin (kπ ) sin
2
| {z2 } | {z }
=0
=0
=0
Interpolation: Choosing the nodes
en (x ) = 0 has n different roots given by
The equation T
1
π for k = 1, 2, ..n
nθ = k −
2
This means that
For k = 1
π
2n
3 π
θ2 =
2 n
θ1 =
For k = 2
For k = n
θn =
2n − 1
2
π
n
Interpolation: Choosing the nodes
Roots for cos (nθ ) where θ ∈ [0, π ] :
k=1
k=2
k=3
...
k=n
n=1
n=2
n=3
π
π
π
2
4
3π
4
6
3π
6
5π
6
...
n
π
2n
3
2n π
5
2n π
2n−1
2n
π
Interpolation: Choosing the nodes
Plooting cos (j θ ) for θ ∈ [0, π ] and j = 1, 2, ..., n
go to inneficient matlab script!
cheby nodes.m
Interpolation: Choosing the nodes
We want the roots of the monic chebyshev and we have the roots of
the cosine function:
1
nθ = k −
π for k = 1, 2, ..n
2
but θ = arccos x, thus
arccos x =
2k − 1
2n
π
for k = 1, 2, ..n
then the roots of the chebyshev polynomials are
2k − 1
xk = cos
π
for k = 1, 2, ..n
2n
en (x ) are the same as the roots of Tn (x ) .
Notice that the roots of T
Interpolation: Choosing the nodes
Ploting chebyshev nodes
go to inneficient matlab script!
cheby nodes.m
Interpolation: Choosing the nodes
The following theorem summarizes some characteristics of chebyshev
polynomials
Theorem
The Chebyshev polynomial Tn (x ) of degree n ≥ 1 has n zeros in [−1, 1]
at
2k − 1
xk = cos
π
for k = 1, 2, ..n
2n
Moreover, Tn (x ) assumes its extremum at
kπ
xk∗ = cos
for k = 0, 1, .., n
n
with
Tn (xk∗ ) = (−1)k
for k = 0, 1, .., n
Interpolation: Choosing the nodes
Corollary
en (x ) has the same zeros and
The monic Chebyshev polynomial T
extremum points as Tn (x ) but with extremum values given by
k
en (x ∗ ) = (−1) for k = 0, 1, ..., n
T
k
2n − 1
Interpolation: Choosing the nodes
Extrema of Chebyshev polynomials:
Tn (x ) = cos (n arccos x )
then
dTn (x )
= Tn0 (x )
dx
n
= − sin (n arccos x ) − √
1 − x2
n sin (n arccos x )
√
=
1 − x2
Notice that Tn0 (x ) is a polynomial of degree n − 1 with zeros given
by
Tn0 (x ) = 0
Interpolation: Choosing the nodes
Excluding the endpoints of the domain, x = −1 or x = 1, then,
extremum points occurs when
sin (n arccos x ) = 0
or when
sin (nθ ) = 0
for θ ∈ (0, π ). Thus
nθk = kπ
for all k = 1, 2, ..., n − 1
Solving for x yields
kπ
n kπ
=⇒ xk∗ = cos
n
θ = arccos x =
for k = 1, 2, ..., n − 1
Obviusly, extrema also occur at the endpoints of the domain (i.e,
x = −1 or x = 1), that is when k = 0 or when k = n.
Interpolation: Choosing the nodes
The extremum values of Tn (x ) occurs when
Tn (x ∗ ) = cos (n arccos x ∗ )
kπ
= cos n arccos cos
n
kπ
= cos n
n
= cos (kπ )
= (−1)k
for k = 0, 1, ..., n
Notice that we are including the endpoints of the domain.
The above result implies
max |Tn (x )| = 1
x ∈[−1,1]
Interpolation: Choosing the nodes
Extrema for monic Chebyshev polynomials are characterized by the
same points since
en (x ) = Tn (x )
T
2n−1
thus
en0 (x ∗ ) = Tn0 (x ∗ ) = 0 for k = 0, 1, ..., n
T
k
k
en (x ) are given by
But the extremum values of T
Tn xk∗
∗
en (x ) =
for k = 0, 1, ..., n
T
k
2n−1
(−1)k
= n−1 for k = 0, 1, ..., n
2
Therefore
1
e
max T
n (x ) = n −1
2
x ∈[−1,1]
Interpolation: Choosing the nodes
An important property of monic Chebyshev polynomials is given by
the following theorem
Theorem
If p
en (x ) is a monic polynomial of degree n defined on [−1, 1], then
1
e
max T
en (x )|
n (x ) = n−1 ≤ max |p
2
x ∈[−1,1]
x ∈[−1,1]
for all monic polynomials of degree n.
Interpolation: Choosing the nodes
Recall that we want to choose the interpolation nodes {x0 , ..., xn } in
order to solve
)
(
n
min
max ∏ (x − xk )
{x0 ,...,xn } x ∈[a,b ] k =0
Choosing the interpolation nodes is the same as choosing the zeros
n
of
∏ ( x − xk ) .
k =0
n
Notice that
∏ (x − xk ) is a monic polynomial of degree n + 1.
k =0
Therefore it must be the case that
n
1
e
max Tn+1 (x ) = n ≤ max ∏ (x − xk )
2
x ∈[−1,1]
x ∈[−1,1] k =0
Interpolation: Choosing the nodes
n
The smallest value that maxx ∈[−1,1] ∏ (x − xk ) can take is 21n .
k =0
Therefore
n
1
max ∏ (x − xk ) = n
2
x ∈[−1,1] k =0
e
= max T
x
(
)
n +1
x ∈[−1,1]
which implies that
n
∏ (x − xk ) = Ten+1 (x )
k =0
n
Therefore the zeros of
∏ (x − xk ) must be the zeros of Ten+1 (x )
k =0
which are given by
2k + 1
xk = cos
π
2 (n + 1)
for k = 1, 2, ..n + 1
Interpolation: Choosing the nodes
n
Since maxx ∈[−1,1] ∏ (x − xk ) = 21n then the maximum
k =0
interpolation error becomes
1
max |f (x ) − pn (x )| ≤
(n + 1) !
x ∈[a,b ]
max |f (x ) − pn (x )| ≤
x ∈[a,b ]
1
(n + 1) !
max f (n+1) (ξ n )
ξ n ∈[a,b ]
n
max ∏ (x
x ∈[a,b ] k =0
1 max f (n+1) (ξ n )
2n
ξ n ∈[a,b ]
Chebyshev nodes eliminate violent oscillations for the error term
compared to uniform spaced nodes.
Interpolation with Chebyshev nodes has better convergence
properties.
It is possible to show that pn (x ) → f (x ) as n → ∞ uniformly. This
is not guaranteed under uniform spaced nodes.
Interpolation: Choosing the nodes
Runge’s example with chebyshev nodes
runge example cheby nodes.m
Interpolation: Choosing the nodes
Figure 6.2 of Miranda and Flecker book:
Comparing the interpolation errors of f (x ) = exp(−x ) defined in
x ∈ [−5, 5] with 10-node polynomial approximation
example miranda cheby nodes
Interpolation through regresssion
The interpolation conditions require to have the same number of
data points (interpolation data) and unknown coefficients in order to
proceed.
But we can also have the case, where the data points exceed the
number of unknow coefficients.
For this case, we can use discrete least squares: Use m
interplation points to find n < m coefficients.
The omitted terms are high degree polynomials that may produce
undesirable oscillations.
The result is a smoother function that approximates the data.
Interpolation through regresssion
Objective: Construct a degree n polynomial , b
f (x ) , that
approximates the function f for x ∈ [a, b ] using m > n interpolation
nodes.
n
b
f (x ) = ∑ cj Tj (xk )
j =0
Interpolation through regresssion
Algorithm:
Step 1: Compute the m Chebyshev interpolation nodes on
[−1, 1]:
2k − 1
zk = cos
π
for k = 1, ..., m
2m
As if we want an m −degree Chebyshev interpolation.
Step 2: Adjust the nodes to the interval [a, b ] :
b−a
xk = ( zk + 1 )
+ a for k = 1, ..., m
2
Step 3: Evaluate f at the nodes:
yk = f (xk ) ...for k = 1, ..., m
Interpolation through regresssion
Algorithm (Cont.):
Step 4: Compute the Chebyshev least squares coefficients
The coefficients that solve the discrete LS problem
"
#2
m
min
∑
n
yk −
∑ cj Tj (zk )
j =0
k =1
are given by
m
∑ yk Tj (zk )
cj =
k =1
m
∑
for j = 0, 1, ..., n
Tj (zk )
2
k =1
where zk is the inverse transformation of xk :
zk =
2xk − (a + b )
b−a
Interpolation through regresssion
Finally, the LS Chebyshev approximating polynomial is given by
n
b
f (x ) =
∑ cj Tj (z )
j =0
where z ∈ [−1, 1] and it is given by
z=
2x − (a + b )
b−a
and the cj are estimated using the LS coefficients
m
∑ yk Tj (zk )
cj = km=1
∑
k =1
for j = 0, 1, ..., n
Tj (zk )
2
Piecewise linear approximation
If we have interpolation data given by
{(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xn , f (xn ))}
We can divide the interpolation nodes in subintervals of the form
[xi , xi +1 ] for i = 0, 1, ..., n − 1
Then we can perform linear interpolation in each subinterval:
Interpolation conditions for each subinterval:
f (xi ) = a0 + a1 xi
f (xi +1 ) = a0 + a1 xi +1
Piecewise linear approximation
Linear interpolation in each subinterval yields [xi , xi +1 ]:
The interpolating coefficients:
a0 = f (xi ) −
a1 =
f (xi +1 ) − f (xi )
xi +1 − xi
xi
f (xi +1 ) − f (xi )
xi +1 − xi
Piecewise linear interpoland:
x − xi
pi (x ) = f (xi ) +
(f (xi +1 ) − f (xi ))
xi +1 − xi
Piecewise linear approximationPiecewise polynomial
approximation: Splines
Example: (x0 , f (x0 )) , (x1 , f (x1 )) , (x2 , f (x2 ))
Then we have two subintervals
[x0 , x1 ] and .[x1 , x2 ]
The interpolating function is given by:
f (x ) + x −x0 (f (x ) − f (x ))
0
1
0
x −x
b
1 0
f (x ) =
f (x1 ) + xx2−−xx11 (f (x2 ) − f (x1 ))
for x ∈ [x0 , x1 ]
for x ∈ [x1 , x2 ]
Piecewise polynomial approximation: Splines
A spline is any smooth function that is piecewise polynomial but also
smooth where the polynomial pieces connect.
Assume that the interpolation data is given by
{(x0 , f (x0 )) , (x1 , f (x1 )) , ..., (xm , f (xm ))} .
A function s (x ) defined on [a, b ] is a spline of order n if:
s is C n−2 on [a, b ]
s (x ) is a polynomial of degree n − 1 on each subinterval [xi , xi +1 ]
for i = 0, 1, .., m − 1
Notice that an order 2-spline is the piecewise linear interpolant
equation.
Piecewise polynomial approximation: Splines
A cubic spline is a spline of order 4:
s is C 2 on [a, b ]
s (x ) is a polynomial of degree n − 1 = 3 on each subinterval
[xi , xi +1 ] for i = 0, 1, .., m − 1
s (x ) = ai + bi x + ci x 2 + di x 3 for x ∈ [xi , xi +1 ], i = 0, 1, .., m − 1
Piecewise polynomial approximation: Splines
Example of cubic spline: Assume that we have the following 3 data
points: (x0 , f (x0 )) , (x1 , f (x1 )) , (x2 , f (x2 ))
There are two subintervals: [x0 , x1 ] and .[x1 , x2 ].
A cubic spline is a function s such that
s is C 2 on [a, b ]
s (x ) is a polynomial of degree 3 on each subinterval:
s0 (x ) = a0 + b0 x + c0 x 2 + d0 x 3 for x ∈ [x0 , x1 ]
s (x ) =
s1 (x ) = a1 + b1 x + c1 x 2 + d1 x 3 for x ∈ [x1 , x2 ]
Notice that in this case we have 8 unknowns:
a0 , a1 , b0 , b1 , c0 , c1 , d0 , d1
Piecewise polynomial approximation: Splines
Example (Cont.): We need 8 conditions
Interpolation and continuity at interior nodes conditions
y0 = s0 (x0 ) = a0 + b0 x0 + c0 x02 + d0 x03
y1 = s0 (x1 ) = a0 + b0 x1 + c0 x12 + d0 x13
y1 = s1 (x1 ) = a1 + b1 x1 + c1 x12 + d1 x13
y2 = s1 (x2 ) = a1 + b1 x2 + c1 x22 + d1 x23
Piecewise polynomial approximation: Splines
Example (Cont.): We need 8 conditions
First and second derivatives must agree at the interior nodes
s00 (x1 ) = s10 (x1 )
b0 + 2c0 x1 + 3d0 x12 = b1 + 2c1 x1 + 3d1 x12
00
00
s0 (x1 ) = s1 (x1 )
2c0 + 6d0 x1 = 2c1 + 6d1 x1
Piecewise polynomial approximation: Splines
Up to know we have 6 conditions, we need two more conditions
3 ways to obtain the additional conditions:
Natural spline: s 0 (x0 ) = s 0 (x2 ) = 0
Hermite spline: If we have information on the slope of the original
function at the end points:
f 0 (x0 ) = s 0 (x0 )
f 0 (x2 ) = s 0 (x2 )
Secant Hermite spline: use the secant to estimate the slope at the
end points
s (x1 ) − s (x0 )
s 0 (x0 ) =
x1 − x0
s (x2 ) − s (x1 )
0
s (x2 ) =
x2 − x1
Piecewise polynomial approximation: Splines
Generalization of cubic splines:
See Judd’s book!!!