8.2 - Orthogonal Polynomials and Least Squares
Approximation
8.2 - Orthogonal Polynomials and Least Squares Approximation
Least Squares Approximation of Functions
Motivation
Suppose f ∈ C[a, b], find a polynomial Pn (x) of degree at most n to
Rb
approximate f such that a (f (x) − Pn (x))2 dx is a minimum.
Let polynomial Pn (x) be Pn (x) =
Pn
k=0 ak x
Z
k
which minimizes the error
b
f (x) −
E ≡ E(a0 , a1 , . . . , an ) =
a
n
X
!2
ak xk
dx.
k=0
The problem is to find a0 , · · · , an that will minimize E. The necessary
∂E
condition for a0 , · · · , an to minimize E is ∂a
= 0, which gives the normal
j
equations
n
X
k=0
Z
ak
b
x
a
j+k
Z
dx =
b
xj f (x) dx
for j = 0, 1, · · · , n
a
8.2 - Orthogonal Polynomials and Least Squares Approximation
Example Find the least squares approximating polynomial of degree 2 for
f (x) = sin πx on [0, 1].
Solution Let P2 (x) = a0 + a1 x + a2 x2 .

R1
R1
R1 2
R1

 a0 R0 1dx + a1 R0 xdx + a2 0R x dx = 0R sin πxdx
1
1
1
1
a0 0 xdx + a1 0 x2 dx + a2 0 x3 dx = 0 x sin πxdx

 a R 1 x2 dx + a R 1 x3 dx + a R 1 x4 dx = R 1 x2 sin πxdx
0 0
1 0
2 0
0
(1)

1
1
 a0 + 2 a1 + 3 a2 = 2/π
1
a0 + 31 a1 + 41 a2 = 1/π
 21
1
1
π 2 −4
3 a0 + 4 a1 + 5 a2 = π 3
a0 = −0.050465, a1 = 4.12251, a2 = −4.12251.
8.2 - Orthogonal Polynomials and Least Squares Approximation
Linearly Independent Functions
Definition
The set of functions {φ0 , · · · , φn } is said to linearly independent on [a, b]
if, whenever c0 φ0 (x) + c1 φ1 (x) + · · · + cn φn (x) = 0, for all x ∈ [a, b] we
have c0 = c1 = · · · = cn = 0. Otherwise the set of functions is said to be
linearly dependent.
Theorem
Suppose that, for each j = 0, 1, · · · , n, φj (x) is a polynomial of degree j.
Then {φ0 , · · · , φn } is linearly independent on any interval [a, b].
Theorem
Suppose that {φ
Q of linearly independent
Q0 (x), · · · , φn (x)} is a collection
polynomials in n . Then any polynomial in n can be written uniquely as
a linear combination of φ0 (x), · · · , φn (x).
8.2 - Orthogonal Polynomials and Least Squares Approximation
Orthogonal Functions
Definition
{φ0 , · · · , φn } is said to be an orthogonal set of functions for the interval
[a, b] with respect to the weight function w if
(
Z b
0,
when j 6= k,
w(x)φj (x)φk (x) dx =
αk > 0, when j = k.
a
If also αk = 1 for each k = 0, . . . , n, the set is orthonormal.
Theorem
If {φ0 , . . . , φn } is orthogonal
Pn on [a, b], then the least squares approximation
to f on [a, b] is P (x) = k=0 ak φk (x) where
Rb
w(x)φk (x)f (x) dx
1
=
ak = aR b
2
αk
a w(x)[φk (x)] dx
Z
b
w(x)φk (x)f (x) dx.
a
8.2 - Orthogonal Polynomials and Least Squares Approximation
Legendre Polynomials
The set of Legendre polynomials {Pn (x)} is orthogonal on [−1, 1] w.r.t. the
weight function w(x) = 1.
P0 (x) = 1,
α0 = 2
P1 (x) = x,
α1 = 2/3
1
P2 (x) = x2 − ,
3
3
P3 (x) = x3 − x,
5
6
3
4
P4 (x) = x − x2 + ,
7
35
α2 = 8/45
α3 = 8/175
α4 = 128/11025
8.2 - Orthogonal Polynomials and Least Squares Approximation