MAT 4725
Numerical Analysis
Section 8.2
Orthogonal Polynomials
and Least Squares
Approximations (Part II)
http://myhome.spu.edu/lauw
Preview


Inner Product Spaces
Gram-Schmidt Process
Recall: Least Squares
Approximation of Functions
Given f  C[a, b],
approximate f ( x) by
n
Pn ( x)   ak x k
f ( x)
k 0
Pn ( x)
Find ak such that
b
E    f ( x )  Pn ( x )  dx
2
a
b
a
is minimized
A Different Technique for Least
Squares Approximation


Computationally Efficient
Once 𝑃𝑛(𝑥) is known, it is easy to
determine 𝑃𝑛+1 (𝑥)
Recall (Linear Algebra)

General Inner Product Spaces
Inner Product
Example 0
Let 𝑓, 𝑔𝐶[𝑎, 𝑏]. Show that
b
f , g   f  x  g  x  dx
a
is an inner product on 𝐶[𝑎, 𝑏]
Norm, Distance,…
Orthonormal Bases
A basis 𝑆 for an inner product space 𝑉 is
orthonormal if
1. For 𝑢, 𝑣  𝑆, < 𝑢, 𝑣 >= 0.
2. For 𝑢  𝑆, 𝑢 is a unit vector.
Gram-Schmidt Process
v1 , v2 ,
, vn  Basis

w1 , w2 ,
, wn  Orthogonal Basis

u1 , u2 ,
, un  Orthonormal Basis
Gram-Schmidt Process
Gram-Schmidt Process


The component in 𝑣2 that is “parallel” to
𝑤1 is removed to get 𝑤2.
So 𝑤1 is “perpendicular” to 𝑤2.
Simple Example
v2  i  j
v1  2i
Specific Inner Product Space
Definition 8.1
0 , 1 ,
, n  is said to be linearly independent
on [a, b] if ,
n
whenever
 c  ( x)  0, x  [a, b],
j 0
j
j
we have ck  0 j  0,1,..., n.
(Otherwse, linearly dependent.)
Theorem 8.2
If  j ( x) is a polynomial of degree j ,
then 0 , 1 ,
, n  is linearly independent
on any interval [a, b].
Demo
Theorem 8.2
If  j ( x) is a polynomial of degree j ,
then 0 , 1 ,
, n  is linearly independent
on any interval [a, b].
0 ( x)  2

1 ( x)  x  3

2
2 ( x)  x  2 x  7
Definition
 n  the set of polynomials of degree  n
Theorem 8.3
If 0 ( x), 1 ( x),
, n ( x) is linearly independent in  n
Then Q   n ,  unique c j such that
n
Q ( x)   c j  j ( x)
j 0
Example 1
0 ( x)  2

1 ( x)  x  3

2

(
x
)

x
 2x  7
 2
Express Q( x)  a0  a1 x  a2 x 2 as a
linear combination of 0 ( x), 1 ( x), and 2 ( x).
Definition (Skip it for the rest)
Weight function w( x) on an interval I :
(a ) integrable
(b) w( x)  0 x
(c) w( x)  0 on any subinterval of I
Weight Functions

to assign varying degree of importance
to certain portion of the interval
1
Modification of the Least
Squares Approximation
Recall from part I
Least Squares Approximation of
Functions
Given f  C[a, b],
f ( x)
approximate f ( x) by
n
Pn ( x)
Pn ( x)   ak x
k 0
a
b
k
Least Squares Approximation of
Functions
Find ak such that
b
E    f ( x )  Pn ( x )  dx
f ( x)
2
a
Pn ( x)
a
is minimized
b
Normal Equations
 b k j  b
j
a
x
dx

f
(
x
)
x
dx, j  0,1,
 

k 
k 0
a
 a
n
Solve for ak
,n
Modification of the Least
Squares Approximation
Given f  C[a, b],
Given f  C[a, b],
approximate f ( x) by
approximate f ( x) by
n
Pn ( x)   ak x
k 0
k
n
P( x)   akk  x 
k 0
Modification of the Least
Squares Approximation
Find ak such that
Find ak such that
b
E    f ( x )  Pn ( x )  dx
2
a
is minimized
b
E    f ( x)  P ( x)  dx
2
a
is minimized
Modification of the Least
Squares Approximation
For j  0,1,
, n, solve for ak
 b k j  b
j
a
x
dx

f
(
x
)
x
dx
 

k 
k 0
a
 a
n
b
 b
ak   k ( x) j ( x)dx    f ( x) j ( x)dx

k 0
a
 a
n
Where are the Improvements?
b
 b
ak   k ( x) j ( x)dx    f ( x) j ( x)dx

k 0
a
 a
n
Where are the Improvements?
b
 b
ak   k ( x) j ( x)dx    f ( x) j ( x)dx

k 0
a
 a
n
Find k  such that
 0
a k ( x) j ( x)dx   j  0
Then......
b
jk
jk
Definition 8.5
0 , 1 ,
, n  is said to be an orthogonal
set of functions on [ a, b] with respect to w if
jk
 0
a k ( x) j ( x)dx   j  0 j  k
(Orthonormal if all  j =1)
b
Theorem 8.6
b
ak 

f ( x)k ( x)dx

a
b
 k ( x)
2
dx

𝑎𝑘 are easier to
solve
𝑎𝑘 are “reusable”
a

1
k
b

a
f ( x)k ( x)dx
n
P( x)   akk ( x)
k 0
Where to find Orthogonal Poly.?

the Gram-Schmidt Process
Gram-Schmidt Process
w( x)  1
0 ( x)  1
b
 x 0 ( x) dx
2
1 ( x)  x  B1 where B1 
a
b
 0 ( x) dx
2
a
For k  2, k ( x)   x  Bk  k 1 ( x)  Ckk  2 ( x)
b
 x 
k 1
where Bk 
( x)  dx
a
b
 k 1 ( x) dx
2
a
b
2
 x
k 1
and Ck 
( x)k  2 ( x)dx
a
b
 k 2 ( x) dx
2
a
Special Case: on [-1,1],
Legendre Polynomials
P0 ( x)  1
[-1,1]
1
 x  P0 ( x) dx
2
P1 ( x)  x  B1  x where B1 
1
1
  P0 ( x) dx
0
2
1
1
P2 ( x)   x  B2  P1 ( x)  C2 P0 ( x)  x 
3
2
1
 x  P ( x)
2
1
where B2 
1
1
1
dx
  P1 ( x) dx
2
1
 xP ( x) P ( x)dx
1
 0 and C2 
0
1
1
  P0 ( x) dx
2
1
1

3
Legendre Polynomials w( x)  1, [-1,1]
P0 ( x)  1
P1 ( x)  x
1
P2 ( x)  x 
3
2
3
P3 ( x)  x  x
5
3
Example 2
Find the least squares approx. of
𝑓(𝑥) = sin(𝜋𝑥)
on [−1,1] by the Legendre Polynomials.
Example 2
1
1
 f ( x) P ( x)dx  sin  x 1dx
b
0
a0 

1
1
  P0 ( x)
2
1
dx
1
1
 1
2
1
dx
 f ( x) ( x)dx
k
0
ak 
a
b
 k ( x)
2
a
dx
Example 2
a0  0
b
1
a1 

f ( x) P1 ( x)dx

1
1
  P ( x)
2
1
1
dx
 f ( x) ( x)dx
k
1
 sin( x)  xdx
1
1
  x
2
1
dx
ak 
a
b
 k ( x)
2

3

a
dx
Example 2
a0  0, a1 
3
b

k
ak 
 2 1
1 sin( x)   x  3  dx
2
 2 1
1  x  3  dx
1
a
b
 k ( x)
2
1
a2 
 f ( x) ( x)dx
a
0
dx
Example 2
a0  0, a1 
3

b
, a2  0
 3 3 
1 sin( x)   x  5 x  dx
k
ak 
2
 3 3 
1  x  5 x  dx
1
a
b
 k ( x)
2
1
a3 
 f ( x) ( x)dx


35 15   2
2 3

a
dx
Example 2
a0  0, a1 
3

, a2  0, a3 

35 15   2

2 3
P( x)  a0 P0 ( x)  a1P1 ( x)  a2 P2 ( x)  a3 P3 ( x)
Example 2
𝑓 𝑥 = sin(𝜋𝑥)
𝑃(𝑥)
Homework

Download Homework