1
eigencos.mcd
Eigenvalue-eigenvector of the second derivative operator d 2 /dx2 . This leads to Fourier series (sine,
cosine, Legendre, Bessel, Chebyshev, etc). This is an example of a systematic way of generating
a set of mutually orthogonal basis vectors via the eigenvalues-eigenvectors to an operator. We
are often less interested in the solution of the ODE or the second derivative operator itself. Main
objective: we are often more interested in the resulting set of eigenvectors that possess "nice"
properties (orthogonality, recursive relationships, derivative & integral relationships, etc).
Instructor: Nam Sun Wang
Problem Statement. Find the eigenvalues and eigenvectors (eigenfunctions) for the second
derivative operator L defined in x=[-1 1]. We will use the terms eigenvectors and eigenfunctions
interchangeably because functions are a type of vectors.
2
L. y D . y
d
2
y λ' . y
y( 1 ) y( 1 ) 1
or any symmetric boundary condition
d x2
By "symmmetric BC" above, we mean it as a part of a symmetric linear transformation L that does
not change scalar product upon swapping an input vector to the operator. Do not confuse the word
"symmetric" with an "even" function symmetrical wrt to x=0)
( L . y , z ) ( y , L . z ) ... definition of a symmetric linear transformation (we swap any y and z in the LVS))
The function that satisfies the differential portion is:
2
d
y exp λ' . x
Check:
exp λ' . x λ' . y
d x2
If we loosely consider just the ODE, there are an infinity number of eigenvalues and infinite number of
eigenvectors -- basically any exponential function exp(λ'*x) with any constant λ' will do. Specifically,
the derivative operator comes with initial conditions or boundary conditions. If we apply the given
symmetrical boundary condition and confine ourselves strictly to one-dimensional invariant
subspace, there is no eigenvector that satisfies the boundary condition. If we relax the
one-dimensional invariant subspace restriction and allow two-dimensional invariant subspace to be
considered, we have the following eigenvalues and eigenvectors (depending on the sign of λ').
for λ' > 0
y exp
λ' . x
and
y exp
λ' . x
λ' . x
which is equivalent to a combination of y sinh
for λ' 0
y 1
and
for λ' < 0
y exp
λ' . x
and y cosh
λ' . x
2
λ .x
exp
y x
exp
y exp( i . λ. x ) and
2
λ .x
exp
1 . λ. x
y exp( i . λ. x )
λ' . x
y exp
where
2
λ
exp
λ'
The square root of a negative number is an imaginary number. The exponential functions with
imaginary arguments, exp(i*λ*x) and exp(-i*λ*x), are equivalent to a combination of
for λ' < 0
y sin( λ. x )
&
y cos( λ. x )
because
e
i .λ.x
cos( λ. x )
i . sin( λ. x )
1 . λ. x
2
eigencos.mcd
In general, for each eigenvalue of the second derivative operator, there are two linearly independent,
but not necessarily orthogonal eigenvectors. The two eigenvectors is the result of being in a
two-dimensional invariant subspace. (Remember, two-dimensional space means, by definition, that
there are two linearly independent vectors.) For each eigenvalue, the eigenvectors are not unique.
We can have different combinations of two eigenvectors, and the resulting linearly independent
vectors are also eigenvectors. Of these eigenvectors, hyperbolic sine, and x are odd functions
(antisymmetric with respect to x=0, f(-x)=-f(x)) that do not satisfy the symmetric boundary condition
of the differential equation. Hyperbolic cosine function is even (symmetric with respect to x=0,
f(-x)=f(x)), but it has a minimum value of 1 at x=0 and does not satisfy the B.C. Only the cosine and
sine functions are valid. Of these, only the cosine functions satisfy y(-1)=y(1)=1, and only the sine
functions satisfy y(-1)=y(1)=0. Without loss of generality, we can write the eigenvalue-eigenvector
equation in terms of -λ2 , rather than simply λ'. Doing so saves us from having to write the square root
sign. Thus, strictly speaking, the set {λ'} are the eigenvalues for the second-derivative operator.
However, we commonly refer to the set {λ} as the eigenvalues; it is just a more general use of the
term. To satisfy the symmetric B.C., we must choose λ to be integer multiples of π.
λ0 0
y0 1
λj π. j
j 1 , 2 .. ∞
y j cos λ j. x
After eliminating sinh and cosh functions, there still remain an infinite number of eigenvalues and an
equally infinite number of the corresponding eigenvectors. Although we first discuss cosine
eigenvectors, sine eigenvectors are similar. Cosine eigenvectors are all mutually orthogonal to one
another; furthermore, they are already normalized (except for cos(0*x)=1). The following are a few
examples.
1
Define scalar product
prod( f , g )
f( x ) . g( x ) dx
1
1
1
cos( 1 . π. x ) . cos( 2 . π. x ) dx = 0
cos( 1 . π. x ) . cos( 1 . π. x ) dx = 1
1
1
1
1
cos( 1 . π. x ) . cos( 3 . π. x ) dx = 0
cos( 2 . π. x ) . cos( 2 . π. x ) dx = 1
1
1
1
1
cos( 2 . π. x ) . cos( 3 . π. x ) dx = 0
cos( 3 . π. x ) . cos( 3 . π. x ) dx = 1
1
1
In general,
for
j 0
1
1 dx = 2
→ Normalize →
1
2
1
for
1
cos( j. π. x ) . cos( k . π. x ) dx 0
j k
1
j 1 , 2 .. ∞
y j( x ) . y k ( x ) dx 0
1
1
1
for
1
y0
cos( j. π. x ) . cos( j. π. x ) dx 1
1
y j( x ) . y j( x ) dx 1
1
3
x
eigencos.mcd
1 , 0.99 .. 1
Cosine Eigenfunctions
1
cos( 0. π. x )
cos( 1. π. x )
cos( 2. π. x )
0
cos( 3. π. x )
0
1
1
0.5
0
0.5
1
x
Now, with these eigenvectors, we can express any twice-differentiable even function in x=[-1 1] as a
linear combination of these eigenvectors. This is a huge, nontrivial statement, because it means the
series has to converge for the statement to hold, and we can state it with confidence even without
any kind of convergence test or formal proof. (Actually, it turns out, with rigorous proof, the sort that
mathematicians like, that we can relax the LVS to include all continuous functions that has at most
a finite number of discontinuities in x=[-1 1].) Since there are an infinite number of eigenvectors, we
need an infinity number of terms in theory (pure mathematics). However, we have to truncate to a
finite number of terms N in practice (engineering mathematics).
∞
∞
a .y
f
j
j
j=0
1
a 0.
2
N
aj. cos λj. x
→ In practice →
j=1
1
f a 0.
2
aj. cos λj. x
j=1
Since the eigenvectors are all mutually orthogonal, the following projection formula apply.
1
f( x ) . cos λj. x dx
f , yj
1
aj
yj , yj
1
cos λj. x . cos λj. x dx
1
The cosine eigenvectors are also normalized; thus, we can eliminate the denominator.
1
aj
f( x ) . cos λj. x dx
f , yj
1
We apply the second derivative operator and estimate the second derivative of any twice-differentiable
function in x=[-1 1] that satisfies f(-1)=f(1)=1. The second derivative of f is,
d
2
d x2
f
d
2
d x2
∞
∞
aj. y i
j=0
j=0
2
d
aj.
yj
d x2
∞
∞
aj. λj
j=0
2.
2
aj. λj . y j
yj
j=1
4
eigencos.mcd
Example. Express a constant function f(x)=1 as a linear combination of the eigenvectors.
∞
1
1
.
1 a
a . cos λ . x
where a f , y
1,y
1 . cos λ . x dx
0
j
j
j
j
j
2
j=1
Let us approximate f(x)=1 with limited number of terms.
N
10
j
1 .. N
f( x )
λ0
0
a0
j
1
1
1
f( x ) .
1
dx
a0 = 1.414
... "average" of f(x)
2
1
1
λj
π. j
f( x ) . cos λj. x dx
aj
1
a0
f approx( x )
2
N
aj. cos λj. x
j=1
1.1
f approx( x )
1
0.9
1
0
1
x
Actually, since f(x)=1=y 0 , which is in turn orthogonal to all the cosine eigenvectors, all the
coefficients other than a 0 are 0. Thus, this case is rather trivial. The point of this exercise is to
demonstrate how we can represent any twice-differentiable function as a a linear combination of the
eigenvectors. It is always a good practice to start with something simple to make sure the method
works. We try another function below.
5
eigencos.mcd
Example. Express the following function as a linear combination of the eigenvectors.
∞
1
1
.
α 25 f( x )
→ rewrite as → f( x ) a0
aj. cos λj. x
2
.
1 αx
2
j=1
N 5
j 1 .. N ← Change the number of terms N and see how the approximation changes.
1
λ0
0
f( x ) .
a0
1
dx
a0 = 0.388
2
1
1
λj
π. j
f( x ) . cos λj. x dx
aj
1
f approx( x )
N
a0
aj. cos λj. x
2
j=1
1
f approx( x )
0.5
f( x )
0
1
0.5
0
0.5
1
x
Fourier Series Approximation
Given Function
The first derivative of the given function is,
Analytical solution:
2 . α. x
f'( x )
1
2
α. x
2
N
Linear combination of eigenvectors:
aj. λj . sin λj. x
f' approx( x )
j=1
The second derivative of the given function is,
Analytical solution:
f''( x )
2 . α.
2
3 . α. x
1
2
α. x
1
3
N
Linear combination of eigenvectors:
2
aj. λj . cos λj. x
f'' approx( x )
j=1
6
eigencos.mcd
First Derivative
Second Derivative
5
50
f' approx( x )
f'' approx( x )
0
f'( x )
0
f''( x )
5
50
1
0
1
1
0
x
1
x
Fourier Series Approximation
Given Function
Fourier Series Approximation
Given Function
Example. Express the following Bessel's function as a linear combination of the eigenvectors.
∞
1
.
f( z ) J0( z ) for z=[α, β] α 0 β 20 → rewrite as → f( x ) a
a . cos λ . x
0
xx( z )
N
α)
2. ( z
10
β
1
α
j
β
z( x )
α.
(x
1)
j
2
α
j
j=1
2
← Change the number of terms N and see how the approximation changes.
1 .. N
1
λ0
0
f( z( x ) ) .
a0
1
dx
2
1
1
λj
π. j
1
f( z( x ) ) . cos λj. x dx
aj
f( z( x ) ) . sin λj. x dx
bj
1
a0
f approx( x )
2
1
N
N
aj. cos λj. x
j=1
b j. sin λj. x
zz
j=1
1
α, α
0.1 .. β
1
0.5
0.5
f approx( x )
f approx( xx( zz ) )
f( z( x ) )
f( zz )
0
0
0.5
0.5
1
0
1
x
Fourier Series Approximation
Given Function
0
10
20
zz
Fourier Series Approximation
Given Function
7
eigencos.mcd
Actually the eigenvalues λ do not need to be integer multiples of π. The eigenvectors only needs to
satisfy the following general symmetric B.C. in the Sturm-Liouville probem. And there are many
ways to satisfy it.
p y'm. y n y'n. y m
p y'm. y n y'n. y m
p 1
a 1
b 1
x a
x b
One way is:
c. y
y'
y'm c. y m
at x=a=-1 and y' c. y at x=b=1, where c=any constant
y'n c. y n
p c. y m . y n
c. y n . y m
c
( λ. sin( λ. x )
2
c. y n . y m
x a
p c. y m . y n
c. y n . y m
0 0
x b
root( λ. sin( λ )
1
... this will involve both sine and cosine.
x b
1
⎯→
λ. sin( λ. 1 )
⎯→
λ. sin( λ. 1 ) c. cos( λ. 1 )
c. cos( λ. 1 ) ⎯→
λ. sin( λ )
c. cos( λ )
⎯→
λ. sin( λ ) c. cos( λ )
λj
lambda λj
c. cos( λ ) , λ )
λ0 = 1.077
lambda( 1 )
d. yn . ym
⎯→ cos(λ⋅x) alone (without sin(λ⋅x)) or sin(λ⋅x) alone will do.
( λ. sin( λ. x ) c. cos( λ. x ) ) x
λ0
p d. ym . yn
x a
c. cos( λ. x ) ) x
lambda( λ )
c. y m . y n
p
y' c. y at x=a=-1 and y' d . y at x=b=1
Another way is:
Example.
⎯→
N
10
j
0 .. N
1
1
π
0
1
2
3
4
5
6
7
8
9
T
λ = 0 1.077 3.644 6.578 9.63 12.722 15.834 18.955 22.081 25.212 28.345
Approximate
f( x )
cos λj. x . cos λk. x dx
coscosj , k
1
1
coscos = 1 0
2 0
1
0
1.042
0
0
N
1
aj. cos λj. x
f approx( x )
1
2
0
1.116 0
3 0
f( x ) . cos λj. x dx
aj
0
1
0 1.388 0
1
Check orthogonality
j 0 .. N k 0 .. N
cos λj. x . cos λj. x dx
j=0
1
Cosine Eigenfunctions
1.1
cos λ 0. x
f approx( x )
1
cos λ 1. x
1
cos λ 2. x
0
cos λ 3. x
0.9
1
0
1
x
Because c can be any constant, the number
of possible eigenvalue-eigenvector sets is
infinite.
0
1
1
0.5
0
x
0.5
1
8
eigencos.mcd
Fourier Series. In the above development, we applied the the idea of eigenvalue-eigenvector to
represent any given vector from the LVS. For a general function (not necessarily odd nor even), we
keep both the cosine and sine eigenvectors. In addition, for simplicity, we take the eigenvector for
λ=0 to be 1 (which is not normalized), and shift the normalization factor of 2 to the a 0 term. In
mathematical literature, this is called the Fourier series. When the infinite series is truncated to a
finite number of terms, it is called the truncated Fourier series.
∞
∞
a0
f( x )
aj. cos λj. x
b j. sin λj. x
2
j=1
j=1
where
λj π. j
j 1 , 2 .. ∞
1
a0
1
f( x ) dx
1
aj
1
f( x ) . cos λj. x dx
f , yj
bj
f( x ) . sin λj. x dx
f , zj
1
1
Cosine Series. For even functions, keep only the cosine terms.
∞
1
a0
f( x )
aj. cos λj. x
aj
f( x ) . cos λj. x dx
2
1
j=1
Sine Series. For odd functions, keep only the sine terms.
∞
1
a0
f( x )
b j. sin λj. x
b j f , zj
f( x ) . sin λj. x dx
2
1
j=1
λj π. j
j 1 , 2 .. ∞
λj π. j
j 1 , 2 .. ∞
Note that a 0 is the average of the given function f(x) within the interval x=[-1, 1]. For problems
where the given interval is not within x=[-1, 1], we can always transform the given interval from
x=[α, β] to x=[-1, 1]. The example above demonstrates this.
Fourier Series in Complex Form. cos and sin can be expressed in terms of exp, and vice versa.
e
i .θ
cos( θ )
cos( θ )
e
i . sin( θ )
i .θ
e
... Euler's formula
i .θ
sin( θ )
2
e
i .θ
e
2. i
i .θ
where i2 =i⋅i=-1 is the imaginary number, not
to be confused with an index.
Substituting the above cos and sin expressions into the Fourier series formula, we obtain,
∞
∞
a0
exp i . λj. x
exp i . λj. x
exp i . λj. x
exp i . λj. x
f( x )
aj.
b j.
2
2
2. i
j=1
j=1
∞
f( x )
1.
2
A0 a0
∞
Aj. exp i . λj. x
A0
j=1
Bj. exp i . λj. x
j=1
Aj aj
i . bj
Bj aj
i . bj
9
1
A0
eigencos.mcd
1
f( x ) dx
1
f( x ) . exp i . λj. x dx
Aj
1
Bj
f( x ) . exp i . λj. x dx
f , zj
1
1
Note that Aj is the coefficient for the exp(i*λj *x) term, but Aj is the scalar product of f(x) and
exp(-i*λj *x), not exp(i*λj *x). Likewise for Bj . Why do we interchange the terms in the integral that
defines the scalar product? On the surface, it seems this is not following the projection formula.
However, we are dealing with complex numbers here, and a scalar product definition that is valid for
real Euclidean space is invalid for complex space. We need to modify the definition slightly for
complex space so that it obeys the general rules of scalar products.
β
f( x ) . g( x ) dx
( f, g )
where f( x ) is the complex conjugate of f(x).
f Re( f )
i . Im( f )
α
or
β
f( x ) . g( x ) dx
( f, g )
α
Detour. For columns of complex numbers, either one of the following definitions may be suitable
(but the definition is not unique).
T
(x,y)
x
.y
T
( x , y ) x .y
or
where x is the complex conjugate of x.
Example.
1
x
i
T
T
.y
Definition #1.
(x,y)
Definition #2.
T
( x , y ) x .y
x
x
T
Definition #3 (bad). ( x , y ) ( x ) . y
.x = 2
(1
1
i ).
T
( x ) .x = 2
( 1 i ).
T
x .x = 0
( 1 i ).
i
1
i
1
i
=2
... ok
=2
... ok
... bad example.
=0
The 3rd definition is invalid, because the length of a nonzero vector should be positive for the
definition to be valid.
Example. The above two definitions give two different scalar products that are complex
conjugate of each other. However, the lengths from both definitions are identical.
x
1
2
i
i .3
y
1
i
1
i .3
T
.y
Definition #1.
(x,y)
Definition #2.
T
( x , y ) x .y
x
T
x
.y = 9
T
3i
T
x . y = 9 + 3i
Complex conjugate ↑
x
. x = 15
T
x . x = 15
T
y
. y = 12
T
y . y = 12
↑ Identical ↑
10
eigencos.mcd
The eigenfunctions exp(i*λj *x) and exp(-i*λj *x) are mutually orthogonal.
1
1
exp i . λj. x , exp i . λk. x
exp i . λj. x . exp i . λk. x dx
1
1
exp i . λj. x , exp i . λj. x
1
1
1
exp( 0 ) dx
1
1 dx 2
1
1
exp i . λj. x . exp i . λk. x dx
1
1
exp i . λj. x , exp i . λj. x
λk . x dx 0
1
1
exp i . λj. x . exp i . λj. x dx
exp i . λj. x , exp i . λk. x
exp i . λj
exp i . λk
λj . x dx 0
1
1
exp i . λj. x . exp i . λj. x dx
1
exp 2 . i . λj. x dx 0
1
We can further compact the linear combination expression into a simpler form.
∞
1
1.
f( x )
Aj. exp i . λj. x
Aj
f( x ) . exp i . λj. x dx
2
1
j= ∞
Or, equivalently,
∞
1.
f( x )
Bj. exp i . λj. x
2
j= ∞
1
f( x ) . exp i . λj. x dx
Bj
1
In the Fourier series approach, we break up a given function into various frequency components. The
coefficient Aj tells us how much of the given function f(x) can be expressed as a sinusoidal oscillation
of frequency λj . Thus, Fourier series allows us to extract the various frequency components
imbedded in the given function. The set of coefficients {Aj } are the amplitudes at the corresponding
frequencies, and the whole set is commonly referred to as the spectrum of f(x). Fourier series is
very important in many aspects of engineering analysis: harmonic oscillation, spectral analysis,
digital signal processing, noise filtering, ...
A closely related idea is Fourier transform, which, too, breaks a given continuous function into
continuous frequency components, rather than just at frequencies corresponding to eigenvalues of a
harmonic oscillator. We extend the integration limit between -∞ and +∞ and replace the summation
sign in the last expression with an integral in a continuous domain.
∞
F( f( x ) ) F( ω )
exp( i . 2 . π. ω. x ) . f( x ) dx
forward Fourier transform (analogous to the
coefficients {Aj }.
∞
∞
F( F( ω ) )
1
exp( i . 2 . π. ω. x ) . F( ω ) dω
f( x )
∞
Inverse Fourier transform (analogous to the
linear combination equation.
11
Many other versions of Fourier transform and inverse transform pairs exist.
eigencos.mcd
∞
F( f( x ) ) F( ω )
exp( i . 2 . π. ω. x ) . f( x ) dx
∞
∞
F( F( ω ) )
1
exp( i . 2 . π. ω. x ) . F( ω ) dω
f( x )
∞
In the following pair, we factor out the 2*π term and distribute it equally between the forward and
inverse transform pairs.
F( f( x ) ) F( ω )
∞
1 .
exp( i . ω. x ) . f( x ) dx
∞
2. π
F( F( ω ) )
1
∞
1 .
f( x )
exp( i . ω. x ) . F( ω ) dω
∞
2. π
In the following pair, we shift the 2*π term to the inverse transform. We can also shift the 2*π term to
the forward transform.
∞
F( f( x ) ) F( ω )
exp( i . ω. x ) . f( x ) dx
∞
F( F( ω ) )
1
f( x )
∞
1 .
2. π
exp( i . ω. x ) . F( ω ) dω
∞
Our Fourier series here most closely resembles the following transform pairs.
∞
F( f( x ) ) F( ω )
exp( i . π. ω. x ) . f( x ) dx
∞
F( F( ω ) )
1
f( x )
1.
2
∞
exp( i . π. ω. x ) . F( ω ) dω
∞
Discretized version of Fourier transform:
n
1 .
k
Fj
fk. exp i . 2 . π. j.
... Forward Fourier transform
n
n k=0
n
fj
1 .
k
Fk. exp i . 2 . π. j.
n
... Inverse Fourier transform
n k=0
Examples of Fourier transforms: Our ears and the associated nerve cells convert mechanical sound
waves in the time-domain into frequency (pitch in sound) and the associated amplitude (loudness).
Likewise, our eyes transform electromagnetic waves in the visible region into frequency (i.e., colors)
and the associated amplitude (brightness). In terms of common household items, a radio extracts a
specific frequency component, and the television also works in a similar fashion.
12
eigencos.mcd
Fourier series is a method of expressing a given function with linearly independent functions taken
from the same LVS; it is not a linear transform. In other words, Fourier series, extended to infinite
number of terms, is just a representation of the same function; it does not change one function to
another. It is expressing a vector with a set of linearly independent basis vectors. Fourier transform,
on the other hand, converts one function to another function. In other words, it is a function of
functions (vectors); thus, Fourier transform is a linear transform.
Fourier series is analogous to Taylor's series expansion of a given function. Taylor's series is based
on the various derivatives evaluated at one single point. Whereas, the Fourier series is based on
various integrals over an interval. Taylor's series is exact at one single point around which expansion
is made, and the approximation degrades as we move farther away from that point of expansion.
Fourier series gives a good approximation over the entire interval.