Orthogonal polynomials
Gérard MEURANT
October, 2008
1
Definition
2
Moments
3
Existence
4
Three-term recurrences
5
Jacobi matrices
6
Christoffel-Darboux relation
7
Examples of orthogonal polynomials
8
Variable-signed weight functions
9
Matrix orthogonal polynomials
Definition
[a, b] = finite or infinite interval of the real line
Definition
A Riemann–Stieltjes integral of a real valued function f of a real
variable with respect to a real function α is denoted by
Z
b
f (λ) dα(λ)
(1)
a
and is defined to be the limit (if it exists), as the mesh size of the
partition π of the interval [a, b] goes to zero, of the sums
X
f (ci )(α(λi+1 ) − α(λi ))
{λi }∈π
where ci ∈ [λi , λi+1 ]
I
if f is continuous and α is of bounded variation on [a, b] then
the integral exists
I
α is of bounded variation if it is the difference of two
nondecreasing functions
I
The integral exists if f is continuous and α is nondecreasing
In many cases Riemann–Stieltjes integrals are directly written as
Z
b
f (λ) w (λ)dλ
a
where w is called the weight function
Moments and inner product
Let α be a nondecreasing function on the interval (a, b) having
finite limits at ±∞ if a = −∞ and/or b = +∞
Definition
The numbers
Z
µi =
b
λi dα(λ), i = 0, 1, . . .
(2)
a
are called the moments related to the measure α
Definition
Let P be the space of real polynomials, we define an inner product
(related to the measure α) of two polynomials p and q ∈ P as
Z
hp, qi =
b
p(λ)q(λ) dα(λ)
a
(3)
The norm of p is defined as
b
Z
12
p(λ) dα(λ)
2
kpk =
(4)
a
We will consider also discrete inner products as
hp, qi =
m
X
p(tj )q(tj )wj2
(5)
j=1
The values tj are referred as points or nodes and the values wj2 are
the weights
We will use the fact that the sum in equation (5) can be seen as
an approximation of the integral (3)
Conversely, it can be written as a Riemann–Stieltjes integral for a
measure α which is piecewise constant and has jumps at the nodes
tj (that we assume to be distinct for simplicity), see Atkinson;
Dahlquist, Eisenstat and Golub; Dahlquist, Golub and Nash


if λ < t1
0P
i
2
α(λ) =
if ti ≤ λ < ti+1 i = 1, . . . , m − 1
j=1 [wj ]

Pm
2
if tm ≤ λ
j=1 [wj ]
There are different ways to normalize polynomials:
A polynomial p of exact degree k is said to be monic if the
coefficient of the monomial of highest degree is 1, that is
p(λ) = λk + ck−1 λk−1 + . . .
Definition
I
The polynomials p and q are said to be orthogonal with
respect to inner products (3) or (5), if hp, qi = 0
I
The polynomials p in a set of polynomials are orthonormal if
they are mutually orthogonal and if hp, pi = 1
I
Polynomials in a set are said to be monic orthogonal
polynomials if they are orthogonal, monic and their norms are
strictly positive
The inner product h·, ·i is said to be positive definite if kpk > 0
for all nonzero p ∈ P
A necessary and sufficient condition for having a positive definite
inner product is that the determinants of the Hankel moment
matrices are positive


µ0
µ1 · · · µk−1
 µ1
µ2 · · ·
µk 


det  .
.
..  > 0, k = 1, 2, . . .
.
.
 .
.
. 
µk−1 µk
· · · µ2k−2
where µi are the moments of definition (2)
Existence of orthogonal polynomials
Theorem
If the inner product h·, ·i is positive definite on P, there exists a
unique infinite sequence of monic orthogonal polynomials related
to the measure α
See Gautschi
Minimization properties
Theorem
If qk is a monic polynomial of degree k, then
Z
min
qk
b
qk2 (λ) dα(λ),
a
is attained if and only if qk is a constant times the orthogonal
polynomial pk related to α
See Szegö
We have defined orthogonality relative to an inner product given
by a Riemann–Stieltjes integral but, more generally, orthogonal
polynomials can be defined relative to a linear functional L such
that L(λk ) = µk
Two polynomials p and q are said to be orthogonal if L(pq) = 0
One obtains the same kind of existence result, see the book by
Brezinski
Three-term recurrences
The main ingredient is the following property for the inner product
hλp, qi = hp, λqi
Theorem
For monic orthogonal polynomials, there exist sequences of
coefficients αk , k = 1, 2, . . . and γk , k = 1, 2, . . . such that
pk+1 (λ) = (λ − αk+1 )pk (λ) − γk pk−1 (λ), k = 0, 1, . . .
p−1 (λ) ≡ 0, p0 (λ) ≡ 1.
where
αk+1 =
γk =
hλpk , pk i
, k = 0, 1, . . .
hpk , pk i
hpk , pk i
, k = 1, 2, . . .
hpk−1 , pk−1 i
(6)
Proof.
A set of monic orthogonal polynomials pj is linearly independent
Any polynomial p of degree k can be written as
p=
k
X
ωj pj ,
j=0
for some real numbers ωj
pk+1 − λpk is of degree ≤ k
pk+1 − λpk = −αk+1 pk − γk pk−1 +
k−2
X
j=0
Taking the inner product of equation (7) with pk
hλpk , pk i = αk+1 hpk , pk i
δj pj
(7)
Multiplying equation (7) by pk−1
hλpk , pk−1 i = γk hpk−1 , pk−1 i
But, using equation (7) for the degree k − 1
hλpk , pk−1 i = hpk , λpk−1 i = hpk , pk i
we multiply equation (7) with pj , j < k − 1
hλpk , pj i = δj hpj , pj i
The left hand side of the last equation vanishes
For this, the property hλpk , pj i = hpk , λpj i is crucial
Since λpj is of degree < k, the left hand side is 0 and it implies
δj = 0, j = 0, . . . , k − 2
There is a converse to this theorem
It is is attributed to J. Favard whose paper was published in 1935,
although this result had also been obtained by J. Shohat at about
the same time and it was known earlier to Stieltjes
Theorem
If a sequence of monic orthogonal polynomials pk , k = 0, 1, . . .
satisfies a three–term recurrence relation such as equation (6) with
real coefficients and γk > 0, then there exists a positive measure α
such that the sequence pk is orthogonal with respect to an inner
product defined by a Riemann–Stieltjes integral for the measure α
Orthonormal polynomials
Theorem
For orthonormal polynomials, there exist sequences of coefficients
αk , k = 1, 2, . . . and βk , k = 1, 2, . . . such that
p
p
βk+1 pk+1 (λ) = (λ − αk+1 )pk (λ) − βk pk−1 (λ), k = 0, 1, . . .
(8)
Z
p
p−1 (λ) ≡ 0, p0 (λ) ≡ 1/ β0 , β0 =
a
where
αk+1 = hλpk , pk i, k = 0, 1, . . .
and βk is computed such that kpk k = 1
b
dα
Relations between monic and orthonormal polynomials
Assume that we have a system of monic polynomials pk satisfying
a three-term recurrence (6), then we can obtain orthonormal
polynomials p̂k by normalization
p̂k (λ) =
pk (λ)
hpk , pk i1/2
Using equation (6)
kpk+1 kp̂k+1 =
hλpk , pk i
λkpk k −
kpk k
p̂k −
kpk k2
p̂k−1
kpk−1 k
After some manipulations
kpk k
kpk+1 k
p̂k+1 = (λ − hλp̂k , p̂k i)p̂k −
p̂k−1
kpk k
kpk−1 k
Note that
hλp̂k , p̂k i =
and
p
βk+1 =
hλpk , pk i
kpk k2
kpk+1 k
kpk k
Therefore the coefficients αk are the same and βk = γk
If we have the coefficients of monic orthogonal polynomials we just
have to take the square root of γk to obtain the coefficients of the
corresponding orthonormal polynomials
Jacobi matrices
If the orthonormal polynomials exist for all k, there is an infinite
symmetric tridiagonal matrix J∞ associated with them


√
α1
β1 √
√
 β1 α2

β2 √


√
J∞ = 

β
α
β
2
3
3


..
..
..
.
.
.
Since it has positive subdiagonal elements, the matrix J∞ is called
an infinite Jacobi matrix
Its leading principal submatrix of order k is denoted as Jk
Orthogonal polynomials are fully described by their Jacobi matrices
Christoffel–Darboux relation
Theorem
Let pk , k = 0, 1, . . . be orthonormal polynomials, then
k
X
pi (λ)pi (µ) =
p
βk+1
i=0
k
X
pi2 (λ) =
pk+1 (λ)pk (µ) − pk (λ)pk+1 (µ)
, if λ 6= µ
λ−µ
(9)
0
βk+1 [pk+1
(λ)pk (λ) − pk0 (λ)pk+1 (λ)]
p
i=0
Corollary
For monic orthogonal polynomials we have
k
X
i=0
γk γk−1 · · · γi+1 pi (λ)pi (µ) =
pk+1 (λ)pk (µ) − pk (λ)pk+1 (µ)
, if λ 6= µ
λ−µ
Properties of zeros
Let
T
Pk (λ) = p0 (λ) p1 (λ) . . . pk−1 (λ)
In matrix form, the three-term recurrence is written as
λPk = Jk Pk + ηk pk (λ)e k
(10)
where Jk is the Jacobi matrix√of order k and e k is the last column
of the identity matrix (ηk = βk )
Theorem
(k)
The zeros θj of the orthonormal polynomial pk are the
eigenvalues of the Jacobi matrix Jk
Proof. If θ is a zero of pk , from equation (10) we have
θPk (θ) = Jk Pk (θ)
This shows that θ is an eigenvalue of Jk and Pk (θ) is a
corresponding (unnormalized) eigenvector
Jk being a symmetric tridiagonal matrix, its eigenvalues (the zeros
of the orthogonal polynomial pk ) are real and distinct
Theorem
The zeros of the orthogonal polynomials pk associated with the
measure α on [a, b] are real, distinct and located in the interior of
[a, b]
see Szegö
Examples of orthogonal polynomials
Jacobi polynomials
dα(λ) = w (λ) dλ
a = −1, b = 1,
w (λ) = (1 − λ)δ (1 + λ)β , δ, β > −1
Special cases:
Chebyshev polynomials of the first kind: δ = β = −1/2
Ck (λ) = cos(k arccos λ)
They satisfy
C0 (λ) ≡ 1, C1 (λ) ≡ λ,
Ck+1 (λ) = 2λCk (λ) − Ck−1 (λ)
The zeros of Ck are
λj+1 = cos
2j + 1 π
k 2
, j = 0, 1, . . . k − 1
The polynomial Ck has k + 1 extremas in [−1, 1]
jπ
0
λj = cos
, j = 0, 1, . . . , k
k
and Ck (λ0j ) = (−1)j
For k ≥ 1, Ck has a leading coefficient 2k−1


0 i 6= j
< Ci , Cj >α = π2 i = j 6= 0


π i =j =0
2
1.5
1
0.5
0
−0.5
−1
−1.5
−2
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Chebyshev polynomials (first kind) Ck , k = 1, . . . , 7 on [−1.1, 1.1]
Let πn1 = { poly. of degree n in λ whose value is 1 for λ = 0 }
Chebyshev polynomials provide the solution of the minimization
problem
min max |qn (λ)|
qn ∈πn1 λ∈[a,b]
The solution is written as
C 2λ−(a+b) n
b−a
1
min max |qn (λ)| = max =
a+b
a+b λ∈[a,b] qn ∈πn1 λ∈[a,b]
Cn b−a
Cn b−a
see Dahlquist and Björck
Chebyshev polynomials of the second kind
δ = β = 1/2
sin(k + 1)θ
, λ = cos θ
sin θ
They satisfy the same three–term recurrence as the Chebyshev
polynomials of the first kind but with initial conditions
Uk (λ) =
U0 ≡ 1, U1 ≡ 2λ
Of all monic polynomials qk , 2−k Uk gives the smallest L1 norm
Z 1
kqk k1 =
|qk (λ)| dλ
−1
5
4
3
2
1
0
−1
−2
−3
−4
−5
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Chebyshev polynomials (second kind) Uk , k = 1, . . . , 7 on [−1.1, 1.1]
Legendre polynomials
δ=β=0
(k+1)Pk+1 (λ) = (2k+1)λPk (λ)−kPk−1 (λ), P0 (λ) ≡ 1, P1 (λ) ≡ λ
The Legendre polynomial Pk is bounded by 1 on [−1, 1]
2
1.5
1
0.5
0
−0.5
−1
−1.5
−2
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Legendre polynomials Pk , k = 1, . . . , 7 on [−1.1, 1.1]
Laguerre polynomials
The interval is [0, ∞] and the weight function is e −λ
The recurrence relation is
(k + 1)Lk+1 = (2k + 1 − λ)Lk − kLk−1
L0 ≡ 1, L1 ≡ 1 − λ
30
20
10
0
−10
−20
−30
−2
0
2
4
6
8
10
12
14
16
18
20
Laguerre polynomials Lk , k = 1, . . . , 7 on [−2, 20]
Hermite polynomials
2
The interval is [−∞, ∞] and the weight function is e −λ
The recurrence relation is
Hk+1 = 2λHk − 2kHk−1
H0 ≡ 1, H1 ≡ 2λ
1000
800
600
400
200
0
−200
−400
−600
−800
−1000
−10
−8
−6
−4
−2
0
2
4
6
8
10
Hermite polynomials Hk , k = 1, . . . , 7 on [−10, 10]
Variable-signed weight functions
What happens if the measure α is not positive?
Theorem
Assume that all the moments exist and are finite
For any k > 0, there exists a polynomial pk of degree at most k
such that pk is orthogonal to all polynomials of degree ≤ k − 1
with respect to w
see G.W. Struble
The important words in this result are: “of degree at most k”
In some cases the polynomial pk can be of degree less than k
C (k) = set of polynomials of degree ≤ k orthogonal to all
polynomials of degree ≤ k − 1
C (k) is called degenerate if it contains polynomials of degree less
than k
If C (k) is non-degenerate it contains one unique polynomial (up
to a multiplicative constant)
Theorem
Let C (k) be non-degenerate with a polynomial pk
Assume C (k + n), n > 0 is the next non-degenerate set. Then pk
is the unique (up to a multiplicative constant) polynomial of lowest
degree in C (k + m), m = 1, . . . , n − 1
dk −dk−1 −1
pk (λ) = (αk λdk −dk−1 +
X
βk,i λi )pk−1 (λ) − γk−1 pk−2 (λ), k = 2, .
i=0
(11)
p0 (λ) ≡ 1,
p1 (λ) = (α1 λ
d1
+
dX
1 −1
β1,i λi )p0 (λ)
i=0
The coefficient of pk−1 contains powers of λ depending on the
difference of the degrees of the polynomials in the non-degenerate
cases
The coefficients αk and γk−1 have to be nonzero
Matrix orthogonal polynomials
We would like to have matrices as coefficients of the polynomials
For our purposes we just need 2 × 2 matrices
Definition
For λ real, a matrix polynomial pi (λ), which is a 2 × 2 matrix, is
defined as
i
X
(i)
pi (λ) =
λj Cj
j=0
(i)
where the coefficients Cj are given 2 × 2 real matrices
If the leading coefficient is the identity matrix, the matrix
polynomial is said to be monic
The measure α(λ) is a matrix of order 2 that we suppose to be
symmetric and positive semi–definite
We assume that the (matrix) moments
Z
Mk =
b
λk dα(λ)
(12)
a
exist for all k
The “inner product” of two matrix polynomials p and q is defined
as
Z b
hp, qi =
p(λ) dα(λ)q(λ)T
(13)
a
Two matrix polynomials in a sequence pk , k = 0, 1, . . . are said to
be orthonormal if
< pi , pj >= δi,j I2
(14)
where δi,j is the Kronecker symbol and I2 the identity matrix of
order 2
Theorem
Sequences of matrix orthogonal polynomials satisfy a block
three–term recurrence
pj (λ)Γj = λpj−1 (λ) − pj−1 (λ)Ωj − pj−2 (λ)ΓT
j−1
p0 (λ) ≡ I2 ,
(15)
p−1 (λ) ≡ 0
where Γj , Ωj are 2 × 2 matrices and the matrices Ωj are symmetric
The block three-term recurrence can be written in matrix form as
λ[p0 (λ), . . . , pk−1 (λ)] = [p0 (λ), . . . , pk−1 (λ)]Jk + [0, . . . , 0, pk (λ)Γk ]
(16)
where


Ω1 ΓT
1
 Γ1 Ω2 ΓT

2




..
..
..
Jk = 

.
.
.


T

Γk−2 Ωk−1 Γk−1 
Γk−1 Ωk
is a block tridiagonal matrix of order 2k with 2 × 2 blocks
Let P(λ) = [p0 (λ), . . . , pk−1 (λ)]T
Jk P(λ) = λP(λ) − [0, . . . , 0, pk (λ)Γk ]T
Theorem
For λ and µ real, we have the matrix analog of the
Christoffel–Darboux identity,
(λ − µ)
k−1
X
T
T
pj (µ)pjT (λ) = pk−1 (µ)ΓT
k pk (λ) − pk (µ)Γk pk−1 (λ)
j=0
(17)
F.V. Atkinson, Discrete and continuous boundary problems,
Academic Press, (1964)
C. Brezinski, Biorthogonality and its applications to
numerical analysis, Marcel Dekker, (1992)
T.S. Chihara, An introduction to orhogonal polynomials,
Gordon and Breach, (1978)
G. Dahlquist and A. Björck, Numerical methods in
scientific computing, volume I, SIAM, (2008)
G. Dahlquist, S.C. Eisenstat and G.H. Golub,
Bounds for the error of linear systems of equations using the
theory of moments, J. Math. Anal. Appl., v 37, (1972),
pp 151–166
G. Dahlquist, G.H. Golub and S.G. Nash, Bounds for
the error in linear systems. In Proc. of the Workshop on
Semi–Infinite Programming, R. Hettich Ed., Springer (1978),
pp 154–172
W. Gautschi, Orthogonal polynomials: computation and
approximation, Oxford University Press, (2004)
G.W. Struble, Orthogonal polynomials: variable–signed
weight functions, Numer. Math., v 5, (1963), pp 88–94
G. Szegö, Orthogonal polynomials, Third Edition, American
Mathematical Society, (1974)