On the maximum errors of polynomial approximations defined
by interpolation and by least squares criteria
By M. J. D. Powell*
A function f(x) is to be approximated by a polynomial of degree n or less over the
interval a < x < b. It is proved that the maximum errors of approximations defined by interpolation and by least squares criteria are within factors, independent off(x), of the least maximum
error that can be achieved. Expressions for these factors are given; they are evaluated for
approximations obtained by truncating the expansion of f(x) in Chebyshev polynomials and for
approximations obtained by interpolation at the zeros of a Chebyshev polynomial. The resultant
numbers are not large: for example, if interpolation is used to evaluate a polynomial approximation of degree 20 it may be guaranteed that the resultant maximum error does not exceed the
minimax error by more than a factor of four.
that the redundancy allows higher order polynomial
approximations to be obtained without additional
values of f{x), but we stress that, if f(x) is calculated
precisely, it is not reasonable to expect a least squares
approximation to be substantially more accurate
(accuracy being measured by the maximum value of the
error function) than one obtained by interpolation at
the recommended points.
The minimax approximation, p*(x), is defined by
1. Introduction
A common reason for wishing to approximate f(x) is
that many values of the function are required, but the
evaluation of each one is laborious. In this case it may
prove economic to use function values to obtain an
approximation to replace f(x) in the main calculation.
Often the approximating function is chosen to be a
polynomial, and we suppose that this choice has been
made. The suitability of polynomials for this purpose
is not questioned in this paper, rather we present and
exploit theorems comparing the maxima of the error
functions arising from approximations denned by interpolation, least squares and minimax criteria.
There is an excellent motive for preferring the polynomial approximation, p+(x) say, obtained by interpolating/(x) at (n + 1) prescribed values of the variable;
calculating it requires the least number of evaluations of
f(x).
However, straightforward interpolation is seldom
used, perhaps because the well-known Runge phenomenon (see Todd, 1962, for instance) has led to the
belief that the interpolation process may introduce
extravagant errors. We shall establish that this is not
the case for practical values of n (say n < 1000) if the
interpolation points are chosen to be the zeros of a
Chebyshev polynominal.
Least squares methods (Clenshaw and Hayes, 1965)
are probably the most popular. The criterion depends
on a non-negative weight function, co(x), the required
approximation being pw{x), where
fu>(x)[f(x)-p«(x)]2dx< \boj(x)[f(x)-p(x)]2dx,
max |/(x) - p*{x)\ < max | f(x) - p(x)\,
(2)
where p(x) is any polynomial of degree n. Calculating
it (Fraser, 1965) is an iterative process, and one must
balance extra evaluations of f(x) with the reduction they
may cause in the maximum error of the current
approximation. The theorems bound the gains that
may be obtained by preferring a minimax approximation
to one defined by least squares or interpolation.
I am indebted to Drs. D. Kershaw and W. Cheney for
the following observations. Theorem 2 is a corollary of
the theorem numbered 4.5.1 in Alexits (1961). Equation (25) relates u(n) to the Lebesgue constant, several
properties of which are given by Hardy (1942). Luttmann and Rivlin (1965) conjecture that v(n), defined by
(35), is obtained for 6 = 0; we prove the conjecture.
2. The theorems
We use the notation x0, xu ..., xn for the interpolation
points defining p +{x). We define the error functions
(1)
e+(x)=f(x)-p+(x) 1
e»(x) = / ( * ) - p»(x) \
e*(x)=f{x)-p*{x\
J
p(x) being any polynomial of degree n. In the algorithms
the integrals are replaced by numerical quadratures, so,
as in interpolation, the approximation depends on specific
evaluations of/(x). The number of evaluations will probably exceed (n + 1), so, usually, p"(x) will not reproduce
the function values; thus the magnitude of the error of
the approximation is suggested. A further advantage is
(3)
and reserve h+, h" and h* for the maximum values of
\e+(x)\, \ea{x)\ and |e*(x)|, a < x < b. Therefore
* Mathematics Branch, Theoretical Physics Division, Atomic Energy Research Establishment, Harwell, Berkshire.
404
Polynomial approximation
Theorem 1
h+/h* < 1 + max
i=0
h* {l +
where /,(x) is the polynomial of degree n satisfying
/,(*,) = 8y, j = 0, 1, . . ., 77.
(6)
e*(x) - e+(x) = p+(x) - p*(x),
(7)
£o<f>,(.x)<f>Ay) dy].
(16)
Again the inequality holds for a < x < 6, which proves
Theorem 2.
Note that an alternative form of (11) may be obtained
by using the identity (Szego, 1939)
Proof
which is a polynomial of degree n or less satisfying
e*(Xj) - e+(Xj)
= e*(Xj),
j = 0, 1, . . . , « .
(
(8)
V
>7
(17)
where A:n is the coefficient of x" in <f>n(x). (17) aids the
numerical evaluation of (11) because, on account of the
modulus signs, it is expedient to divide the range of
integration at the zeros of the integrand. These zeros
are the values of y satisfying
Therefore
e*(x) - «+(*) = f e*(x;)/,.(x),
n+l
(9)
1= 0
from which we obtain
\e+(x)\ = \e*(x) - S e*(*,)/,(*)|
;=o
<f>n+\{x)
;=o
It may be established that the theorems provide least
upper bounds on h+/h* and h^/h* in the event that the
function f(x) may be any bounded square-integrable
function defined on a < x < b.
(10)
|
i=0
Since (10) holds for a < x < b, the theorem follows.
3. Application to truncated Chebyshev expansions
In this case the range of approximation is—1 < x < 1,
Theorem 2
h<*lh* < 1 + max
S 4>,{
»=0
, (11)
co(x) = (l - x 2 ) - ' / 2 ,
where <f>0(x), <f>x(x), . . . is the sequence of orthonormal
polynomials satisfying
= Su.
#0(JC) = 1/Vw
2 rf
u(n) = 1 + max o e ^ J
(13)
2'
i=o
|o
cos (tO) cos
(21)
-..(22)
where the prime on the summation sign indicates that the
first term is to be halved. Now
V'
2J
« (txi
COS (/A)
,=o
sin {(H +
=
—^—:—rr-jT
2 sin {i<f>}
so
(14)
1
w(«) = 1 + max =-
Therefore, using (12), (13) and (14),
a, = fo,(y)e*(y)Uyyy.
(20)
*,(«» 0) = V(2/7r) cos (t0), t=l,2,...
Hence the bound provided by Theorem 2 is
(12)
where the numbers a0, a,, . . ., a n are to be determined.
Since eu(x) is the error function of the least squares
approximation,
fa>(x)e%x)Ux)dx = 0,» = 0, 1, . . ., ».
(19)
and
Proof
We may write
e*(x) — e"(x) = S <*/<£,(*)>
(18)
0
B
&
77
'
(15)
sin {K« + </<)}
4-
sin {(n + i)(« sin {i(0 -
Hence
= 1 + max ,.477
\e»(x)\ = e*(x) - •£
sin {i(0 + tfi)}
, sin {(n + i)(0 sin {MB 405
(23)
Polynomial approximation
Table 1
1
< 1 + max -r-J
The dependence of u and v on n
sin{K0+ «/-)}
/I
u(n)
f(«)
1
2
3
4
5
6
7
8
9
10
15
20
25
50
100
200
500
1000
2-436
2-642
2-778
2-880
2-961
3-029
3-087
3138
3183
3-223
3-381
3-494
3-583
3-860
4139
4-419
4-789
5-070
2-414
2-667
2-848
2-989
3-104
3-202
3-287
3-362
3-429
3-489
3-728
3-901
4-037
4-466
4-901
5-339
5-920
6-361
sin {±(0-</-)}
sin {(« +
sin
(24)
As this upper bound is attained when 0 = 0,
u{n) = 1 + -
sin {(« + *)</-}
(25)
In order to tabulate u(n) we re-employ (23), noting
that in (25) the zeros of the integrand occur at
•, j = 1, 2, . . ., n.
(26)
Therefore, denning if>0 — 0 and *]>„+1 = it,
cos
2 "
= i+-
T
S(-iy
7T y = 0
l,(x) is derived by appealing to the identity, valid for
#+
ij = 0, 1, . . ., n.
L
4 "
(27)
from which it follows that
Now
2
"
7 Zi cos
n + 1 ,=o
Hence the required bound is
(28)
S ( - i y sin
and
sin 0 ^ ) =
cos ( e )
/,(cos 0) =
cos {j<f>} — cos {(« +
2 sin
r max
-(34)
£
i O^B^-H
Hence
'
;=o ( = 0
(35)
Using (23) again,
1
Using (30) values of u(n) were calculated; they are
presented in the second column of Table 1.
It is apparent that the asymptotic dependence is
logarithmic, and from (25) we may derive
4
w(n)'— —j log «.
max
i=o
sin {(« + WO + 6,)}
sin {}(0 + 0,)}
sin {(« + i)(d - 0,)}
sin {±(0-0,-)}
where
(31)
, (36)
(37)
0, =.(«
Since
4. Application to interpolation at the zeros of a Chebyshev
polynomial
= sin {(« + 1)0 ± (/ + i)7r - i(0 ± »/)}
= ± ( - D ' c o s {(« + 1)0 -H8±
0,)}, (38)
We apply Theorem 1 to bound h+/h* in the event that
the range of the approximation is — 1 < x < 1 and the
interpolation points are
we may obtain from (36)
(32)
v{n) =
406
1
(« + 1) max
(39)
Polynomial
approximation
where
asymptotic dependence on n is
y(0) = | cos {(« + 1)0}| £ | cot
+ 0,)}
- cot {±(0 - 0,)}|.
v(ri) ~ - log n.
(40)
We now establish that y(0) attains its maximum value
at 0 = 0.
Consider the effect on x(0) of adding or subtracting
an integral multiple of -, ,-•; to 0. Only the terms
under the summation sign of (40) are altered, and the
definition of 0, is such that the same 2(« + 1) cotangents
occur, the change in 0 causing them to be paired differently. Since y(0) is an even function of 0 and since if
0<
0<
(41)
both |(0 + 6,) and -}(# — 0,) are in the
[0, in,] for i = 0, 1, . . ., n, we deduce that the
maximum value of y(0) is attained subject
Now the zeros of /,(cos 0) are all real, and
occur in the interval
2(II +
1)"^
^
interval
required
to (41).
they all
142)
Consequently, within the interval (41) each term
n+
(43)
of (35) is monotonic decreasing. We conclude
max X(0)
O=S8 <7t
= y(0)
(44)
(46)
TT
5. Remarks
We have not lost generality in supposing, in Sections 3
and 4, that the range of approximation is — 1 < x < 1.
If the range is a < x < b it may be established, through
the transformation
/ = ( a + b - 2x)/(a - b),
(47)
that the third column of Table 1 is appropriate to the
interpolation points
b) + {b-a) cos
j = 0, 1, . . ., n. (48)
The basis of Theorems 1 and 2 is that the difference
between the approximations that are being compared
is a polynomial whose coefficients are determined from
the properties of the approximations. This device has
several applications.
The Chebyshev weighting function (19) is not optimal
if the criterion is adopted that OJ{X) is to be chosen to
minimize (11); indeed the optimal OJ(X) may depend on
n. Golomb (1965) asserts that no choice of weighting
function can improve the asymptotic dependence given
by (31) by more than a factor of two.
This paper is not directly relevant to deciding whether
a polynomial of degree n can provide an adequate
representation of/(x). Rather we have established that
if the polynomial obtained by interpolation at the zeros
of a Chebyshev polynomial is not satisfactory, it is
unlikely that the minimax polynomial approximation of
degree n is acceptable.
Therefore the required bound is
6. Acknowledgements
I am indebted to Mr. A. R. Curtis for many profitable
discussions and to Mr. M. J. Hopper for programming
the calculation of Table 1.
it is tabulated in the third column of Table 1. The
References
ALEXITS, G. (1961). Convergence Problems of Orthogonal Series, Pergamon Press, London.
CLENSHAW, C. W., and HAYES, J. G. (1965). "Curve and surface fitting," / . Inst. Maths. Applies., Vol. 1, p. 164.
FRASER, W. (1965). "A survey of methods for computing minimax and near minimax polynomial approximations for functions
of a single independent variable," / . Ass. Comp. Mach., Vol. 12, p. 295.
GOLOMB, M. (1965). "Optimal and nearly-optimal linear approximation" in Approximation of Functions, editor H L Garabedian
Elsevier Publishing Co., p. 83.
'
HARDY, G. H. (1942). "Note on Lebesgue's constants in the theory of Fourier series," Jnl. Lon. Math. Soc Vol 17 p 4
LUTTMANN F. W., and RIVLIN, T. N. (1965). "Some numerical experiments in the theory of polynomial interpolation
"IBM
Jnl. of Res. and Devel., Vol. 9, p. 187.
SZEGO, G. (1939). "Orthogonal polynomials," Amer. Math. Soc. Colloquium Publications, Vol. 23.
TODD, J. (1962). A Survey of Numerical Analysis, McGraw-Hill, New York.
407