On Solving Fredholm Integral Equations of the First Kind
In&an Institute of Technology, Bombay, Indm
illustrated The solution f(x) of the integral equation is assumed to be a sample function of a wide-sense
stationary random process with known autocorrelaUon function. From the set of permissible solutions, the
solution that "best" satisfies the statistical properties of the random process is admitted as the correct
solution With a kernel matrix A, the search for this solution is carried out by introducing the orthogonal
frame of reference of the symmetrized matrix ArA and then suitably adjusting the components along the
principal axes with small eigenvalues ofATA (1 e small singular values ofA), The method is illustrated for an
example first considered by Philhps and also for another problem from the area of image processing
KEY WORDS AND PHRASES" Fredhoim integral equations of the first kind, numerical solution, random
function, ill-conditioning, singular values, autocorrelatlon function
1. Introduction
I n m a n y physical p r o b l e m s , s u c h as t h e m e a s u r e m e n t of s p e c t r a l d i s t r i b u t i o n , c o s m i c
r a d i a t i o n , a n d t h e a n g u l a r v a r i a t i o n of s c a t t e r e d light, i n d i r e c t s e n s i n g devices a r e u s e d
[3, 5, 6]. H e r e t h e r e l a t i o n b e t w e e n t h e q u a n t i t y o b s e r v e d a n d t h e q u a n t i t y to b e
m e a s u r e d IS g i v e n b y a F r e d h o l m i n t e g r a l e q u a t i o n o f t h e first k i n d :
f b K(y, x)f(x)dx = g(y),
a -< y -< b.
(1)
T h e d e g r a d a t i o n o f a n i m a g e c a u s e d by t h e p o i n t s p r e a d f u n c t i o n o f t h e o p t i c a l s y s t e m
is also a n e x a m p l e r e p r e s e n t e d b y t h e s a m e e q u a t i o n .
I n all s u c h cases f(x) c a n b e o b t a i n e d b y solving (1) w h e n g(y) is t h e o b s e r v a t i o n at
v a r i o u s y v a l u e s a n d t h e k e r n e l K(y, x) is k n o w n . T h e p r o b l e m of solving (1) is in m a n y
cases difficult b e c a u s e t h e s o l u t i o n m a y b e e x t r e m e l y s e n s i t i v e t o t h e p r e s e n c e of no~se in
g(y), s u c h as m e a s u r e m e n t e r r o r s o r r o u n d o f f e r r o r s . W i t h t h e p r e s e n c e o f n o i s e , (1)
s h o u l d b e r e w r i t t e n as
f ~ K(y , x)f(x)dx = g(y) + E(y),
a -< y-< b.
(2)
H e r e t h e e r r o r f u n c t i o n ~(y) is a r b i t r a r y e x c e p t for s o m e r e s t r i c t i o n o n its m a g n i t u d e . T h e
family o f a l l o w e d f u n c t i o n s ¢(y) d e f i n e s a f a m i l y F o f a l l o w e d functionsf(x), b u t m o s t o f
t h e s e s o l u t i o n s will n o t b e physically m e a n i n g f u l .
Phillips [8] a s s u m e d t h e d e s i r e d s o l u t i o n to b e r e a s o n a b l y s m o o t h a n d t r i e d to find
t h e f u n c t i o n f(x) ~ F t h a t is s m o o t h e s t in s o m e s e n s e . H e c h o s e as t h e s m o o t h n e s s
c r i t e r i o n t h a t t h e s e c o n d d e r i v a t i v e b e m i n i m u m . T w o m e y [11, 12] simplified a n d
g e n e r a l i z e d t h e m e t h o d o f Phillips; his m e t h o d IS s i m p l e b e c a u s e it i n v o l v e s o n l y o n e
m a t r i x i n v e r s i o n r a t h e r t h a n two. T w o m e y also s h o w e d h o w t o g e n e r a l i z e t h e m e t h o d
in order to use a priori constraints other than Phillips's smoothness constraint. However,
in both cases no systematic method of determining the required amount of smoothing is
given. Strand and Westwater [10] and Helstrom [41 have given similar procedures for
solving (2) They assume f ( x ) and ~(y) to be independent stochastic Gaussian processes
with zero mean and known covariances. Vemuri and Fang-Pai [13] have given an initial
value method which gives a family of solutions, and to select one they have to use some
smoothing constraint. In their method also, the amount of smoothing required is not
determined. In the method described below the constrained function f ( x ) is assumed to
be a sample function of a wide-sense stationary process w~th known autocorrelation function, which can be the case in many practical problems. The method automatically incorporates the amount of smoothing required. We also show how the constraints suggested
by Phillips and Twomey can be interpreted in terms of the autocorrelation function.
2. Method
Letf(x) be a sample function of a wide-sense stationary process with known autocorrelation function m the interval a -< x -< b and let K ( y , x) be a continuous or a piecewise
continuous function of x and y. To solve (2) numerically, it is necessary to make the
variables discrete and replace the integral equation by a set of finite linear equations.
Continuous variables x and y can be replaced by sets of finite mesh points
a -< Yl < Y2 < " "" < Ynl-I < Ynl ~- b,
a-<x~ < x 2 < " " <xn2-1 < x . 2 - < b ,
where nl --> nz. The integral equation can then be replaced by a set of equations
g(y,) + ~(y,) =
K ( y , x)f(x)dx = ~ wjK(y,, x,)f(xj),
(3)
J=l
where t = 1, 2, ... , nl and Wl, wz, ... , wn2 are the weighting coefficients for the
quadrature formula used. Vectors f, g, and ~ and matrix A can be defined as
f=f(xj),
e=e(y,),
g=g(y,),
and
A = w , K ( y , xj),
where i = 1, 2, ... , n~ a n d / = 1, 2, ... , n2. With the above notation, (3) can be rewritten
as
A f = g + ~.
(4)
Let H be the g-inverse of matrix A ; then by introduction of the orthogonal frame of
reference of the symmetrized matrix A r A , H can be decomposed into U[D, 0IV [9],
where U and V are square orthogonal matrices, the columns of U are eigenvectors of
A r A , and the rows of V are eigenvectors of A A r. [D, 0] represents an n, x n l matrix
and D = diag[1/toa, 1/to, . . . . . 1/ton2], where col, toz . . . . . ton2 are singular values of A .
Hence the "true" solution of (2) can be written as
n2
f, = ~ UoflJ¢o~,
i = 1, 2 . . . . .
n2,
(5)
J=l
where 13 = Vg.
Here the components of vector /3 are not known to any desired degree of accuracy
because of the presence of errors e, in measurement of g,. Even small inaccuracies in
the components of/3 corresponding to small singular values cause considerable variations
in the values o f f . However, f is relatively insensitive to the errors in the components of
13 corresponding to large singular values. Therefore these components may be presumed
to be known on the basis of measurements of g,. In eq. (6) they are written as known
constants C~, C~ . . . . , Cn,, but the components of 13 corresponding to small singular
values are considered to be unknowns. They are to be determined on the basis of the a
priori knowledge about f that we have m the known autocorrelation function o f f . Let
the n u m b e r of unknown 13 components be m ; then (5) can be rewritten as
626
f t = Ci "~ Ui,vl2-mqF1/3ll2-m+l/(Onl-)~l-t-1 -~ "''
+ U,,n,-1/3n2-1/oJn:-I + U,,n~/3nJcon:,
i = 1, 2 . . . . .
n 2,
(6)
The criterion used for "smallness" of a singular value is as follows. In solving the
system of (4), if ¢ is the error in g, then ~, the error in the solution f , is given by [14]
II,1 II/Ill II <- (@.a~/em,.) (11~ II/11 g II),
(7)
where tOraaxand totaln are the largest and the smallest singular values of A , respectively. If
we choose IIx II -- [ ~ , xfl 1'~ and ~,n~l e~ _< e ~ , then (7) may be rewritten as
II~ II/IIf II <- C(e/IIg II),
(8)
where the condition number C is the ratio of Omax to corot,. In (8) the ratio 11o II/Ill II may
now be interpreted as the usual noise/signal ratio. In a physical situation, the upper
bound to the noise/signal ratio is usually known. This gives us the permissible upper limit
to the condition number C. Equation (8) gives the upper bound of 11o II when the error
vector ¢ is oriented in favor of the principal axis of A r A corresponding to the smallest
singular value. If the error vector is oriented along the principal axis corresponding to
any other co,, then the corresponding condition number would be O.,)max/(O t. The criterion
for the smallness of a singular value is that all a), for which the condition number (Omax/O0z
exceeds the permissible limit should be considered as small and the corresponding terms
in expansion (6) should be treated as unknown.
However, the maximum number of components of/3 that may be treated as unknowns
is limited. The vector f is a finite sample of a stochastic process. For a good point
eshmate of the autocorrelation function, the number of lag points for which it is
estimated should not, as a rule of thumb, exceed n 2 / 5 , where n2 is the total number of
sample points. Preferably it should be limited to n 2 / l O [1]. If the number of small
singular values is less than n 2 / 5 , then the system'of (6) can be an overdetermined system.
On the other hand, if the number of small singular values is more than n 2 / 5 , then the
number of unknowns in (6) has to be restricted to terms corresponding to the smallest
n2/5 singular values. Since the error bound on f given by (7) is rather pessimistic [14],
even in such a case the possibility of successful solution is quite good.
If the points x, in (4) are chosen to be equally spaced, then the estimated ad~ocorrelation function can be written as
/~
R(i) =
AA+~ + f~A+, + "'" + f n z - f n 2
f ~ + f [ + "'" + f~2
,
i = 1, 2 . . . . .
(9)
k,
A
where m -< k -< n z / 5 . By equating the estimated autocorrelation function R(i) With the
known autocorrelation function R(i), we get
R(i) - R(i) = O,
i = 1, 2 . . . . .
(10)
k.
Equation (10) can be rewritten with the help of (6) and (9) for each value of/, yielding
F~(fln:-m+l . . . . .
/3n=-l, fin) = O,
i = 1, 2 . . . . .
k,
(11)
where FI, F2 . . . . , Fk represent quadratic equations in unknown/3 values which are then
determined as solutions of (11). Substituting these fl values in[o (6) gives the required f.
Consider the function
k
S(/3nz-m+i . . . . .
/3nz-1, /3nz) = ~ [F,(/3nz-ra+l, ... , /3he-l, /3nz)] 2.
It is obvious that each solution of system (11) reduces the function S to zero;
contrariwise the /3 values for which the function S is zero are the roots of the system
(11). Hence the solution of system (11) can be obtained by finding the minimum of the
function S in m-dimensional space Em = (/3,~-m+~ . . . . . /3,~_~,/3.,). The system of (11)
could also be an overdetermined system. In the illustrative examples presented here,
for minimizing S a modified sequential simplex method has been e m p l o y e d [2].
It can easily be shown that Phillips's constraint for choosing an f that satisfies the
condition
[y"(x)] ~ dx = min
[f"(x)] 2 dx
/'EF
is essentially choosing an f which satisfies the condition mm~F [3R(0) - 4R(1) + R(2)].
The constraints of minimizing higher derivatives can also be expressed in an analogous
manner.
3. Numerical Results
Example 1 (Phillips's problem).
g(y) =
K(x - y)f(x) dx,
6
K(z) = 1 + cos(Trz/3)
= 0
for [z I _m 3,
for Izl > 3,
g(y) = (6 + y)(1 - (1~2)cos(cry~3)) - (9/2~-)sin(cry/3)
=0
for
for
[Y[ -< 6,
[y[ > 6.
The variables x and y were made discrete by using 25 equally spaced points. The values
of g, were truncated at the third decimal point, yielding I~,1 -< 0.01, which givese ~ = 25 x
1 0 - ' . It was desired to keep the noise/signal raUo less than ten times the noise/signal
ratio in the measurement g,. This forces the condition number in (8) to be equal to 10.
The number of small singular values, according to this criterion, is 18. This is far in
excess of the maximum number of permissible unknowns, which Is n2/5 = 5. Hence the
number of unknowns in (6) was chosen to be equal to 5. Since in this e x a m p l e f is not a
sample function of a stochastic process, its autocorrelation function was obtained from
the known analytical solution. Estimated f and the true solution and the solution
obtained by simple i n v e r s i o n f = A -1 (g + E) are shown in Figure 1. The closeness of the
estimated solution to the true solution is mdicauve of the pessimistic error bound in (7).
Example 2.
g(Y) =
f ll2
~-,12
K(y - x)f(x)dx,
where K(z) = 1/Ctcos(sm-~(z/Cz)) and C~, Cz are constants.
The kernel K(y - x) chosen in this example ~s a point spread function arising in
certain image processing problems where the degradation of the image is due to
vibratory motion of the imaging device. The variables x and y were m a d e discrete by
using 64 equally spaced points. The error in g, was such that I~,1 -< 0.25, yielding e = 4.
The noise/signal ratio criterion used was the same as in Example 1; as a consequence
the number of unknowns in (6) is 8. This is less than the maximum permissible number
of unknowns, which is 12 in the present case. Hence the system of (11) is overdetermined (12 equaUons). The sequence f for this example was generated as an output of a
difference equation driven by white noise [7], and hence the autocorrelation funcUon
of f is a known function in terms of the constants of the difference equation. The
estimated solution and the true solution and the solution due to numerical i n v e r s i o n f =
A - ~ ( g + E) are shown in Figure 2.
4. Summary
A method for solving Fredholm integral equations of the first kind in the presence of
628
True
.....
solution
Solution
f = , ~ l (9+¢)
• , • Estimoted
solution
A
I I
L2"VM,.;,
,H,!
f
.A
e,
y
'K
,^,
-^~
V
X
-1
-2
FIG
.
1
True solution
....
Solution
f = A I (9~.E)
• • • Estimoted
solution
10
I,
8
lit
lit
6
I. ,
2
F,
A
eof~
~'Jp..~
iJ
II
I ....
iI
I ~,
•
i ,
#I
( ! ,el If- !l '~
'.
=,
2...1,:,
.~,
,'"~ ~, r.. l ' l
=...~
1 ' - 4',
, ~. ~ . ';?
v
,"
,1
;," L'.l' "~ vl~t
11/7- - I I
.,,
"r-q,"°"
"" VII
"It
r~i
I It ~'I
~ l-
'
"r
.
/L
~,
,:
'~
I
,,
~ :
, :
,.,_l.
~!
o
N
!
-2
X
-4
-6
FiG
2
measurement errors in g ( y ) has been derived and illustrated. To solve numerically, the
integral equation is first made discrete, yielding a system of linear equations of the type
A f = g + ~. The v e c t o r f is then expressed in terms of singular values o f A . The condition
number C for the desired noise/signal ratio of the solution f is then evaluated. The
s i n g u l a r v a l u e s f o r w h i c h t h e r a t i o tOmax/~O, e x c e e d s t h e c o n d i t i o n n u m b e r C a r e c o n s i d ered as small singular values, and the corresponding components/9~g + ~ are treated as
u n k n o w n s . T h e s e a r e t h e n d e t e r m i n e d b y m i n i m i z i n g S = ~ = ~ [R(i) - R ( i ) ] ~ • T h e s o l u t i o n is c o m p u t e d s u b s e q u e n t l y f r o m t h e s e o b t a i n e d c o m p o n e n t s .
REFERENCES
1 BENDAT,J S , AND PIERSOL, A G Measurement and Analysts o f Random Data Wiley, New York, 1966.
2. BEVERIDGE, G S G , AND SCHECHTER, R S Opnmlzatton: Theory and PracUce. McGraw-Hill, New
York, 1970
3 GILLE, J C Inversion of radiometrlc measurements Bull. Amer. Meteorol Soc. 49, 9 (1968), 903-912.
4 HELSTROM,C W Image restoration by the method of least squares. J Opt Soc. Amer. 57 (1967), 918922
5 HUNt, B The inverse problem of radiograph lnt J Math Btosct. 8 (1970), 161-169.
6 KAHN, P D The correction of observational data for instrumental band width. Proc. Cambridge Philos.
Soc. 51 (1955), 519-525
7 NAHI, N E. Est~manon Theory and Apphcattons. Wiley, New York, 1969
8 PHILLIPS, D L A technique for the numerical solution of certain integral equations of the first kind. J.
ACM 9, 1 (Jan 1962), 84-97
9. RUST, B W., AND BURRUS, W R Mathematical Programmmg and the Numertcal Solution o f Linear
Equations American Elsevier, New York, 1972
10 STRAND, O N , AND WESTWATER, E R Statistical estimation of the numerical solution of a Fredholm
integral equation of the first kind. J. ACM 15, 1 (Jan. 1968), 100-114
11 TWOMEY,S. On the numerical solution of Fredholm integral equations of the first kind by the inversion of
the linear system produced by quadrature J ACM 10, 1 (1963), 97-101.
12 TWOMEY, S The application of numerical hltermg to the solution of integral equations encountered in
redirect sensing measurements J. Franklin lnst Tech 279, 2 (1965), 95-109.
13 VEMURI, V., AND FANG-PAl On solving Fredholm integral equations of the first kind. J. Frankhn Inst.
Tech 297, 3 (1974), 187-200
RECEIVED DECEMBER 1 9 7 4 ,
REVISED NOVEMBER 1 9 7 6
Journal of the Associationfor ComputingMachinery,Vol 24, NO 4, Octobcg1977.