A stabilized inversion for limited angle tomography
Tim Olsony
Department of Mathematics, Bradley Hall
Dartmouth College, Hanover, NH 03755
(603) 646 - 3173
Contents
1 Introduction
2 Background and Denitions
2
2
3 Limitations of the Singular Value Decomposition
5
2.1 The Radon transform : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2
2.2 Tomography and Limited Angle Tomography : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4
2.3 Physical Motivation and Prior Work : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4
3.1 Unbounded Inverses and Approximate Identities : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5
3.2 Uncorrelated, Exact Bases and Induced Correlations : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6
3.3 Decreasing Signal To Noise Ratio : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6
4 Mollication Methods
4.1
4.2
4.3
4.4
4.5
4.6
Continuous Approximation to a Discrete Operator : : :
Discrete Approximations to the Continous Operator : :
Szego's theory for nite Toeplitz operators : : : : : : : :
Limited angle spectra : : : : : : : : : : : : : : : : : : :
Uncertainty principles and signal recovery : : : : : : : :
Non-linear constraints, induced correlations, and POCS
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
6
6
7
8
9
10
10
5 The algorithm
6 Implementation
11
12
7 Conclusion
12
6.1 Numerical results : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12
This work was supported in part
y Thanks are due to Dennis Healy,
by DARPA as administered by the AFOSR under contract AFOSR-90-0292.
Reese Prosser, Dan Rockmore, and Richard Zalik for their advice and encouragement.
1
Abstract
In this paper we present a method for reconstructing a function f : R2 ! R from limited angle tomographic
data. This reconstruction problem occurs in many physical systems, where physical limitations prohibit the
gathering of tomographic data a certain angles.
We begin by reviewing some of the classical work on singular value decompositions and POCS in the context
of this problem. We then review some of the classical work by G. Szego and others on nite Toeplitz operators.
We consider the implications of this work toward a classical inversion of the problem.
We then introduce an alternate inversion technique which utilizes multiresolution analysis, induced correlations
caused by non-linear constraints, and non-linear ltering to mollify the reconstruction process. We show that the
uncertainty principles generated in recent works of Donoho, Stark et al guarantee the invertibility but of this
alternative inversion technique. We also utilize the noise reduction techniques of Donoho and Johnstone to reduce
the eects of noise on the process.
Finally we present numerical examples of the algorithm. We accurately recover images from limited angle
tomographic data.
1 Introduction
Many problems in applied mathematics involve recovering a function f from measurements of Lf, where L is a
known operator. If L is a linear operator, with a bounded inverse, then f can recovered from noisy data Lf + n via
standard techniques with little diculty. The recovery of f from noisy data Lf + n is much more dicult if L is an
operator whose inverse is unbounded.
In this paper we will study the recovery of a function f : R2 ! R, from limited knowledge of its Fourier transform
^f. Thus we are interested in reconstructing f from Lf where L is an operator which reects the limited knowledge
of the Fourier transform of f. This problem is motivated by the limited angle tomography problem. This recovery
problem is ill-posed and unsolvable without a priori assumptions about f. For example, if f is compactly supported,
then f^ will be an analytic function, and therefore f^ will be uniquely determined by its values on any region containing
a limit point [3].
Classical analytic continuation techniques are not numerically feasible [25], and the operator L, will generally not
have a bounded inverse. A technique for doing analytic continuation was introduced by Papoulis in [22]. This algorithm depends upon the eigenfunctions of the continuous time and band-limiting operators, which were extensively
studied in the classical works of Slepian, Pollack, and Landau [33]. The convergence of the algorithm is dependent
upon the eigenvalues of the operator, and except in extreme cases, this algorithm is not feasible numerically [10].
Most data recovery problems can be modelled as the inversion of a compact operator L ( for limited data ); where
^
Lf = XS (Xd
C f);
S is the common support of the functions under consideration, C is the set where we can measure the Fourier
transform of f, and XO is a characteristic, or indicator function on the set O.
We will to study the inversion of L, from a discrete signal processing perspective. In this context, the continous
operator L is naturally viewed as the the limiting case of a series of discrete operators. Since L will generally be
compact and self-adjoint, its spectrum can been analyzed via standard techniques. Real-world problems are generated
from discretely sampled data sets. Therefore we believe that it more informative to study the spectra of the discrete
approximations to L rather than the spectrum of L itself.
Our main tool for analyzing the spectra of these discrete approximations to L will be the theory of nite Toeplitz
forms, originally introduced by Szego [4]. We will show that the study of these nite Toeplitz forms will allow us to
construct an accurate, stable inversion for L, even when the continuous spectra of L suggests that it is not invertible.
We will then apply these results to the extensively studied ill-conditioned problem of limited angle tomography.
2 Background and Denitions
2.1 The Radon transform
We will review some of the basic properties of the Radon transform. For details and proofs, we recommend Natterer
[25]. We let S 1 denote the unit circle, i.e.
S 1 = f~ = (cos ; sin )g;
2
where we let denote the angle generated by ~ and the x-axis. We dene a corresponding vector perpendicular to ~
by ~? (? sin ; cos ). Throughout we will use the notation Fn to denote the Fourier transform on L2 (Rn) dened
by
Z
1
^
Fn(f)(~s ) f(~s) = (2)(n?1)=2
f(~x)e?i~s~xdx:
R
We will refer to the band-limiting operators Bl which are most easily dened by their action restricting the Fourier
transform of f, i.e.
^ s) ! f(~
^ s)Xj~sj<l (~s):
Bl : f(~
The fundamental theoretical tool behind tomography has always been the Radon transform [31, 25]. The Radon
transform, which is sometimes referred to as the x-ray transform or the projection transform, of a function f : R2 ! R
is dened by
Z
(1)
Pf (~; ~ ~x) = f(~x + t~? )dt:
n
R
It follows that Pf (~; ~ ~x) is the line integral of f through ~x 2 R2 and perpendicular to ~. We will often abuse this
notation by associating the vector ~ with the angle , between ~ and the positive x-axis.
An essential property of the Radon transform is that the one-dimensional Fourier transform of Pf (~; ~ ~x) with
respect to the variable t ~ ~x is a line through the two-dimensional Fourier transform of f, i.e.
^ ~):
F1 Pf (~; ~ ~x) = F2(f)(s~ ) f(s
(2)
Using this fact and a polar Fourier inversion formula, one can easily relate the Radon transform Pf (~; ~ ~x) to f(~x)
through the reconstruction formula
f(~x) =
Z Z
S1
R
^ ~)jsjei(~xs~) dsd =
f(s
Z Z
S1
R
F1 Pf (~; ~ ~x) (s)jsjei(~xs~) dsd
(3)
The inner integral in (3) is simply a one-dimensional inverse Fourier transform. Insertion of an appropriate bandlimiting window into (3) yields the standard ltered backprojection formula [25]
fw (~x) =
Z Z
S1
R
F1 Pf (~; ~ ~x) (w(s)jsj)ei(~x~)s dsd =
Z
S1
Pf (~; ~ ~x) F1?1(w(s)jsj)d:
(4)
Figure 1: The Radon transform isR illustrated above. The notation ~? stands for the vector which is perpindicular
to ~. The line integral Rf (; r) = f(r~ + t~? )dt is taken along the illustrated line at a distance r from the origin.
3
2.2 Tomography and Limited Angle Tomography
In this paper, tomography refers to the recovery of a two-dimensional density f : R2 ! R from one-dimensional
line integrals, via an inversion formula such as (4). Tomography will generally refer to the recovery of such a two
dimensional density from a full complement of one-dimensional line integrals, i.e. Pf (~; ~ ~x) for all ~ 2 S 1 , or an
evenly subsampled version of these line integrals. Typical sampling algorithms conditions can be found in Natterer
[25].
Limited angle tomography refers to the recovery of a function f from a proper subset of its line integrals. More
specically, in limited angle tomography only the line integrals Pf (~; ~ ~x) where jj < < =2 are available for the
inversion. From (2) we know that the one-dimensional Fourier transform of each projection corresponds to a line
through the origin of the two dimensional Fourier transform of the original image.
The diculty of recovering a function from limited angle tomographic data can be understood through the
reconstruction formulas (2),(3), and (4). We can see via (2) that if the projection data Pf (~; ~ ~x) is only available
^ ~) will only
for a restricted angular range, ? < < , where < =2 then the Fourier data F2(f)(s~) f(s
be available for the restricted angular range ? < < . We know from (3), and (4) that all of the Fourier, or
projection data is needed to accurately reconstruct f. Thus we are forced to try to recover an unknown portion of
the Fourier transform of f, as is illustrated in Figure 2.
Known
Fourier Data
Unknown
Unknown
Fourier Data
Fourier Data
Known
Fourier Data
Figure 2: The limited data problem generated by limited angle tomography is illustrated above. In order to accurately
reconstruct the original density, we must recover an unknown portion of the Fourier transform of f.
If we know nothing about the objective function f : R2 ! R, then this reconstruction would certainly be
impossible. Usually, though, the support of the object is nite and known. It is also generally the case that the
function is positive and real. Therefore, we assume throughout that we are interested in reconstructing a bounded
function f : S \ R2 ! R, from the values of its Fourier transform, where S is a compact set, containing the origin.
This will imply that the Fourier transform of f will be an entire function of nite exponential type, and therefore
the continuous problem is solvable. Classical analytic continuation methods, however, are not numerically feasible
[10, 25].
The creation of a stable numerical inversion for this problem is the subject of this paper.
2.3 Physical Motivation and Prior Work
The limited angle tomography problem arises in a number of contexts. In biology, transmission electron microscopy
is used to image the structure of chromatin bers in tissue. Because of the rapid absorbtion of the electrons an
extremely thin slice of tissue is prepared. This slice becomes prohibitively thick when the angle of incidence of the
rays goes beyond a certain threshold. Therefore only projections at angles within approximately 60 degrees of the
normal to the slice can be utilized.
Another application occurs in the non-destructive testing of welds in critical structures such as nuclear reactors,
rockets, and missiles. In this case, as before, data can only be collected at angles which do not cause severe attenuation
of the beams. From these examples we are able to generalize and see that there are a great number of applications
in which an accurate limited angle tomography algorithm would be extremely useful.
4
Limited angle tomography has been extensively studied. It has been proven [7, 9, 16] that when the singular
values of the associated operator are viewed as a function of the angular limit , they converge to zero extremely
quickly and as a result the problem is highly ill-conditioned.
One approach to the problem of reconstructing a density function from limited angle data is to compute the
projection of the known data onto the the range of the Radon transform [25]. The result is a completed data
set, and this data set is then processed via typical reconstruction algorithms to produce an approximation to the
original density function. The range theorems for the Radon transform make it possible to recover the low frequency
components of the image through this process. The crucial step in this process is the determination of the coecients
of a trigonometric polynomial, from non-uniform samples. The high frequency components of the image require the
recovery of high order trigonometric polynomials, however. This process is notoriously ill-conditioned, and as a result
the algorithm will not perform well when the angular limit is not close to =2 and a high resolution reconstruction
is desired.
In [32] a alternative reconstruction algorithm was proposed. It was speculated that it might be close to optimal.
It was shown in [34], however, that this algorithm formed a commutative diagram with ltered backprojection, and
as a result was of little use in limited angle tomography.
A recent paper by Quinto [30], uses the microlocal analysis concept of wavefront sets to quantify the information
which would be dicult to recover in limited data tomography. This work reinforces with the general feeling in
the area that the unknown high frequency information, which is generated by singularites, will be very dicult to
recover. There has been recent work by Ramm and Zaslovsky [26], which proposed the recovery of singularities
of a two-dimensional density from limited angle tomographic data using the Legendre transform. There were also
theoretical stability results. We are not aware of any numerical application of this result, however.
Finally, a thorough review of other approaches to the limited angle tomography problem can be found in Natterer
[25].
3 Limitations of the Singular Value Decomposition
3.1 Unbounded Inverses and Approximate Identities
The standard numerical method for inverting a linear operator is the singular value decomposition (SVD). The
singular value decomposition
of a linear operator L consists of the normalized eigenfunctions of L L, f n g, and the
p
functions fng = f1= n L ng wherepn is the eigenvalue corresponding to n . The vectors n and
p n are the
singular vectors, and the values n = n are the singular values. It follows that both f n g and f1= n A ng are
orthonormal sets for the domain and range of L. Most important is the fact that these vectors allow a decomposition
of L given by
X
Lf = nhf; n in:
(5)
n
Moreover, the singular value decomposition also yields an inversion formula
f=
X ?1
n hLf; n i n :
n
(6)
A great number of problems in applied mathematics can be reduced to the inversion of a linear operator, and as a
result, the singular value decomposition has been extremely important in applied and numerical mathematics.
Unfortunately, since the operator L is usually compact, the values n ! 0 as n ! 1 [1]. Thus when only nite
precision is available in both the measurements and computer facilities, the usefulness of (6) is somewhat limited
for two reasons. The rst is that the storage of the data Lf with nite precision will cause the higher order terms
in the sum (5) to be beyond the wordsize of the machine, and thus they will be rounded to zero. In addition, the
multipliers n?1 in (6) will approach 1 as n ! 1. This makes the computation of the higher order terms in the sum
in (6) very unstable or impossible and therefore an altered, truncated version of (6) is often used.
These mollied methods, such as the truncated SVD, and Tikhonov-Phillips [25] regularization can usually be
represented in the form
X
Mf = w(n)n?1 hLf; ni n
n
where the windowing function w(n) is chosen so that w(n)n?1 2 l2. For instance the truncated SVD would dene
w(n) = 1 for n < N and zero for n N. Thus an approximate inversion is obtained, which is computationally
stable, but not exact.
5
The second problem with the SVD is that if there are two distinct classes of singular values: those which are close
to a xed value b > 0 and those which are nearly 0, then the inversion formulas will all reduce to an approximation
of a truncated SVD. If in addition, the operator is self-adjoint, the functions f ng will be equal to the functions
fng, and will be eigenvectors. In this case the truncated SVD will act as an approximate identity, since division by
the singular values which are all in the vicinity of b will only scale the function Lf, and the singular values which
are nearly 0 will truncated from the expansion. Thus the eort of constructing and inverting the SVD will not cause
a noticeable improvement in the nal result.
As we will show in Section 4.3, this is the case in limited angle tomography. The operator is compact and selfadjoint, and the singular values, or eigenvalues, of the operator separate distinctly into two classes, those which are
nearly 2 and those which are nearly 0.
3.2 Uncorrelated, Exact Bases and Induced Correlations
The SVD produces exact orthonormal bases for the range and domain of any compact linear operator. Although
there are many instances when an exact representation is useful, an exact representation is not desireable in this
type of situation.
When reconstruction problems, or inversion problems are ill-conditioned, we believe that an overcomplete basis,
which is correlated and redundant, is more useful than an exact basis. It is well known that in limited angle
tomography, and many other imaging problems, low resolution information can easily be reconstructed. With an
exact, uncorrelated basis, this low resolution information will not help to reconstruct the high frequency details which
are dicult to recover. If a correlated, redundant basis, is used, however, then the low resolution information can
aid in recovering the high frequency details.
We will utilize non-linear or linear constraints to induce correlations between the coecients of an orthonormal
basis. In our case, the initial non-linear constraints are positivity, compact support, and the known data. Once
we reconstruct the low-resolution information in the image, we will then use this low-resolution information as an
additional non-linear constraint.
3.3 Decreasing Signal To Noise Ratio
The linear POCS, algorithm, as outlined in [22], uses the singular functions of the time- and band-limitingoperators to
construct an approximate
inverse to the problem. More specically, if f is the desired function, with the eigenfunction
P
expansion for f = cnen , and we observe a blurred version of f, namely Lf, then the K th step of POCS yields the
result
X
P K (Lf) = (1 ? (1 ? n)K )cnen :
We observe that as K ! 1 this converges to f.
P
In the presence of white noise, however, we will observe Lf = (n cn + gn)en , where gn are i.i.d. observations
of white noise. Now the reconstruction procedure on the K th step will yield
X
P K (f + g) = (1 ? (1 ? n )K )(cn + 1 gn)en :
n
As K ! 1, it follows that the signal to noise ration of the solution will go to zero ( assuming that L was an innite
dimensional compact operator ). Moreover, the solution will converge to
X
(cn + 1 gn)en :
n
Since it is usually the case that n ! 0, this solution is not desireable.
4 Mollication Methods
4.1 Continuous Approximation to a Discrete Operator
A singular value decomposition for the operator was constructed in [9], and the asymptotics of the singular values of
the operator were investigated. It was proven that these singular values, when viewed as a function of the angular
limit , converge to zero extremely quickly. The singular value decomposition was also studied in [16]. There it was
shown that the condition number grows only linearly, for near =2, but grows much more quickly for smaller .
These studies concentrated on the continuous operator, rather than its discrete realization, however.
6
We will study the behavior of the spectra of the discrete approximations to the continuous operator L, for xed ,
as a function of the discretization. We will show that the an appropriate Hilbert-Schmidt norm of these discretizations
only grows linearly, and therefore, these discretized operators are not horribly ill-conditioned.
It was noted by Davison in [9] that \Our results should not be construed as proof that limited angle tomography
is impossible". Unfortunately, those results and the work of others on the SVD for the limited angle problem, have
led some to believe that limited angle tomography is impossible. It was also noted in [9] that \In practice there
are often important a priori constraints on the unknown function f", and that these constraints \have no natural
expression in terms of the singular functions." We will incorporate these constraints into our algorithm, in order to
help stabilize the problem.
We will present a method which uses a series of singular value decompositions, at increasingly ne resolutions,
to construct a stabilized inversion of the limited angle tomography problem. This stabilized inversion signicantly
outperforms the singular value decomposition for the problem.
The importance of this type of approach is partially due to the fact that it concentrates on the actual discrete
inversion of the operator in question, rather than a continuous approximation to the discrete inversion, or vice versa.
In this context, the continuous operator is naturally seen as a limiting case of the discrete operators. Moreover, the
spectrum of the continuous operator is necessarily worse than its discrete approximations, and thus we might be led
to believe that the problem is intractable, when in fact it is not as bad as it might seem.
4.2 Discrete Approximations to the Continous Operator
We will now outline the conversion of the limited angle tomography problem into a discrete signal processing problem.
The reconstruction process will begin with the collection of a nite number of measurements of the Radon transform.
A fundamental principle of signal processing is that innite frequency resolution is not possible, given only nite
measurements. In addition, the nal outcome of any imaging process is a N N pixel array of values which are
displayed for the scientist, radiologist, etc. This N N array should represent the properly sampled values of the
function f.
Since there were can only be a nite number of input measurements, and the output is a nite array, it follows
that only a nite number of frequencies are present in the reconstruction, (although some may be due to aliasing.).
^ [?; ] [?; ]. This is not inconsistent
Therefore we assume that f is a band-limited function, i.e. supp(f)
with our assumption of compact support in the former section. Rather, we should think of f as being the image of
a compactly supported function, under the band-limiting operator.
We also need an assumption concerning the essential support of the function f in time. We will be concerned with
the recovery of functions f 2 L2 (R2) which are band-limited, and are essentially compactly supported in the following
^ = [?; ] [?; ],
way. Since we have assumed that the functions under consideration are band-limited, i.e. supp(f)
we can represent them as sums of sinc functions, i.e.
XX
(x ? m) sin (y ? n) :
f(x; y) =
f(m; n) sin(x
? m) (y ? n)
We assume that all of the functions f are essentially compactly supported in the sense that they have nite sinc
expansions:
f(x; y) =
N X
N
X
1
1
(x ? m) sin (y ? n) :
f(m; n) sin(x
? m) (y ? n)
We will refer to the size of the pixel array, N, for an N N as above the discretization size of the operator. These
assumptions are consistent with the reconstruction of a compactly supported object.
If we restrict ourselves to the case where we know only 90 degrees of projection, or equivalently Fourier data as
is illustrated in Figure 3. If we assume that the rst and third quadrant of the Fourier transform is given then the
operator assumes the form
Lf = XS
N X
N
X
?N ?N
2 (x ? m) sin 2 (y ? n) cos ((x ? m) + (y ? n)):
f(m; n) sin (x
? m)
(y ? n)
2
2
2
(7)
This formula can be most easily realized by expanding the Fourier transform of f in an exponential series on
[?; ] [?; ], and seeing that the terms in the sum are translated versions of the inverse Fourier transform of the
characteristic function on the known data in Figure 3.
7
Unknown
Known
Fourier
Fourier
Data
Data
Known
Unknown
Fourier
Fourier
Data
Data
Figure 3: We illustrate the known data in a 90 degree Fourier reconstruction problem.
Since we are only interested in the values attained by Lf at the integers, which will represent our pixel values,
this naturally generates a linear operator from the original set of sample values f(m; n) to the new values Lf(m; n).
This operator can most eciently be viewed as a truncated discrete convolution of
2 (m) sin 2 (n) cos (m + n)
K(m; n) = sin (m)
(n)
2
2
2
sampled at the integers against the array ff(m; n)g.
We will now study the behavior of nite convolution, or Toeplitz operators of this type.
4.3 Szego's theory for nite Toeplitz operators
In this section we will study the spectral properties of discrete Toeplitz operators generated from compact operators
of the type
Lf = XS F2?1(f^XC ):
We will study the spectral properties of these discrete operators, as a function of the discretization N. We will then
use this study to introduce an altered inversion formula which will prove to be stable.
Szego's one-dimensional theory of Toeplitz operators extends, essentially with a change of notation, to ndimensions. For completeness, we will present a small portion of this here. What follows can be found in the
one-dimensional case in [5].
We begin with a convolution kernel K(m; n) which is a symmetric, square summable function in l2 (Z2), and
^ It follows that K^ is an integrable real function, and for
denote its Fourier transform on [?; ] [?; ] by K.
notational purposes we dene
Z
1
^ s)ei~s ds
K() = c = j
j K(~
^ or the values of K, and 2 Z Z.
to be the Fourier coecients of the K,
Let SN be a subset of the lattice of integers Z2 R2, which represents the eective support of our function, i.e.
SN = [0; 1; 2; :::N ? 1] [0; 1; 2; :::N ? 1] = ZN ZN where N is our discretization.
The operator (7) can then be viewed as the nite Toeplitz, or convolution operator
TN (K)(s) =
X
2
SN
c ? s ;
where K^ = XC . We will assume throughout that XC is symmetric about the origin, and it then follows that TN (K)
is a symmetric operator, and thus has real eigenvalues. If we let v be an eigenvector, with unit norm and eigenvalue
, it follows that
X X
(8)
= TN (K)(v) v =
c ?v v :
2
2
S N SN
8
Utilizing the denition of c along with a basic combinatorial identity we can express (8) in the following form:
Z X
X X
^
=
c ? v v = j
1 j j
v ei sj2K(s)ds:
(9)
2S 2S
2S
N
N
N
Assuming that m ess inf K < ess sup K M, it then follows from basic measure theory arguments that
m < < M:
In our case, we had normalized the kernel so that K(~0) = 1, and thus the Fourier transform of the kernel was
K^ = 2XC , where C is the set where we know the Fourier transform. Thus 0 = m < < M = 2 for each eigenvalue
and we can now see that our operators are non-singular, for each discretization N.
Much more is known about the eigenvalues of Toeplitz forms [5]. As the discretization N ! 1, the eigenvalues
become distributionally equivalent to K^ in the following sense. The distribution of the eigenvalues fN g2S of
nite Toeplitz operators, converges to the distribution of the values of K(~s), i.e.
N
lim
N !1
jfN ja < < bgj 1
^ s) < bgj;
= j
j jf~s ja < K(~
jSN j
as long as the measure of the sets where F(~s) = a or F(~s) = b is zero. In words, the proportion of eigenvalues in a
given interval, converges to the measure of the proportion of the set where F is in the corresponding interval.
To gain some insight into the ill-conditioned nature of inverting any limited angle problem, we will apply (10) to
the nite Toeplitz forms associate with the Fourier transform of our kernel K^ = 2XC . Since all of the eigenvalues
are positive, it follows that
jfN j0 < < gj
jfN j ? < < gj 1
^ s) < gj = j
j ? jC j :
lim
=
lim
= j
j jf~s j ? < K(~
(10)
N !1
N !1
jSN j
jSN j
j
j
Thus the proportion of eigenvalues which eventually tend toward 0 is exactly the same as the proportion of where
K is 0, or the measure of n C. More explicitly,
lim
N !1
jfN j0 < < gj = jUnknown Dataj :
jSN j
jNecessary Dataj
(11)
In the case which we have been studying, i.e. the 90 degree case, half of the eigenvectors must approach 0, and half
must approach 2.
This discussion highlights the problems associated with using a truncated or mollied SVD to invert this type of
operator, when the discretization N is large. There will be eigenvalues which are essentially zero, which will represent
the unknown data, and eigenvalues which are near 2, which represent the known data. If N is large, the truncated
SVD will only rescale the nal image, by dividing the known data by 2. The unknown coecients will be truncated
to zero, and no new information will be gathered.
4.4 Limited angle spectra
We have noted above that the spectra of limited data operators separate into two classes. There are eigenvalues
which are essentially one, and eigenvalues which are essentially zero. Because of the above discussion of the geometry
of the eigenvectors, it is not at all surprising that half of the eigenvalues are greater than 1=2, and nearly 1, and half
of the eigenvalues are less than 1=2, and nearly 0.
For the 90 degree limited angle problem, 8944 out of 16384 eigenvalues the are greater than .01, or 54%. Thus
a truncated singular value decomposition with a tolerance of .01 will recover only 4% more information than was
present in the original 50% of the known data. The dierence in image quality due to this recovered information is
negligible. Thus a truncated singular value decomposition, used at at discretization of 128x128 will act essentially
as an identity operator on the blurred image, rather than an eective inversion tool.
We now consider the spectra of the limited angle operators from smaller discretizations. In Figure 4, we have
plotted the spectra of the limited angle operators at various discretizations (note that normalizing factors cause them
to tend toward either 0 or 2. From the gure, we see that multiresolution analysis can be used to mollify of the
inversion. The low resolution operators are not massively ill-conditioned, and we should be able to reverse the eect
of the blurring at low resolutions. We want to use this information to help reverse the blurring at higher resolutions.
9
4.5 Uncertainty principles and signal recovery
Another way to understand the diculties of signal recovery, and the advantages of our approach, is via the work of
Donoho and Stark on uncertainty principles [10]. The classical uncertainty principle states that if a function f(t) is
^ is essentially zero outside an interval
essentially zero outside and interval of length t and its Fourier transform f(w)
of length w, then
tw 1:
A basic tenet of [10] is the following. Suppose that you are trying to recover a signal f(t) which are non-zero on
^ Suppose further that for some reason you do not
a region of size t, from knowledge of the Fourier transform f(!).
^
know the values of f(!) on some interval I where
1:
jI j < w t
^ can not be essentially zero outside of I, and thus you can observe f(t) from
Then by the uncertainty principle f(!)
^ where ! 62 I. Since this problem is linear, this is equivalent to stating that the problem is invertable if and only
f(!)
if the kernel of the operator is 0.
Donoho and Stark constructed discrete analogues of the uncertainty principle which are of use here. The rst of
these is the following
Lemma 1 Let fxtg be a discrete sequence with at most Nt non-zero elements, and let fx^!g be its discrete Fourier
transform. Then fx^! g cannot have Nt consecutive zeros.
This Lemma can be generalized to an n?dimensional discrete Fourier transform, with a proof essentially identical
to that in [10], except that the Vandermonde matrix of [10] will be replaced with a tensor product of Vandermonde
matrices. We state this below.
Lemma 2 Let xt be a two dimensional sequence on S = ZN ZN, and let x^! be its two dimensional discrete Fourier
transform. If xt is non-zero only on a block of the type R = [r1; r1 + 1; :::r1 + n] [r2; r2 + 1; :::r2 + n] S then x^!
cannot vanish on a block Q = [q1; q1 + 1; :::q1 + n] [q2; q2 + 1; :::q2 + n] S .
We use this to construct the following two-dimensional result, which applies directly to our problem.
Suppose that a function ft , where t 2 S is non-zero only on the interior N=2N=2 region of S. Then we can recover
it from the values of its discrete Fourier transform on the region C = [0; 1; ::::N=2 ? 1] [0; 1; ::::N=2 ? 1] [ [N=2; N=2+
1; ::::N ? 1] [N=2; N=2+1; ::::N ? 1]. This result follows directly from the above Lemma, since the problem is linear
^ cannot vanish on either [0; 1; :::N=2 ? 1] [N=2;N=2+1; ::::N ? 1] or [N=2; N=2+1; ::::N ? 1] [0; 1;:::N=2 ? 1],
and f(!)
thus the problem must be non-singular. This is of interest in our case, because the set C is the discrete analogue of
the rst and third quadrants in the continuous case.
Thus we nd that if we oversample the discrete Fourier transform of a function ft by a factor of 2, the recovery
of the original function from limited angle Fourier data is non-singular. This corresponds with having a \true"
rather than a modulo convolution on the support of the image. The problem is very ill-conditioned, however, if the
discretization is large.
We once again return to the \moral" argument that a signal recovery problem is invertible, if the data to be
retrieved cannot vanish on the unknown region. Although the data cannot vanish on the unknown region, it can
essentially vanish, if the discretization is large. Thus, if we try to recover ne details from the beginning, the problem
will be very ill conditioned. If however, we can recover a low-resolution reconstruction rst, and use this low resolution
reconstraction to mollify the ne-resolution reconstructions, then we might have some hope.
4.6 Non-linear constraints, induced correlations, and POCS
We assume throughout that the functions which we are recovering are compactly supported, and have nite support,
which is known to some degree. Moreover, we know a portion of the Fourier transform of these functions. Finally,
we assume that there is some maximum density which can be observed, i.e. gold markers in electron microscopy.
Thus our algorithm begins with the constraints on our image f:
1. 0 f M.
2. f^XC is known.
3. suppf 10
The techniques which we will use to stabilize the algorithm are highlighted below.
Multiresolution analysis: It is well known that a low resolution version of f can be reconstructed accurately.
This low resolution version of f is then added to the constraints. If we utilize a projection operators Pk for
the projections onto these resolution subspaces, then it follows that we are interested in using our ability to
recover P1f, or a low resolution version of f, to aid us in the recovery of f on higher subspaces.
Preservation of non-linear constraints: We would like the non-linear constraints outlined earlier to hold
on these subspaces. In other words, we would like for Pnf to be positive, and compactly supported. This is
the case if we use the Haar functions for our reconstruction.
Induced correlations: We want to preserve the non-linear constraints on the low-resolution subspaces, so
that the dierence subspaces Dn = Pn n Pn?1 will be correlated to Pn?1. In the case of the Haar subspaces,
positivity implies that if the scaling coecient at level n ? 1 is given
by sk;n?1, then the dierence coecients
on this interval will be bounded bounded above and below by s : ?1
Similarly, the support constraint induces correlations between coecients at dierent scales. This is of utmost
importance to this algorithm. If we think only of the linear problem, then one would have to retrieve cnen when
one has only observed the noisy coecient (n cn + gn )en . If n is small, this is a very ill-conditioned problem.
But if there is a great deal of correlation between scales, then it implies a great deal of correlation between
singular functions. Thus we can observe cn en from the coecients ck ek where k < n. The \low resolution"
information will allow us to recover the \high resolution information".
Signal to Noise Recovery As outlined previously, the linear POCS algorithm will necessarily produce a very
noisy image, if the input data is noisy. More specically, the signal to noise ratio's will be essentially driven
to zero as n ! 1 (although this assumes the existence of truly white noise, which would require innite noise
power, the approximation in nite dimensions is essentially true).
To combat this we want to denoise the reconstruction periodically, in order to restore the dominant features and
remove the noise. We do this with the non-linear thresholding techniques of Donoho and Johnstone [11, 12]. We
utilize this type of technique because images are generally dominated by their edges. Non-linear thresholding
is designed, to allow edges to remain sharp, while polynomially smooth regions can be smoothed or denoised.
Since this is incorporated into the more general POCS algorithm, we will use the \hard" non-linear thresholding
algorithm, rather than the slightly more stylish \soft" non-linear thresholding.
k;n
5 The algorithm
Our algorithm consists of three dierent procedures.
Stable low-resolution reconstruction We have shown that when the resolution of the problem, or the size
of the numerical grid SN is suciently large, the spectrum of the problem will make a truncated or mollied
SVD essentially useless. Thus we will not be able to eectively invert an operator of the type
Lf = XB F2?1(f^XC )
unless the discretization is suciently coarse, or the frequency resolution is very low.
If the discretization is suciently coarse, however, the spectra of the discrete operators is quite dierent, as can
be see in Figure 4. Therefore we will begin by recovering the image on a very low-resolution Haar subspace,
i.e. we begin by recovering local averages f1.
This will serve as the rst approximation to the desired object function f. We reiterate that f1 will be the
image of f in a low-resolution Haar subspace.
Stable iterative improvement: Once we have a rst approximation f1 to f, the natural question to ask is
whether we can use f1 and our original data Lf to reconstruct a higher resolution reconstruction of f, which
we will call f2 .
As outlined before, the Haar subspaces preserve the non-linear constraint of positivity, as well as the linear
constraint of compact support. These constraints induce correlations between the scales, and allow the low
resolution subspace to mollify the reconstruction on the higher resolution subspace.
Thus we use the induced non-linear constraint given by the solution on the former subspace, as well as positivity,
which induces a bound on the size of the coecients in the dierence subspace. We also utilize the compact
support for further stabilization.
11
Iterative error reduction: As outlined before, any type of algorithm which yields an actual inverse to a
noisy ill-conditioned problem, will drive the signal to noise ratio in the nal image to zero. Therefore we need
to remove noise from the image along the way.
We utilize both (POCS), and non-linear shrinkage to accomplish this goal.
Projection onto convex sets, or (POCS) [23, 2], is a standard iterative algorithm for utilizing a priori knowledge
in a reconstruction algorithm. We assume throughout that 0 f M, supp f BN , and that we know f^XC .
All of these assumptions are consistent with tomography, where positive, compactly supported densities are
recovered. Moreover, the maximum density of the image is also well known, (i.e. bone when doing medical
imaging, gold tracers when doing electron microscopy, etc.). Thus we have three convex sets, corresponding to
these three assumptions, and we want our answer to lie in the intersection of these convex sets.
This technique allows us to reduce the residual computational error during the algorithm, and stabilize the
nal outcome.
6 Implementation
We will now explicitly outline the algorithm.
1. Using POCS, with the constraints outlined above, on the input data. After a short while you will discover that
you have a good low resolution Haar approximation to your function.
2. After attaining a good low resolution approximation, use the low resolution approximation as an additional
constraint, and once again, allow POCS to run, with the additional constraint.
3. As each successive approximation converges on each subspace, reduce the noise in the image via non-linear
\hard" shinkage as in [11, 12].
4. Iterate steps 2 and 3, at higher and higher resolutions.
6.1 Numerical results
We will illustrate our algorithm with three separate numerical experiments. We do not feel that these experiments
are all inclusive, but we do feel that they demonstrate the potential of this type of algorithm. All of the experiments
begin with limited angle projection data, and use the second Fourier reconstruction algorithm, outlined in Natterer
[25], to construct the \known" portion of the Fourier data. The algorithm is then run as is described above. In all
experiments, the known \full" data set consists of 256 evenly spaced angles, and 256 samples per angle. The limited
angle data sets consist of correspondingly less angles.
In the rst experiment, we use 145 degrees of projection data. The only \noise" which the algorithm will be
subject to the is \noise" created by the Fourier reconstruction algorithm. In this rst experiment, we assumed that
knowledge concerning the support of the object is not exact. We assumed only that the support of the object is
embedded in a circle inscribed the reconstruction square. Nevertheless, the algorithm recovered the edges correctly,
although their magnitudes were not exactly reconstructed. The results of this experiment can be seen in Figure 6
Note that there is one irritating portion of the reconstruction, namely the widened edge at the top right.
In the second experiment, we use 145 degrees of noisy projection data. We also assumed that we knew the support
of the object fairly exactly. The results of this experiment can be seen in Figure 7.
The third experiment utilizes only 90 degrees of projection data, and once again assumes knowledge of the support
of the object. The reconstruction can be seen in Figure 8.
We should note at this time, that it seems that utilizing the complete knowledge of the support of the object, is
useful, but also can cause artifacts. If the data is not exact, a full data reconstruction will not produce an object
with exactly the support of the original image. As a result, insisting on this constraint can cause artifacts. This
constraint seemed necessary, however, if one was to recover the image from the extreme limited angle case of 90
degrees.
7 Conclusion
Upon examining the procedure we get the feeling that we are getting something for nothing. The explanation for
why we are able to extrapolate into the missing region of the Fourier transform is simple. In order to get the true
12
convolution of the image against our kernel, we have to zero pad, and therefore oversample the Fourier transform
of the function f(x; y). Since f has compact support this is equivalent to interpolating with a sinc series which is
very non-local. Therefore we are implicitly getting non-local information about our Fourier transform. This is the
information which we are utilizing in order to recover the missing region of the Fourier transform.
This conforms with the constraints of the limited angle tomography problem, because it is equivalent to zero
padding the projections, at each angle, or oversampling the Fourier transform of the projections.
There are a number of issues which we have not addressed. We have utilized empirical timing schedules to decide
when to impose the low resolution constraints on our algorithm. We believe, however, that these timing schedules
can either be replaced with xed schedules, or with adaptive schedules as in simulated annealing. We have also
utilized empirical results to ne tune the non-linear ltering. The non-linear ltering has been extensively studied
and this does not seem to be a concern.
A recent paper by Quinto, [30], states that \if a singularity is not stably visible from limited data, no algorithm
can reconstruct it stably". Our results support the fact that these are the dicult edges to recover, but they are
quite a bit more optimistic. The results of [30] are generated from Sobolev norms and microlocal analysis, and are
very elegant. In practical numerical analysis, however, decay at innity is not really an issue. It is well known that
the low frequencies can be reconstructed [25]. We use the low frequencies to build outward and recover edge detail.
In a certain sense, we are solving a series of problems which are much easier than the limited angle tomography
problem. We certainly agree, however, that the edges which are tangent to the view directions will be much easier
to reconstruct.
Another issue which should be addressed is that in most applications where limited angle tomography is needed,
the object is not eectively compactly supported. The wall of a nuclear reactor is better modeled as innitely long for
all practical purposes. A weld is generally a local non-uniformity in a uniform substance, however, so the projections
can be changed synthetically, after they are gathered, in order to make them correspond to the local, compactly
supported region of interest. In electron microscopy, however, the solution to this problem is not obvious.
References
[1] M.Reed and B.Simon, \Functional Analysis I", Acad. Press, 1980.
[2] D.C. Youla, \Mathematical Theory of Image Restoration by the Method of Convex Projections", in Image
Recovery, Theory and Applications, AP, San Diego, 1987.
[3] I.M. Gelfand and G.E. Silov, \Fourier transforms of rapidly increasing functions and questions of the uniqueness
of the solution of Cauchy's Problem.", Am. Math. Soc. Trans., Vol. 15, pp. 221-283.
[4] G.Szego, \Ein Grenzwertsatz uber die Toeplitzschen Determinanten einer reellen positiven Funktion",
Math.Ann. 76, 490-503, 1915.
[5] U.Grenander and G.Szego, Toeplitz Forms and Their Applications, Univ. of Cal. Press, 1958.
[6] I.I.Hirshman, Jr. and D.E. Hughes, \Extreme Eigen Values of Toeplitz Operators", Lecture Notes in Mathematics, No. 618, Springer-Verlag, 1977.
[7] F.A.Grunbaum, \A study of Fourier space methods for \Limited Angle" image reconstruction," Num Func
Anal Opt, (1980), 2(1), 31-42.
[8] M.E.Davison and F.A.Grubaum, \Tomographic reconstructions with arbitrary directions", Comm. Pure and
Appl. Math, 34, (1981), 77-120.
[9] M.E.Davison, \The ill-conditioned nature of the limit angle tomography problem," SIAM J.Appl.Math, 43,
April 1983, 428-448.
[10] D.L. Donoho and P.B. Stark, \Uncertainty Principles and Signal Recovery", SIAM J. Appl Math, Vol. 49,
No.3, pp. 906-931, June, 1989.
[11] D.L. Donoho and I.M. Johnstone, \Minimax estimation via wavelet shrinkage", Tech. Rep. 402, Dept. of Stats.,
Stanford University, Stanford, CA.
[12] D.L. Donoho and I.M. Johnstone, \Ideal spatial adaptation by wavelet shrinkage" ", Tech. Rep. 400, Dept. of
Stats., Stanford University, Stanford, CA.
13
[13] D. Jackson, \U ber die Genauigkeit der Annaherung Stetiger Funtionen Durch Gegebener Ordnung", dissertation, Gottingen, 1911.
[14] W.D.Cheney, Introduction to Approximation Theory, Chelsea, 1966.
[15] A.K.Louis, \Picture reconstruction from projections in restricted range", Math. Meth. in the Appl.Sci.,2 (1980),
209-220.
[16] A.K. Louis, \Incomplete data problems in x-ray computerized tomography I", Num.Mathematik, 48, (1986),
251-262.
[17] A.K. Louis and A. Reider, \Incomplete Data Problems in X-Ray Computerized Tomography II",
Num.Mathematik, 56, (1989), 371-383.
[18] P.Maass, \The X-ray transform: singular value decomposition and resolution", Inverse Problems, 3 (1987),
729-741
[19] W.R.Madych, Summability and Approximate Reconstruction from Radon Transform Data, Contemporary
Mathematics, 113, pp. 189-219, 1990.
[20] W.R.Madych and S.A.Nelson, \Reconstruction from restricted Radon transform data: resolution and illconditionedness", SIAM J.Math.Anal, 17, 1447-1453.
[21] S.Mallat, \Multiresolution representation and wavelet orthonormal bases of L2 (R)", Trans.Amer.Math.Soc.,
315 (1989), 69-87.
[22] A.Papoulis, \A New Algorithm in Spectral Analysis and Band-Limited Extrapolation", IEEE Trans on Cir
and Sys, 9, Sept. 1975.
[23] M.I.Sezan and Henry Stark, \Applications of Convex Projections Theory to Image Recovery in tomography
and Related Areas", Image Recovery: Theory and Application, Acad Press 1987, Henry Stark, Ed.
[24] R.A.Horn and C.R.Johnson, Matrix Analysis, Cambridge, 1990.
[25] F.Natterer, The Mathematics of Computerized Tomography, John Wiley, 1986.
[26] A.G.Rammand A.I.Zaslavsky, \Reconstructing singularities of a function from its Radon transform,", preprint.
[27] A.G.Ramm, \Inversion of limited-angle tomographic data", Comp. and Math. with Appl., 22, 1991, 101-112.
[28] E.T.Quinto, \Singular value decompositions and inversion methods for the exterior Radon transform and a
spherical transform", J.Math.Anal.Appl., 95 (1983), 437-448.
[29] E.T.Quinto, \Tomographic reconstruction from incomplete data- numerical inversion of the exterior Radon
transform", Inverse Problems, 4, (1988), 867-876.
[30] E.T.Quinto, \Singularities of the x-ray transform and limited data tomography in R2 and R3.", SIAM
J.Math.Anal. to appear, 1993.
[31] J.Radon, \U ber die Bestimmung von Funktionen durch ihre Integralwerte langs gewisser Mannigfaltigkeiten.",
Berichte Sachsishe Akademie der Wissenschaften, Liepzig, Math.-Phys.Kl, 69,262-267.
[32] J.A. Reeds and L.A. Shepp, \Limited Angle Reconstruction in Tomography via Squashing", Trans. on Med.
Im., Vol. MI-6, No. 2, June, 1987, 89-97.
[33] D. Slepian, and H.O. Pollack, \Prolate spheroidal wave functions, Fourier analysis and uncertainty, I and II",
Bell Syst. Tech. J., 40, (1961), pp. 43-84.
[34] T. Olson and J. Jae, \An explanation of the eects of Squashing in limited angle tomography", Trans. on
Med. Im., Vol. 9, No. 3, Sept. 1990.
14
N=6
N=8
2
2
1
1
0
5
10
0
15
10
N=12
2
1
1
20
40
0
60
40
N=24
2
1
1
80
160
0
240
160
N=48
2
1
1
320
640
120
320
480
N=64
2
0
80
N=32
2
0
30
N=16
2
0
20
0
960
600
1200
1800
Figure 4: Above we illustrate the spectra of the 90 degree limited angle operators. Notice that as the discretization
becomes large, the eigenvalues converge to either 0 or 1.
15
(a)
(b)
Figure 5: The discrete invertability result is illustrated above. In (a), we illustrate the support of f, on the discrete
lattice SN = ZN ZN . In (b), the known portions of the discrete Fourier transform f,^ are illustrated. These
known regions correspond to the known regions of the Fourier transform in limited angle tomography. The discrete
invertability result implies that f^ cannot vanish on these known regions.
Figure 6: Above we illustrate the reconstruction of the Shepp-Logan phantom from 145 degrees of data. The full
data reconstruction is shown at top left. The reconstruction using only the limited data is shown at top right.
The full data reconstruction is shown again, for reference at bottom left, and the reconstruction using our limited
angle algorithm is shown at bottom right. Notice that the edges are recovered in their proper places, although their
magnitudes are somewhat lower. The only irritating artifact is the edge at the top right, which is somewhat enlarged.
16
Figure 7: Above we illutrate the reconstruction of the Shepp-Logan phantom from 145 degrees of noisy data. The
full data reconstruction from this noisy data is shown at top left. The reconstruction using only the limited data, is
shown at top right. The full data reconstruction is shown again, for reference at bottom left, and the reconstruction
using our limited angle algorithm is shown at bottom right. Notice that the noise level is somewhat elevated, but
that most of the structure is recovered. The noise is white gaussian, and the signal to noise ratio is 20:1.
17
Figure 8: Above we illutrate the reconstruction of the Shepp-Logan phantom from 90 degrees of data. The full data
reconstruction from this noisy data is shown at top left. The reconstruction using only the limited data, is shown at
top right. The full data reconstruction is shown again, for reference at bottom left, and the reconstruction using our
limited angle algorithm is shown at bottom right. Although the reconstruction is not perfect, this is not surprising
given that only half of the angular data was utilized.
18