Abstract
It is well known that the phase problem (a signal is represented
only by the phase or the magnitude of its Fourier transform) plays
a major role in applied mathematics, physics and signal processing.
The problem can be solved by an iterative algorithm using POCS
(Projection Onto Convex Sets), is described in this paper and its application is shown. In the case of restoration from magnitude there is
a nonconvex set involved, so general projections are used. Both possibilities, restoration from phase and restoration from magnitude, are
explained and files for a simulation in MATLAB are included. At last
a multiresolution algorithm for the restoration from magnitude problem is discussed and examples for all algorithms are computed with
different signals by using MATLAB. From the simulation results we
observe that for both, reconstruction from magnitude and reconstruction from phase, signals of symmetric form have the worst convergence
while signals of geometric rate converge best. Due to this fact it is
better to have localized information (STFT).
1
Contents
1 Introduction
1.1 Phase Problem in X-ray Crystallography . . . . . . . . . . . .
1.2 Phase Problem in Optical Astronomy . . . . . . . . . . . . . .
1.3 Error-Reducing Algorithm . . . . . . . . . . . . . . . . . . . .
4
4
6
7
2 Mathematical Theory of the Method of Convex Projections 9
2.1 Convex Sets in Hilbert Space . . . . . . . . . . . . . . . . . . 9
2.2 Nonexpansive Maps and their Fixed Points . . . . . . . . . . 15
2.3 Iterative Techniques for the Phase-Problem
in a Hilbert Space Setting . . . . . . . . . . . . . . . . . . . . 24
2.4 Useful Projections . . . . . . . . . . . . . . . . . . . . . . . . . 37
3 Application
3.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 The Method of POCS . . . . . . . . . . . . . . . . . . . . .
3.2.1 Optimization of the Two-Step POCS Algorithm . . .
3.2.2 Restoration from Phase (RFP) . . . . . . . . . . . .
3.2.3 Per-Step Optimization of the Relaxation Parameters
3.2.4 Per-Cycle Optimization of the Relaxation Parameters
3.3 The Method of Generalized Projections . . . . . . . . . . . .
3.3.1 General Background . . . . . . . . . . . . . . . . . .
3.3.2 Restoration by Generalized Projections . . . . . . . .
3.4 Signal Recovery from Magnitude . . . . . . . . . . . . . . .
3.4.1 General . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.2 Generalized Projection Algorithm for the Restoration
from Magnitude Problem . . . . . . . . . . . . . . . .
3.4.3 Optimization of the Relaxation parameter λi . . . . .
3.5 Traps and Tunnels . . . . . . . . . . . . . . . . . . . . . . .
3.6 Iterative Multiresolution Algorithm for Reconstruction from
Magnitude . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.1 General . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.2 Multiresolution Algorithm . . . . . . . . . . . . . . .
3.6.3 Description of the Procedure . . . . . . . . . . . . . .
2
.
.
.
.
.
.
.
.
.
.
.
40
40
41
41
42
43
44
45
45
47
49
49
. 50
. 52
. 54
.
.
.
.
57
57
58
59
3.7
Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . 61
3.7.1 Results from RFP . . . . . . . . . . . . . . . . . . . . . 61
3.7.2 Results from RFM . . . . . . . . . . . . . . . . . . . . 65
A MATLAB Files
A.1 Restoration from Phase . . . . . . . . . . . . . . . . . . . . .
A.1.1 POCS Algorithm without Relaxation . . . . . . . . .
A.1.2 POCS Algorithm with Relaxation . . . . . . . . . . .
A.1.3 Projection Operator P2 . . . . . . . . . . . . . . . . .
A.1.4 Reconstruction from Localized Phase . . . . . . . . .
A.2 Restoration from Magnitude . . . . . . . . . . . . . . . . . .
A.2.1 Generalized Projection Algorithm without Relaxation
A.2.2 Generalized Projection Algorithm with Relaxation . .
A.2.3 Multiresolution Algorithm . . . . . . . . . . . . . . .
A.2.4 Error Reducing Algorithm . . . . . . . . . . . . . . .
A.2.5 Projection Operator P2 . . . . . . . . . . . . . . . . .
3
.
.
.
.
.
.
.
.
.
.
.
67
67
67
69
70
71
72
72
73
75
76
77
1
Introduction
The analysis of signal representation by magnitude-only or phase-only information of the Fourier transform is of interest in several fields such as signal
processing [3], image coding [9], speech analysis [2], X-ray crystallography
[4], holography, optical and radio astronomy [8] and others. Two examples
from physics are discussed in detail to show that this is not only a problem
of mathematical interest.
1.1
Phase Problem in X-ray Crystallography
Crystal structure analysis is usually based on diffraction phenomena caused
by the interaction of matter with X-rays.
Assume that there are two scattering centres at O and O′ and r =OO′ .
If a plane wave excites them they become sources of secondary spherical
waves which mutually interfere. Let s0 be the unit vector associated with
the direction of propagation of the primary X-ray beam. The phase difference
between the wave scattered by O′ in the direction defined by the unit vector
s and that scattered by O in the same direction is
δ=
2π
(s − s0 ) · r = 2πr∗ ·r
λ
where
r∗ = λ−1 (s − s0 ).
On the other hand
r = (x, y, z) = xa + yb + zc
and
r∗ = (h, k, l) = ha∗ + kb∗ + lc∗
with a, b, c the lattice constants or basic vectors of the crystal and a∗ , b∗ , c∗
defined as the reciprocal basic vectors i.e.
a∗ =
b×c
abc
b∗ =
c×a
abc
as can be shown by means of linear algebra.
4
c∗ =
a×b
abc
If AO is the amplitude of the wave scattered by the material point O
(its phase is assumed to be zero) the wave scattered by O′ is described by
AO′ exp(2πir∗ ·r). If there are N point scatterers along the path of the incident
plane wave we have
N
X
∗
∗
F (r ) =
Aj e2πir ·rj
j=1
where Aj is the amplitude of the wave scattered by the jth scatterer.
In our case it is more convenient to express the intensity I scattered by
a given object (for example, an atom) in terms of intensity Ie scattered by a
free electron. The ratio f = I/Ie is called scattering factor of the object. To
give an example, let us imagine a certain number of electrons concentrated
at O′ . In this case fO′ expresses the number of electrons.
So we get
N
X
∗
∗
fj e2πir ·rj .
F (r ) =
j=1
If the scattering centres constitute a continuum, the element of volume dr
will contain a number of electrons equal ρ(r)dr where ρ(r) is their density.
The wave scattered on the element dr is given, in amplitude and phase , by
∗
ρ(r)dre2πir ·r and the total amplitude of the scattered wave will be
Z
∗
∗
F (r ) =
ρ(r)e2πir ·r dr =ρ̂(r) =F[ρ(r)]
V
where ρ̂ represents the Fourier transform and F the Fourier transform operator.
In crystallography the space of the r∗ vectors is called the reciprocal
space. The equation constitutes an important result: the amplitude of the
scattered wave can be considered as the Fourier transform of the density
of the elementary scatterers. If these are electrons, the amplitude of the
scattered wave is the Fourier transform of the electron density ρ. From the
theory of Fourier transforms we also know that
Z
∗
ρ(r) =
F (r∗ )e−2πir ·r dr∗ =F−1 [F (r∗ )]
V∗
where F−1 represents the inverse Fourier transform.
So one could think to get the shape of crystal lattice which is represented
by the electron density ρ, it is only necessary to apply the inverse Fourier
5
transform on the scattering image. But it is not so easy, indeed. Because
F (r∗ ) is a complex function it can be written as
∗)
F (r∗ ) = |F (r∗ )| eiφ(r
where |F (r∗ )| is the magnitude and φ(r∗ ) is the phase. The measured intensity
I(r∗ ) =c · F F ∗ (r∗ ) = c · |F (r∗ )|2
is an absolute value. So the phase information has been lost but it is necessary
for the inverse Fourier transformation to have the phase, too. So the task is
to reconstruct the phase φ from the magnitude information.
1.2
Phase Problem in Optical Astronomy
The conventional view of an image is that it consists of an set of points
of different brightness. The alternative view of an image that its intensity
distribution is constituted by the superposition of sine waves of different
frequencies and amplitudes, which may be expressed as
I(x, y) =
Z∞Z
ˆ v)e2πi(ux+vy) dudv.
I(u,
−∞
ˆ v) is the Fourier transform of
Here u and v are the spatial frequencies. I(u,
I(x, y) and given by
ˆ v) =
I(u,
Z∞Z
I(x, y)e−2πi(ux+vy) dxdy.
−∞
Imperfection in the optics and inhomogeneities in the atmosphere deteriorate
the image of a point source. Instead of the diffraction pattern centered at
the position (x0 , y0 ) a light distribution P (x − x0 , y − y0 ) around the point
(x0 , y0 ) is observed. This point spread function weighted according to the
brightness of this point, by incoherent addition if these functions yields the
image. Mathematically this procedure is described by the convolution
I(x, y) =
Z∞Z
O(x′ , y ′ )P (x − x′ , y − y ′ )dx′ dy ′
−∞
6
or, in shorter notation
I(x, y) = O(x, y) ∗ P (x, y)
where O(x, y) is the object as it would appear in the xy plane under diffractionfree perfect imaging. In the Fourier space, according to the convolution
theorem this transforms to the simple relation
ˆ v) = Ô(u, v) · P̂ (u, v).
I(u,
Here, imaging is a process of linear filtering.
In some cases only the phase φ(u, v) of the Fourier transformed image
¯
¯
¯
¯ˆ
ˆ
I(u, v) = ¯I(u, v)¯ eiφ(u,v)
turns out to be undistorted or to be accessible. The distortion may be caused
through fluctuations in the transmission medium and turbulences in the atmosphere. So the task is to reconstruct the image I(x, y) from only the
phase of its Fourier transform. For this constellation also the term ”blind
deconvolution” is used. This means a desired signal is to be recovered from
an observation which is the convolution of the desired signal with some unknown signal. Since little is usually known about either the desired signal
or the distorting signal, deconvolution of the two signals is generally a very
difficult problem. However in the special case in which the distorting signal
is known to have a zero phase Fourier transform, the spectral phase of the
desired signal is undistorted. Such a situation occurs, at least approximately
in long-term exposure to atmospheric turbulence or when images are blurred
by severely defocused lenses. In this case the phase of the observed signal is
the same as the phase of the desired signal and, therefore, it is of interest to
consider reconstructing the signal from phase information alone.
1.3
Error-Reducing Algorithm
As mentioned earlier one is usually interested in determining an object, f (t),
which is related to its Fourier transform, F (u), by
F (u) = |F (u)| eiφ(u) = F[f (t)]
=
Z∞
f (t)e−2πiu·t dt
−∞
7
where t is a an m-dimensional spatial coordinate, and u is an m-dimensional
spatial frequency coordinate. The error-reducing algorithm which is also
called Gerchberg-Saxton algorithm consists of the following four steps:
(1) Fourier transform an estimate of the object;
(2) replace the modulus of the resulting computed Fourier transform with
the measured Fourier modulus to form an estimate of the Fourier transform;
(3) inverse Fourier transform the estimate of the Fourier transform;
(4) replace the modulus of the resulting computed image by zero where it
violates the condition f (t) ≥ 0. Described in the form of equations this is,
for the kth iteration
Gk (u) = |Gk (u)| eiφk (u) = F[gk (t)]
(1)
G′k (u) = |F (u)| eiφk (u)
(2)
where |F (u)| is the measured intensity
gk′ (t) = F−1 [G′k (u)]
′
gk+1
(t)
=
½
gk′ (t)
0
(3)
t∈
/S
t∈S
(4)
where S is the set of points at which gk′ (t) violates the object-domain constraints, i.e., where gk′ (t) < 0.
The iterations continue until the computed Fourier transform satisfies the
Fourier-domain constraints; then one has found a solution. The convergence
of the algorithm can be monitored by computing the squared error for the
kth iteration which is
Bk = kGk − G′k k22 =
Z∞
2
|Gk (u) − G′k (u)| du.
(5)
−∞
About the convergence could be said that Bk+1 ≤ Bk . So convergence is not
guaranteed, because it is possible that error doesn’t become smaller and the
iterative process stagnates.
The same procedure could be done if only the phase of the Fourier transform is known and one wants to get the magnitude. In general one can say
that the phase of signals carries more information than the magnitude. Let
us assume the following : we have two 2-dimensional functions (e.g. two
8
images with different contents). Now we Fourier transform both images take
the magnitude of the first and the phase of the second and do the inverse
transform. The image obtained by this procedure contains many of the features of the second image but nearly none of the first. If we change the role
of the first and the second image then the effect in the result is reversed, too.
2
2.1
Mathematical Theory of the Method of Convex Projections
Convex Sets in Hilbert Space
In this chapter H denotes a Hilbert space.
Definition 1 A subset C of H is said to be convex if together with any x1
and x2 it also contains λx1 + (1 − λ)x2 ∀λ ∈ [0, 1].
Theorem 2 Let C denote any closed convex subset of H and let f be any
element of H. Then there exists a unique x0 ∈ C such that
inf kf − xk = kf − x0 k.
x∈C
i.e., x0 is the element in C closest to f in norm.
Proof. Suppose that
inf kf − xk = δ = kf − x0 k = kf − y0 k
x∈C
where x0 , y0 ∈ C. Then by the identity
°
°
°
°
° x + y °2 ° x − y °2 kxk2 + kyk2
°
° +°
°
° 2 ° =
° 2 °
2
using f − x0 , f − y0 for x, y we obtain
°2
°2
°
°
°
°
°
°
x
+
y
x
−
y
0
0
0
0
2
° = δ2 − °
°
°f −
°
° 2 ° ≤δ .
2 °
9
(6)
But because C is convex,
x0 +y0
2
°
°
°f
°
∈ C so that
°2
x0 + y0 °
° ≥ δ2.
−
2 °
(7)
Hence, kx0 − y0 k = 0 and x0 = y0 .
By definition of infinum there exists a sequence (xn ) ⊆ C such that
lim kf − xn k = δ.
Again, using Eq.6 with f − x0 , f − y0 for x, y we have
°
°2
°
°
x
+
x
n
m
2
2
° .
f
−
kxn − xm k = 2(kf − xn k + kf − xm k ) − 4 °
°
°
2
Thus, since
xn +xm
2
(8)
(9)
∈ C for n, m = 1, 2, ... , 7 and 9 yield
0 ≤ lim kxn − xm k2 ≤ 4δ 2 − 4δ 2 = 0,
n,m→∞
which implies that
lim kxn − xm k = 0.
n,m→∞
The sequence (xn ) is therefore Cauchy and converges to a limit x0 ∈ C
because C is closed. From 8 kf − xn k = δ.
Theorem 2 is important as it leads to the notation of a projection operator: For any x ∈ H the projection operator PC onto C maps x to the
element in C closest to x. If C is closed and convex, PC x is well defined from
the minimality criterion
kx − PC xk = min kx − yk.
y∈C
This rule, which assigns to every x ∈ H its nearest neighbor in C, defines
the possibly nonlinear projection operator PC : H → C without ambiguity.
Corollary 3 Let C be a closed linear subspace of H and H = C ⊕ C ⊥ .
Then, every x ∈ C possesses a unique decomposition of the form x = x1 + x2 ,
where x1 = PC x ∈ C and x2 ∈ C ⊥ . The projection operator PC then is linear.
10
Proof. Suppose that x = x1 + x2 = x3 + x4 , where x1 , x3 ∈ C and x2 , x4 ∈
C ⊥ . Then, x1 − x3 = x4 − x2 ∈ C ∩ C ⊥ . Thus x1 = x3 , x2 = x4 , and the
decomposition, if it exists, must be unique.
Choose x1 = PC x and set x2 = x − PC x. Then, x = x1 + x2 and it only
remains to show that x2 is orthogonal to C. That is, (x2 , y) = 0, ∀y ∈ C.
From the definition of PC and the convexity of C,
λy + (1 − λ)PC x ∈ C, ∀y ∈ C, ∀λ ∈ (0, 1)
so that
kx − PC xk2 ≤ kx − λy − (1 − λ)PC xk2 .
Or, upon expansion of the right-hand side of
kx − PC xk2 = kx − PC xk2 + λ2 ky − PC xk2 − 2λRe(x − PC x, y − PC x).
Hence,
2Re(x − PC x, y − PC x) ≤ λky − PC xk2
and by letting λ → 0 we find that
Re(x − PC x, y − PC x) ≤ 0, ∀y ∈ C.
(10)
Now, µy ∈ C, ∀µ ∈ R and 10 yields,
µRe(x − PC x, y) ≤ Re(x − PC x, PC x), ∀µ ∈ R.
This inequality can be true only if
Re(x − PC x, y) = 0, ∀y ∈ C
If now y in 11 is replaced by iy, we get
Im(x − PC x, y) = 0
so that finally (x − PC x, y) = 0, y ∈ C.
11
(11)
Corollary 4 Let C be a closed convex set. Then, for any x ∈ H,
Re(x − PC x, y − PC x) ≤ 0
(12)
for every y ∈ C. Conversely, if some z ∈ C has the property
Re(x − z, y − z) ≤ 0
(13)
for all y ∈ C, then z = PC x.
Proof. A review of the proof of Corollary 3 reveals that 10 is valid for any
closed convex set C. From 13 we have for any y ∈ C :
kx − yk2 = kx − z + z − yk2 = kx − zk2 −2 Re(x−z, y−z)+kz − yk2 ≥ kx − zk2 .
Thus, by Theorem 2, z = PC x.
Definition 5 A sequence (xn ) ⊂ H is said to converge to its weak limit x if
lim (xn , y) = (x, y) ∀y ∈ H.
n→∞
Weak convergence and its link to weak topology is an important fact in
functional analysis but not for our further explanations. So we do not discuss
it here [16],[10].
Definition 6 A set C is said weakly compact if every sequence (xn ) ⊂ C has
a subsequence (xn′ ) which converges to its weak limit x, and all such weak
limits belong to C.
Corollary 7 Any closed bounded convex set C is weakly compact.
Proof. Since C is bounded every sequence (xn ) is contained in C. Since
H is weakly compact, there exists a subsequence (xn′ ) of (xn ) such that
xn′ ⇀ f, f ∈ H. By 12 we have
Re(f − PC f, xn′ − PC f ) ≤ 0, ∀xn′
Since xn′ ⇀ f we have
0 ≥ Re(f − PC f, f − PC f ) = kf − PC f k2 ≥ 0.
12
Thus, kf − PC f k = 0 or f = PC f ∈ C.
Due to the fact that a closed convex set, bounded or unbounded, contains
all its weak limits it is weakly closed. Since a strong limit is automatically
weak, we see that weak closure of any set implies its strong closure, since a
strong limit is automatically weak, but the converse is in general false.
Theorem 8 Let C be a closed convex set. Then for every pair x, y ∈ H we
have
kPC x − PC yk2 ≤ Re(x − y, PC x − PC y).
(14)
Proof. Since PC x, PC y ∈ C it follows by using the identity 12 that
Re(x − PC x, PC y − PC x) ≤ 0
(15)
and
Re(x − PC y, PC x − PC y) ≤ 0.
The result follows by addition.
Corollary 9 Projection operators onto closed convex sets C are nonexpansive and, therefore, continuous.
Proof. Schwartz’s inequality applied to 14 yields
kPC x − PC yk ≤ kx − yk , ∀x, y ∈ H.
Definition 10 A convex set C is said to be strictly convex if x, y ∈ C with
x 6= y implies that (x + y)/2, the midpoint of the connection between x and
y, is an interior point of C.
Definition 11 A convex set is said to be uniformly convex if there exists a
function δ(τ ), positive for τ > 0, zero only for τ = 0 such that for x, y ∈ C
and
x+y
kz −
k ≤ δ(kx − yk)
(16)
2
imply z ∈ C. If in addition δ(τ ) = µτ 2 for µ > 0, C is said to be strongly convex.
13
Evidently, strong convexity implies uniform convexity, which implies strict
convexity.
Example 12 If C is a closed half-circle, then it is convex but not strictly
convex. On the other hand, the full closed circle is a strongly convex set.
Theorem 13 (Gubin)
If C is strictly convex, x ∈
/ C, y ∈ C and y 6= PC x, then we have
Re(x − PC x, y − PC x) < 0.
Proof. By definition, y ∈ C, PC x 6= y ⇒ ∃ε > 0
w=z+
PC x + y
∈ C, ∀z : kzk ≤ ε.
2
Hence, if x ∈
/ C we can choose
z=
ε(x − PC x)
kx − PC xk
and then replace y in 12 by w to obtain
Re(x − PC x, y − PC x) ≤ −2ε kx − PC xk < 0.
(17)
Corollary 14 If C is strongly convex, then
kPC x − yk ≤ ρkx − yk
where ρ = 1/(1 + 2µkx − PC xk).
If C is uniformly convex, x ∈
/ C, y ∈ C, then
Re(x − PC x, y − PC x) ≤ −2δ(ky − PC xk) · kx − PC xk.
(18)
Proof. If y = PC x, 17 is correct with ε = 0. To derive 18 we rewrite 17 in
the form
kPC x − yk2 ≤ Re(x − y, PC x − y) − 2ε kx − PC xk ,
14
and then use Schwartz’s inequality to obtain
kPC x − yk2 − kPC x − yk · kx − yk + 2ε kx − PC xk ≤ 0.
Or, with ε replaced by µ kPC x − yk2 ,
kPC x − yk(kPC x − yk + 2µkPC x − yk · kx − PC xk − kx − yk) ≤ 0,
so that always,
kPC x − yk ≤
2.2
kx − yk
.
1 + 2µ kx − PC xk
Nonexpansive Maps and their Fixed Points
Definition 15 A mapping T : C → H is said to be a contraction if there
exists a positive constant L ∈ (0, 1), such that
kT x − T yk ≤ Lkx − yk ∀x, y ∈ C.
Theorem 16 (Banach)
If C is a nonempty closed subset of H, any contraction mapping T of C
into itself possesses a unique fixed point x∗ ∈ C. Starting from any element
x0 ∈ C, T n x0 → x∗ as n → ∞.
Corollary 17 Let T l , l ∈ N\{1}, be a contraction of a nonempty closed set
C into itself. Then T possesses a unique fixed point x∗ obtainable as the limit
of any sequence of the form (T n x0 ), x0 ∈ C.
Proof. According to Theorem 16, there exists a unique x∗ ∈ C such that
T l x∗ = x∗ . Hence,
T x∗ = T (T l x∗ ) = T l (T x∗ )
and T x∗ ∈ C is also a fixed point of T l . By uniqueness, T x∗ = x∗ and x∗
is therefore a fixed point of T. Consider the sequence (T n x0 ) as n → ∞ and
write n = ql + r where q is an integer and 0 ≤ r < l. Then, as n, q → ∞ and
kT n x0 − T x∗ k = kT r T ql x0 − T r x∗ k ≤ kT ql x0 − x∗ k → 0.
15
Consequently, T n x0 → x∗ as n → ∞.
A weaker condition than the strong requirement of contraction is the
following
Definition 18 A mapping T : C → H is said to be strictly nonexpansive if
kT x − T yk < kx − yk∀x, y ∈ C
with x 6= y.
Theorem 19 A strictly nonexpansive map T of a compact set C into itself
always possesses a fixed point.
Proof. The numerical function kT x − xk is obviously continuous on C.
Hence, since C is compact there exists an element x∗ ∈ C such that
inf kT x − xk = kT x∗ − x∗ k.
x∈C
But we must have T x∗ = x∗ because otherwise
kT (T x∗ ) − T x∗ k < kT x∗ − x∗ k,
a contradiction.
Definition 20 A mapping T : C → H is said to be nonexpansive if
kT x − T yk ≤ kx − yk ∀x, y ∈ C.
Theorem 21 Let T : C → C be a nonexpansive map whose domain C is a
nonempty closed bounded set. Then T has at least one fixed point.
Proof. Let y0 be any preselected member of C and let the set C0 = {x :
x = y − y0 , y ∈ C}. The translate C0 is evidently also a closed bounded set.
Clearly, every x ∈ C0 possesses a unique decomposition x = y − y0 , y ∈ C.
Let F : C0 → C0 be defined by
F x = T y − y0 .
16
This map is nonexpansive because x1 = y1 − y0 and x2 = y2 − y0 imply that
kF x1 −F x2 k = kT y1 −T y2 k ≤ ky1 −y2 k = k(y1 −y0 )−(y2 −y0 )k = kx1 −x2 k.
The map G = kF is a contraction C0 → C0 , ∀k ∈ (0, 1).
kF x = k(F x) + (1 − k)0 ∈ C0 , ∀x ∈ C0
where 0∈ C0 is the zero vector and
kGx1 − Gx2 k = kkF x1 − F x2 k ≤ kkx1 − x2 k.
Hence, invoking Theorem 16, there exists a unique xk ∈ C0 such that
xk = kFk , ∀k ∈ (0, 1).
(19)
We have now to show that xk → g as k → 1 from below, then by the
continuity of F and g ∈ C0 , it is seen from 19 that g = F g. Or, since
g = f − y0 , f ∈ C, we obtain f = T f, so f is a fixed point of T. We shall
actually proof that lim xk = g where g is the unique fixed point of F in C0
k→1
k∈(0,1)
of minimum norm.
Assume that 0 < k < l ≤ 1, xk = kF xk , xl = lF xl and let h = xl − xk .
Then since
kF x1 − F x2 k ≤ kx1 − x2 k
we obtain
or
µ
µ
1 1
−
l
k
¶2
xk + h xk xk + h xk
− ,
−
l
k
l
k
2
kxk k +
µ
¶
≤ khk2
¶
µ
¶
1 1
1
2
− 1 khk ≤ 2l
−
Re(xk , h).
l2
k
l
Thus, Re(xk , h) ≥ 0 which, together with the identity
kxl k2 ≥ kxk + hk2 = kxk k2 + khk2 + 2Re(xk , h),
17
leads to the inequality
kxl k2 ≥ kxk k2 + kxl − xk k2 .
(20)
To sum up, for any choice of sequence 0 < k1 < k2 < · · · such that ki → 1,
the sequence (kxki k) is monotone nondecreasing and bounded. It therefore
converges and, in particular, from 20
kxl + xk k2 ≤ kxl k2 − kxk k2 → 0, l, k → ∞.
By the completeness of H, xki → g ∈ C0 because C0 is closed.
Let e be any fixed point of F in C0 . Then, e = 1 · F e and we can apply
20 with xl = e, l = 1, xk = xki , k = ki , i = 1, 2, .... As i → ∞, xki → g, so
kek2 ≥ kgk2 + ke − gk2 ≥ kgk2 .
Therefore, kgk = inf kek , as e ranges over the fixed points of F in C0 .
Lemma 22 The set of fixed points T of a nonexpansive mapping T with
closed convex domain C and range H is a closed convex set.
Proof. Let xi = T xi , i = 1, 2, ..., and suppose that xi → x. Since (xi ) ⊆ C,
which is closed, x ∈ C and T x is well defined. Thus, invoking nonexpansitivity,
kT x − xk = kT x − T xi + xi − xk ≤ 2 kx − xi k → 0,
so T x = x and T is closed. To establish convexity we need the identity
kx − yk2 − kT x − T yk2 = 4Re(P x − P y, (I − P )x − (I − P )y), ∀x, y ∈ C,
where P =
I+T
.
2
But T is nonexpansive, hence
Re(P x − P y, (I − P )x − (I − P )y) ≥ 0, ∀x, y ∈ C.
18
(21)
Since P and T the same fixed points it suffices to show that the set of fixed
points of P is convex.
Let y be any fixed point of P. Then 21 reduces to
Re(P x − y, x − P x) ≥ 0, ∀x ∈ C.
(22)
Conversely, if some y ∈ C satisfies 22 it satisfies it for x = y, which implies
that ky − P yk ≤ 0 or y = P y. The set of fixed points of T is the set of all
y ∈ C that satisfy 22. But this is obviously convex.
Corollary 23 The map T with closed convex domain C is nonexpansive
⇔ P = I+T
satisfies 21.
2
Lemma 24 Let (xn ) ⊆ H be a sequence.
xn ⇀ x0 ⇒ lim inf kxn − xk > lim inf kxn − x0 k , ∀x 6= x0 .
n→∞
n→∞
(23)
Proof. Since a weakly convergent sequence is bounded, both limits in 23 are
finite. Thus, to prove this inequality it suffices to note that in the estimate
kxn − xk2 = kxn − x0 + x0 − xk2
= kxn − x0 k2 + kx0 − xk2 + 2Re(xn − x0 , x0 − x)
the last term tends to zero as n → ∞.
Definition 25 A map T : C → H is said to be demiclosed if
(xn ) ⊆ C, xn ⇀ x0 , x0 ∈ C, T xn → y0 ⇒ T x0 = y0 ,
i.e., T is demiclosed if for any sequence (xn ) which converges weakly to x0 ,
the strong convergence of the sequence (T xn ) to y0 implies that T x0 = y0 .
Lemma 26 Let T be any nonexpansive map with closed convex domain C ⊆
H. Then, I − T is demiclosed.
19
Proof. Let (xn ) ⊆ C, xn ⇀ x0 , xn −T xn → y0 . Then, since T is nonexpansive
lim inf kxn − x0 k ≥ lim inf kT xn − T x0 k = lim inf kT xn − xn + xn − T x0 k
n→∞
n→∞
n→∞
= lim inf kx0 − y0 − T x0 k = lim inf kxn − x0 k
n→∞
n→∞
by Lemma 24. Hence, again by Lemma 24 x0 = y0 − T x0 or (I − T )x0 = y0
so that I − T is demiclosed.
Unlike Theorem 21 the compactness assumption on C is replaced by the
much weaker condition of convexity and boundedness. Nevertheless, in many
applications even this assumption of boundedness is to much because the
numerical bounds are not known. However, if the existence of a fixed point
is known in advance from physical conditions, the boundedness requirement
on C can be dropped. This fact is taken in consideration in the next theorem.
Definition 27 A map T : C → C is said to be asymtotically regular if
T n x − T n+1 x → 0 , ∀x ∈ C
Theorem 28 Let T : C → C be an asymtotically regular nonexpansive
map with closed convex domain C ⊂ H whose set of fixed points T ⊂ C
is nonempty. Then
(T n x) ⇀ x∗ , ∀x ∈ C
for some x∗ ∈ T .
Proof. The sequence (dn ) = (kT n x − yk) is nonincreasing ∀y ∈ T because
°
°
dn+1 = °T n+1 x − y ° = kT (T n x) − T yk ≤ kT n x − yk = dn .
Thus,
d(y) = lim kT n x − yk < ∞, ∀y ∈ T .
n→∞
(24)
According to Lemma 22, T is a closed convex subset of C and it follows
that
Tδ = {y ∈ T : d(y) ≤ δ}, ∀δ ≥ 0
is a closed bounded convex subset of T which is nonempty for δ large enough.
20
Indeed, convexity and closure are obvious from 24, and boundedness is implied by the inequalities
kyk = ky − T n x + T n xk ≤ kT n x − yk + kT n xk
and
kT n xk = kT n x − y0 + y0 k ≤ kT n x − y0 k + ky0 k , ∀y0 ∈ T .
Explicitly,
kyk ≤ δ + d(y0 ) + ky0 k , ∀y ∈ Tδ .
Since bounded closed convex sets are weakly compact, the intersection of
all such nonempty Tδ is a nonempty closed bounded convex set Tδ0 . Clearly,
δ0 is the smallest value of δ for which Tδ is nonempty. The set Tδ0 can contain
only one element, say y0 , for if we suppose that it also contains y1 6= y0 , the
identity
°2
°
°
°
°
° n
¡ n
¢ ° y0 − y1 °2
1
y
+
y
0
1
2
2
n
° =
°
°T x −
kT x − y0 k + kT x − y1 k − °
° 2 °
°
2 °
2
¢
¡
1
yields d y0 +y
< δ0 , which contradicts the meaning of δ0 .
2
T n x ⇀ y0 , since this sequence is bounded, it suffices to prove that all
possible weak limits of its various subsequences equal y0 . Assume that such
is the case but T n x 6⇀ y0 . Then, for some f ∈ H it is true that the sequence
′
(T n x, f ) fails to converge to (y0 , f ). Hence, there exists a subsequence (T n x)
′
of (T n x) such that lim(T n x, f ) exists and is unequal to (y0 , f ). But by hy′
′′
pothesis the sequence (T n x) itself contains a subsequence (T n x) converging
weakly to y0 , i.e.,
′′
′
(y0 , f ) = lim(T n x, f ) = lim(T n x, f ) 6= (y0 , f ),
′
a contradiction. Let T n x ⇀ y1 6= y0 . Then, from the asymtotic regularity of
T,
′
′
′
T n x − T n +1 x = (I − T )T n x → 0
21
and invoking the demiclosed character of I − T , (I − T )y1 =0, i.e., y1 is a
fixed point of T. By Lemma 24,
°
°
°
°
° n′
°
° n′
°
δ0 = lim °T x − y0 ° > lim °T x − y1 ° = d(y1 ),
which is incompatible with the meaning of δ0 . Thus, the sequence (T n x) is
weakly convergent to a fixed point of T ∀ x ∈ C.
Corollary 29 The sequence (T n x) converges strongly to x∗ if and only if at
least one of its subsequences converges strongly.
Proof. We know that T n x ⇀ y0 , a fixed point of T, ∀x ∈ C. Clearly, since
weak and strong limit of a sequence must coincide, the only possible strong
limit of any subsequence of (T n x) is y0 . Now, from the equality
kT n x − y0 k2 = kT n xk2 + 2Re(T n x, y0 ) + ky0 k2
we find that
lim kT n xk2 = d2 (y0 ) + ky0 k2 .
n→∞
′
In particular, for any subsequence (T n x) of (T n x),
°
°
° n′ °2
lim °T x° = d2 (y0 ) + ky0 k2 .
′
But, if any subsequence (T n x) converges strongly, it converges strongly to
y0 and therefore
°
°
° n′ °2
lim °T x° = ky0 k2
so that necessarily, d2 (y0 ) = 0 and
T n x → y0 .
Definition 30 A mapping T : C → C is said to be a reasonable wanderer if
∞
X
kT n x − T n+1 xk < ∞ ∀x ∈ C.
n=0
22
It is clear that a reasonable wanderer is automatically asymtotically regular.
Example 31 Let C = [−1, 1] and T defined by T x = −x. T is nonexpansive,
maps C into C and its only fixed point is x = 0. Then (T n x − T n+1 x) =
2 · (−1)n x 6→ 0 as n → ∞ unless x = 0. Therefore T is a nonexpansive
operator that is not asymtotically regular.
Theorem 32 If T : C → C is a nonexpansive map with closed convex domain C whose set of fixed points is not empty then Tλ : C → C with
Tλ = λ · I + (1 − λ)T
for any fixed λ ∈ (0, 1) is a reasonable wanderer and the sequence (Tλn x)
converges weakly to a fixed point of T ,∀x ∈ C.
Proof. xn = Tλn x, ∀x ∈ C is well defined. In addition, for λ 6= 1 the
fixed points of T and Tλ coincide and it only remains to prove that Tλ is a
reasonable wanderer since it is obviously nonexpansive. Let y ∈ C be a fixed
point of T. Then, y = T y = Tλ y and
kxn+1 − yk2 = kλxn + (1 − λ)T xn − yk2
= kλ(xn − y) + (1 − λ)(T xn − y)k2
= λ2 kxn − yk2 + 2λ(1 − λ)Re(xn − y, T xn − y) + (1 − λ)2 kT xn − yk2 .
Similarly,
kxn − T xn k2 = kxn − y − (T xn − y)k2
= kxn − yk2 − 2Re(xn − y, T xn − y) + kT xn − yk2 ,
which, after multiplication by λ(1 − λ) and addition to 25, yields
kxn+1 − yk2 + λ(1 − λ) kxn − T xn k2
= λ kxn − yk2 + (1 − λ) kT xn − yk2 ≤ kxn − yk2 .
Hence,
λ(1 − λ) kxn − T xn k2 ≤ kxn − yk2 − kxn+1 − yk2
23
(25)
so that
N
N
X
°
λ X°
°Tλn x − T n+1 x°2
λ(1 − λ)
kxn − T xn k =
λ
1 − λ n=0
n=0
2
≤ kx − yk2 − kxN +1 − yk2 ≤ kx − yk2 .
Consequently, since λ = 0,
∞
X
kTλn x − Tλn+1 xk2 < ∞.
n=0
Definition 33 A mapping T : C → H is said to be demicompact if (xn ) ⊂ H
is a bounded sequence and (T xn − xn ) is strongly convergent then there is a
subsequence (xni ) ⊂ (xn ) which is strongly convergent.
Theorem 34 Let T : C → C nonexpansive, demicompact, then the set T
⊂ C of fixed points of T is a nonempty convex set and
(Tλn x) → x∗ ∈ T ∀x ∈ C and any λ ∈ (0, 1).
2.3
Iterative Techniques for the Phase-Problem
in a Hilbert Space Setting
There exists a class of reconstruction problems in which the unknown funcm
T
Ci of well defined
tion f can be assumed to lie in an intersection C0 =
i=1
closed convex sets Ci , i = 1, ..., m. C0 is also a closed convex set containing
f , and we shall denote the projection operator onto Ci by Pi , i = 0, ..., m.
Now we define an operator Ti = I + λi (Pi − I) with the relaxation constants
λ1 , ..., λm . Clearly that f ∈ Co is a fixed point of the composition operator
T = Tm Tm−1 · · · T1 .
Theorem 35 Let C0 be nonempty. Then the sequence (T n x) converges
weakly to a point of C0 for every x ∈ H and every choice of λi ∈ (0, 2), i =
1, ..., m.
24
Proof. For λi ≥ 0, every
Ti = I + λi (Pi − I) = (I − λi )I + λi Pi
is nonexpansive. The assertion is obviously correct for λi ∈ [0, 1], but if
λi > 1 we have 1 − λi < 0 and it is necessary to reason differently. With the
aid of Eqs. 15 and 16, it is found that
kTi x − Ti yk2 = k(1 − λi )(x − y) + λi (Pi x + Pi y)k2
= (1 − λi )2 kx − yk2 + 2λi Re(x − y, Pi x − Pi y) + λ2i kPi x − Pi yk2
≤ (1 − λi )2 kx − yk2 + (λ2i + 2λi (1 − λi )) kPi x − Pi yk2
= (1 − λi )2 kx − yk2 + λi (2 − λi ) kPi x − Pi yk2
≤ λi (2 − λi ) + (1 − λi )2 kx − yk2 = kx − yk2
and nonexpansivity is established. Now it is shown that T is a reasonable
wanderer.
For m = 1, we have T = T1 , C0 = C1 , and
kx − T xk2 = λ21 kx − P1 xk2 .
(26)
Moreover, for any y ∈ C0 , T y = P1 y = y and
kT x − yk2 = kx − y + λ1 (P1 x − x)k
= kx − yk2 + 2λi Re(x − y, P1 x − x) + λ21 kx − P1 xk2
= kx − yk2 − λ1 (2 − λ1 ) kx − P1 xk2 + 2λ1 Re(x − P1 x, y − P1 x)
2
2
≤ kx − yk − λ1 (2 − λ1 ) kx − P1 xk
(27)
(28)
because the last term in Eq.27 is nonpositive. Thus, by combining 26 and
28 it is found that
¢
λ1 ¡
kx − yk2 − kT x − yk2 , λ1 ∈ (0, 2).
(29)
kx − T xk2 ≤
2 − λ1
For arbitrary m ≥ 1, a straightforward induction on m yields the inequal-
ity
¡
¢
kx − T xk2 ≤ bm 2m−1 kx − yk2 − kT x − yk2 ,
25
(30)
where
bm = sup
1≤i≤m
½
λi
2 − λi
¾
.
Let T = Tm K, where
K = Tm−1 Tm−2 · · · T1 ,
and observe that for m ≥ 2,
kx − T xk2 = kx − Kx + Kx − T xk2
≤ (kx − Kxk + kKx − T xk)2
≤ 2(kx − Kxk2 + kKx − T xk2 )
≤ 2(kx − Kxk2 + 2m−2 kKx − Tm Kxk2 ).
Thus, by the induction hypothesis, bm ≥
λm
2−λm
and bm ≥ sup1≤i≤m−1
kx − T xk2 ≤
n
λi
2−λi
o
,
≤ 2bm (2m−2 kx − yk2 −2m−2 kKx − yk2 +2m−2 kKx − yk2 −2m−2 kT x − yk2 ) =
= 2m−1 bm (kx − yk2 − kT x − yk2 ),
the desired inequality.
It now follows immediately that
∞
X
kT n x − T n+1 xk2 ≤ 2m−1 bm kx − yk2 < ∞
n=0
and T is therefore a reasonable wanderer and asymptotically regular. By
Theorem 28, the sequence (T n x) converges weakly to a fixed point of T.
m
T
However, the fixed points of T coincide with the points of C0 = Ci . It
i=1
is obvious that x ∈ C0 implies x = T x since x ∈ Ci , i = 1, ..., m. Conversely,
if x = T x and y ∈ C0 ,
kx − yk = kT x − T yk ≤ kT1 x − T1 yk = kT1 x − yk ≤ kx − yk ,
26
hence kx − yk = kT1 x − yk . This is only possible if x = T1 x so that x =
Tm Tm−1 · · · T2 x, and a repetition of the argument finally leads to x ∈ Ci , i =
1, ..., m, i.e., x ∈ C0 .
We are now able to address the question of strong convergence. Let
T : C → C denote a nonexpansive asymtotically regular operator with closed
convex domain C ⊂ H whose set T of fixed points is nonempty. From the
corollary 29, the sequence (T n x) converges strong to an element of T if at
least one of its subsequences converges strongly.
It is important to note that if all Ci in Theorem 35 are closed linear
manifolds, the sequence of iterates (T n x) not only converges strongly but
actually converges to the projection of x onto C0 .
Corollary 36 Under the conditions of Theorem 35, if every Ci is a closed
linear manifold, T n converges strongly to P0 , the orthogonal projection onto
the closed linear manifold C0 .
Proof. Note that all Pi and P0 are bounded linear self-adjoint operators.
Therefore
Ti∗ = I + λi (Pi∗ − I) = I + λi (Pi − I) = Ti
is self-adjoint and linear, i = 1, ..., m, and T n is also linear ∀n ∈ N0 . Moreover,
since T is nonexpansive and T 0 = 0 ,
kT xk = kT x − T 0k ≤ kx − 0k = kxk , ∀x ∈ H
so that kT k and therefore kT n k ≤ 1, n ∈ N0 .
As already been shown in Theorem 35
T n x − T n+1 x = T n (I − T )x → 0,
hence T n y → 0; ∀y ∈ R(I − T ). Now R(I − T ) = N ⊥ (I − T )∗ and
(I − T )∗ = I − (Tm Tm−1 · · · T1 )∗ = I − T1 T2 · · · Tm ,
27
from which it is deduced that
x ∈ N ⊥ (I − T )∗ ⇔ x = T1 T2 · · · Tm x.
Thus, in view of Theorem 35, N ⊥ (I − T )∗ = C0 so that H = C0 ⊕ R(I − T ).
Let y ∈ R(I − T ) ⇒ ∃(yi ) ⊆ R(I − T ) : yi → y. Clearly,
kT n yk = kT n yi + T n y − T n yi k ≤ kT n yi k + ky − yi k , n ∈ N,
which immediately gives
lim sup kT n yk ≤ ky − yi k
n→∞
because T n yi → 0, ∀i ∈ N. But ky − yi k → 0 so that T n y → 0, n → ∞, ∀y ∈
R(I − T ). Finally, every x ∈ H admits a unique decomposition x = z + y,
where z ∈ C0 , y ∈ R(I − T ). Consequently, T n z = z, ∀n ∈ N and
T n x = T n z + T n y = z + T n y → z = P0 x, n → ∞.
Therefore, by definition, T n converges strongly to the orthogonal projection
operator P0 onto C0 .
Definition 37 A linear variety C is the set of all vectors x = g + h with
fixed g ∈ H and y ∈ S where S is a closed linear manifold.
Clearly that a linear variety is closed and convex.
Corollary 38 If each Ci is a linear variety, i = 1, ..., m, and if C0 =
6= ∅ then the sequence (T n x) converges strongly to P0 x ∀x ∈ H.
m
T
Ci
i=1
Proof. To avoid complication in notation we assume m = 3. Thus, Ci =
gi + Si with fixed g ∈ H and Si is a given closed linear manifold, i = 1, 2, 3.
It is easily shown that for each i
Pi x = Si x + Ri gi , ∀x ∈ H,
28
where Si is the orthogonal projection operator onto Si and Ri is that onto
Si⊥ . Clearly then,
Ti x = x + λi (Si x + Ri gi − x) = (1 − λi )x + λi Si x + λi Ri gi = Li x + λi Ri gi
where
Li = (1 − λi )I + λi Si , i = 1, 2, 3.
Thus,
T x = T3 T2 T1 x = λ3 R3 g3 + λ2 L2 R2 g2 + λ1 L3 L2 R1 g1 + L3 L2 L1 x
= h + Lx,
where
h = λ3 R3 g3 + λ2 L2 R2 g2 + λ1 L3 L2 R1 g1
and
L = L3 L2 L1 .
It now follows by iteration of Eq.31 that
n
T x=
n−1
X
Lr h + Ln x, n ∈ N.
r=0
In particular, for x = f, f a fixed point of T, T n f = f,
f=
n−1
X
Lr h + Ln f
r=0
and
T n x − f = Ln (x − f ), n ∈ N.
According to Corollary 36, Ln (x − f ) → PS (x − f ), hence
T n x → f + PS (x − f ) ≡ w,
29
(31)
where PS is the orthogonal projection onto the closed linear manifold
S=
3
T
Si .
i=1
Obviously, w = lim T n x is independent of the choice of f ∈ C0 .
n→∞
It is clear from f ∈ C0 that w ∈ C0 , and
x − w = x − f − PS (x − f ) = QS (x − f ),
where QS is the projection onto S ⊥ . Hence,
kx − wk = kQS (x − f )k ≤ kx − f k .
But f is an arbitrary member of C0 , so that necessarily, T n x → w = P0 x.
Lemma 39 Let T be any nonexpansive map whose set of fixed points includes
a given closed convex set T , and let PT denote the projection operator onto
T . Then
Re(x − T x, x − PT x) ≥ 0
(32)
and
kx − T xk ≤ 2kx − PT xk.
Proof. Clearly,
kT x − T yk2 − kx − yk ≤ 0, ∀x, y ∈ dom(T ).
In particular, if y = PT x,
kT x − PT xk2 ≤ kx − PT xk2
because PT x ∈ T and T PT x = PT x, by hypothesis. Thus,
kx − T x − (x − PT x)k2 ≤ kx − PT xk2
30
(33)
and an easy simplification yields
kx − T xk2 ≤ 2Re(x − T x, x − PT x), x ∈ dom(T ).
(34)
From Eq. 34 and Schwartz’s inequality we now obtain the result immediately.
Inequality 32 states that T x lies on the side of the support plane through
x, parallel to that through PT x, containing T .
Lemma 40 The iterated sequence (T n x) of Theorem 28 converges strongly
if and only if
kT n x − PT T n xk → 0, n → ∞.
Proof. As shown in Theorem 28, T n x ⇀ x∗ ∈ T . Evidently, if the convergence is actually strong,
d(T n x, T ) = kT n x − PT T n xk ≤ kT n x − x∗ k → 0, n → ∞.
Conversely, suppose that d(T n x, T ) → 0. Then, since T k is nonexpansive
∀k ∈ N and includes T in its set of fixed points, Eq. 38 yields,
kx − T n xk ≤ 2kx − PT xk, ∀x ∈ dom(T ), k ∈ N0 .
In particular, if x is replaced by T n x we obtain
kT n x − T n+k xk ≤ 2kT n x − PT T n xk → 0, n → ∞.
(35)
Thus, the sequence (T n x) is Cauchy and T n x → x∗ .
According to Lemma 40, convergence in Theorem 35 of the sequence
n
(T x) to a point of C0 is strong if and only if
d(T n x, C0 ) = kT n x − P0 T n xk → 0, n → ∞
(36)
Therefore we can see that the set of fixed points of T = Tm Tm−1 ···T1 coincides
with C0 . We can show without imposing additional constraints that
d(T n x, Cr ) = kT n x − Pr T n xk → 0, n → ∞, r = 1, ..., m
31
(37)
To see this, note that as a special case of Eq. 30, in which we identify T
with Pr Pr−1 · · · P1 and set m = r and λi = 1, i = 1, ..., r,
kT n x − Pr Pr−1 · · · P1 T n xk2 ≤ 2r−1 (kT n x − yk2 − kPr Pr−1 · · · P1 T n x − yk2 )
≤ 2r−1 (kT n x − yk2 − kT n+1 x − yk2 ) → 0
∀y ∈ C0 , r = 1, ..., m
because
kT n+1 x − yk ≤ kPr Pr−1 · · · P1 T n x − yk
and
lim kT n x − yk
n→∞
exists. Thus, since Pr Pr−1 · · · P1 T n x ∈ Cr ,
d(T n x, Cr ) ≤ kT n x − Pr Pr−1 · · · P1 T n xk → 0, n → ∞
and Eq. 37 is established. So we have to find some restrictions , which
together with Eq.37 implies Eq.36. The next theorem complements Theorem
35 and supplies two sufficient conditions for strong convergence.
Theorem 41 Either of the two conditions (i) or (ii) stated below guarantees
the strong convergence of the sequence (T n x) of Theorem 35 to its weak limit
x∗ .
(i) At least one of the Ci is uniformly convex and does not contain x∗ in
its interior Ci◦ .


◦
(ii) For some j, 1 ≤ j ≤ m, the intersection Cj ∩ 
m
T
i=1
i6=j
Ci  is nonempty.
In this case the strong convergence (T n x) → x∗ is at a geometric rate.
Proof. (i) Let us suppose that C1 is uniformly convex and contains x∗ on
its norm boundary. As shown in Eq. 37
d(T n x, C1 ) = kT n x − P1 T n xk → 0, n → ∞.
32
Clearly, every yn ∈ P1 T n x and yn ⇀ x∗ , because
|(T n x, z) − (yn , z)| = |(T n x − yn , z)| ≤ kT n x − yn k kzk → 0, n → ∞
and lim (yn , z) = lim (T n x, z) = (x∗ , z). Now if the convergence of the
n→∞
n→∞
sequence (yn ) to x∗ is not strong
∃ε > 0, (yn′ ) ⊆ (yn ) : kyn′ − x∗ k ≥ ε, ∀n′ .
Thus, since x∗ ∈ C0 ⊆ C1 and C1 is uniformly convex,
zn′ =
yn′ + x∗
+ h ∈ C1 , ∀n′ , ∀h : 0 < khk ≤ δε .
2
But for any such fixed choice of h, zn′ ⇀ x∗ + h, which belongs to C1 because
the latter is weakly closed. Consequently, x∗ is the center of a sphere of radius
δε > 0 contained in C1 , and this contradicts the assumption that x∗ ∈
/ C1◦ .
Therefore the convergence of (yn ) and hence of (T n x) to x∗ must be strong
because
kT n x − x∗ k ≤ kT n x − yn k + kyn − x∗ k → 0, n → ∞.
◦

m
T
(ii) Let z ∈ Cj ∩  Ci  . Then z ∈ C0 and for some δ > 0 it is true
i=1
i6=j
that
h ∈ Ci , i 6= j, ∀h : kh − zk ≤ δ.
Let y = Pj T n x. Then,
w=
ε
δ
z+
y ∈ C0 , ∀ε > 0
ε+δ
ε+δ
for all sufficiently large n.
In fact, since z, y ∈ Cj and Cj is convex, w ∈ Cj . Now observe that
ky − Pi yk = d(y, Ci ) ≤ ky − Pi T n xk
≤ kT n x − yk + kT n x − Pi T n xk
ε ε
= d(T n x, Cj ) + d(T n , Ci ) ≤ + = ε
2 2
33
for n large enough according to Eq. 37 for i = 1, ..., m, lim d(T n , Ci ) = 0.
n→∞
Thus, for all such n
δ
h = z + (y − Pi y) ∈ Ci , i 6= j,
ε
because kh − zk ≤ δ. From the identity
¶
µ
ε
δ
ε
δ
δ
w=
Pi y =
h+
Pi y
z + (y − Pi y) +
ε+δ
ε
ε+δ
ε+δ
ε+δ
and the convexity of Ci , we also conclude that w ∈ Ci , i 6= j, i.e., w ∈ Ci , i =
1, ..., m, and w ∈ C0 .
Consequently,
d(T n x, C0 ) ≤ kT n x − wk ≤ kT n x − yk + ky − wk
ε
ky − zk
= d(T n x, Cj ) +
µ ε+δ ¶
ε εR
2R ε
cε
≤ +
= 1+
= ,
2
δ
δ
2
2
where
c=
µ
2R
1+
δ
(38)
¶
and R is an upper bound on
ky − zk = kPj T n x − zk ≤ kT n x − zk, ∀z ∈ C0 .
But the latter is a monotone increasing function of n and is therefore bounded
by some finite number R. Since ε is arbitrary, d(T n x, C0 ) → 0, n → ∞ and
therefore (T n x) → x∗ . It remains to establish that convergence is at geometric
rate.
Observe first that 2ε in Eq. 38 is by construction, and for sufficiently large
n, an upper bound on the m distances d(T n x, Ci ), i = 1, ..., m :
ε
d(T n x, Ci ) = kT n x − Pi T n xk ≤ , i = 1, ..., m .
2
34
Hence, for all large enough n,
d(T n x, C0 ) = c sup kT n x − Pi T n xk.
i=1,...,m
Choose the positive numbers ε1 and ε2 so that ε1c2ε2 < 1 and 0 < ε1 ≤
λi ≤ 2 − ε2 , i = 1, ..., m. Then from 29 and 26 with T replaced by Ti , x by
T n x, and y by P0 T n x we obtain, for every i = 1, ..., m,
d2 (T n x, C0 ) ≤ c2 d2 (T n x, Ci )
¡ 2 n
¢
c2
≤
d (T x, C0 ) − kTk T n x − P0 T n xk2
λi (2 − λi )
¢
c2 ¡ 2 n
d (T x, C0 ) − kT n+1 x − P0 T n xk2
≤
ε1 ε2
¢
c2 ¡ 2 n
d (T x, C0 ) − d2 (T n+1 x, C0 ) .
≤
ε1 ε2
Therefore,
d(T
n+1
r
ε1 ε2
x, C0 ) ≤ 1 − 2 d(T n x, C0 ),
c
and by iteration
r³
ε1 ε2 ´ n
d(T n x, C0 ) ≤
1− 2
d(x, C0 )
c
r³
ε1 ε2 ´ n
1− 2
.
≤R
c
From Eq. 35, with T = C0 and PT = P0 , Eq. 39 yields,
r³
ε1 ε2 ´ n
n
n+k
1− 2
, ∀k ∈ N.
kT x − T
xk ≤ 2R
c
(39)
(40)
But for fixed n, T n+k x → x∗ , k → ∞, so that a passage to the limit in Eq.
40 gives
r³
ε1 ε2 ´ n
n
∗
kT x − x k ≤ 2R
,
1− 2
c
35
i.e., T n x → x∗ at least as fast as 2Ran where
r
ε1 ε2
a = 1 − 2 < 1.
c
Apart from the important uniform convexity requirement, (i) succeeds
because the weak limit x∗ is on the boundary of at least one Ci , whereas the
success of (ii) depends on the fact that at least one point of C0 is interior of the
intersection of some Ci . In general, (ii) is of limited applicability because for
the class of problems we have in mind, the unknown f is usually a boundary
point and convex set with interiors are hard to come by.
Example 42 a)Let H = L2 (R) and F (u) Fourier transform, the correspondence is denoted by f (t) ↔ F (u). For any prescribed b > 0 let Cb
denote the subset of H composed of all functions bandlimited to b, i.e., f
∈ Cb ⇔ F (u) = 0, |u| > b. It is clear that Cb is a closed linear manifold
devoid of any interior points. For any given ε > 0 and any f ∈ Cb ∃g ∈
/ Cb
such that
Z∞
2
kf − gk =
|f (t) − g(t)|2 dt < ε2 .
−∞
Thus, as measured in terms of energy content, any small neighborhood of
a bandlimited signal contains signals that are not bandlimited.
b)For any prescribed a > 0, let Ca denote the set of all functions whose
orthogonal projection Pa f onto the interval [−a, a] is a given function g(t) =
0, |t| > a. It is easily seen that Ca is a closed linear manifold whose interior
is empty.
Thus Ca and Cb are two examples of sets of functions with compact support that occur in many areas of signal processing as well as phase retrieval.
Let C1 denote the closed unit sphere in H and let g be any fixed element
with kgk = 1. Then g lies on the boundary of C1 , a closed uniformly convex
set. Let C2 denote a closed half space whose boundary is tangent to C1 at g.
Obviously, g is the only point common to the intersection C1 ∩ C2 . To find g
we can generate the iterates T n x, where T = P2 P1 and x is arbitrary. Since
x∗ = g lies on the boundary of C1 , criterion (i) guarantees that T n x → g.
(ii) fails because C2 ∩ C1◦ is empty.
36
2.4
Useful Projections
Definition 43 A convex set C is said to be a cone with vertex φ, if f ∈ C ⇒
µf ∈ C ∀µ ≥ 0, where φ is the zero function in L2 .
(1) Let Bb denote the subset from example 42 a).
(2) Let Ta denote the subset from example 42 b).
(3) Let Bb+ denote the subset of Bb composed of those functions whose
Fourier transform are nonnegative. It can be shown that Bb+ is a closed convex
cone with vertex φ. For if f (t), f1 (t), f2 (t) ∈ Bb+ and µ ≥ 0, µf (t) ↔ µF (u)
is clearly in Bb+ . Similarly, if 0 ≤ µ ≤ 1, then µf1 (t) + (1 − µ)f2 (t) ↔
µF1 (u) + (1 − µ)F2 (u) ≥ 0 and Bb+ is at least a convex cone. To demonstrate
closure we must prove that (fn ) ⊆ Bb+ and fn → f ⇒ f ∈ Bb+ . Let fn (t) ↔
Fn (u), n → ∞, and let f (t) ↔ F (u) := F1 (u) + iF2 (u), where F1 (u) and
F2 (u) are real. Then, since all Fn are real, Parseval’s formula yields
Z∞
|Fn (u) − F1 (u)|2 du +
−∞
Z∞
|F2 (u)|2 du → 0, n → ∞.
(41)
−∞
Thus, F2 (u) = 0 and F (u) = F1 (u) is real. It now follows from 41 and
the vanishing of all Fn for |u| > b that
Z
2
|Fn (u) − F (u)| du +
|u|≤b
Z
|F (u)|2 du → 0.
(42)
|u|>b
Consequently, F (u) = 0 in |u| > b, f (t) is bandlimited to b, and
Z
|Fn (u) − F (u)|2 du → 0.
(43)
|u|≤b
Finally, since all Fn are nonnegative, the assumption that F (u) < 0 on
some set D of positive measure entails the inequality
Z
2
|Fn (u) − F (u)| du ≥
Z
D
|u|≤b
37
|F (u)|2 du > 0∀n,
which contradicts Eq. 43. Therefore, f ∈ Bb+ .
(4) Let Bb+ (ρ) denote the subset of Bb+ composed of all functions that
satisfy
Zb
1
F (u)du = ρ.
2π
−b
Of course, f (t) ↔ F (u) ≥ 0 and ρ ≥ 0 is prescribed. That Bb+ (ρ) is
convex is evident, and as regards closure, suppose again that f ⊆ Bb+ (ρ) and
fn → f. Invoking (3), f ∈ Bb+ , hence f (t) ↔ F (u), Schwarz’s inequality
yields
¯2
¯2
¯ b
¯
¯
¯
¯Z
¯
Zb
¯
¯ 1 ¯
¯
¯
¯
¯ρ − 1
(Fn (u) − F (u)) du¯¯
F (u)du¯
¯
¯
2
2π
¯
¯ 4π ¯
¯
−b
−b
2
 b
Z
1
≤ 2  |Fn (u) − F (u)| du
4π
−b
b
≤ 2
2π
Zb
|Fn (u) − F (u)|2 du → 0,
−b
i.e.
1
2π
Rb
F (u)du = ρ and Bb+ (ρ) is closed.
−b
(5) Let Bb+ (ρ2 ) denote the subset of Bb+ of all functions that satisfy the
energy constraint
Z∞
−∞
1
|f (t)| dt =
2π
2
Zb
|F (u)|2 du ≤ ρ2 .
−b
Since kµf1 + (1 − µ)f2 k ≤ µkf1 k + (1 − µ)kf2 k ≤ µρ + (1 − µ)ρ = ρ,
∀f1 , f2 ∈ Bb+ (ρ2 ), 0 ≤ µ ≤ 1, we conclude that Bb+ (ρ2 ) is convex. In addition
fn → f implies that ρ ≥ kfn k → kf k and closure is also obvious. Bb+ , Bb+ (ρ),
and Bb+ (ρ2 ) are subsets without interiors of Bb .
(6) For a given real function φ(u) (don’t mix it up with the vertex φ
defined earlier)defined on a prescribed set D, let Cφ (D) denote the collection
38
of all functions f (x) ∈ L2 whose Fourier transforms are of the form
F (u) = A(u)eiφ(u) ,
in D, where A(u) = |F (u)| . Clearly, Cφ (D) is a convex cone. To establish
closure, note that in the transform domain, fn → f goes into
An (u)eiθn (u) → A(u)eiθ(u) ,
or, letting D′ denote the complement of D,
Z
D
¯
¯
¯An (u)eiφ − A(u)eiθ ¯2 du +
Z
|Fn − F |2 du → 0.
(44)
D′
Since An → A, Eq. 44 remains valid if θ(u) is replaced by φ(u) in D
and left unchanged for u ∈ D′ . Thus, if this modified exponential is denoted
by eiβ(u) , then Fn → Aeiβ . But a mean-square limit is essentially unique,
whence, eiθ(u) = eiβ(u) , the desired result.
(7) Let T + (D) denote the subset of all functions which are nonnegative
on a prescribed set D. Clearly, T + (D) is a closed convex set with vertex φ.
Theorem 44 Let the projection operators onto the 7 closed convex sets defined in (1)-(7) be denoted by Pi , i = 1, ..., 7. Let an arbitrary element be
denoted by f = f1 + if2 ↔ Aeiθ = F1 + iF2 , where f1 , f2 , F1 , F2 , A and θ are
all real functions. The 7 Pi are synthesized by means of the following rules.
(1) Onto Bb :
½
F (u)
|u| ≤ b
.
P1 f ↔
0
otherwise
(2) Onto Ta :
(3) Onto Bb+ :
where
½
f (t)
P2 f =
0
|t| ≤ a
.
otherwise
½ +
F1 (u)
P3 f ↔
0
F1+ (u)
|u| ≤ b
,
otherwise
½
F1 (u)
=
0
F1 (u) ≥ 0
.
otherwise
39
(4) Onto Bb+ (ρ) :
½
P4 f ↔
(F1 (u) + c)+
0
|u| ≤ b
,
otherwise
where the real constant c is chose so that
Zb
1
(F1 (u) + c)+ du = ρ.
2π
−b
(5) Onto Bb+ (ρ2 ) :
where
c=
½ +
cF1 (u)
P5 f ↔
0




 1




+
s
|u| ≤ b
,
otherwise
ρ2 ≥
1
2π
ρ
1
2π
Rb
Rb
(F1+ (u))2 du
−b
otherwise
.
(F1+ (u))2 du
−b
(6) Onto Cφ (D) :
½ iφ(u)
e
A(u) cos+ [θ(u) − φ(u)]
P6 f ↔
F (u)
u∈D
u 6∈ D
.
(7) Onto T + (D) :
½
f1 (t)
P7 f =
0
t∈D
.
t 6∈ D
Only P1 and P2 are linear projection operators but nevertheless all 7
projection operators are frequently occurrent in a wide variety of signal processing and phase retrieval applications.
3
3.1
Application
General
We can distinguish between reconstruction using POCS (e.g. reconstruction
from phase) and using general projections (e.g. reconstruction from magnitude). While in the convex case only tunnels (regions of slow convergence)
40
may occur in the other case also traps (fixed points which are not a solution
of the problem) are possible (see Chapter 3.5). To avoid traps or reduce
their number to a minimum a multiresolution algorithm (see Chapter 3.6) is
presented.
3.2
The Method of POCS
3.2.1
Optimization of the Two-Step POCS Algorithm
The two-step-per-cycle realization of the POCS algorithm is given by
f1n = T1 fn ,
fn+1 = T2 f1n ,
f0 arbitrary,
or as a composition
fn+1 = T2 T1 fn ,
f0 arbitrary.
(45)
The estimate f1n is an intermediate result obtained in each iteration after
the first step, i.e., after applying T1 on fn . In what follows we consider the
algorithm in Eq.45 with
T1 = I + λ1n (P1 − I),
T2 = I + λ2n (P2 − I),
and we attempt to develop some theory and procedures for determining the
relaxation parameters λ1n , λ2n for improving the convergence rate. The first
subscript identifies the λ with the operator and the second refers to the
iteration number. Thus λin is the relaxation parameter associated with Pi
at the n-th iteration.
To distinguish between the two different criteria for choosing the relaxation parameters we first define the error after the n-th iteration by
en = fn − f,
(46)
where f ∈ C0 is a valid solution and define the intermediate-step error e1n
by
e1n = f1n − f.
(47)
It is easy to show that the sequence of normed per-cycle errors satisfy
ken k ≤ ke1,n−1 k ≤ ken−1 k ≤ ke1,n−2 k ≤ · · ·.
41
The proof is based on the following facts: since f is a fixed point of
Ti , i = 1, 2 we can write f = T1 f = T2 f and the projection operators are
nonexpansive and hence kTi fn − Ti f k ≤ kfn − f k, i = 1, 2.
We define our criteria as follows:
1) Find λ1n , λ2n that sequentially minimize the normed errors after each
step, i.e., so-called per-step optimization.
2) Find λ1n , λ2n that minimize the normed errors after each iteration
(cycle), i.e., so-called per-cycle optimization.
In 1) we seek to sequentially minimize ke1n k for a given fn , according to
which we minimize ken+1 k for a given f1n . In 2) we seek to minimize ken+1 k
for a given fn . Intuitively we expect criterion 2) to give better results because
it considers the minimization for a whole cycle.
3.2.2
Restoration from Phase (RFP)
Theorem 45 Let f (t) be a finite time signal whose Fourier transform has
no complex conjugate zeros. The signal f (t) is uniquely determined (within
a positive multiplicative constant) by its Fourier phase φ(u).
The two sets of principal interest in the RFP problem are
C1 = {h(t) : h(t) = 0, |t| > a}
and
C2 = {h(t) : arg[H(u)] = φ(u)}.
C1 is the set of space-truncated functions and C2 is the set of all h with a
prescribed phase. Both C1 and C2 are closed convex sets. With Pi denoting
the projection operator onto Ci it is not difficult to show that P1 and P2 are
realized by
½
g(t)
|t| ≤ a
P1 g =
0
otherwise
and
½
|G(u)| cos[φ(u) − ψ(u)]eiφ(u)
cos[φ(u) − ψ(u)] ≥ 0
P2 g ↔
0
otherwise
(48)
where φ(u) is the prescribed phase, ψ(u) = arg[G(u)], and g(t) is an arbitrary
element of H. Another useful generalization of C2 is Cφ (D) defined by
C2 = {h(t) : arg[H(u)] = φ(u) over a set D}
42
and

 |G(u)| cos[φ(u) − ψ(u)]eiφ(u) cos[φ(u) − ψ(u)] ≥ 0, u ∈ D
P2 g ↔
0
otherwise for u ∈ D

G(u)
u∈
/D
For the reconstruction of f (t) from its Fourier transform phase φ(u) we can
use the two-step POCS algorithm given by 45 ,i.e.,
fn+1 = T2 T1 fn with f0 arbitrary and Ti = I+λin (Pi −I), i = 1, 2; λin ∈ (0, 2).
The relaxation parameters λin may be chosen by the per-step or per-cycle
optimization method.
In 45, T1 and T2 can be interchanged without affecting the ultimate convergence, although the actual rate of convergence may depend on the ordering.
3.2.3
Per-Step Optimization of the Relaxation Parameters
To simplify the notation we omit the subscript n on λ. From 47 and 45, we
obtain
ke1n k = kfn + λ1 (P1 − I)fn − f k2 =
kfn − f k2 + 2λ1 Re[(fn − f, (P1 − I)fn )] + λ21 k(P1 − I)fn k2
from which it is easily verified by differentiation that ke1n k2 is a minimum
for any fixed fn when
λ1 = λ1,opt =
Re[(f − fn , (P1 − I)fn )]
.
k(P1 − I)fn k2
Similarly, if f1n is fixed, ken+1 k is a minimum when
λ2 = λ2,opt =
Re[(f − f1n , (P2 − I)f1n )]
.
k(P2 − I)f1n k2
(49)
When C is a linear subspace then λopt = 1.[1] Since P1 projects onto a linear
subspace, we have λ1,opt = 1. To compute λ2,opt , we use 48 for P2 . To simplify
the notation, we make the following definitions
yn = P2 fn ,
Yn (u) = F[yn (x)],
43
so that
½
|Fn (u)| cos[φ(u) − ψn (u)]eiφ(u)
Yn (u) =
0
u ∈ Ωcn
u ∈ Ωn
where
ψn = arg[Fn (u)], ψ = arg[F (u)], Ωn = {u : cos[φ(u) − ψn (u)] ≥ 0}.
λ2,opt is given by
λ2,opt = 1 −
R
Ωcn
|F (u)| |Fn (u)| cos[φ(u) − ψn (u)]du
kFn k2 − kYn k2
.[1]
We see that λ2,opt ≥ 1 since the integration is over Ωcn , for which the term on
the right of the minus sign is negative. Since |F (u)| is unknown, we approximate λ2,opt by replacing |F (u)| with |Fn (u)| wherever the former appears.
3.2.4
Per-Cycle Optimization of the Relaxation Parameters
Here we seek to minimize ken+1 k by appropriate choice of λ1 , λ2 , where
en+1 = fn+1 − f = T2 T1 fn − f.
First we observe that regardless of what value λ1 has, we must still compute
λ2,opt from 53, namely
λ2,opt =
Re[(f − T1 fn , P2 T1 fn − T1 fn )]
,
kP2 T1 fn − T1 fn k2
(50)
since λ2,opt will minimize ken+1 k for any given fn and λ1 . However, λ2,opt
cannot be evaluated directly because we have as yet not determined λ1,opt on
which λ2,opt depends and we do not know f. We note that ken+1 k2 can be
written as
ken+1 k2 = k[I + λ2,opt (P2 − I)]T1 fn − f k2 .
(51)
We shall consider how per-cycle optimization of the relaxation parameters is
affected for the two algorithms obtained by commuting the operators T1 and
T2 . Thus we consider the optimization of 1) fn+1 = T2 T1 fn and 2) fn+1 =
T1 T2 fn .
1) Finding the relaxation parameters of fn+1 = T2 T1 fn . The value of λ2
that minimizes ken+1 k2 given by 51 is given by 50. To minimize 51 we can
44
minimize some suitable upper bound on Eq.51 or minimize an approximation
of Eq.51 directly. In either of these alternatives λ1,opt can be determined
approximately by a simple search technique. This estimate of λ1,opt is then
used in Eq.50 with the latest estimate of |F | replacing the actual |F | to
compute an estimate of λ2,opt .
2) Finding the relaxation parameters of fn+1 = T1 T2 fn . It is easy to show
that λ1,opt = 1 since C1 is a closed linear subspace and for any f ∈ C1 the
error is minimized by projecting orthogonally onto the subspace. With this
result, we can compute λ2,opt explicitly as
λ2,opt = 1 +
kP2 fn − P1 P2 fn k2 Re[(f − P2 fn , P2 fn − fn )]
+
.[1]
kP1 P2 fn − fn k2
kP1 P2 fn − fn k2
(52)
It is not difficult to show that the third term on the right in this equation is
nonnegative. Therefore we obtain as a lower bound
λ2,opt ≥ 1 +
kP2 fn − P1 P2 fn k2
= λL .
kP1 P2 fn − fn k2
(53)
We note that
ken+1 k2 = kP1 fn − f + λ2 P1 (P2 − I)fn k2
is a quadratic function of λ2 . Therefore, there is only one minimum and it
occurs at the value of λ2,opt given by Eq.52. However, in the range 0 ≤ λ ≤
λ2,opt , the error is a monotonically decreasing function of λ even if λL ≥ 2
because in this way we minimize approximately ken+1 k2 . However, at some
point and thereafter in the algorithm, we should restrict λL ≤ 2 to ensure
convergence.
The projection P2 fn is calculated in the Fourier domain by using Eq.
48 and λ2,opt is approximated by λL of Eq. 53. Since λ1,opt = 1 we have
fn+1 = P1 T2 fn and therefore fn ∈ C1 , ∀n 6= 0. As a result we have P1 fn = fn ,
which leads to
fn+1 = P1 [I + λ2,opt (P2 − I)]fn = (1 − λ2,opt )fn + λ2,opt P1 P2 fn .
3.3
3.3.1
The Method of Generalized Projections
General Background
There are also important restoration problems involving nonconvex constraints, e.g., digitizing a signal is not equivalent to projecting onto a convex
45
set. Neither is constraining a signal (nor its Fourier transform) to have a
prescribed magnitude.
But when the number of sets is two a lot of interesting conclusions can
be drawn. This is not as serious a restriction as might at first be supposed
because other constraint sets can be combined to form single, more complex
sets for which it is still possible to derive a projection operator. Indeed,
the investigation of the algorithm fn+1 = Tn fn when C1 and C2 are not
necessarily convex is the main consideration in this section.
In general, for all closed sets- not just convex ones- we call g = Pi h the
projection of h onto Ci if g ∈ Ci and
kg − hk = min ky − hk.
y∈Ci
Remark 46 1. The projection defined in the equation above is a unique
point if Ci is a convex set. When Ci is nonconvex there may be a set of
points that satisfy the definition of a projection. However, in practice, we
can also find a procedure for uniquely choosing one of these points, usually
through the demand of satisfying another condition. This eliminates the ambiguity that would otherwise result from nonsingleton projection points, e.g.,
in the restoration from magnitude problem, in projecting onto the set of functions with prescribed Fourier magnitude, the phase of the estimate at the n-th
iteration uniquely defines the projection.
2. Projections always exist for convex sets. For nonconvex sets, the proof
of the existence of a projection apparently does not exist, although it is difficult
to conjecture a realistic situation where this would be the case. Nevertheless,
to be on the safe side, we shall assume that all sets that appear in problems
for which the method of generalized projections may be useful have at least
one projection.
One easily verifiable result is that the Gerchberg-Saxton algorithm for
restoration of a function from magnitude data does involve projection onto
nonconvex sets. It can be written as
fn+1 = P1 P2 fn
where P1 is the projection operator onto the set C1 of functions which are
space-limited and P2 the projection operator onto C2 of functions which have
a Fourier transform magnitude equal to some prescribed magnitude.
46
3.3.2
Restoration by Generalized Projections
Since convergence of the equation fn+1 = T1 T2 fn is not assured, we need
some measure of performance that will allow us to compute the performance
of the algorithm during the iteration process.
Remark 47 The distance of a point x from a closed set C, denoted by
d(x, C), could be written as
d(x, C) = min ky − xk = kP x − xk
y∈C
using the projector P.
Definition 48 Let C1 , C2 be any two sets with projection operators P1 , P2 .
Let fn be the estimate of f. The performance measure at fn , denoted by J(fn ),
is the sum of the distances between the point fn and the sets C1 and C2 . Thus
the performance is measured by a criterion which we call the summed distance
error (SDE). Specifically,
J(fn ) = kP1 fn − fn k + kP2 fn − fn k
(54)
and similarly
J(T2 fn ) = kP1 T2 fn − T2 fn k + kP1 T2 fn − T2 fn k.
Remark 49 a) J(fn ) ≥ 0
b) J(fn ) = 0 ⇔ fn ∈ C1 ∩ C2
Theorem 50 The recursion fn+1 = T1 T2 fn has the property
J(fn+1 ) ≤ J(T2 fn ) ≤ J(fn )
∀λi : 0 ≤ λi ≤
A2i + Ai
,
A2i + Ai − 12 (Ai + Bi )
where
kP1 T2 fn − T2 fn k
,
kP2 T2 fn − T2 fn k
kP2 fn − fn k
A2 =
kP1 fn − fn k
A1 =
47
(55)
and
(P2 T2 fn − T2 fn , P1 T2 fn − T2 fn )
,
kP2 T2 fn − T2 fn k2
(P1 fn − fn , P2 fn − fn )
B2 =
.
kP1 fn − fn k2
B1 =
Remark 51 1. It can easily be verified, by using Cauchy’s inequality, that
Ai ≥ Bi ,
1
Ai ≥ (Ai + Bi ) ≥ 0.
(56)
2
Therefore by using Eq. 56 in Eq. 55 we conclude that the value of unity will
always be included in the range of λi as given in Eq. 55. This means that
the algorithm given by
fn+1 = P1 P2 fn ,
f0 arbitrary,
will always have the property of set distance reduction. However, that does
not make the algorithm optimal.
2. Theorem 50 is also true if Ai → ∞, in which case the range of λi as
given in Eq. 55 becomes 0 ≤ λi ≤ 1, e.g., kP1 fn − fn k → 0 implies A2 → ∞.
Despite the fact that the theorem is not valid for more than two sets,
the algorithm fn+1 = T1 T2 fn is not so restrictive in practice. As already
stated, the theorem does not restrict the complexity of the sets and therefore
C1 and C2 can indeed include signals with multiple constraints. But as the
sets become more complex, so do the projection operators. We can usually
combine several properties of the signal f and associate them with one set.
As a rule we can combine those properties of the signal which are easily
expressed in the space domain to one set C1 whose associated projector P1 can
be calculated without too much effort. Similarly, the properties of the signal,
which are easily expressed in the transform domain, can be combined into
a second set C2 and the corresponding projector P2 can again be calculated
without too much effort.
Example 52 Let C1 be the set of functions g(t) of compact support which
are zero ∀t ∈
/ S1 and have a space domain magnitude which is equal to some
given positive function h(t) in a set S2 with S2 ⊂ S1 . Thus
C1 = {g(t) : g(t) = 0, ∀t ∈ S1c and |g(t)| = h(t), ∀t ∈ S2 },
48
Figure 1: Original signal (a) and error comparison (b)
It is easily shown that for any arbitrary f (x)

0
t ∈ S1c



f (t)
t ∈ S1 ∩ S2
P1 f =
h(t)
t
∈
S2 , f (t) > 0



−h(t)
t ∈ S2 , f (t) ≤ 0
3.4
3.4.1
Signal Recovery from Magnitude
General
This problem, called restoration from magnitude (RFM) or phase retrieval,
appears e.g. in x-ray-crystallography, where the available information about
a signal comes from intensity measurement in the space domain and from
some a priori knowledge such as the fact that the signal is space-limited and
the signal function is nonnegative.
The related uniqueness problem refers to the question of whether a function can be uniquely defined by its Fourier transform magnitude. When
dealing with uniqueness in the phase retrieval problem it must be noted that
49
Figure 2: POCS method without relaxation
all functions f (t), −f (t), f (t−t0 ) and f (−t) have the same Fourier transform
magnitude function and therefore uniqueness in this case is up to a sign, shift
in coordinates, or coordinate reversal.
Theorem 53 If f (t) is a complex finite time signal and its Fourier transform
has no zeros in the upper half plane or lower half plane, then f (t) is uniquely
determined by its Fourier magnitude.
3.4.2
Generalized Projection Algorithm for the Restoration from
Magnitude Problem
The two sets involved in the phase retrieval problem are C1 the set of spacelimited functions (a positive level constraint, i.e., f (t) ≥ 0, ∀t ∈ [−a, a],
can be added easily), and C2 , the set of all functions which have a Fourier
transform magnitude equal to some real positive given function M (u). Thus
C1 = {g(t) : g(t) = 0 for |t| > a},
(57)
C2 = {g(t) ↔ G(u) : |G(u)| = M (u)∀u}.
(58)
50
Figure 3: POCS method with relaxation
As mentioned earlier C1 is convex and C2 is nonconvex. The projections P1
and P2 onto C1 and C2 , respectively are given by
½
g(t),
|t| < a,
P1 g =
(59)
0,
|t| ≥ a.
and
P2 g ↔ M (u)eiφ(u) ,
(60)
where φ(u) is the phase of G(u). P2 g is uniquely defined by the Eq. 60
although C2 is nonconvex. The general restoration algorithm is given by
fn+1 = T1 T2 fn with Ti defined by Ti = I + λi (Pi − I). This algorithm has
the property of set-distance reduction, or the property that (J(fn ))∞
n=0 is a
nonincreasing sequence, for those values of λ1 , λ2 that satisfy the inequality
55.
Remark 54 When λ1 = λ2 = 1 then fn+1 = T1 T2 fn with P1 and P2 as
defined in Eqs. 59 and 60, reduces to the error reducing algorithm, and the
property that (J(fn ))∞
n=0 is a nonincreasing sequence becomes equivalent to
the nonincreasing error property.
51
Figure 4: Original signal (a) and error comparison (b)
3.4.3
Optimization of the Relaxation parameter λi
The summed distance error J(fn ) is the best performance measure that we
have for this problem, because it is always nonnegative and is zero if and
only if a solution is achieved. Therefore, it is natural to try to choose the
λi in order to maximize the rate of reduction of the sequence (J(fn )). We
use the per-cycle optimization in which we try to minimize J(fn+1 ) with
respect to λ1 and λ2 simultaneously for a given fn . To avoid too much effort
and not to waste computing time, we make the approximation λ1,opt = 1
for the algorithm fn+1 = T1 T2 fn . This approximation becomes exact if the
projection operator P1 is linear which it is in our case (Eq. 59). With this
approximation we have
fn+1 = P1 T2 fn
f0 arbitrary.
This equation implies that fn ∈ C1 ∀n 6= 0; therefore kP1 fn+1 − fn+1 k = 0.
Now, by using the definition for the SDE we obtain for J(fn+1 ) the following
relation
J(fn+1 ) = kP2 P1 T2 fn − P1 T2 fn k = kP2 fn+1 − fn+1 k.
52
Figure 5: POCS method without relaxation
To calculate λ2,opt let us denote fn+1 (t) ↔ |Fn+1 | eiφn+1 (u) a Fourier transform pair. From the definition of P2 given by Eq.60, we obtain P2 fn+1 =
M (u)eiφn+1 (u) and therefore by using Parseval’s relation kf − gk = kF − Gk
we obtain
Z∞
1
2
(M (u) − |Fn+1 (u)|)2 du,
(61)
J (fn+1 ) =
2π
−∞
where
Fn+1 (u) = F{P1 [I + λ2 (P2 − I)]fn (t)}.
(62)
To find λ2,opt we have to minimize J(fn+1 ) in Eq.61, where Fn+1 (u) is given
in Eq. 62. In general this can be done by a search through a range of values
of λ2 . This search may be relatively fast if P1 is linear, since in this case
Eq.62 reduces to
Fn+1 (u) = (1 − λ2 )Fn (u) + λ2 F{P1 P2 fn },
where the fact that P1 fn = fn is used. This means that when P1 is linear
the search process does not involve the Fourier transform operation. When
P1 is not linear we have to use the Fourier transform operator once for each
tested value of λ2 , and therefore the search of λ2,opt will be much slower.
53
Figure 6: POCS method with relaxation
3.5
Traps and Tunnels
We define a trap as a fixed point of the composition operator T1 · · · Tm which
is not a fixed point of every individual Ti , i = 1, ..., m, i.e., a point which fails
to satisfy one or more of the a priori constraints yet satisfies
fn+1 = T1 T2 · · · Tm fn = fn
(63)
We say that a point fn is a tunnel if Eq.63 is almost satisfied, which means
that the change in fn from one iteration to the next is negligible. Traps can
never occur when only convex sets are involved, but tunnels (i.e., extremely
slow convergence toward a valid solution) may occur. In general, when at
least one nonconvex set is involved, traps may occur.
However, the existence of traps is supported by the difficulties one encounters in restoring some signals from their magnitudes. The mathematical
conditions that must be satisfied for traps to exist can be explored in the
restoration from magnitude problem by considering the error reducing algorithm
fn+1 = P1 P2 fn ,
f0 arbitrary.
54
Figure 7: Reconstruction from localized phase
This is the simplest form of the more general algorithm fn+1 = P1 P2 fn and
corresponds to the unrelaxed case (i.e., when λ1 = λ2 = 1) involving pure
projections. P1 and P2 and their associated sets C1 and C2 are given by Eqs.
59, 60 and Eqs. 57, 58, respectively. By using the notation fn (t) ↔ Fn (u) =
An (u)eiφn (u) , where An (u) = |Fn (u)| , we can write P2 fn ↔ M (u)eiφn (u) .
Assuming f to be a correct solution, the indication of a trap in this case is
fn 6= f and fn is a fixed point of P1 P2 , i.e.,
P1 P2 fn = fn .
We can replace this condition by
P2 fn (t)Π
where
Π(x) =
½
µ
t
2a
¶
= fn (t),
|x| ≤ 12 ,
elsewhere.
1
0
55
(64)
Figure 8: Original signal and error comparison
We can Eq. 64 in the frequency domain by taking the Fourier transform of
both sides as
Z∞
M (v)eiφn (v)
2 sin((u − v)v)dv
= An (u)eiφn (u) .
u−v
−∞
Invalid (traps) and valid solutions to the RFM must be solutions of the form
An (u)eiφn (u) of this integral equation. The correct solution f ↔ An (u)eiφn (u)
satisfies An (u) = M (u). For traps a much weaker restriction applies: An (u) 6=
M (u) and An (u) ≥ 0.
The SDE J(fn ) can be used to detect traps when we observe no change
in J(fn ) > 0 from iteration to iteration. The condition
J ∗ (fn+1 ) = J(fn ) ⇔ fn+1 = fn
where J ∗ (fn+1 ) denotes the minimum of J(fn+1 ), determines the existence
of a trap. One can infer a trap when J ∗ (fn+1 ) > 0 and kfn+1 − fn k = 0.
56
Figure 9: Original signal (a) and error comparison (b)
3.6
3.6.1
Iterative Multiresolution Algorithm for Reconstruction from Magnitude
General
The multiresolution adaptation of the error-reducing algorithm that attempts
to solve the stagnation is based on decomposing the problem of signal reconstruction into different resolutions. By using the solution obtained from the
iterative algorithm at a lower resolution as an initial guess for the next finer
level, then following a coarse-to-fine strategy, the approach enables the iterative algorithm to escape local minima by providing a better initial phase
estimate. If several iterations of the error-reducing algorithm are performed
on lower resolutions of the signal, then the number of local minima, as well
as the computations required, will be reduced. This indicates that it is more
likely to find a good initial point that is close to the global minimum by
coarser grid iterations rather than with a finer grid. Then, by locally interpolating the result to a finer grid and using this result as the new estimate,
rapid convergence can be achieved.
57
Figure 10: POCS method without relaxation
3.6.2
Multiresolution Algorithm
In the multiresolution algorithm a successively condensed representation of
the input information is provided. What condensed may be signal intensity, so that the successive levels are reduced-resolution versions of the input
signal. However, this condensed representation also represents a smoothed
or subsampled version of any information of the input, so that each level
represents a reduced entropy of the input.
Given an input signal f0 of size N = 2p , applying the averaging process
yields a reduced signal f1 of size 2p−1 . Applying the process again to the
reduced signal yields a still smaller signal f2 of size 2p−2 and so on till we
reach fM −1 of size 1, which represents the average value of the input signal.
The restriction operator R is defined such that,
fl = Rfl−1 , 0 < l < M
where fl is a reduced-resolution version of fl−1 .
In a similar manner a prolongation operator P is defined such that,
fl = Pfl+1 , 0 ≤ l < M − 1.
58
Figure 11: POCS method with relaxation
3.6.3
Description of the Procedure
The first step in the signal reconstruction procedure outlined here is to construct an approximated representation of the measured Fourier transform
magnitude for each level. Since a precise measurement for the Fourier transform at all the levels (i.e. |Fl (u)| , 0 < l < M ) is not available, an estimate
of these measurements is sought. An approximation of the Fourier transform magnitude at coarse grids can be obtained by first downsampling the
measured autocorrelation function, followed by Fourier transforming the subsampled autocorrelation and taking the square root of its real part. Using
the restriction operator R the above procedure can be expressed as
¯
¯
¯e ¯ p
F
(u)
¯ l ¯ = Re (F (Rrl−1 (t))), 0 < l < M,
¡
¢
where r0 (t) = r(t) = F−1 |F (u)|2 is the autocorrelation. The approximation of the Fourier transform magnitude at a lower resolutions are equal
to a Fourier transform magnitude of a low-pass version of the signal to be
reconstructed. The reconstruction procedure proceeds from the lowest to
the highest resolution level by performing several cycles of the basic error59
Figure 12: Original signal (a) and error comparison (b)
reduction algorithm on the lowest resolution level (M − 1) until the number
of iterations reaches an arbitrary limit that depends on the complexity of the
signal. The reconstructed signal gM −1 is then transferred to the next level
(M − 2) with double the resolution by applying the interpolation process P,
that is
gl−1 = Pgl , 1 ≤ l < M.
The same basic iterative procedure is performed on the new level using the
last signal estimate computed from the previous level as the new initial guess
for the algorithm. This process is repeated following a coarse-to-fine strategy until level zero is reached, where the signal is now reconstructed at the
original resolution.
The number of levels M for a signal of size N is typically chosen to be
equal log2 N.
60
Figure 13: Generalized projections without relaxation
3.7
3.7.1
Simulation Results
Results from RFP
The signal
f (t) =
½
¡
¢
¡ ¢
0.6 + 1 − 10t sin 20t
0
t = 0, ..., 255
,
t = 256, ...511
shown in Fig.1(a), is Fourier transformed and to the phase information of
the obtained complex valued function the POCS algorithm with pure projections is applied. The reconstructed signal after 100 iterations of the POCS
algorithm without relaxation is shown in Fig.2(a) as solid line. For reasons of
comparison the signal is shown in normalized form. The normalized original
signal f (t) is shown as broken line. The logarithm of the normalized error
°
°
°
f
1 °
f
n
°
°,
−
en =
kf k ° kfn k kf k °
where fn is the result after the n-th iteration and f is the original signal,
61
Figure 14: Generalized projections with relaxation
is shown in 2(b). To make signals of different length and magnitude better
comparable the error is divided by kf k .
In Fig. 3 the results for the POCS algorithm with relaxation are shown.
A comparison of both error curves is done in Fig.1(b), where the broken
line is the logarithmic error of the POCS algorithm with pure projections and
the solid line is that one with relaxation. As we can see the error reduces
in the second case much faster than in the case without relaxation. This
fact we can watch in Fig. 3(a), too. In this diagram the original and the
reconstructed signal are equal.
In Figs. 4 - 6 the same is done for a random signal.
The Convergence Angle and Localized Phase The angle α between
the two convex sets C1 and C2 is responsible for the speed of convergence
of the algorithm. The consequence of a small angle is a tunnel and the
convergence rate becomes slow. If the angle even equals zero we have the
slowest convergence rate. Based on the convergence angle α, which is equal
to the eigenvalue of the operator P1 P2 , it can be shown that the highest value
α = π4 is achieved by a special type of signals a geometric sequence of the
62
Figure 15: Multiresolution algorithm
form
£
¤
c 1, q, q 2 , ..., q N −1 , c, q ∈ R+ ,
(65)
where N is the length of the signal. For all other signals is α ≤ π4 .
For a symmetric or partially symmetric sequence whose Fourier amplitudes are all non-zero, the convergence angle α = 0. [?]
Based on the above results, we conclude that a significantly higher convergence rate is achieved in the case of localized phase compared to the global
phase representation. If the signal is bisected, each further bisection can only
increase the ratio of monotonic to non-monotonic sequences. Smaller sections
are more likely to be monotonic than larger sections, and therefore likely to
be closer , on the average, to geometric.
In Fig. 7 the reconstruction of two signals from localized phase is shown.
The signal is divided into 10 segments and as window used for the STFT a
box function is with length N/10 is taken. In other words, the segmentation
process is the STFT simply applied on the original signal. The output of this
process is a matrix from which only the phase information is taken. With this
localized phase the reconstruction process is done with an iteration number
63
Figure 16: Signal (a) and error comparison (b)
of 10 for each segment. In the recombination process all segments are linked
together by multiplying each segment with an appropriate factor so that a
continuous signal is obtained. This is necessary because each segment is
reconstructed within an arbitrary positive scale factor.
Above all in the case of symmetric signals this method shows better results
than the method of global phase as we can see if we compare Fig. 7(b) with
Fig. 10(a) or 11(a). The result can be improved once again if more effort
is invested in the recombination process (e.g. using the mean energy or the
standard deviation of the segments to calculate the appropriate factors) or if
the number of segments is chosen depending on the signal (e.g. 2 segments
for the signal in Fig.7(b) ). A simple recombination algorithm is shown in
A.1.4.
In Fig. 8 reconstruction of a geometric signal is shown. Both algorithms,
with and without relaxation, give the same result and convergence is very
fast. As signal the form defined by Eq. 65 with c = 1, q = 0.94 and N = 64
was used.
In Figs. 9 - 11 the results for a symmetric signal are shown. Both methods
stagnate at approximately same level of error and the reconstructed signal is
64
Figure 17: Signal (a) and error comparison (b)
deviated from the original.
Finally could be said that the method with relaxation is very useful because it gives much better results and needs only 10% more computing time.
3.7.2
Results from RFM
The three methods, generalized projection without relaxation, generalized
projection with relaxation and the multiresolution algorithm are tested and
the results are compared. As test signals the same signals as for the RFP
algorithm were used. As error measure the SDE (Eq.54) was used. To
make signals of different length and magnitude better comparable the SDE
J(fn ) is normalized by kf k . For the majority of signals the observation was
made that the multiresolution algorithm gives the best results and needs
only 10% more computing time than the general projecting algorithm without
relaxation while the general projection algorithm with relaxation needs about
four times more computing time. This is explainable through the fact that
the multiresolution algorithm needs only e few iterations on a coarser grid
which does not cost to much computing time while the projection algorithm
65
with relaxation needs a search strategy for the relaxation parameter and this
is very time expensive. In general the same conclusions like in RFP can
be done, namely that signals of geometric form have the best convergence
while symmetric signals are the worst case. In Figs. 12-17 the results are
shown. In the error comparison diagrams in Figs.12, 16 and 17 the error of
the generalized algorithm without relaxation is shown as solid line, the error
of the generalized algorithm with relaxation as dashed line and the error of
the multiresolution algorithm as dashdotted line.
66
A
A.1
MATLAB Files
Restoration from Phase
The files are shown for 100 iterations. As input the phase information of the
Fourier transform is expected. As output the reconstructed original signal is
given.
A.1.1
POCS Algorithm without Relaxation
function r = rfp1(ph,iter)
% rfp1 is the pocs-algorithm for reconstructing the
% signal (f) from the phase of its fourier transform (ph)
% without relaxation
if nargin < 2
% set default iteration number
iter = 100;
end;
n = length(ph);
n2 = n/2;
% initial guess of f
f = [ones(1,n2),zeros(1,n2)];
% p1....projection onto c1
p1 = [ones(1,n2),zeros(1,n2)];
% the algorithm
for k = 1:iter,
% p2....projection onto c2 (procedure)
f = p2(f,ph);
f = p1.*f;
end;
67
r = f;
68
A.1.2
POCS Algorithm with Relaxation
function r = rfp2(ph,iter)
% rfp2 is the pocs-algorithm for reconstructing the
% signal (f) from the phase of its fourier transform (ph)
% with relaxation
if nargin < 2
%set default iteration number
iter = 100;
end;
n = length(ph);
n2 = n/2;
% initial guess of f
f = [ones(1,n2),zeros(1,n2)];
% p1....projection onto c1
p1 = [ones(1,n2),zeros(1,n2)];
% the algorithm
for k = 1:iter,
% calculating lambda 2
% p2....projection onto c2 (procedure)
p2v = p2(f,ph)
p1p2 = p1.*p2v;
l2 = ((norm(p2v-p1p2))^2)/((norm(p1p2-f))^2);
f = (1-l2)*f+l2*p1p2;
end;
r = f;
69
A.1.3
Projection Operator P2
function r = p2(f,ph)
% p2 is the projection onto c2
n = length(f);
ft = fft(f);
% angle of fourier transform
x_ph = angle(ft);
amp = abs(ft);
% projection in fourier space
orton = cos(ph-x_ph);
orton = orton.*(orton>=0);
amp = amp.*orton;
% go back into original space
[x,y] = pol2cart(ph,amp);
cart = x+i*y;
r = real(ifft(cart));
70
A.1.4
Reconstruction from Localized Phase
function r = reconstr(seg)
% recombination process for reconstruction from
% localized phase
% input parameter is the matrix containing the segments
% of the local reconstruction from phase
% calculate matrix dimension
[m,n] = size(seg);
% initialize first segment
s = seg(1,1:n/2);
% iteration over all segments
for k = 2:m,
% take last sample of previous signal
ps = s((k-1) * n/2);
% take first sample of current segment
cs = seg(k,1);
% multiply current segment to fit with
% previous signal
seg(k,:) = seg(k,:) * ps/cs;
% add current segment to previous signal
s = [s,seg(k,1:n/2)];
% normalize signal
s = s/norm(s);
end;
r = s;
71
A.2
Restoration from Magnitude
The files are shown for 100 iterations. As input the magnitude information
of the Fourier transform is expected. As output the reconstructed original
signal is given.
A.2.1
Generalized Projection Algorithm without Relaxation
function r = rfm1(amp,iter)
% rfm1 is the algorithm for reconstructing the signal (f)
% from the magnitude of its fourier transform (amp) without
% relaxation
if nargin < 2
%set default iterartion number
iter = 100;
end;
n = length(amp);
n2 = n/2;
% initial guess of f
f = [ones(1,n2),zeros(1,n2)];
% p1....projection onto c1
p1 = [ones(1,n2),zeros(1,n2)];
% the algorithm
for k = 1:iter,
% p2....projection onto c2 (procedure)
f = p2(f,amp);
f = p1.*f;
end;
r = f;
72
A.2.2
Generalized Projection Algorithm with Relaxation
function r = rfm2(amp,iter)
% rfm2 is the algorithm for reconstructing the signal (f)
% from the magnitude of its fourier transform (amp) with
% relaxation
if nargin < 2
%set default iteration number
iter = 100;
end;
n = length(amp);
n2 = n/2;
% initial guess of f
f = [ones(1,n2),zeros(1,n2)];
% p1....projection onto c1
p1 = [ones(1,n2),zeros(1,n2)];
Fn = fft(f);
% the algorithm
for k = 1:iter,
% calculate l2opt (lambda 2 opt)
fftp = fft(p1.*p2(f,amp));
jmin = realmax;
for l2 = 0:0.1:2,
Fn = (1-l2)*Fn+l2*fftp;
j = norm(amp-abs(Fn))/(2*pi);
if j < jmin
jmin = j;
l2opt = l2;
end;
end;
% p2....projection onto c2 (procedure)
73
f = (1-l2opt)*f+l2opt*p2(f,amp);
f = p1.*f;
end;
r = f;
74
A.2.3
Multiresolution Algorithm
function r = rfm3(amp,iter)
% multiresolution algorithm for phase retrieval
% from the magnitude of the fourier transform
if nargin < 2
%set default iteration number
iter = 100;
end;
% amp....magnitude of fourier transform (measured)
n = length(amp);
% st is the number of steps needed to build the
% multiresolution pyramid
st = log2(n)-1;
% calculate the autocorrelation of magnitude
ac = abs((ifft(amp.^2)));
% re.....restriction operator
rr = re(ac);
% built the pyramid
for l = 1:st,
k = 2^(st-l+1);
f(l,1:k) = abs(sqrt(real(fft(rr))));
rr = re(rr);
end;
% initial guess
guess = [1,0];
for l = st:-1:1,
k = 2^(st-l+1);
guess = gs(f(l,1:k),guess,iter/10);
% inter....linear interpolation
guess = inter(guess);
end;
r = gs(amp,guess,iter);
75
A.2.4
Error Reducing Algorithm
function r = gs(amp,guess,iter)
% gs is the algorithm for reconstructing the signal (f)
% from the magnitude of its fourier transform (amp) without
% relaxation (error reducing or gerchberg-saxton algorithm)
n = length(amp);
n2 = n/2;
% initial guess of f
f = guess;
% p1....projection onto c1
p1 = [ones(1,n2),zeros(1,n2)];
% the algorithm
for k = 1:iter,
% p2....projection onto c2 (procedure)
f = p2(f,amp);
f = p1.*f;
end;
r = f;
76
A.2.5
Projection Operator P2
function r = p2(f,amp)
% p2 is the projection onto c2
n = length(f);
ft = fft(f);
ph = angle (ft);
% take phase of ft and give magnitude
[x,y] = pol2cart(ph,amp);
cart = x+i*y;
% do inverse fourier transform
r = real(ifft (cart));
77
Acknowledgement
I would like to thank my advisor Prof. Hans G. Feichtinger for his help
during my work on this thesis. Without his support it would have been
impossible to write this thesis. Special thanks also to Johann Lutz who
introduced me in solid state physics.
78
References
[1] H. Stark A. Levi. Image restoration by the method of generalized projections with application to restoration from magnitude. J. Opt. Soc.
Am. 1(2), pages 932–943, 1984.
[2] J.C. Anderson. Speech analysis/synthesis based on perception. Technical report, MIT Lincoln Laboratory, 1984.
[3] A.S. Willsky A.V. Oppenheim. Signals and Systems. Prentice-Hall,
1983.
[4] Bergmann-Schaefer. Lehrbuch der Experimentalphysik - Festkoerper.
Walter de Gruyter, 1992.
[5] C. Giacovazzo. Fundamentals of Crystallography. Oxford University
Press, 1992.
[6] H. Heuser. Funktionalanalysis. Teubner, 1992.
[7] N.E. Hurt. Phase Retrieval and Zero Crossing. Kluwer Academic Publishers, 1989.
[8] P.Lena I. Appenzeller, H.J. Habing. Evolution of Galaxies, Astronomical
Observations. Springer-Verlag, 1989.
[9] Y.Y. Zeevi J. Behar, M. Porat. Image reconstruction from localized
phase. IEEE Transactions on Signal Processing Vol.40, No.4, pages
736–743, 1992.
[10] B. Simon M. Reed. Methods of Modern Mathematical Physics. I: Functional Analysis. Academic Press, 1980.
[11] A.A. Pakhomov. Optimization of an image restoration algorithm from
its fourier spectrum phase. Optical Engineering, Vol.35 No. 4, pages
1044–1045, Apr. 1996.
[12] W. Rudin. Functional Analysis. McGraw-Hill, 1991.
79
[13] H. Stark. Image Recovery. Academic Press, 1987.
[14] S.V.W. Beckwith T.P. Ray. Star Formation and Techniques in Infrared
and mm-Wave Astronomy. Springer-Verlag, 1994.
[15] A.R. Weeks W.A. Rabadi, H.R. Myler. Iterative multiresolution algorithm for image reconstruction from the magnitude of its fourier transform. Optical Engineering, Vol. 35 No. 4, pages 1015–1024, Apr. 1996.
[16] D. Werner. Funktionalanalysis. Springer, 1995.
80