E. Somersalo
Truncated SVD and Deconvolution
Let f : [0, 1] → R be a signal to be estimated from noisy samples of the
convolution integral,
Z 1
g(s) =
a(s − t)f (t)dt + e(s),
0
where a is a known convolution kernel. Discretization:
1 X
yj = g(sj ) ≈
a(sj − tk )f (tk ) + e(sj ).
N
k
Denote xk = f (tk ), k = 1, 2, . . . , N .
Computational Methods in Inverse Problems, Mat–1.3626
0-0
E. Somersalo
Example: Optical blur
yj
yj+n
a0
ak
x
x
j−k j
xj+n
Computational Methods in Inverse Problems, Mat–1.3626
0-1
E. Somersalo
Matrix model



x=

x1
x2
..
.



,




y=

xN
Symmetric kernel:

a−L
a−L+1
..
.



a=

 aL−1
aL
y1
y2
..
.



.

yN




 ∈ R2L+1 .


Computational Methods in Inverse Problems, Mat–1.3626
0-2
E. Somersalo
Matrix model
Write the matrix equation
y = Ax + e,
where A ∈ RN ×N is the Toeplitz matrix,

a0 a−1 · · · a−L

 a1 a0

 ..
..
 .
.
A=

..
 aL
.


..

.
aL · · ·

..
.
a−L
..
.
a0
a1
a−1
a0





.





The parameter L defines the bandwidth of the matrix.
Computational Methods in Inverse Problems, Mat–1.3626
0-3
E. Somersalo
Defining the blurring kernel an true signal
Gaussian blurring kernel,
µ
¶
1
1 2
√
a(t) =
exp − 2 t .
2
2w
2πw
The true signal is a boxcar function,
½
0, if j < n1 or j > n2 ,
xj =
1, if n1 ≤ j ≤ n2 .
Computational Methods in Inverse Problems, Mat–1.3626
0-4
E. Somersalo
In Matlab
N = 60;
t = linspace(0,1,N);
width = 0.1;
a = 1/sqrt(2*pi*width^2)*exp(-(1/(2*width^2))*t.^2);
A = (1/N)*toeplitz(a);
n1 = 25;
n2 = 40;
xtrue = zeros(N,1);
xtrue(n1:n2) = ones(n2-n1+1,1);
Computational Methods in Inverse Problems, Mat–1.3626
0-5
E. Somersalo
Adding noise
Gaussian additive noise, the standard deviation (STD) 2% of the maximum
of the noiseless signal:
b0 = A*xtrue;
noiselevel = 0.02*max(b0);
noise = noiselevel*randn(N,1);
b = b0 + noise;
Computational Methods in Inverse Problems, Mat–1.3626
0-6
E. Somersalo
Plot noisy and noiseless signal
1.2
1
0.8
0.6
0.4
0.2
0
−0.2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
plot(t,xtrue,’b-’,’LineWidth’,1.2);
hold on
plot(t,b,’r-’,’LineWidth’,1.2);
hold off
Computational Methods in Inverse Problems, Mat–1.3626
0-7
E. Somersalo
SVD: plot singular values
A = U DV T ,
where
D = diag(d1 , . . . , dn ).
Machine epsilon: smallest non-negative number that the machine recognizes to
be non-zero. Below that level, values are cluttered under the roundoff errors.
In Matlab eps = 2.2204e-016.
Singular values below eps can be treated as zeros.
Computational Methods in Inverse Problems, Mat–1.3626
0-8
E. Somersalo
SVD in Matlab and logarithmic plot
[U,D,V] = svd(A);
d = diag(D);
r = max(find(d>eps));
semilogy(d,’b.’,’MarkerSize’,8);
hold on
semilogy([0,N],[eps,eps],’r-’);
text(r+2,1e-14,[’r = ’ ,num2str(r)]);
hold off
Computational Methods in Inverse Problems, Mat–1.3626
0-9
E. Somersalo
0
10
−5
10
−10
10
r = 34
−15
10
−20
10
0
10
20
30
40
50
60
Computational Methods in Inverse Problems, Mat–1.3626
0-10
E. Somersalo
Calculating TSVD(k)–estimates
x
b(k)
k
X
1 T
=
(uj b)vj .
d
j=1 j
Xk = zeros(N,r);
normX = zeros(r,1);
discr = zeros(r,1);
for k = 1:r
Xk(:,k) = V(:,1:k)*diag(1./d(1:k))*U(:,1:k)’*b;
normX(k) = norm(Xk(:,k));
discr(k) = norm(b - A*Xk(:,k));
end
Computational Methods in Inverse Problems, Mat–1.3626
0-11
E. Somersalo
Plotting the discrepancy curve
Estimate of the noise level: Here, we set
δ = 1.2 kek.
Notice, that in reality, kek is not known and has to be estimated.
plot([1:r],discr,’b.’,’MarkerSize’,8);
hold on
plot([1:r],discr,’k-’,’LineWidth’,0.8)
plot([0,r],[delta,delta],’r-’)
hold off
Computational Methods in Inverse Problems, Mat–1.3626
0-12
E. Somersalo
Plotting the discrepancy curve
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
0
5
10
15
20
25
30
35
Optimal value seems to be k = 6 or k = 7.
Computational Methods in Inverse Problems, Mat–1.3626
0-13
E. Somersalo
Plotting the L–curve
Plot in log–log scale the points
(kb
x(k) k, kAb
x(k) − yk),
k = 1, 2, . . . , r.
loglog(normX,discr,’b.’,’MarkerSize’,8);
hold on
loglog(normX,discr,’k-’,’LineWidth’,0.8)
hold off
Computational Methods in Inverse Problems, Mat–1.3626
0-14
E. Somersalo
Plotting the L–curve
1
10
0
10
−1
10
−2
10
0
10
5
10
10
10
15
10
Optimal k again around 5–7.
Computational Methods in Inverse Problems, Mat–1.3626
0-15
E. Somersalo
Plotting the solutions
1.8
1.2
1.6
1
1.4
0.8
1.2
1
0.6
0.8
0.4
0.6
0.2
0.4
0
0.2
0
−0.2
0
5
10
15
20
25
30
35
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Solutions corresponding to values
k = 5, 6, 7.
Computational Methods in Inverse Problems, Mat–1.3626
0-16
E. Somersalo
Is it important to cut off the singular values?
18
16
14
12
10
8
6
4
2
0
0
10
20
30
40
50
60
Discrepancy curve with all singular values retained
Computational Methods in Inverse Problems, Mat–1.3626
0-17
E. Somersalo
...Maybe
2
10
1
10
0
10
−1
10
−2
10
0
10
5
10
10
10
10
15
20
10
L–curve with all singular values retained.
Computational Methods in Inverse Problems, Mat–1.3626
0-18