Large-Scale Methods in Inverse Problems
Per Christian Hansen
Informatics and Mathematical Modelling
Technical University of Denmark
With contributions from:
• Michael Jacobsen, Toke Koldborg Jensen
- PhD students
• Line H. Clemmensen, Iben Kraglund, Kristine Horn,
Jesper Pedersen, Marie-Louise H. Rasmussen
- Master students
Large-Scale Methods in Inverse Problems
1
Overview of Talk
A survey of numerical methods
for large-scale inverse problems
1.
2.
3.
4.
5.
6.
7.
Some examples.
The need for regularization algorithms.
Krylov subspace methods for large-scale problems.
Preconditioning for regularization problems.
Signal subspaces and (semi)norms.
GMRES as a regularization method.
Alternatives to spectral filtering.
Many details are skipped, to get the big picture!!!
Large-Scale Methods in Inverse Problems
2
Related Work
Many people work on similar problems and algorithms:
•
Åke Björck, Lars Eldén, Tommy Elfving
•
Martin Hanke, James G. Nagy, Robert Plemmons
•
Misha E. Kilmer, Dianne P. Oleary
•
Daniela Calvetti, Lothar Reichel, Brian Lewis
•
Gene H. Golub, Urs von Matt
•
Uri Asher, Eldad Haber, Douglas Oldenburg
•
Jerry Eriksson, Mårten Gullikson, Per-Åke Wedin
•
Marielba Rojas, Trond Steihaug
•
Tony Chan, Stanley Osher, Curtis R. Vogel
•
Jesse Barlow, Raymond Chan, Michael Ng
Recent Matlab software packages:
•
•
•
Restore Tools (Nagy, Palmer, Perrone, 2004)
MOORe Tools (Jacobsen, 2004)
GeoTools (Pedersen, 2005)
Large-Scale Methods in Inverse Problems
3
Inverse Geomagnetic Problems
Large-Scale Methods in Inverse Problems
4
Inverse Acoustic Problems
Oticon/
Rhinometrics
Large-Scale Methods in Inverse Problems
5
Image Restoration Problems
blurring
deblurring
Io (moon of Saturn)
You cannot depend on your eyes when
your imagination is out of focus
– Mark Twain
Large-Scale Methods in Inverse Problems
6
Model Problem and Discretization
Vertical component of
magnetic field from a dipole
Large-Scale Methods in Inverse Problems
7
The Need for Regularization
Regularization:
keep the “good” SVD components
and discard the noisy ones!
Large-Scale Methods in Inverse Problems
8
Regularization – TSVD & Tikhonov
Large-Scale Methods in Inverse Problems
9
Singular Vectors (Always) Oscillate
Large-Scale Methods in Inverse Problems
10
Large-Scale Aspects (the easy case)
Large-Scale Methods in Inverse Problems
11
Large-Scale Aspects (the real problems)
Toeplitz matrix-vector
multiplication flop count.
Large-Scale Methods in Inverse Problems
12
Large-Scale Tikhonov Regularization
Large-Scale Methods in Inverse Problems
13
Difficulties and Remedies I
Large-Scale Methods in Inverse Problems
14
Difficulties and Remedies II
Large-Scale Methods in Inverse Problems
15
The Art of Preconditioning
Large-Scale Methods in Inverse Problems
16
Explicit Subspace Preconditiong
Large-Scale Methods in Inverse Problems
17
Krylov Signal Subspaces
Smiley Crater, Mars
Large-Scale Methods in Inverse Problems
18
Pros and Cons of Regularizing Iterations
Large-Scale Methods in Inverse Problems
19
Projection, then Regularization
Large-Scale Methods in Inverse Problems
20
Bounds on “Everything”
Large-Scale Methods in Inverse Problems
21
A Dilemma With Projection + Regular.
Large-Scale Methods in Inverse Problems
22
Better Basis Vectors!
Large-Scale Methods in Inverse Problems
23
Considerations in 2D
…
…
Large-Scale Methods in Inverse Problems
24
Good Seminorms for 2D Problems
Large-Scale Methods in Inverse Problems
25
Seminorms and Regularizing Iterations
Large-Scale Methods in Inverse Problems
26
Krylov Implementation
Large-Scale Methods in Inverse Problems
27
Avoiding the Transpose: GMRES
Large-Scale Methods in Inverse Problems
28
GMRES and CGLS Basis Vectors
Large-Scale Methods in Inverse Problems
29
CGLS and GMRES Solutions
Large-Scale Methods in Inverse Problems
30
The “Freckles’’
DCT spectrum
spatial domain
Large-Scale Methods in Inverse Problems
31
Preconditioning for GMRES
Large-Scale Methods in Inverse Problems
32
A New and Better Approach
Large-Scale Methods in Inverse Problems
33
(P)CGLS and (P)GMRES
Large-Scale Methods in Inverse Problems
34
Io (moon of Saturn)
Away From 2-Norms
q=2
Large-Scale Methods in Inverse Problems
q = 1.1
35
Functionals Defined on Sols. to DIP
Large-Scale Methods in Inverse Problems
36
Large-Scale Algorithm MLFIP
Large-Scale Methods in Inverse Problems
37
Confidence Invervals with MLFIP
Large-Scale Methods in Inverse Problems
38
Many Topics Not Covered …
• Algorithms for other norms (p and q ≠ 2).
• In particular, total variation (TV).
• Nonnegativity constraints.
• General linear inequality constraints.
• Compression of dense coefficient matrix A.
• Color images (and color TV).
• Implementation aspects and software.
• The choice the of regularization parameter.
Large-Scale Methods in Inverse Problems
39
“Conclusions and Further Work”
I hesitate to give any conclusion –
•
the work is ongoing;
•
•
•
there are many open problems,
lots of challenges (mathematical and numerical),
and a multitude of practical problems waiting to be solved.
Large-Scale Methods in Inverse Problems
40