Compressed Sensing
• Compressible/k-sparse signals
• Stable measurements
• Restricted isometric property
• Signal reconstruction algorithms
• Geometric interpretation
• Constrained/Unconstrained optimization
• Fast algorithms
IP, José Bioucas Dias, IST, 2007
1
K-sparse signals
Definition of k-sparse signals:
is k-sparse if only k coefficients
IP, José Bioucas Dias, IST, 2007
are nonzero
2
Compressible signals
There exists a basis where the representation
has just a few large coefficients and many small
coefficients.
Compressible signals are well approximated by k-sparse
representations
Many natural and man-made signals are compressible
IP, José Bioucas Dias, IST, 2007
3
Example: wavelet representation
4
10
2
10
0
10
-2
10
0
1
2
3
4
5
6
4
x 10
IP, José Bioucas Dias, IST, 2007
4
Example: wavelet representation (cont.)
IP, José Bioucas Dias, IST, 2007
5
Classical transform coding of compressive signals
1. Acquire the full n-sample signal
2. Compute the set of n coefficients
3. Retain the k << n largest coefficients and encode
their locations and values
Shortcommings:
1. Acquire n samples and only k << n coefficients are retained
2. The encoder computes n coefficients
3. The locations, besides the values of the largest k coefficients, have to be
encoded
IP, José Bioucas Dias, IST, 2007
6
Goal of compressive sampling
Measurement matrix
Goal of CS
Design a measurement matrix and a reconstruction algorithm for k-sparse
and compressible signals such that
is of the order of
IP, José Bioucas Dias, IST, 2007
7
Stable measurement matrix
The measurement processes must not erase the information present in the k
non-zero entries of signal
Restricted isometric property (RIP)
For any k-sparse vector
there holds
for some ε > 0
Any
submatrix of
IP, José Bioucas Dias, IST, 2007
is well-conditioned
8
Another way to look at stable measurements
The measurement matrix must be incoherent with the sparsifying basis
in the sense that the vectors
cannot sparsely represent the vectors
and
vice versa.
Stable (in the RIP sense) measurement matrices
random Gaussian
random binary
randomly selected Fourier samples (extra log factors apply)
IP, José Bioucas Dias, IST, 2007
9
Signal reconstruction algorithms
Minimum
-norm reconstruction
NP- complete problem !!!!
(Convex approximation) Replace
Basis pursuit problem (
with
complexity)
Surprise: if the measurement matrix is Gaussian i.i.d, then
a k-sparse vector f is exactly recovered by solving the
optimization problem with
observations
IP, José Bioucas Dias, IST, 2007
10
Geometrical interpretation
k-sparse vectors
IP, José Bioucas Dias, IST, 2007
11
Unconstrained optimization
Minimum
-norm reconstruction
Equivalent
Basis denoising algorithm
Algorithms
• l1_ls
[Kim et al, 2007]
• GPRS [Figueiredo et al, 2007]
• TwIST [Bioucas-Dias & Figueiredo , 2007]
IP, José Bioucas Dias, IST, 2007
12
Example: Sparse reconstruction
is i.i.d. Gaussian
20
40
60
80
100
120
100
200
300
400
500
600
700
800
900
1000
Observed data - g
Original data - f
0.15
1
0.8
0.1
0.6
0.4
0.05
0.2
0
-0.2
0
-0.4
-0.6
-0.05
-0.8
-1
0
200
400
600
IP, José Bioucas Dias, IST, 2007
800
1000
1200
-0.1
0
20
40
60
80
100
120
140
13
Example: Sparse reconstruction
regularization
Pseudo-inverse
1
1
f
f
fpseudo-inv
0.8
0.8
α
0.6
0.6
0.4
0.4
0.2
0.2
0
0
-0.2
-0.2
-0.4
-0.4
-0.6
-0.6
-0.8
-0.8
-1
-1
0
200
400
600
IP, José Bioucas Dias, IST, 2007
800
1000
f
1200
0
200
400
600
800
1000
1200
14
Example: Total variation reconstruction
randomly selected Fourier samples (9%)
IP, José Bioucas Dias, IST, 2007
15
IP, José Bioucas Dias, IST, 2007
16
References
E. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal
Reconstruction from highly incomplete frequency information,” IEEE Trans. Inform.
Theory, vol. 52, no. 2, pp. 489–509, Feb. 2006.
D. Donoho, “Compressed sensing,” IEEE Trans. Inform. Theory, vol. 52, no.
4, pp. 1289– 1306, April 2006.
R. Baraniuk, “Compressed sensing”, IEEE Signal Processing Magazine, 2007
J. Bioucas-Dias, M. Figueiredo, “A new TwIST: two-step iterative
shrinkage/thresholding algorithms for image restoration”, IEEE Transactions on
Image Processing, December 2007.
M. Figueiredo, R. Nowak, S. Wright, “Gradient projection for sparse
reconstruction: application to compressed sensing and other inverse problems”,
IEEE Journal of Selected Topics in Signal Processing: Special Issue on Convex
Optimization Methods for Signal Processing, vol. 1, no. 4, pp. 586-598, 2007.
S Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “A method for
large-scale l1-regularized least squares”, IEEE Journal on Selected Topics
in Signal Processing, vol. 4, no. 1, pp. 606-617, 2007.
IP, José Bioucas Dias, IST, 2007
17