A User s Guide to Compressed
Sensing for Communications Systems
Kazunori Hayashi
Graduate School of Informatics
Kyoto University
[email protected]
Wireless Signal Processing & Networking Workshop: Emerging Wireless Technologies
2
Survey paper on compressed sensing:
K. Hayashi, M. Nagahara and T. Tanaka,
A User s Guide to Compressed Sensing for
Communications Systems,
IEICE Trans. Commun. , Vol. E96-B, No.3, pp.
685-712, Mar. 2013.
3
Agenda
•  A Brief Introduction to Compressed Sensing
•  Warming-up
•  Linear Equations Review
•  Problem Setting
•  Algorithms
•  Applications to Communications Systems
•  Channel Estimation
•  Wireless Sensor Network
•  Network Tomography
•  Spectrum Sensing
•  RFID
4
A Brief Introduction to
Compressed Sensing
5
Warming-up Exercise(1)
-Linear simultaneous equations:
x+y+z =3
x−z =0
2x − y + z = 2
x=1
y=1
z=1
（# of Eqs. = # of unknowns）
Unique solution
6
Warming-up Exercise(2)
-Linear simultaneous equations:
x+y+z =3
x−z =0
2x − y + z = 2
x+y =1
?
（# of Eqs. > # of unknowns）
No solution
7
Warming-up Exercise(3)
-Linear simultaneous equations:
x+y+z
x−z
2x − y + z
x+y
x=1
y=1
z=1
=3
=0
=2
=2
（# of Eqs. > # of unknowns）
Unique solution
8
Warming-up Exercise(4)
-Linear simultaneous equations:
x+y+z =3
x−z =0
（# of Eqs. < # of unknowns）
x=t
y = −2t + 3
z=t
Infinitely many solutions
9
Linear Equations in Engineering Problems
y = Ax
A ∈ Rm×n
Unknown vector：
x ∈ Rn
m
y
∈
R
Measurement vector：
Sensing matrix：
-Answering No solution or Non-unique
solutions is not enough
-Measurement may include noise
y = Ax + v
noise
10
Linear Equations Review:
Conventional approach ( n = m , noise-free)
y = Ax
m×n
A
∈
R
Sensing matrix：
n
x
∈
R
Unknown vector：
m
y
∈
R
Measurement vector：
-Unique Solution (if A
is not singular):
x = A−1 y
Corresponds to warming-up (1)
11
Linear Equations Review:
Conventional approach ( n < m , noise-free)
y = Ax
m×n
A
∈
R
Sensing matrix：
n
x
∈
R
Unknown vector：
m
y
∈
R
Measurement vector：
A is of full column rank):
-Unique Solution (if x = (AT A)−1 AT y
Corresponds to warming-up (3)
12
Linear Equations Review:
Conventional approach ( n > m , noise-free)
y = Ax
m×n
A
∈
R
Sensing matrix：
n
x
∈
R
Unknown vector：
m
y
∈
R
Measurement vector：
-Non-unique solutions
Select x , which has minimum 2 -norm
-Minimum-norm solution：
x̂MN = arg min ||x||22
x
subject to Ax = y
= AT (AAT )−1 y
Corresponds to warming-up (4)
Linear Equations Review:
Conventional approach ( n < m , noisy)
y = Ax
13
m×n
A
∈
R
Sensing matrix：
n
x
∈
R
Unknown vector：
m
y
∈
R
Measurement vector：
-No solution in general
（ y is not included in Img. of A due to noise）
Select x , which minimize squared error
between Ax and y , as a solution
-Least-square (LS) solution：
x̂LS = arg min ||Ax − y||22 = (AT A)−1 AT y
x
Corresponds to warming-up (2)
Linear Equations Review:
Conventional approach ( n > m , noisy)
y = Ax
14
m×n
A
∈
R
Sensing matrix：
n
x
∈
R
Unknown vector：
m
y
∈
R
Measurement vector：
-Non-unique solutions
-Regularized LS solution:
x̂rLS = arg min ||Ax −
x
y||22
+
λ||x||22
= (λI + AT A)−1 AT y
λ：regularization parameter
15
Problem Setting of Compressed Sensing
n > m with a prior
Linear equations of information that the true solution is sparse
y = Ax
m×n
R
Rnm
Sensing matrix (non-adaptive): A ∈
x∈
Unknown vector：
y∈R
Measurement vector：
Underdetermined:
Sparse unknown vector:
n>m
||x||0 n
||x||0 : # of nonzero elements of x
16
Signal Recovery via 0 Optimization -Natural and straightforward approach:
x̂0 = arg min ||x||0
x
subject to Ax = y
> ||x||0 , almost always true solution
- if m
is obtained
- NP hard in general
17
Signal Recovery via 1 Optimization n
|xi |
Replace ||x||0 with ||x||1 =
i=1
x̂1 = arg min ||x||1
x
subject to Ax = y
-It can be posed as linear programing (LP) problem
-True solution can be obtained even when n > m
under some conditions
If A satisfies RIP (restricted isometry property),
only O(k log(n/k)) measurement will be enough for
exact recovery
Intuitive Illustrations of Sparse Signal
Recovery
1 recovery：
x̂1 = arg min ||x||1
x
2
MN solution： x̂MN = arg min ||x||2
x
x̂
0
y = Ax
subject to Ax = y
subject to Ax = y
x̂
0
18
y = Ax
19
Reconstruction with Noisy
Measurement（1/2）
1 reconstruction:
Constrained x̂1 = arg min ||x||1
x
subject to ||Ax − y||2 ≤ ( > 0)
-  Natural constraint for bounded noise
-  A certain guarantee of signal recovery can
be given for Gaussian noise
Reconstruction with Noisy
Measurement（2/2）
1 -2 optimization：
1
x̂1 -2 = arg min
||Ax − y||22 + λ||x||1
x
2
- Equivalent to constrained 1 reconstruction
for a certain choice of λ
-  Analogy to regularized
LS problem:
x̂rLS = arg min ||Ax − y||22 + λ||x||22
x
- LARS (least angle regression) algorithm
20
21
Equivalent approaches to 1 -2 optimization
-Lasso (Least absolute shrinkage and
selection operator)：
x̂Lasso = arg min ||Ax − y||22 subject to ||x||1 ≤ t
x
(t > 0)
-Maximum a posteriori (MAP) estimator for
AWGN with Laplacian prior ∝ exp(−λ||x||1 )
22
Algorithms for Compressed Sensing
Sample program (Scilab):
http://www.msys.sys.i.kyoto-u.ac.jp/csdemo.html
23
Applications to
Communications Systems
24
Key issues in applications
•  Sparsity of signals of interest
  Is it natural to assume the sparsity of them?
  In which domain?
•  Linear measurement of the signals
  Can we formulate with linear equation?
  Is it happy to reduce the number of equations?
25
Channel Estimation
Multipath channel
Receiver
Transmitter
Noise
Tx. signal
sn
Convolution channel
h0 , . . . , hM
yn =
M
vn
Rx. signal
yn
hm sn−m + vn
m=0
Estimation of channel impulse response using Rx. signals
26
Channel Estimation
-Received signal vector： y = [y1 , . . . , yP −M ]T
= Hs + v
Channel matrix：
⎡
hM
⎢
⎢ 0
H=⎢
⎢ .
⎣ ..
0
...
hM
..
.
...
h0
0
..
h0
0
.
hM
...
..
.
..
.
...
⎤
0
.. ⎥
.⎥
⎥
⎥
0⎦
h0
(P − M ) × P
Known pilot signal：
s = [s1 , . . . , sP ]T ∈ RP
T
v
=
[v
,
.
.
.
,
v
]
White noise：
1
P −M
27
Channel Estimation
-Received signal vector (rewritten)：
y = Sh + v
⎡
sM +1
⎢sM +2
⎢
S=⎢ .
⎣ ..
sP
sM
...
...
s1
s2
..
.
sP −1
...
sP −M
sM +1
..
.
⎤
⎥
⎥
⎥ (P − M ) × (M + 1)
⎦
h = [h0 , . . . , hM ]T ∈ RM +1
-Conventional approaches：
T
−1 T
-LS solution (P − M ≥ M + 1) ： ĥ = (S S) S y
-MN solution (P − M < M + 1)： ĥ = ST (SST )−1 y
28
Channel Estimation
-Sparsity of wireless channel impulse response:
Wireless channel is discrete in nature, and the
feature appears as the bandwidth increases．
-Estimation via compressed sensing：
ĥ = arg min ||h||1
h
subject to ||Sh − y||2 ≤ S：Sensing matrix（Random, Toeplitz）
h：Unknown sparse channel impulse response
Merit: Reduction of pilot signals
(Improvement of frequency efficiency)
29
Wireless Sensor Network (Single-Hop)
-Tx. signal@time @
i j -th node:
Ai,j sj
sj
Ai,j
:measurement data
:random coefficients
(generated from node id)
-Rx. signal@time i @ fusion center
yi =
n
Ai,j sj + vi
j=1
y = [y1 , . . . , ym ]T
= As + v
30
Wireless Sensor Network (Single-Hop)
-Sparsity of measurement data：
s = Φx
（ :
Φ invertible transformation matrix, typically
denoting DFT, DCT, wavelet, etc.）
s might be sparse in a certain representation with basis Φ
（due to spatial correlation of data）
-Data recovery via compressed sensing：
x̂ = arg min ||x||1 subject to ||AΦx − y||2 ≤ x
AΦ：sensing matrix
x ：unknown sparse vector
Merit: Reduction of data collection time
31
Wireless Sensor Network (Multi-Hop)
-Rx. signal@sink node in the i -th
multi-hop
communication：
yi =
Ai,j sj
j∈Pi
Pi : index set of nodes included in
the
i -th multi-pop communication
Ai,j : random coefficient of the j -th node
i
in the -th multi-hop communication
/ Pi )
( Ai,j = 0 if j ∈
y = AΦx
Sensing matrix depends on routing algorithm
Merit: Reduction of # of communications
(power consumption)
32
Network Tomography
Router
Link
Path
Terminal
(destination)
Terminal
(source)
Network
-Link-level parameter estimation based on end-end
measurements
-End-End performance evaluation based on link-level
measurements
Delay Tomography
(Link Delay Estimation)
-delay @ link ej：xj
-total delay in the i-th path:
yi =
Ai,j xj
j∈Pi
Pi : set of link index included in the i -th path
1 j ∈ Pi
Ai,j =
0 otherwise
y = Ax
A : Routing Matrix
33
Delay Tomography
(Link Delay Estimation)
34
-Sparsity of link delay:
Only limited number of links have large delay, due to
node failure or some other reasons in typical network
-Estimation via compressed sensing:
x̂ = arg min ||x||1 subject to Ax = y
x
A：sensing matrix (routing matrix)
x ：(approximately) sparse delay vector
Merit: Reduction of # of prove packets
35
Loss Tomography
(Link Loss Probability Estimation)
-packet loss probability @ link ej : pj
-packet loss probability @ i -th path: qi
-packet success probability @ i -th path: 1 − qi =
⎛
− log(1 − qi ) = − log ⎝
=−
(1 − pj )
j∈Pi
⎞
(1 − pj )⎠
j∈Pi
log(1 − pj )
j∈Pi
yi =
j∈Pi
yi = − log(1 − qi )
xj = − log(1 − pj )
Ai,j xj
y = Ax
36
Spectrum Sensing
-Rx. signal (continuous time):
r(t)
t ∈ [0, nTs ]
Ts：inverse of Nyquist rate
-Rx. signal with Nyquist rate sampling：
rt = [r(Ts ), . . . , r(nTs )]T
-Rx. signal with (discrete time) linear measurement：
yt = Ψrt
Ψ ：m × n sensing matrix
m < n 㱺 sub-Nyquist rate sampling）
( 37
Spectrum Sensing
-Sparsity of occupied spectrum：
Primary system uses
rf = Drt
D：DFT matrix
-Spectrum sensing via compressed sensing：
yt = ΨDH rf
ΨDH：Sensing matrix
rf ：Sparse spectrum vector
Merit: Reduction of sampling rate
38
Spectrum Sensing
-Edge spectrum
zs = ΓDΦs rt
Φs: ⎡smoothing function⎤
1
Sparsity of spectrum edge
0
... ... 0
⎢
.. ⎥
⎢−1 1 . . .
⎥
.
⎢
⎥
⎢
⎥
Γ = ⎢ 0 . . . . . . . . . ... ⎥
⎢
⎥
⎢ .
⎥
.
.
.
.
.
.
.
⎣ .
.
.
. 0⎦
0 . . . 0 −1 1
yt = Ψ(ΓDΦs )−1 zs
39
RFID
Reader
Tag
-Each tag sends its own ID by reflecting wireless
signal from reader
-Conventionally, collision avoiding protocols are used
40
RFID Tag Detection
-ID of tag j :
aj (j = 1, . . . , n)
-Received signal @ reader：
y=
aj
j∈P
= [a1 , . . . , an ]x
= Ax
P ：Index set of tags
in reader s range
1
xj =
0
j∈P
otherwise
41
RFID Tag Detection
-Sparsity of RFID tags：
Number of tags in the reader s range is much
smaller than that of whole ID space
(similar to CS-based MUD in the previous talk)
-Tag detection via compressed sensing：
x̂ = arg min ||x||1 subject to Ax = y
x
A：Sensing matrix (ID matrix)
x ：Unknown sparse vector
Merit: Reduction of detection time