Tamkang Journal of Science and Engineering, Vol. 14, No. 3, pp. 217-224 (2011)
217
Signal Sparse Representation Based on the Peak
Transform and Modulus Maximum of
Wavelet Coefficients
Yi-Gang Cen1,2, Rui Xin1,2, Xiao-Fang Chen3*, Li-Hui Cen3,
Ming-Xing Hu4 and Shi-Ming Chen5
1
Institute of Information Science, Beijing Jiaotong University,
Beijing, 100044, P.R. China
2
Beijing Key Laboratory of Advanced Information Science and Network Technology,
Beijing 100044, P.R. China
3
School of Information Science and Engineering, Central South University,
Hunan, Changsha 410083, P.R. China
4
Centre for Medical Image Computing, University College London, UK
5
School of Electrical and Electronic Engineering, East China Jiao Tong University,
Jiangxi, Nanchang 330013, P.R. China
Abstract
It is now well-known that one can reconstruct sparse or compressible signals accurately from a
very limited number of measurements. This technique is known as “compressed sensing” or
“compressive sampling” (CS). A basic requirement of CS is that a signal should be sparse or it can be
sparsely represented in some orthogonal bases. Based on the Peak Transform (PT) and modulus
maximum of wavelet coefficients, a new algorithm was proposed for the signals that are non-sparse
themselves and can not be sparsely represented by wavelet transform such as the Linear Frequency
Modulated signal. According to this algorithm, K-sparse wavelet coefficients can be obtained. For
the peak sequence produced by the Peak Transform, value expansion approach of reversible
watermarking is exploited such that the peak sequence can be embedded into the measurements of the
signal, which avoids increasing additional points for transmission. By using the Peak Transform and
modulus maximum, non-sparse wavelet coefficients can be transformed into K-sparse coefficients,
which improves the reconstruction result of CS. Simulation results showed that our proposed
algorithm achieved better performance comparing with the original CS algorithm.
Key Words: Compressed Sensing, Sparse Representation, Peak Transform, Modulus Maximum,
Wavelet Transform
1. Introduction
Conventional approaches to sampling signals or images follow Shannon’s celebrated theorem: the sampling
rate must be at least twice the maximum frequency present in the signal (the so-called Nyquist rate). In the field
of data conversion, for example, standard analog-to*Corresponding author. E-mail: [email protected]
digital converter (ADC) technology implements the usual
quantized Shannon representation: the signal is uniformly sampled at or above the Nyquist rate. In the recent years, a new theory known as Compressed Sensing
or Compressive Sampling [1,2] has been proposed, which
is a novel sensing/sampling paradigm that goes against
the common wisdom in data acquisition. CS theory asserts that one can recover certain signals and images
from far fewer samples or measurements than traditional
218
Yi-Gang Cen et al.
methods use, which significantly reduces the cost for
sampling, processing, transmission and storage of signals.
By [3], if a signal f is sparse or compressible in the
sense that they have concise representations when expressed in a proper basis Y, then by using few measurements, the signal can be recovered with high accuracy. In
[4], seven types of reconstruction problem and the corresponding solutions were given. It can be seen that a most
important premise of CS is that the signal can be sparsely
represented. In this paper, a novel approach for the Ksparse representation is proposed. Then the interior point
method is employed for solving the Min-l1 with equality
constraints problem as that presented in [4] for the reconstruction of signals. The method of Min-l1 with equality
constraints is very powerful for the reconstruction of
sparse signals. But in general, we almost can not obtain
strict K-sparse coefficients by the wavelet transform for
the real signals, which will lead to large errors for the recovered signals.
Peak Transform (PT) [5] is a nonlinear geometric
transform that can transfer the high frequency components of a signal into the low frequency components.
This property has been successfully applied in the image
compression [5]. Combining PT and wavelet transform,
the sparsity of wavelet coefficients can be greatly increased. Moreover, modulus maximum can be utilized to
obtain sparser wavelet coefficients. Thus, an algorithm
based on the PT and modulus maximum of wavelet coefficients is proposed in this paper such that K-sparse
wavelet coefficients can be obtained. Especially for the
signals that are non-sparse and can not be sparsely represented by the wavelet bases, our proposed algorithm
achieves much better performance for the reconstruction
signals.
2. Compressed Sensing and Peak Transform
2.1 Compressed Sensing
Suppose that x Î RN´1 is a one dimension signal and
it can be expanded by an orthogonal basis (such as a
wavelet basis) Y = {y1, ¼, yN}, i.e.,
(1)
where, yi = < x, yi >, Y Î CN´N and YYH = YHY = IN (the
N ´ N identity matrix). If there are only K (K << N) non-
zero coefficients in y, then Y is called as the sparse basis of x.
In general, most signals are not sparse themselves.
But their wavelet coefficients can be considered to be
sparse or compressible. For example, one can consider
only keeping the largest K coefficients and discarding
the N - K small coefficients without much perceptual
loss. Then the coefficient vector is sparse in a strict sense
since all but a few of its entries are zero. We will call
K-sparse for such objects with at most K nonzero entries.
Multiplying the signal x by an M ´ N sensing matrix
F = {f1, ¼, fM} to obtain M measurements
s = Fx
(2)
Then the signal can be recovered by the M measurements and F. By substituting (1) into (2), we have
s = FYy = Qy
(3)
Here Q = FY is an M ´ N matrix. According to the above
process, the N dimension signal x is reduced into an M
dimension signal s. Since the unknown variable number
is greater than the number of the equations, (2) has not a
determinate solution. But the y in (3) is K-sparse, i.e.,
there are only K nonzero components in y and K < M £ N.
Thus, y can be reconstructed from s by solving the convex optimization problem. Finally, x can be recovered
by (1). Reference [6] pointed out that Q should obey the
Restricted Isometry Property (RIP), i.e.,
(4)
where d > 0. An equivalent condition of RIP is that F
and Y satisfy the incoherence requirement [6,7]. In order to reconstruct the signal x, we need to solve the following convex optimization problem
(5)
This is in general difficult to implement because it is an
NP-hard problem in combinatorial mathematics. But
this problem can be converted into solving [8]
(6)
Signal Sparse Representation Based on the Peak Transform and Modulus Maximum of Wavelet Coefficients
There were some iterative greedy algorithms for solving (6) such as the Primal-Dual Interior-Point Algorithm [4], Matching Pursuit (MP) algorithm [9], Orthogonal Matching Pursuit (OMP) algorithm [7], Gradient Projection for Sparse Reconstruction (GPSR) algorithm [10] etc.
219
3. Sparse Representation based on the PT and
Modulus Maximum
2.2 Peak Transform
Peak transform (PT) [5] is able to convert highfrequency signals into low-frequency ones. Coupled
with wavelet transform and subband decomposition, PT
is able to significantly reduce signal energy in highfrequency subbands and achieve a significant transform
coding gain. This has important applications in efficient
data representation and compression. In the following,
we will illustrate the PT by a simple example.
As shown in the Figure 1(a), suppose that the original signal is f (x). We select seven local minimum and
maximum points, i.e., x0, ¼, x6, which is called as peaks.
Then f (x) can be partitioned into six curve segments by
the peaks, denoted as f1(x), ¼, f6(x). We first cascade
all odd-numbered curve segments then all even-numbered curve segments and form a new curve, denoted as
PT{f (x) | xi}, as shown in the Figure 1(b).
The above transform process is called as the forward
PT. It can be seen that the PT only changes the order of
curve segments and is reversible. The backward transform can be done by simply recascading the curve segments according to their original order. The backward
PT operation is denoted by PT-1{f (x) | xi}.
3.1 Peak Transform for Signals
According to the theory of CS, a basic requirement
of CS is that the signal should be sparse itself or it can be
sparsely represented in an orthogonal basis such as the
wavelet bases. The wavelet decomposition layer should
be as large as possible. This is because that one can obtain a low-pass approximation sub-band and a high-pass
detail sub-band (both the low-pass and high-pass coefficient numbers are respectively only half of the original
signal’s point number) for one dimension signal by one
layer wavelet transform. The high-pass coefficients can
be considered as sparse or compressible. But the lowpass sub-band is an approximation of the original signal.
If the original signal is not sparse, then the low-pass coefficients must not be sparse. Thus, we need to decompose the signal as possible as we can so that there are
only several points in the maximal low-pass layer J.
Then they can be omitted in the future process and all the
wavelet coefficients can be considered sparse or compressible, which just meet the requirements of CS.
But for some signals such as the Linear Frequency
Modulation (LFM) signal, its wavelet coefficients are not
sparse. As shown in the Figure 2, a LFM signal is decomposed by the 4-layer wavelet transform. It can be seen that
the coefficients in each layer are not sparse. Thus, if we
simply apply the CS process on the wavelet coefficients of
LFM signal, the reconstruction error will be increased.
Figure 1. Peak transform.
Figure 2. LFM signal and its coefficients of 4 layers wavelet
decomposition.
220
Yi-Gang Cen et al.
According to section 2.2, a signal will be much
smoother after PT. The zero point (or the points with
very small magnitudes) number will dramatically increase, which means that the sparsity of the coefficients
is improved greatly. Figure 3 is a toy example. We can
see that after the PT, the nonzero high-pass wavelet coefficients of the PT output signal are significantly reduced.
Based on the above discussions, although the wavelet coefficients of some signals are not sparse, but we can
perform PT on the non-sparse signals and pass the PT
output into the wavelet filter banks. Then sparse wavelet
coefficients can be obtained. Experiment results show
that the energy of high-frequency sub-bands with PT is
only about 40% of that without PT.
3.2 Embedment of Peak Sequence into Measurements
In the forward PT, some points are selected as peaks
so that curve segments can be determined, i.e., we need
to know the truncated positions of the original signal. In
the backward PT, the peaks also need to be used. Thus,
the information about the peak positions need to be encoded with the measurements and transferred to the decoder. For a discrete signal f (x) Î RN, the peak sequence
is defined as follows.
the first N/q measurement points is left shifted for q
bits, respectively. Then the N bits of B{f (x)}k are embedded into the last q bits of these N/q measurements.
When the decoder receives the signal, it only needs to
pick up the last q bits from the first N/q measurements
to obtain the peak sequence and then right shift the first
N/q measurements for q bits to obtain the original measurements.
3.3 Modulus Maximum and Hard-Threshold for
Sparse Representation
In the practice, signals are more complex than the triangle wave shown in the Figure 3. The PT output signal
will not as smooth as that shown in Figure 3(c), which
leads to the fact that there are a lot of nonzero wavelet
coefficients with small magnitude. This deteriorates the
sparsity of the wavelet coefficients. In the wavelet theory, de-noising algorithm based on the modulus maximum has been successfully exploited. Here, it will be
used for the sparse representation of signals. We only
keep the modulus maximum of the wavelet coefficients
for sensing. Then, a hard-threshold t is used by
(8)
(7)
Since all elements in B{f (x)}k are either 0 or 1, B{f (x)}k
is a binary sequence with N bits. In order to reduce the
information for transmission, we can encode B{f (x)}k
by using some algorithms such as run-length coding,
Huffman coding and arithmetic coding etc. But these
encoding algorithms will greatly increase the computational cost for the encoder. For example, for a LFM signal with 256 points, its peak sequence also has 256 bits.
If the arithmetic coding is applied, the encoding result
still has 216 bits. But the run-time of arithmetic encoder
and decoder is almost the same as the CS algorithm.
Thus, we give up encoding the peak sequence by these
coding algorithms. For the sake of not increasing the
measurements number for transmission, value expansion approach of reversible watermarking is used directly such that B{f (x)}k can be embedded into the measurements of the signal, i.e., each of the binary values of
where Max | WPT{f (x) | xi}| denotes the modulus maximum of the wavelet coefficients. By the hard-threshold,
Figure 3. Wavelet coefficients of triangle wave and the coefficients of PT output.
Signal Sparse Representation Based on the Peak Transform and Modulus Maximum of Wavelet Coefficients
the sparseness of the modulus maximum sequence will
be significantly improved.
The selection of t is important, if t is too large, many
modulus maximum points are set to zeros. Although the
~
sequence Max WPT { f ( x ) x i is sparse enough so that CS
~
algorithm can recover Max WPT { f ( x ) x i accurately. The
error of the reconstructed signal will increase since a lot
of information of the signal was lost by thresholding. If t
is too small, the sparseness of the modulus maximum sequence will decrease. Then the CS algorithm may not re~
cover Max WPT { f ( x ) x i accurately. This also increases
the error of the reconstructed signal.
~
In the decoder, when Max WPT { f ( x ) x i has been recovered by a convex optimization algorithm, it needs to re~
construct the wavelet coefficients from Max WPT { f ( x ) x i .
Alternate Projection (AP) [11] gave a method to reconstruct the wavelet coefficients but with high computational cost. In [12], a Piecewise Cubic Spline Interpolation (PCSI) algorithm was proposed, which was simple
and fast. More important, the quality of the reconstructed
signal could be improved. Thus, we will use PCSI in the
decoder for the reconstruction of wavelet coefficients.
Finally, we will discuss the property of wavelet decomposition. It is well-known that the low-pass coefficients are the best approximation of the original signal in
the wavelet subspace. But in general, low-pass coefficients are not sparse. Thus, the error of the reconstructed
signal will increase if the CS algorithm is performed on
both the high-pass and low-pass coefficients. According
to the Mallat algorithm of wavelet transform, after onelayer wavelet decomposition, the low-pass coefficient
number is only half of the original one. Thus, in the jth
wavelet decomposition layer, the low-pass coefficient
number is only N / 2j, where N is the length of the original
signal. For example, if N = 1024, then the low-pass coefficients only have 16 points after 6 wavelet decompositions. In order to improve the quality of the reconstructed signal, we can transmit these 16 points to the decoder directly and only perform the CS algorithm on the
high-pass coefficients. When the decoder received the
low-pass coefficients and recovered the high-pass coefficients, the signal will be better reconstructed.
221
3.4 Compressive Sensing Algorithm Based on the
Peak Transform and Modulus Maximum
Based on the above discussion, the CS algorithm
based on the PT and modulus maximum proposed in this
paper is as follows.
Encoder
Step 1. Find the peaks of the original signal f (x) Î RN
and obtain B{f (x)}k. Perform PT on f (x), the output signal is denoted as PT{f (x) | xi}.
Step 2. Perform wavelet transform on PT{f (x) | xi}. The
wavelet coefficients are denoted as aJ, dJ, dJ-1,
¼, d1, where aJ is the low-pass coefficients in
the maximal decomposition layer, dj, 1 £ j £ J, is
the high-pass coefficients of the jth layer. Define
WPT{f (x)} = {dJ, dJ-1, ¼, d1}.
Step 3. Find out the modulus maximum points of
WPT{f (x)} and apply the threshold on them.
Only the largest K modulus maximum points are
kept and the resultant sequence is denoted as
~
Max WPT { f ( x ) x i .
Step 4. Construct the sensing random matrix FM´N, M <
N. The entries of F are drawn from the Gaussian
~
distribution (0,1/N). Then Max WPT { f ( x ) x i is
sensed by F and M measurements can be obtained. Finally, B{f (x)}k is embedded into the
first N/q measurements and transmitted to the
decoder together with the low-pass coefficients aJ.
Decoder
~
Step 1. Extract M measurements of Max WPT { f ( x ) x i ,
the low-pass coefficients aJ and the peak sequence B{f (x)}k from the received signal.
Step 2. Recover the modulus maximum sequence
~
Max r WPT { f ( x ) x i } by the convex optimization
algorithm.
~
Step 3. According to the Max r WPT { f ( x ) x i }, the highpass wavelet coefficients can be reconstructed
by the PCSI algorithm. Then the PT output signal can be recovered by the inverse wavelet
transform. The recovered PT output signal is de~
noted as WPT { f ( x )}.
222
Yi-Gang Cen et al.
Step 4. Perform the backward Peak Transform on
~
WPT { f ( x )} by using B{f (x)}k to obtain the re~
covered signal f ( x ).
4. Simulation Results
In order to evaluate the similarity of the original and
the reconstructed signals, the error between two vectors
is defined as follows.
(9)
~
Here, f is the original signal and f is the recon~
structed signal. The smaller the m( f , f ) is, the more accurate the reconstructed signal is.
We used the LFM signal with 512 points in the simulation. The initial normalized frequency and the final
normalized frequency are 0.1 and 0.3, respectively. The
magnitude was normalized to 1. The selection of wavelet
basis is very important. According to the properties of
wavelet functions, we should select wavelet basis with
orthogonality, compactly support and near symmetry. By
the experiments, we find that coif3~coif5, sym8, db4~
db10 bases achieve better reconstruction results. The
LFM signal was decomposed into 5 layers. Figure 4 illustrates the relations between the measurement number
and the reconstruction error for different threshold values t. Because of the randomness of the sensing matrix,
the reconstruction result of each simulation is different.
Thus in Figure 4, for each M, we conducted the experiment 100 independent trials and calculate the average reconstruction error. According to the figure, it can be seen
that if the threshold value t is too small (for example, t =
0.1), the wavelet coefficients after thresholding are not
sparse enough so that the reconstruction error is large.
But if the threshold value is too large (for example, t =
0.3), too many points with useful information were set to
zeroes. Thus, the error between the recovered signal and
the original signal can not been decreased significantly.
Moreover, in this case, when M > 320, the reconstruction
error will not decrease even the measurement number is
increased. This is because the threshold is too large, the
information contained in the coefficients is limited. After
experiments, we found that t = 0.2 is better in this simulation, as shown by the dotted line with diamond points
in Figure 4. When M is chosen between 250 and 350 and
t = 0.2, our proposed algorithm achieved better performance.
Figure 5 was a simulation result. The algorithm of
Min-l1 with equality constraints presented in [4] was
used for comparison. The wavelet decomposition layer
was J = 5. The wavelet function was chosen as the sym8.
There were total 496 high-pass coefficients in the 5 decomposition layers and 16 low-pass coefficients in the
5th decomposition layer. Let t = 0.2, after thresholding,
there were only 167 nonzero points in the modulus maximum sequence. M was chosen as 300. Thus, in our proposed algorithm, we need to transmit 300 measurements
and 16 low-pass coefficients together to the decoder. For
the decoder, Primal-Dual Interior-Point Algorithm described in [4] was used to recover the modulus maximum
sequence. The reconstruction result was shown in Figure
5(a) and the reconstruction error computed by (7) is m =
6.2551. In order to obtain a fair comparison, for the
simulation by using the CS algorithm presented in [4], M
was set to be 316. The reconstruction result was shown in
Figure 5(b) and the reconstruction error computed by (9)
is m = 7.1342.
In this simulation, 158 points were selected as peaks,
i.e., the corresponding values of these 158 points of
B{f (x)}k are 1. The size of the sensing matrix was 300 ´
512, then 300 measurements were obtained. Thus, each
of the binary values of the first 256 points was left shift
for 2 bits (q = 2). Then the 512 bits of B{f (x)}k could be
embedded into these 256 measurements sequentially for
transmission. This embedment method is non-adaptive
Figure 4. Reconstruction error comparison for different thresholds.
Signal Sparse Representation Based on the Peak Transform and Modulus Maximum of Wavelet Coefficients
223
Figure 5. Reconstruction results of our proposed algorithm and the CS algorithm described in [4]. (a) Comparison of the
recovered signal of our algorithm and the original signal, m = 6.2551. (b) Comparison of the recovered signal of the CS
algorithm described in [4] and the original signal, m = 7.1342.
since if the size of the sensing matrix is changed, the
points used for embedment also need to be changed. But
in the practice, the sensing matrix should be selected optimally in prior. Thus the measurements number M can
be determined. Then the points used for embedding can
also be determined according to the length of B{f (x)}k.
Figure 6 shows the performance of our proposed algorithm comparing with the algorithm of Min-l1 with
equality constraints presented in [4], which illustrated
the relations between the reconstruction errors and the
values of measurement number M. For each M we have
conducted the experiment 100 independent trials. By
Figure 6, the performance of our proposed algorithm is
better than the CS algorithm presented in [4]. Especially
when M is small, the reconstruction error computed by
(9) of our algorithm outperforms the algorithm presented
in [4] by up to 2.
5. Conclusion
Although the theory and applications of CS is very
attractive, there are a lot of problems need to be solved.
Except the sparse representation problem, the choosing
of the sensing matrix is also a hot issue. Since the random
matrices generated in each experiment are different, the
reconstruction performance will be greatly influenced.
Figure 6. Average reconstruction error of our proposed algorithm and the algorithm of Min-l1 with equality
constraints presented in [4].
On the other hand, the recover algorithm is another issue
for the future development. In this paper, an algorithm
based on the Peak Transform and modulus maximum
was proposed for sparse representation of signals, which
was especially effective for the signals that are not sparse
themselves and have no sparse coefficients after wavelet
transform. Combining with the original CS algorithm,
the quality of reconstructed signal has been improved.
224
Yi-Gang Cen et al.
Acknowledgment
This work was supported by National Natural Science Foundation of China (No. 60802045, 60804066,
61073079); the Scientific Research Foundation for Returned Overseas Chinese Scholars ([2009] 1001), the
Fundamental Research Funds for the Central Universities of China (2009JBM028, 201012200159, 20101220
0110); Beijing Municipal Natural Science Foundation
(4113075) and New Teacher Foundation of State Education Ministry (20100162120019).
References
[1] Donoho, D., “Compressed Sensing,” IEEE Trans. Inform. Theory, Vol. 52, pp. 1289-1306 (2006).
[2] Candes, E., “Compressive Sampling,” Proceedings of
the International Congress of Mathmaticians, Madrid,
Spain (2006).
[3] Candes, E. and Tao, T., “Near Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?” IEEE Trans. Inform. Theory, Vol. 52,
pp. 5406-5425 (2006).
[4] Candes, E. and Caltech J. R., l1-MAGIC: Recovery of
Sparse Signals via Convex Programming, Oct., 2005;
http://dsp.rice.edu/cs.
[5] He, Z. H., “Peak Transform for Efficient Image Representation and Coding,” IEEE Transactions on Image
Processing, Vol. 16, pp. 1741-1754 (2007).
[6] Candes, E., Romberg, J. and Tao, T., “Robust Uncer-
tainty Principles: Exact Signal Reconstruction from
Highly Incomplete Frequency Information,” IEEE
Trans. Inform. Theory, Vol. 52, pp. 489-509 (2006).
[7] Tropp, J. A. and Gilbert, A. C., “Signal Recovery from
Random Measurements via Orthogonal Matching Pursuit,” IEEE Trans. Inform. Theory, Vol. 53, pp. 46554666 (2007).
[8] Donoho, D. and Tsaic, Y., “Extensions of Compressed
Sensing,” Signal Processing, Vol. 86, pp. 533-548
(2006).
[9] Tropp, J. A., “Greed is Good: Algorithmic Results for
Sparse Approximation,” IEEE Trans. Inform. Theory,
Vol. 50, pp. 2231-2242 (2004).
[10] Figueiredo, M. A. T., Nowak, R. D. and Wright, S. J.,
“Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse
Problem,” Journal of Selected Topics in Signal Processing: Special Issue on Convex Optimization Methods for Signal Processing, Vol. 1, pp. 586-598
(2007).
[11] Mallat, S. and Zhong, S., “Characterization of Signals
from Multiscale Edges,” IEEE Trans. on PAMI, Vol.
14, pp. 710-732 (1992).
[12] Zhao, R. Z., Song, G. X. and Qu, H. Z., “A New
Piecewise Cubic Spline Interpolating Algorithm for
the Reconstruction of Wavelet Coefficients,” Signal
Processing (Chinese), Vol. 17, pp. 242-277 (2001).
Manuscript Received: Dec. 17, 2010
Accepted: Feb. 15, 2011