Sudocodes Fast measurement and reconstruction of sparse signals Shriram Sarvotham Dror Baron Richard Baraniuk ECE Department Rice University dsp.rice.edu/cs Motivation: coding of sparse data • Distributed delivery of data with sparse representation – Content delivery networks – Peer to peer networks – Distributed file storage systems • E.g. thresholded DCT/wavelet coefficients used in JPEG/JPEG2000 Motivation: coding of sparse data • Distributed coding of sparse data – Can we exploit sparsity? – Efficient? – Low complexity? Sparse signal processing • Signal has non-zero coefficients • Efficient ways to measure and recover ? • Traditional DSP approach: – Acquisition: first obtain measurements – Then exploit sparsity is in the processing stage Sparse signal processing • Signal has non-zero coefficients • Efficient ways to measure and recover ? • Traditional DSP approach: – Acquisition: first obtain measurements – Then exploit sparsity is in the processing stage • New compressive sampling (CS) approach: – Acquisition: obtain just measurements – Sparsity is exploited during signal acquisition [Candes et al; Donoho] Compressive sampling • Signal is -sparse • Measure signal via few linear projections • Enough to encode the signal measurements sparse signal nonzero entries Compressive sampling • Signal is -sparse • Measure signal via few linear projections • Random Gaussian measurements measurements will work! sparse signal nonzero entries CS Miracle: L1 reconstruction • Find the explanation with smallest L1 norm [Candes et al; Donoho] • If then perfect reconstruction w/ high probability measurements sparse signal nonzero entries CS Miracle: L1 reconstruction • Performance – Polynomial complexity reconstruction – Efficient encoding measurements sparse signal nonzero entries CS Miracle: L1 reconstruction • But… is still impractical for many applications • Reconstruction times: N=1,000 N=10,000 N=100,000 measurements t=10 seconds t=3 hours t=~months sparse signal nonzero entries Why is reconstruction expensive? measurements sparse signal nonzero entries Why is reconstruction expensive? Culprit: dense, unstructured measurements sparse signal nonzero entries Fast CS reconstruction • Sudocode matrix (sparse) • Only 0/1 in • Each row of contains measurements randomly placed 1’s sparse signal nonzero entries Sudocodes • Sudocode performance – Efficient encoding – Sub-linear complexity reconstruction • Encouraging numerical results N=100,000 measurements K=1,000 t=5.47 seconds M=5,132 sparse signal nonzero entries Sudocode reconstruction Process each in succession measurements Each can recover some ‘’s sparse signal nonzero entries Case 1: Zero measurement Case 1: Zero measurement • Resolves all coefficients in the support • Can resolve up to coefficients Case 1: Zero measurement • Resolves all coefficients in the support • Can resolve up to coefficients • Reduces size of problem Case 2: #(support set)=1 Case 2: #(support set)=1 • Trivially resolves Case 2: #(support set)=1 • Trivially resolves Case 3: Matching measurements Case 3: Matching measurements • Matches originate from same support • Disjoint support Common support coefficients = 0 contain nonzeros Common support Case 3: Matching measurements • Matches originate from same support • Disjoint support Common support coefficients = 0 contain nonzeros Trigger of revelations • Recovery of can trigger more revelations Trigger of revelations • Recovery of can trigger more revelations • An avalanche of coefficient revelations Trigger of revelations • Recovery of can trigger more revelations • An avalanche of coefficient revelations Sudocode reconstruction Like sudoku puzzles measurements sparse signal nonzero entries Practical considerations • Bottleneck: search for matches – With Binary Search Tree, matches ~ • Re-explain measurements: more data structures Search for matches Design of Sudo measurement matrix • Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed Design of Sudo measurement matrix • Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed Design of Sudo measurement matrix • Choice of Small : Most measurements reveal Many measurements needed • Intuition: Large : Most measurements uninformative Many measurements needed so that Design of Sudo measurement matrix • Choice of Small : Most measurements reveal Many measurements needed • Intuition: Large : Most measurements uninformative Many measurements needed so that Related work • [Cormode, Muthukrishnan] – CS scheme based on group testing – – Complexity • [Gilbert et. al.] Chaining Pursuit – CS scheme based on group testing and iterating – – Complexity – Works best for super-sparse signals Performance comparison L1 reconstruction Chaining Pursuit Sudocodes N=10,000 K=10 M=99 T3 hours M=5,915 t=0.16 sec M=461 t=0.14 sec N=10,000 K=100 M=664 T3 hours M=90,013 t=2.43 sec M=803 t=0.37 sec N=100,000 K=10 M=1,329 Tmonths M=17,398 t=1.13 sec M=931 t=1.09 sec N=10,000 K=1000 M=3,321 T3 hours M>106 t>30 sec M=5,132 t=5.47 sec sparse signal measurements nonzero entries Utility in CDNs • Measurements come from different sources • Needs enough measurements Ongoing work • Statistical dependencies between non-zero coefficients • Irregular degree distributions • Adaptive linear projections • Noisy measurements Conclusions • Sudocodes for CS – highly efficient – low complexity • Key idea: use sparse • Applications to content distribution THE END Compressed sensing webpage: dsp.rice.edu/cs Number of measurements Theorem: With coefficients Proof sketch: , phase 1 requires to exactly reconstruct Two phase decoding is not measured • • • Phase 1: decode coefficients Phase 2: decode remaining coefficients Why? • Phase 2 saves a factor of – When most coefficients are decoded, measurements Phase 2 measurements and decoding • is non-sparse of dimension Phase 2 measurements and decoding • is non-sparse of dimension • Resolve remaining coefficients by inverting the sub-matrix of Phase 2 measurements and decoding • is non-sparse of dimension • Resolve remaining coefficients by inverting the sub-matrix of • Phase 2 complexity = • Key: choose • Phase 2 complexity is Compressive Sampling • Signal is -sparse in basis/dictionary – WLOG assume sparsity in space domain • Measure signal via few linear projections measurements sparse signal nonzero entries • Random sparse measurements will work! Signal model • Signal is strictly sparse • Every nonzero ~ continuous distribution each nonzero coefficient is unique almost surely measurements sparse signal nonzero entries
© Copyright 2026 Paperzz