Jon Dattorro convexoptimization.com prototypical cardinality problem find x subject to Ax  b x0 card x  k Perspectives:  Combinatorial  Geometric 2 Euclidean bodies Permutation Polyhedron P  {X  R nn X 1  1, X 1  1, X  0 } T • n! permutation matrices are vertices in (n-1)2 dimensions. • permutaton matrices are minimum cardinality doubly stochastic matrices. H Hyperplane H  {x  R n x 1  1} T 3 Geometrical perspective Compressed Sensing minimize || x ||1 subject to Ax  b 1-norm ball: 2n vertices, 2n facets Candes/Donoho (2004) 4 Candes demo  %Emmanuel Candes, California Institute of Technology, June 6 2007, IMA Summerschool.    clear all, close all n = 512; m = 64;      k = 0:n-1; t = 0:n-1; F = exp(-i*2*pi*k'*t/n)/sqrt(n); freq = randsample(n,m); A = [real(F(freq,:)); imag(F(freq,:))];     S = 28; support = randsample(n,S); x0 = zeros(n,1); x0(support) = randn(S,1); b = A*x0;        % Solve l1 using CVX cvx_quiet(true); cvx_begin variable x(n); minimize(norm(x,1)); A*x == b; cvx_end   norm(x - x0)/norm(x0) figure, plot(1:n,x0,'b*',1:n,x,'ro'), legend('original','decoded') % Size of signal % Number of samples (undersample by a factor 8) % Fourier matrix % Incomplete Fourier matrix wikimization.org 5 Candes demo find x subject to Fx  Fx0 A F  R m  n b Fx0  R m m  64  2 n  512 k  28  binary mask F Fourier matrix x is sparse 6 k-sparse sampling theorem • Donoho/Tanner (2005) A  R m n 7 two geometrical interpretations find x subject to Ax  b x0 K  {Ax x  0} 8 motivation to study cones  convex cones generalize orthogonal subspaces K  K  R * n  Projection on K determinable from projection on -K* and vice versa. (Moreau)  Dual cone: K  { y x y  0 x  K} * T 9 application - LP presolver  Delete rows and columns of matrix A  columns: smallest face F of cone K containing b  F( K  b)  {a  K a  K  b } *  A holds generators for K find subject to z bT z  0 AT z  0 A(: , i ) T z  1  If feasible, throw A(: , i) away K  {Ax x  0} 10 application - Cartography 11 list reconstruction from distance D a.k.a  metric multidimensional scaling  principal component analysis  Karhunen-Loeve transform  cartography: projection on semidefinite cone 12 minimize DS h || V ( D  H ) Vn ||F T n subject to  V D Vn  0 T n projection on semidefinite cone because S  V S h Vn T n subspace of symmetric matrices is isomorphic with subspace of symmetric hollow matrices 13 minimize DS h || V ( D  H ) Vn ||F T n subject to  V D Vn  0 T n (EY) rank V D Vn   T n  is convex problem (Eckart & Young) (§7.1.4 CO&EDG)  optimal list X from  VnT D Vn (§5.12 CO&EDG) 14 ordinal reconstruction minimize DS h || VnT ( D  O) Vn ||F subject to  VnT D Vn  0 rank V D Vn   T n  vect D  K M  • nonconvex • strategy: break into two problems: (EY) and convex problem minimize  ||    vect D || subject to   K M  • fast projection on monotone nonnegative cone KM+ (Nemeth, 2009) •  R 50202 15 Cardinality heuristics  y  minimize ||  ||0 4M4M R subject to E vec   t 0  minimize vec  , y * subject to E vec   t 0 16 Rank heuristics  trace is convex envelope of rank on PSD matrices  rank function is quasiconcave 17  Idea behind convex iteration tr G  G , I (vector inner product) 18 Convex Iteration 19 application (Recht, Fazel, Parrilo, 2007) (Rice University 2005) 20 one-pixel camera - MIT 21 one-pixel camera - MIT 22 application - MRI phantom • Led directly to sparse sampling theorem MATLAB>> phantom(256) Candes, Romberg, Tao 2004 23 application - MRI phantom • MRI raw data called k-space • aliasing at 4% subsampling • Raw data in Fourier domain vect 1 f 24 application - MRI phantom  P C 216 × 216 (projection matrix) • hard to compute y is direction vector from convex iteration 25 application - MRI phantom 26 application - MRI phantom reconstruction error: -103dB 27