Exercise 5.7
Proof of lemma 5.5.1 part 2
Jules Coret
May 25, 2017
Jules Coret
Exercise 5.7
May 25, 2017
1/6
Theorem
Given an r ∈ (0, 1), there exist constants , c > 0. Such that for all n, s
with s < rn the set of s-sparse vectors in Rn has a cardinality of at least
n ∨ exp(cs log(n/s))
Jules Coret
Exercise 5.7
May 25, 2017
2/6
Proof
Let S be the set of s-sparse vectors whose non-zero coordiantes are equal
√
to 1/ s. Let X , Y be independ random variables uniformly distributed on
S. Since kX − Y k2 = kX − Y k0 /s we have
P(kX − Y k ≤ ) = P(kX − Y k0 ≤ 2 s)
|{y ∈ S : ky − X k0 ≤ 2 s}|
|S|
−1 n
s
n − (1 − 2 )s
=
s
(1 − 2 )s
2 s
−1 n
s
n−E
=
s
E
s −E
s!(n − s)!s!(n − E )!
=
n!E !(s − E )!(s − E )!(n − s)!
P(kX − Y k0 ≤ 2 s|X ) =
Jules Coret
Exercise 5.7
May 25, 2017
3/6
Proof
s!s!(n − E )!E !
n!E !E !(s − E )!(s − E )!
−1 2
n
s
=
E
s −E
−1 2
n
s
=
(1 − 2 )s
2 s
=
2 )s
≤ (n/(1 − 2 )s)(−1+
2 )rn
≤ ((1 − 2 )r )(1−
2 rn
e 2
2s
(e/2 )2
−4
2 rn
= exp{(1 − 2 )rn log(r ) + log(1 − 2 ) + 22 rn − log()42 rn}
≤ exp{rn((log(r )) + 22 − log()42 )}
Jules Coret
Exercise 5.7
May 25, 2017
4/6
Proof
Let X1 , X2 , .., XM be M independent vectors uniform on S then
P(kXi − Xj k ≤ for some i 6= j) ≤ M 2 exp{rn((log(r )) + 22 − log()42 )}
for M = exp(cs log(n/s)) we have
P ≤ exp{2cs log(n/s) + rn(log(r ) + 22 (1 − 2 log()))}
≤ exp{−2crn log(r ) + rn(log(r ) + 22 (1 − 2 log()))}
= exp{rn((1 − 2c) log(r ) + 22 (1 − 2 log()))}
Jules Coret
Exercise 5.7
May 25, 2017
5/6
Proof
P ≤ exp{rn((1 − 2c) log(r ) + 22 (1 − 2 log()))} < 1
(1 − 2c) log(r ) + 22 (1 − 2 log()) < 0
c < 1/2
such that 22 (1 − 2 log()) < − log(r )(1 − 2c)
We also have that
P(kXi − Xj k ≥ for all i 6= j) = 1 − P(kXi − Xj k ≤ for some i 6= j)
Thus we have for these parameters that P(kXi − Xj k ≥ for all i 6= j) > 0
Thus there has to be a set S of s-sparse vectors separated by with
cardinality greater or equal to exp(cs log(n/s))
Jules Coret
Exercise 5.7
May 25, 2017
6/6