Constructions of Almost k-wise Independent
Random Variables
Based on work by N. Alon, O. Goldreich, J. Håstad, R. Peralta
April 21, 2012
1 / 17
Alon et. al.
Almost k-wise Independent Random Variables
k-wise independent sample spaces
• Known constructions.
2 / 17
Alon et. al.
Almost k-wise Independent Random Variables
k-wise independent sample spaces
• Known constructions.
k
• Lower bound of n 2 .
2 / 17
Alon et. al.
Almost k-wise Independent Random Variables
k-wise independent sample spaces
• Known constructions.
k
• Lower bound of n 2 .
• Attempt to beat the bound with an almost.
2 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Almost k-wise independence
Definition
Let Sn ⊆ {0, 1}n be a sample space and X = x1 x2 . . . xn be chosen
uniformly from Sn .
• (max-norm): Sn is (, k) independent if for any k positions
i1 ≤ i2 . . . ≤ ik , and any k-bit string α, we have
|Pr [xi1 xi2 . . . xik = α] − 2−k | ≤ (1)
• (statistical closeness): Sn is -away from k-wise independence
if for any k positions i1 ≤ i2 . . . ≤ ik , we have
Σα∈{0,1}k |Pr [xi1 xi2 . . . xik = α] − 2−k | ≤ 3 / 17
Alon et. al.
Almost k-wise Independent Random Variables
(2)
Linear tests
Some more definitions
Definition
(Bias) A 0 − 1 random variable X is said to be -biased if
|Pr [X = 0] − Pr [X = 1]| ≤ 4 / 17
Alon et. al.
Almost k-wise Independent Random Variables
(3)
Linear tests
Some more definitions
Definition
(Bias) A 0 − 1 random variable X is said to be -biased if
|Pr [X = 0] − Pr [X = 1]| ≤ (3)
Definition
(Bias w.r.t linear tests) A sample space Sn is said to be biased
with respect to linear tests of size at most k is for every n bit
string α 6= {0}n , such that at most k positions in α are 1, the
random variable < α, X > is -biased.
4 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Almost k-wise independence and linear tests of size k
Theorem (Vazirani)
Let Sn ⊆ {0, 1}n be a sample space that is biased with respect to
linear tests of size at most k. Then the sample space Sn is
1
((1 − 2−k ), , k)-independent in max norm and (2k − 1) 2 away in
L1 norm from k-wise independence.
5 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Almost k-wise independence and linear tests of size k
Theorem (Vazirani)
Let Sn ⊆ {0, 1}n be a sample space that is biased with respect to
linear tests of size at most k. Then the sample space Sn is
1
((1 − 2−k ), , k)-independent in max norm and (2k − 1) 2 away in
L1 norm from k-wise independence.
Proof Idea:
• Bias of a random variable X coming from a distro.
D = |Pr [X = 1] − Pr [X = 0]|.
5 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Almost k-wise independence and linear tests of size k
Theorem (Vazirani)
Let Sn ⊆ {0, 1}n be a sample space that is biased with respect to
linear tests of size at most k. Then the sample space Sn is
1
((1 − 2−k ), , k)-independent in max norm and (2k − 1) 2 away in
L1 norm from k-wise independence.
Proof Idea:
• Bias of a random variable X coming from a distro.
D = |Pr [X = 1] − Pr [X = 0]|.
• Bias of linearity tests w.r.t
β, B = |Pr [< X , β >= 0] − Pr [< X , β >= 1]|.
5 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Almost k-wise independence and linear tests of size k
Theorem (Vazirani)
Let Sn ⊆ {0, 1}n be a sample space that is biased with respect to
linear tests of size at most k. Then the sample space Sn is
1
((1 − 2−k ), , k)-independent in max norm and (2k − 1) 2 away in
L1 norm from k-wise independence.
Proof Idea:
• Bias of a random variable X coming from a distro.
D = |Pr [X = 1] − Pr [X = 0]|.
• Bias of linearity tests w.r.t
β, B = |Pr [< X , β >= 0] − Pr [< X , β >= 1]|.
• B = |Pr [X = α∧ < α, β >= 0] − Pr [X = α∧ < α, β >= 1]|
5 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Almost k-wise independence and linear tests of size k
Theorem (Vazirani)
Let Sn ⊆ {0, 1}n be a sample space that is biased with respect to
linear tests of size at most k. Then the sample space Sn is
1
((1 − 2−k ), , k)-independent in max norm and (2k − 1) 2 away in
L1 norm from k-wise independence.
Proof Idea:
• Bias of a random variable X coming from a distro.
D = |Pr [X = 1] − Pr [X = 0]|.
• Bias of linearity tests w.r.t
β, B = |Pr [< X , β >= 0] − Pr [< X , β >= 1]|.
• B = |Pr [X = α∧ < α, β >= 0] − Pr [X = α∧ < α, β >= 1]|
• Intuitively captures the difference in probability mass of two
partitions in the domain S0 , S1 .
5 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
6 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Pr [X = α∧ < α, β >= 0] = Σα∈S0 Pr [X = α]
6 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Pr [X = α∧ < α, β >= 0] = Σα∈S0 Pr [X = α] = Σα∈S0 Pα
6 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Pr [X = α∧ < α, β >= 0] = Σα∈S0 Pr [X = α] = Σα∈S0 Pα =
Pα (−1)<α,β>
6 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Pr [X = α∧ < α, β >= 0] = Σα∈S0 Pr [X = α] = Σα∈S0 Pα =
Pα (−1)<α,β>
• Similarly, −Pr [X = α∧ < α, β >= 1] = −Σα∈S1 Pr [X = α] =
−Σα∈S1 Pα = Σα∈S1 Pα (−1)<α,β>
6 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Pr [X = α∧ < α, β >= 0] = Σα∈S0 Pr [X = α] = Σα∈S0 Pα =
Pα (−1)<α,β>
• Similarly, −Pr [X = α∧ < α, β >= 1] = −Σα∈S1 Pr [X = α] =
−Σα∈S1 Pα = Σα∈S1 Pα (−1)<α,β>
• Hence, B = Σα Pα (−1)<α,β>
6 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Consider P as a boolean map from {0, 1}k → [0, 1].
7 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Consider P as a boolean map from {0, 1}k → [0, 1].
• Recall, φβ (α) = (−1)<α,β> forms a fourier basis.
7 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Consider P as a boolean map from {0, 1}k → [0, 1].
• Recall, φβ (α) = (−1)<α,β> forms a fourier basis.
• P = Σβ cβ φβ .
7 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Consider P as a boolean map from {0, 1}k → [0, 1].
• Recall, φβ (α) = (−1)<α,β> forms a fourier basis.
• P = Σβ cβ φβ .
• Where cβ = 2−k Σα Pα φβ (α)
7 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Consider P as a boolean map from {0, 1}k → [0, 1].
• Recall, φβ (α) = (−1)<α,β> forms a fourier basis.
• P = Σβ cβ φβ .
• Where cβ = 2−k Σα Pα φβ (α)
• Which precisely equals B.
7 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Consider P as a boolean map from {0, 1}k → [0, 1].
• Recall, φβ (α) = (−1)<α,β> forms a fourier basis.
• P = Σβ cβ φβ .
• Where cβ = 2−k Σα Pα φβ (α)
• Which precisely equals B.
• So, cβ ≤ 2−k .
7 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Recall again,
|Pα − 2−k |
8 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Recall again,
|Pα − 2−k | = |Σβ cβ φβ (α) − 2−k |
8 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Recall again,
|Pα − 2−k | = |Σβ cβ φβ (α) − 2−k | = |Σβ6=0 cβ φβ (α)|
8 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Recall again,
|Pα − 2−k | = |Σβ cβ φβ (α) − 2−k | = |Σβ6=0 cβ φβ (α)|
• This is at most 2−k (2k − 1), which proves the bound.
8 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Recall again,
|Pα − 2−k | = |Σβ cβ φβ (α) − 2−k | = |Σβ6=0 cβ φβ (α)|
• This is at most 2−k (2k − 1), which proves the bound.
• For the other part, standard idea using Parseval’s theorem and
Cauchy-Schwartz inequality.
8 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
• Recall again,
|Pα − 2−k | = |Σβ cβ φβ (α) − 2−k | = |Σβ6=0 cβ φβ (α)|
• This is at most 2−k (2k − 1), which proves the bound.
• For the other part, standard idea using Parseval’s theorem and
Cauchy-Schwartz inequality.
Moral of the story: To show k-wise independence, show small bias
for linear tests.
8 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Let p be a prime number. Some definitions:
• Quadratic Residue: An integer x is said to be a quadratic
residue mod p if there is an integer y such that y 2 ≡ x (mod
p).
9 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Let p be a prime number. Some definitions:
• Quadratic Residue: An integer x is said to be a quadratic
residue mod p if there is an integer y such that y 2 ≡ x (mod
p).
• Quadratic Character: of x mod p denoted by χp (x) equals 0
when x is a multiple of p
9 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Let p be a prime number. Some definitions:
• Quadratic Residue: An integer x is said to be a quadratic
residue mod p if there is an integer y such that y 2 ≡ x (mod
p).
• Quadratic Character: of x mod p denoted by χp (x) equals 0
when x is a multiple of p, equals 1 when x is a quadratic
residue mod p
9 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Let p be a prime number. Some definitions:
• Quadratic Residue: An integer x is said to be a quadratic
residue mod p if there is an integer y such that y 2 ≡ x (mod
p).
• Quadratic Character: of x mod p denoted by χp (x) equals 0
when x is a multiple of p, equals 1 when x is a quadratic
residue mod p, −1 otherwise.
9 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Theorem (Weil)
Let p be an odd prime and f (t) be a polynomial over GF (p) which
is not the square of another polynomial and has precisely n distinct
zeros. Then,
√
|Σx∈GF (p) χp f (x)| ≤ (n − 1) p
10 / 17
Alon et. al.
Almost k-wise Independent Random Variables
(4)
Quadratic Character Construction
Construction:
• p strings of length n.
11 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Construction:
• p strings of length n.
• x th string is r (x) = r0 (x)r1 (x) . . . rn−1 (x)
11 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Construction:
• p strings of length n.
• x th string is r (x) = r0 (x)r1 (x) . . . rn−1 (x)
• ri (x) =
11 / 17
1−χp (x+i)
2
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Correctness:
Lemma
For any α, the random variable < α, r > is
selected uniformly at random.
n−1
√
p
+
n
p
biased for an r
• The bias of linear tests w.r.t α is Σβ Pβ (−1)<α,β> .
12 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Correctness:
Lemma
For any α, the random variable < α, r > is
selected uniformly at random.
n−1
√
p
+
n
p
biased for an r
• The bias of linear tests w.r.t α is Σβ Pβ (−1)<α,β> .
• Here, Pβ = p1 .
12 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Correctness:
Lemma
For any α, the random variable < α, r > is
selected uniformly at random.
n−1
√
p
+
n
p
biased for an r
• The bias of linear tests w.r.t α is Σβ Pβ (−1)<α,β> .
• Here, Pβ = p1 .
• Bias = p1 Σx∈GF (p) (−1)Σi αi ri (x)
12 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Correctness:
Lemma
For any α, the random variable < α, r > is
selected uniformly at random.
n−1
√
p
+
n
p
biased for an r
• The bias of linear tests w.r.t α is Σβ Pβ (−1)<α,β> .
• Here, Pβ = p1 .
• Bias = p1 Σx∈GF (p) (−1)Σi αi ri (x)
• Except when x + i is a multiple of p , (−1)ri (x) = χp (x + i).
12 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Quadratic Character Construction
Correctness:
Lemma
For any α, the random variable < α, r > is
selected uniformly at random.
n−1
√
p
+
n
p
biased for an r
• The bias of linear tests w.r.t α is Σβ Pβ (−1)<α,β> .
• Here, Pβ = p1 .
• Bias = p1 Σx∈GF (p) (−1)Σi αi ri (x)
• Except when x + i is a multiple of p , (−1)ri (x) = χp (x + i).
• Observe χp (xy ) = χp (x)χp (y ).
12 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
Correctness:
• (−1)αi ri (x) = χp (x + i)αi
13 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
Correctness:
• (−1)αi ri (x) = χp (x + i)αi
• which equals χp (f (x, i)). for f (x, i) = (x + i)αi
13 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
Correctness:
• (−1)αi ri (x) = χp (x + i)αi
• which equals χp (f (x, i)). for f (x, i) = (x + i)αi
• For good x, (−1)Σi αi ri (x) = χp (f (x)) for f (x) = Πi (x + i)αi
13 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
Correctness:
• (−1)αi ri (x) = χp (x + i)αi
• which equals χp (f (x, i)). for f (x, i) = (x + i)αi
• For good x, (−1)Σi αi ri (x) = χp (f (x)) for f (x) = Πi (x + i)αi
• For bad x, for which there is an i such that x + i = t.p for
some i such that αi = 1, there are at most n of them.
13 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
Correctness:
• (−1)αi ri (x) = χp (x + i)αi
• which equals χp (f (x, i)). for f (x, i) = (x + i)αi
• For good x, (−1)Σi αi ri (x) = χp (f (x)) for f (x) = Πi (x + i)αi
• For bad x, for which there is an i such that x + i = t.p for
some i such that αi = 1, there are at most n of them.
• So, the bias is p1 |Σx χp (f (x))| + pn
13 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Proof contin..
Correctness:
• (−1)αi ri (x) = χp (x + i)αi
• which equals χp (f (x, i)). for f (x, i) = (x + i)αi
• For good x, (−1)Σi αi ri (x) = χp (f (x)) for f (x) = Πi (x + i)αi
• For bad x, for which there is an i such that x + i = t.p for
some i such that αi = 1, there are at most n of them.
• So, the bias is p1 |Σx χp (f (x))| + pn , which using Wiel’s theorm,
gives us the desired bound.
13 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Construction:
• For GF (2m ), imagine elements being represented as m bit
strings.
• For x, y ∈ GF (2m ), r (x, y )i =< x i , y >
14 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
• = Σi < αi x i , y >..linearity of inner product
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
• = Σi < αi x i , y >..linearity of inner product
• Which equals < Σi αi x i , y >= B(say)
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
• = Σi < αi x i , y >..linearity of inner product
• Which equals < Σi αi x i , y >= B(say)
• Now, we analyse B for a fixed x and a random y .
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
• = Σi < αi x i , y >..linearity of inner product
• Which equals < Σi αi x i , y >= B(say)
• Now, we analyse B for a fixed x and a random y .
• B is unbiased if x is not a root of Pα (z) = Σi αi z i and y is
random.
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
• = Σi < αi x i , y >..linearity of inner product
• Which equals < Σi αi x i , y >= B(say)
• Now, we analyse B for a fixed x and a random y .
• B is unbiased if x is not a root of Pα (z) = Σi αi z i and y is
random.
• Bias happens only when x is a root of Pα (z), which happens
at most n − 1 times.
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Powering Construction
Correctness:
Lemma
For any α, < α, r > is (n − 1)2−m biased when r is selected
uniformly at random.
Proof:
• < α, r (x, y ) >= Σi αi r (x, y )i = Σi αi < x i , y >
• = Σi < αi x i , y >..linearity of inner product
• Which equals < Σi αi x i , y >= B(say)
• Now, we analyse B for a fixed x and a random y .
• B is unbiased if x is not a root of Pα (z) = Σi αi z i and y is
random.
• Bias happens only when x is a root of Pα (z), which happens
at most n − 1 times.
• Hence, Bias is at most n−1
2m
15 / 17
Alon et. al.
Almost k-wise Independent Random Variables
On lower bounds
16 / 17
Alon et. al.
Almost k-wise Independent Random Variables
On lower bounds
Via error correcting codes
• Define E : n → m via a generator matrix A such that A(i, j) is
the j th bit of the i th string in the sample space.
16 / 17
Alon et. al.
Almost k-wise Independent Random Variables
On lower bounds
Via error correcting codes
• Define E : n → m via a generator matrix A such that A(i, j) is
the j th bit of the i th string in the sample space.
• Claim: distance is between ( 12 + ) and ( 12 − ) for bias
parameter .
So, bounds from the world of coding theory apply.
16 / 17
Alon et. al.
Almost k-wise Independent Random Variables
On lower bounds
Via error correcting codes
• Define E : n → m via a generator matrix A such that A(i, j) is
the j th bit of the i th string in the sample space.
• Claim: distance is between ( 12 + ) and ( 12 − ) for bias
parameter .
So, bounds from the world of coding theory apply.
Best obtained bound Ω(min{ 2 (logn(1/)) , 2n })
16 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Queries/comments...
17 / 17
Alon et. al.
Almost k-wise Independent Random Variables
Queries/comments...
Thank You :-)
17 / 17
Alon et. al.
Almost k-wise Independent Random Variables