0368.416701 Sublinear Time Algorithms
January 20, 2015
Lecture 12
Lecturer: Ronitt Rubinfeld
1
Scribe: Nicholas Boffi, Gal Rotem
Estimating the Sum of Powers of Degree One Fourier Coefficients
1.1
Reminders
Recall from the last lecture that, for any Boolean function f : {−1, 1}m → {−1, 1}, we have that
X
f (x) =
fˆ(s)χS (x)
(1)
S⊂[n]
where the χS are given by
χS (x) =
Y
xi
(2)
i∈S
and the Fourier coefficients fˆ(S) are inner products:
fˆ(S) = hf, χS i = 2−n
X
f (x)χS (x).
(3)
x
1.2
Estimating Powers
Say we want to estimate the sum of powers of degree one Fourier coefficients. We say that a coefficient
is “degree one” if fˆ(S) is such that S is a singleton set, i.e, S = {i} for some i ∈ [n]. Why might we be
interested in doing this? One example is in testing if a function is a dictator, where such sums can be
highly indicative that a function is determined by a single bit. Furthermore, the ability to estimate the
sums of degree one powers will provide us with an iterative process to estimate sums of any degree. For
example, if we can estimate sums of degree two and less, and we can estimate the sums of degree one,
then we can subtract the two to find out the sum of degree two terms only. We will show how to perform
such a process explicitly later when we prove the upcoming algorithm for degree one coefficients.
First, we proceed with some notation. Denote fˆ({i}) by fˆ(i). Then the question becomes, can we
estimate:
n
X
fˆ(i)p
(4)
i=1
for some power p?
Data: A power p
1) Pick x(1) , x(2) , ... , x(p−1) ∈ {±1}n
2) Pick a “noise vector” µ such that Pr[µi = 1] = 12 + η2 and Pr[µi = −1] =
constant η < 1.
3) Set y = f (x(1) ) · f (x(2) ) · ... · f (x(p−1) ) · f (x(1) x(2) ... µ).
4) return y.
Algorithm 1: A “random process”.
Claim 1 E[y] =
P
S⊂[n]
η |S| fˆ(S)p
1
1
2
−
η
2
for some
Proof
Note that we can write
E[y] = E[f (x(1) ) · f (x(2) ) · ... · f (x(p−1) ) · f (x(1) x(2) ... µ)].
(5)
We can expand this as
X
E[y] =
fˆ(S1 )fˆ(S2 )...fˆ(Sp )E[χS1 (x(1) )χS2 (x(2) )...χSp−1 (x(p−1) )χSp (x(1) x(2) ... µ)]
(6)
S1 S2 ...Sp
by writing each f (x) in terms of its Fourier coefficients and using linearity of expectation. Recall now
two useful relations from last lecture that χS (a b) = χS (a)χS (b), and that χA (x)χB (x) = χA4B (x)
where A4B denotes the symmetric difference of A and B, A4B = (A − B) ∪ (B − A). Then we have
that:
E[χS1 (x(1) )χS2 (x(2) )...χSp−1 (x(p−1) )χSp (x(1) x(2) ...µ)] = E[χS1 4Sp (x(1) )...χSp−1 4Sp (x(p−1) )χS (µ)]
(7)
where we have used the first of the two useful relations. Employing the second one to the product, we
can now write
E[χS1 4Sp (x(1) )...χSp−1 4Sp (x(p−1) )χS (µ)] =
p−1
Y
E[χSi 4Sp (x(i) )] E[χS (µ)].
(8)
i=1
As in the last lecture, E[χSi 4Sp (x(i) ) = 1 for Si = Sp and 0 otherwise. Also, E[χS (µ)] = E
Q
Q
|S|
. Putting this all together, we have:
i∈S E[µi ] =
i∈S η = η
E[y] =
X
η |S| fˆ(S)p .
Q
i∈S
µi =
(9)
S⊂[n]
Okay, so the claim is true, but why do we care? The point is that η |S| decays rapidly as |S| increases.
Thus, the the terms that contribute most to the sum are the Fourier coefficients of degree one and the
Fourier coefficient of degree zero. With a proper choice of η, we can “tune” E[y] so that we can use it for
an approximation for degree one Fourier coefficients once we get rid of that pesky coefficient of degree
zero. Here’s how:
Data: A power p
Result: An estimation of the sum of Fourier coefficients of degree one to the power p.
1) Estimate ω = E[f (x(1) ...f (x(p) )] = fˆ(∅)p to additive ±η 2 . Note that xp = x(1) ... x(p−1) µ
with η = 0 for this step.
2) Use Alg. 1 to estimate E[y] with y as before.
P
3) return E[y] − ω = S⊂[n],|S|>0 η |S| fˆ(S)p = γ.
P
First, note that hf, χ∅ i = fˆ(∅) = x f (x) 21n = Ex [f (x)] by definition.
P
Claim 2 γ/η is a good estimate of |S|=1 fˆ(S)p .
Proof
X
|S|=1
η |S| fˆ(S)p
=
X
η |S| fˆ(S)p −
|S|>0
X
|S|>1
2
η |S| fˆ(S)p
(10)
≤ γ − η2
X
fˆ(S)p
(11)
|fˆ(S)p |
(12)
|S|>1
≤ γ − η2
X
|S|>1
≤ γ−η
2
sX
fˆ(S)2 ·
|S|>1
≤
γ − η 2 · (1) ·
sX
(fˆ(S)(p−1) )2
(13)
|S|>1
sX
fˆ(S)2
(14)
|S|>1
≤ γ − η 2 · (1) · (1)
(15)
≤ γ − η2
(16)
Where we have used that the sum of positive and negative numbers is less than the sum of their absolute
values to go from (11) to (12), Cauchy-Schwartz to go from (12) to (13), Boolean Parseval’s to reduce
the first square root from (13) to (14) and noting that fˆ(S)p−1 ≤ fˆ(S)2 for p ≥ 2 to reduce the second
square root. Finally, we employ Boolean Parseval’s again to go from (14) to (15). Now note that
P
P
|S| ˆ
f (S)p = |S|=1 η fˆ(S)p Dividing both sides by η, we have:
|S|=1 η
X
η fˆ(S)p ≤
|S|=1
2
γ
− η.
η
(17)
Interactive Proofs
Assume we have some user U who wants to compute a function f on an input x. Furthermore assume
the user is computationally bounded and cannot compute f (x) on his own, but can outsource the
computation to a server S. S will return y to the user and claims that y = f (x). To prevent “cheating”,
U demands that S sends both y and some reliable proof that y = f (x).
x
User
y
Server
Proof that f(x)=y
Example 1 U owns a website, and a company C claims that at least k clicks were made through their
website to enter U’s website. We assume that given a “description” x of a click, U can efficiently (in
constant time) verify that x is indeed a valid click.
U’s goal is to determine if the number of valid clicks is greater than k given the proof provided by U.
We ask that U return “PASS” with high probability in such a case, and if the number of valid clicks is
less than (1 − )k return “FAIL” with high probability.
3
For our first try, we assume that C sends U a “big array” X with a description of all k clicks:
X = x1 , x2 , ..., xk . Then U samples O(1/) xi ’s and checks if there are less then k/2 invalid click
descriptions. If so, he says “PASS”, otherwise he returns “FAIL”. Note that in this simple protocol, C
can easily fool U by sending the same description of a click multiple times. Thus we need to also check
if there are at most k/2 duplicate click descriptions.
We need to modify the proof C sends to U, and we will assume that we don’t care how “big” the
proof is.
C will build a table T with all possible clicks description, t1 , t2 , ..., tn and send both tables T and X,
where X is the array x1 , ...xk . C will also send forward and back pointers both from T to X and from
X to T in the following way:
1. For each cell ti ∈ T , ti is some click’s possible description. If we have such a click in table X,
then ti will hold a pointer to that cell in X. Thus, we have a pointer from ti to xj if xj ’s click
description is exactly ti . If we don’t have a click in X with ti ’s description, then ti will hold a
NULL pointer.
2. For each cell in xj ∈ X, we will hold a back-pointer to the appropriate cell ti such that ti is the
description of click xj .
T
All possible clicks
description
X
Actual clicks
NULL
Now U will verify the proof with the following procedure:
1) Repeat O(1/) times:
1.1) pick j ∈ [k] at random
1.2) l ← X[j]
1.3) if T (l) = xj continue, otherwise return F AIL
2) return P ASS
Algorithm 2: Verifying the proof
The intuition is that if there are any duplicates in X, then T won’t “know” on which xi to point.
Thus, we can easily see that if there are ≥ k/2 duplicates in X, then in each round we fail with
probability ≥ /2, so in O(1/) rounds, we will catch a false proof with constant probability.
Example 2 Consider the graph problem MAX-CUT. In this problem, we are given a graph, G = (V, E)
and we are interested in whether there exists a cut in the graph with at least k crossing edges, this problem
is known to be N P-complete.
4
We are only interested in verifying a solution: in such a case, some prover, S, declares that he found
such a cut. We can verify his proof in sublinear-time using the previous example: this time T serves as
a boolean array of size |V |, and each cell indicates on which side of the cut the corresponding vertex is
in. The table X is of size k as before, and holds all the crossing edges of the cut. A simple verifier can
use the following algorithm to check for the proof’s correctness:
1) Repeat O(1/) times:
1.1) pick an edge (vi , vj ) ∈ [k] at random
1.2) query T for both vertices vi , vj
1.3) if T (vi ) = T (vj ) return F AIL (they are on the same side of the cut), otherwise- continue
2) return P ASS
Algorithm 3: Verifying max-cut
As before, it is clear that with high probability we accept only correct proofs.
Example 3 Consider the Bin-Packing problem: we are given a positive integer B, a set of n positive
elements x1 , x2 , ..., xn where each xi ∈ [B], and k bins of size B. We want to know whether there exists
a legal packing of these elements into the k bins, i.e, each element is associated with a bin and the total
size of the elements in each bin is less than B. This is also a well-known N P-complete problem.
As before, we are interested in verifying a solution for this problem in sublinear-time. We want an
algorithm that returns “PASS” with probability 1 on all correct proofs, and if at most (1 − )n of the
elements fit, returns “FAIL” with high probability. Consider the following proof: We will have k arrays
(one for each bin) A1 , ..., Ak and each array will be of size B. If xi is of weight w and appears in bin Aj ,
then Aj will contain w consecutive instances of xi . In addition, we will have an extra array X of size n
such that X[j] indicates the number of bins i in which xj is packed and the offset m in Ai at which xj
starts to appear w consecutive times.
To verify this proof, the verifier will use the following procedure:
1) Repeat O(1/) times:
1.1) pick an element xi of size w , i ∈ n at random
1.2) query X[i] to get the number of bin j, and the offset m, for xi .
1.3) verify that xi appears w consecutive times in Aj starting at index m, if it is not - return
F AIL, else-continue
2) return P ASS
Algorithm 4: Verifying bin-packing
A problem may arise: it could be that xi is ”heavy-weighted”, and so checking w consecutive indices
could be costly. A possible solution could be to use a monotonicity tester, but we won’t elaborate any
further.
5