Computational Complexity Theory, Fall 2008
November 21
Lecture 18: Error correcting codes. Local decoding and hardness
amplification.
Lecturer: Kristoffer Arnsfelt Hansen
1
Scribe: Thomas Dueholm Hansen
Error correcting codes
Definition 1 (Fractional Hamming distance) We define the fractional Hamming distance between two strings x, y ∈ Σm as:
∆(x, y) =
1
|{i : xi 6= yi }|.
m
Definition 2 (Error correcting code) We say that E : Σn → Γm is an error correcting code
(EEC) of distance δ if for x 6= y we have ∆(E(x), E(y)) ≥ δ.
We present the following theorem regarding the existence of error correcting codes without a
proof.
Theorem 3 (Gilbert-Varshamov Bound) For every δ < 21 and sufficiently large n there exist
an E : {0, 1}n → {0, 1}n/(1−H(δ)) that is an ECC of distance δ, where H(δ) = δ log2 (1/δ) + (1 −
δ) log2 (1/(1 − δ)).
However, we also need explicit codes and efficient encodings and decondings of such codes. The
following is examples of such EEC’s.
Definition 4 (Walsh-Hadamard code) We define the Walsh-Hadamard code W H : {0, 1}n →
n
{0, 1}2 as W H(x) = z, where zy = x · y, y ∈ {0, 1}n , with · being the inner product modulo 2.
We note that W H is a code of distance 21 (proven at lecture 17). Also, W H is a linear code,
meaning that W H(x + y) = W H(x) + W H(y).
Definition 5 (Reed-Solomon code) Let F = {α0 , α1 , . . . } be a finite field. Let nP≤ m ≤ |F |.
n−1
i
We define the Reed-Solomon code RS : F n → F m as RS(a) = z, where zj =
i=0 ai (αj ) ,
j = 0, . . . , m − 1. That is, a is used as coefficients of a degree n − 1 polynomial over F .
Proposition 6 RS is a linear ECC of distance 1 −
n+1
m .
Proof We observe that:
RS(a + b)j =
n−1
X
i=0
(ai + bi )(αj )i =
n−1
X
ai (αj )i +
i=0
n−1
X
bi (αj )i = RS(a)j + RS(b)j
i=0
Hence, RS is linear.
Let a 6= b ∈ F , then RS(a − b) = RS(a) − RS(b) 6= 0 is a polynomial over F of degree ≤ n − 1,
= 1 − n+1
meaning that it has at most n − 1 roots. Thus, RS has distance at least m−n+1
m
m .
1
Theorem 7 (Code concatenation) Let E1 : Σn → Γm have distance δ1 and E2 : Ωl → ∆k have
distance δ2 , where |Ω|l ≥ |Γ|. Identify Γ with a subset of Ωl . Define E = E2 ◦ E1 : Σn → ∆k·m as
E(x) = E2 (E1 (x)1 )E2 (E1 (x)2 ) · · · E2 (E1 (x)m ).
Then E has distance δ1 · δ2 .
Proof Let x 6= y ∈ Σn . Then E1 (x) and E1 (y) differ in at least δ1 m coordinates. When
E1 (x)j 6= E1 (y)j then E2 (E1 (x)j ) and E2 (E1 (y)j ) differ in at least δ2 k coordinates. Thus, E(x)
and E(y) differ in at least δ1 m · δ2 k = δ1 δ2 · mk coordinates.
We can then concatenate W H and RS:
RS : {0, 1}n·log |F | ' F n → F m
W H : {0, 1}log |F | → {0, 1}2
log |F |
= {0, 1}|F |
W H ◦ RS : {0, 1}n·log |F | → {0, 1}m|F |
to get a code of distance
1
2
· (1 −
n+1
m )
=
1
2
−
n+1
2m .
Definition 8 (Reed-Muller code) Let F be a finite field. Let l and d be given with d < |F |. We
l+d
l
define the Reed-Muller code RM : F ( d ) → F |F | , where RM takes as input a l-variate degree d
polynomial
X
P (x1 , . . . , xl ) =
ci1 ,...,il xi11 · · · xil l .
i1 +···+il ≤d
I.e., the input is the coefficients ci1 ,...,il and the output is the evaluation of P on all of F l .
Proposition 9 RM is a linear code of distance 1 −
d
|F | .
Proof We know from the Schwartz-Zippel Lemma that the probabilty that a nonzero multivariate
degree d polynomial over a field F evaluates to zero for a randomly selected input is at most |Fd | .
The proof then goes as the proof of Proposition 6.
We note that the encoding of RM is polynomial time computable in the size of the output since
we just evaluate the polynomium.
2
Decoding of error correcting codes
We have the following theorem about decoding of the Reed-Solomon code:
Theorem 10 (Berlekamp-Welch procedure) There is a poly-time algorithm that given (a1 , b1 ),
. . . , (am , bm ) ∈ F 2 , such that there exists a degree d polynomial G : F → F satisfying G(ai ) = bi
for t pairs where t > m+d
2 , recovers G. (Note that such a G is unique.)
Proof We will show that the following procedure works.
(1) Find a nonzero polynomial E(x) of degree ≤ m−t and a polynomial C(x) of degree ≤ d+m−t,
such that C(ai ) = bi E(ai ) for all i = 1, . . . , m.
2
(2) Divide C(x) by E(x), such that C(x) = E(x)P (x) + R(x), where deg(R) < deg(C).
(3) Output P (x).
Q
First, we need to show that we can find a solution in step (1). Let E(x) := i∈{i:G(ai )6=bi } (x−ai ),
and let C(x) := G(x)E(x). Then deg(E) ≤ m − t since |{i : G(ai ) 6= bi }| ≤ m − t, and deg(C) ≤
d + m − t. Also, C(ai ) = bi E(ai ) for all i = 1, . . . , m, because either G(ai ) = bi or E(ai ) = 0.
Hence, we know that a solution exists in which E(x) is nonzero.
E and C can be found by solving the system of m linear equations where for all i = 1, . . . , m :
C(ai ) = bi E(ai ). This system of linear equations has m−t+1 unknowns coming from the coefficients
of E(x) and d + m − t + 1 unknowns coming from the coefficients of C(x). To find a solution with a
nonzero E(x) we repeatedly fix one of the coefficients of E to 1, and try to solve the corresponding
system of linear equations. We know that we will succeed at some point, because the above solution
exists also when E has been scaled such that this coefficient is 1.
After step (1) we know that C(x) = G(x)E(x) for at least t values of x, because G(ai ) = bi
for t of the pairs (a1 , b1 ), . . . , (am , bm ) ∈ F 2 . Thus, C(x) − G(x)E(x) has at least t roots, but
deg(C(x) − G(x)E(x)) ≤ d + m − t < 2t − t = t, implying that C(x) − G(x)E(x) = 0. Hence,
C(x) = G(x)E(x) for all x, and it follows that P (x) = G(x) (and R(x) = 0).
3
Local decoding
Definition 11 (Local decoder) Let E : Σn → Γm be an ECC. A local decoder for E handling
error rate ρ is a probabilistic algorithm D that has random access to a y ∈ Γm , where ∆(y, E(x)) <
ρ. On input j ∈ {1, . . . , n} D then runs in time (log m)O(1) and outputs xj with probability ≥ 23 .
Theorem 12 Suppose that there is a local decoder for an ECC, handling error rate ρ, E : {0, 1}N →
{0, 1}M where E(x) is computable in polynomial time. Let f : {0, 1}n → {0, 1}, where 2n = N , be
a function that requires boolean circuits of size S(n), where S is monotonely increasing.
Then there is a c > 0 and a fˆ ∈ E (linear exponential time), such that no circuit of size
S(n/c)/nc can compute fˆ on a 1 − ρ fraction of the inputs.
Proof Let 2m = M . Define fˆ : {0, 1}m → {0, 1} by fˆ = E(f ) where f and fˆ are viewed as
truth-tables. That is, f : {0, 1}n → {0, 1} can be expressed as a bitstring of length 2n , meaning
that E(f ) is a bitstring of length 2m which we use as a truth-table for fˆ.
Suppose C is a circuit of size T (m) computing fˆ on a 1 − ρ fraction of the inputs.
Since E is locally decodeable with error rate ρ there is a circuit of size (log M )k = mk implementing the local decoder, taking as input ≤ mk bits of the truth-table description of fˆ.
This gives a circuit for f of size mk T (m) ≥ S(n). There exists a k such that M ≤ N k , i.e.
n ≥ m/k. Thus, mk T (m) ≥ S(n) ≥ S(m/k) ⇒ T (m) ≥ S(m/k)/mk .
3