Locality in Coding Theory
Madhu Sudan
Harvard
April 9, 2016
Skoltech: Locality in Coding Theory
1 of 28
Error-Correcting Codes
• (Linear) Code 𝐶 ⊆ 𝔽𝑛𝑞 .
–
–
–
–
𝔽𝑞 : Finite field with 𝑞 elements.
𝑛 ≝ block length
𝑘 = dim(𝐶) ≝ message length
𝑅(𝐶) ≝ 𝑘/𝑛: Rate of 𝐶
– 𝛿 𝐶 ≝ min {𝛿 𝑢, 𝑣 ≝ Pr[𝑢𝑖 ≠ 𝑣𝑖 ]}.
𝑥≠𝑦∈𝐶
𝑖
• Basic Algorithmic Tasks
– Encoding: map message in 𝔽𝑘𝑞 to codeword.
– Testing: Decide if 𝑢 ∈ 𝐶
– Correcting: If 𝑢 ∉ 𝐶, find nearest 𝑣 ∈ 𝐶 to 𝑢.
April 9, 2016
Skoltech: Locality in Coding Theory
2 of 28
Locality in Algorithms
• “Sublinear” time algorithms:
–
–
–
–
Algorithms that run in time o(input), o(output).
Assume random access to input
Provide random access to output
Typically probabilistic; allowed to compute output on
approximation to input.
• (Property testing ≜ Sublinear time for Boolean functions)
• LTCs: Codes that have sublinear time testers.
– Decide if 𝑢 ∈ 𝐶 probabilistically.
– Allowed to accept 𝑢 if 𝛿(𝑢, 𝐶) small.
• LCCs: Codes that have sublinear time correctors.
– If 𝛿(𝑢, 𝐶) is small, compute 𝑣𝑖 , for 𝑣 ∈ 𝐶 closest to 𝑢.
April 9, 2016
Skoltech: Locality in Coding Theory
3 of 28
LTCs and LCCs: Formally
• 𝐶 is a (ℓ, 𝜖)-LTC if there exists a tester that
– Makes ℓ(𝑛) queries to 𝑢.
– Accept 𝑢 ∈ 𝐶 w.p. 1
– Reject 𝑢 w.p. at least 𝜖 ⋅ 𝛿(𝑢, 𝐶).
• C is a (ℓ, 𝜖)-LCC if exists decoder 𝐷 s.t.
– Given oracle access 𝑢 close to 𝑣 ∈ 𝐶, and 𝑖 LDC if only
coordinates of
– Decoder makes ℓ(𝑛) queries to 𝑢.
message can be
– Decoder 𝐷𝑢 (𝑖) usually outputs 𝑣𝑖 .
• ∀𝑖, Pr[ 𝐷𝑢 𝑖 ≠ 𝑣𝑖 ] ≤ 𝛿(𝑢, 𝑣)/𝜖
recovered.
𝐷
1
3
• Often: ignore 𝜖 (fix to ) and focus on ℓ
April 9, 2016
Skoltech: Locality in Coding Theory
4 of 28
Outline of this talk
•
•
•
•
•
Part 0: Definitions of LTC, LCC
Part 1: Elementary construction
Part 2: Motivation (historical, current)
Part 3: State-of-the-art constructions of LCCs
Part 4: State-of-the-art constructions of LTCs
April 9, 2016
Skoltech: Locality in Coding Theory
5 of 28
Part 1: Elementary Construction
April 9, 2016
Skoltech: Locality in Coding Theory
6 of 28
Main Example: Reed-Muller Codes
• Message = multivariate polynomial;
Encoding = evaluations everywhere.
– RM 𝑚, 𝑟, 𝑞 ≝
{𝑓 𝛼
|
𝛼∈𝔽𝑚
𝑞
𝑓 ∈ 𝔽q 𝑥1 , … , 𝑥𝑚 , deg 𝑓 ≤ 𝑟}
• Locality? Say 𝑟 < 𝑞
– Restrictions of low-degree
polynomials to lines yield
low-degree (univ.) polys.
– Random lines sample 𝔽𝑚
𝑞
uniformly (pairwise ind’ly)
April 9, 2016
Skoltech: Locality in Coding Theory
7 of 28
LCCs and LTCs from Polynomials
• Decoding (𝑟 < 𝑞):
– Problem: Given 𝑓 ≈ 𝑝, 𝛼 ∈ 𝔽𝑚
𝑞 , compute 𝑝 𝛼 .
– Pick random 𝛽 and consider 𝑓|𝐿
where 𝐿 = {𝛼 + 𝑡 𝛽 | 𝑡 ∈ 𝔽𝑞 } is a random line ∋ 𝛼.
– Find univ. poly ℎ ≈ 𝑓|𝐿 and output ℎ(𝛼)
• Testing (𝑟 < 𝑞/2):
Analysis non-trivial
– Verify deg 𝑓|𝐿 ≤ 𝑟 for random line 𝐿
• Parameters:
–𝑛=
April 9, 2016
1
1𝑟
Ideas
can
be
extended
to
𝑚
𝑚
𝑞 ; ℓ = 𝑞 = 𝑛 ; 𝑅 −1
𝐶 ≈
Locality ≈ poly(𝛿 ) 𝑚!
Skoltech: Locality in Coding Theory
> 𝑞.
8 of 28
Part 2: Motivations
April 9, 2016
Skoltech: Locality in Coding Theory
9 of 28
Motivation – 1 (“Practical”)
• How to encode massive data?
– Solution I
• Encode all data in one big chunk
• Pro: Pr[failure] = exp(-|big chunk|)
• Con: Recovery time ~ |big chunk|
– Solution II
• Break data into small pieces; encode separately.
• Pro: Recovery time ~ |small|
• Con: Pr[failure] = #pieces X Pr[failure of a piece]
– Locality (if possible): Best of both Solutions!!
April 9, 2016
Skoltech: Locality in Coding Theory
10 of 28
Aside: LCCs vs. other Localities
• Local Reconstruction Codes (LRC):
– Recover from few (one? two?) erasures locally.
– AND Recover from many errors globally.
• Regenerating Codes (RgC):
– Restricted access pattern for recovery: Partition
coordinates and access few symbols per partition.
• Main Differences:
– #errors: LCCs high vs LRC/RgC low
– Asymptotic (LCC) vs. Concrete parameters (LRC/RgC)
April 9, 2016
Skoltech: Locality in Coding Theory
11 of 28
Motivation – 2 (“Theoretical”)
• (Many?) mathematical consequences:
– Probabilistically checkable proofs:
• Use specific LCCs and LTCs
– Hardness amplification:
• Constructing functions that are very hard on average from
functions that are hard on worst-case.
• Any (sufficiently good) LCC ⇒ Hardness amplification
– Small set expanders (SSE):
• Usually have mostly small eigenvalues.
• LTCs ⇒ SSEs with many big eigenvalues [Barak et al., Gopalan
et al.]
April 9, 2016
Skoltech: Locality in Coding Theory
12 of 28
Aside: PCPs (1 of 4)
• Familiar task: Protect massive data 𝑚 ∈ 0,1
𝑚
𝑘
𝐸(𝑚)
𝐸𝜙 (𝑚)
𝜙 𝑚
• PCP task: Protect 𝑚 + analysis 𝜙 𝑚 ∈ {0,1}.
– 𝜙(𝑚) is just one bit long – would like to read &
trust 𝜙(𝑚) with few probes.
– Can we do it? Yes! PCPs!
– “Functional Error-correction”
April 9, 2016
Skoltech: Locality in Coding Theory
13 of 28
PCPs (2 of 4) - Definition
0
W
HTHHTH
V
1
1
PCP Verifer: Given 𝑊 ∈ 0,1
𝑁
1. Tosses random coins
2. Determines query locations
3. Reads locations. Accepts/Rejects
𝑊 ≈ 𝐸𝜙 𝑚 with 𝜙 𝑚 = 1 ⇒ V accepts w.h.p.
𝑊 far from every 𝐸𝜙 𝑚 with 𝜙 𝑚 = 1 ⇒ rejects w.h.p.
Distinguishes 𝜙 −1 1 ≠ ∅ from 𝜙 −1 1 = ∅
April 9, 2016
Skoltech: Locality in Coding Theory
14 of 28
PCPs (3 of 4): “Polynomial-speak”
•
•
•
•
𝑚 → 𝑀(𝑥) low-degree (multiv.) polynomial
𝜙 → Φ : local map from poly’s to poly’s
𝜙 𝑚 = 1 ⇔ ∃ 𝐴, 𝐵, 𝐶 s. t. Φ 𝑀, 𝐴, 𝐵, 𝐶 ≡ 0
𝐸𝜙 𝑚 = ( 𝑀 , 𝐴 , 𝐵 , 𝐶 ) (evaluations)
• Local testability of RM codes ⇒ can verify 𝐸𝜙 (𝑚)
syntactically correct. ( 〈𝑀〉, 〈𝐴〉, 〈𝐵〉, 𝐶 ≈ polynomials )
• Distance of RM codes + Φ 𝑀, 𝐴, 𝐵, 𝐶 𝑎 = 0 for
random 𝑎 ⇒ Semantically correct (𝜙 𝑚 = 1)).
April 9, 2016
Skoltech: Locality in Coding Theory
15 of 28
PCP (4 of 4)
• 𝑚 ≡ 𝐸: 𝑉 × 𝑉 → 0,1 (edges of graph).
≡ 𝐸: 𝔽2𝑞 → 𝔽𝑞 with deg 𝐸 ≤ 𝑛 ≝ |𝑉|
• 𝜙 𝐸 = 1 ⇔ graph 3-colorable.
– ∃ 𝜒: 𝑉 → {−1,0,1} s.t. ∀𝑦, 𝑧 ∈ 𝑉, 𝐸 𝑦, 𝑧 = 1 ⇒ 𝜒 𝑦 ≠ 𝜒 𝑧
• Φ 𝜒, 𝐴, 𝐴1 , 𝐵, 𝐵1 , 𝐵2 , 𝑡 =
𝐴(𝑦) − 𝜒 𝑦 + 1 ⋅ 𝜒(𝑦) ⋅ 𝜒(𝑦) + 1
+ 𝑡 𝐴 𝑦 − 𝐴1 𝑦 ⋅ 𝑎∈V(𝑦 − 𝑎)
+ 𝑡 2 𝐵 𝑦, 𝑧 − 𝐸 𝑦, 𝑧 ⋅
+𝑡 3 𝐵 𝑦, 𝑧 − 𝐵1 𝑦, 𝑧
April 9, 2016
𝑗∈{−2,−1,1,2}
𝑎∈𝑉
𝜒 𝑦 −𝜒 𝑧 −𝑗
𝑦 − 𝑎 − 𝐵2 𝑦, 𝑧
Skoltech: Locality in Coding Theory
𝑏∈𝑉(𝑧
− 𝑏)
16 of 28
Part 3: Recent Progress on LCCs + LTCs
April 9, 2016
Skoltech: Locality in Coding Theory
17 of 28
Summary of Recent Progress
• Till 2010: locality 𝑛 = 𝑜 𝑛 ⇒ 𝑅𝑎𝑡𝑒 <
• 2016:
– LCC locality 𝑛 = 2~
⇒ ℓ 𝑛 = 2~
log 𝑛
log 𝑛
1
.
2
& 𝑅𝑎𝑡𝑒 → 1
meeting Singleton Bound
⇒ ℓ 𝑛 = 2~ log 𝑛 binary, Zyablov bound.
(locally correcting half-the-distance!)
– LTC locality 𝑛 = ~poly log 𝑛 & 𝑅𝑎𝑡𝑒 → 1
⇒ … Singleton Bound, Zyablov bound …
April 9, 2016
Skoltech: Locality in Coding Theory
18 of 28
Main References
2
• Multiplicity codes [KoppartySarafYekhanin’10]
• See also
– Lifted Codes [GuoKoppartySudan’13]
1
– Expander codes [HemenwayOstrovskyWootters’13]
• Tensor codes [Viderman ‘11] (see also [GKS’13] )
• Above + Alon-Luby composition:
3
[KoppartyMeirRon-ZewiSaraf’16a]
• A “Zig-Zag” construction with above ingredients:
[KMR-ZS’16b]
4
April 9, 2016
Skoltech: Locality in Coding Theory
19 of 28
Lifted Codes
• Codes obtained by inverting decoder:
– Recall decoder for RM codes.
– What code does it decode?
– 𝐶𝑚,𝑟,𝑞 = 𝑓: 𝔽𝑚
𝑞 → 𝔽𝑞 deg 𝑓 |𝐿 ≤ 𝑟 ∀ 𝐿}
– What we know: RM 𝑚, 𝑟, 𝑞 ⊆ 𝐶𝑚,𝑟,𝑞
• Theorem [GKS’13]: 𝛿(𝐶𝑚,𝑟,𝑞 ) ≈ 𝛿 RM 𝑚, 𝑟, 𝑞
𝑅𝑎𝑡𝑒 𝐶𝑚,𝑟,𝑞 → 1 if 𝑞 = 2𝑡 and 𝑡 → ∞
• Local decodability by construction.
• Local testability [KaufmanS’07,GuoHaramatyS’15].
April 9, 2016
Skoltech: Locality in Coding Theory
20 of 28
Why does Rate 𝐶𝑚,𝑟,𝑞 → 1?
• Example: 𝑚 = 2
• Some monomials OK!
𝑞
;
2
2𝑡 ;
– E.g. 𝑞 =
𝑟 = 𝑓 𝑥, 𝑦 = 𝑥 𝑟 𝑦 𝑟 satisfies
deg 𝑓|𝑙𝑖𝑛𝑒 ≤ 𝑟 for every line!
• 𝑥 𝑟 𝑎𝑥 + 𝑏
𝑟
= 𝑏 𝑟 𝑥 𝑟 + 𝑎𝑟 𝑥 (mod 𝑥 𝑞 − 𝑥)
• As 𝑟 → 𝑞 many more monomials
– Proof: Lucas’s lemma + simple combinatorics
April 9, 2016
Skoltech: Locality in Coding Theory
21 of 28
Multiplicity Codes
• Basic example
• Message = (coeffs. of) poly 𝑝 ∈ 𝔽𝑞 𝑥, 𝑦 .
• Encoding = Evaluations of
2
Length = 𝑛 = 𝑞 ; Alphabet =
𝜕𝑝 𝜕𝑝
𝑝, ,
𝜕𝑥 𝜕𝑦
𝔽3𝑞 ;
Rate →
• Local-decoding via lines. Locality = O
• More multiplicities ⇒ Rate → 1
• More derivatives ⇒ Locality → 𝑛𝜖
April 9, 2016
Skoltech: Locality in Coding Theory
over 𝔽2𝑞 .
2
3
𝑛
22 of 28
Multiplicity Codes - 2
•
•
2
Why does Rate → ?
3
𝜕𝑝 𝜕𝑝
Every zero of 𝑝, ,
𝜕𝑥 𝜕𝑦
≡ two zeroes of 𝑝
• Can afford to use 𝑝 of degree → 2𝑞.
• Dimension ↑ × 4 ; But encoding length ↑× 3
(Same reason that multiplicity improves radius of listdecoding in [Guruswami,S.])
April 9, 2016
Skoltech: Locality in Coding Theory
23 of 28
State-of-the-art as of 2014
• ∀𝜖, 𝛼 > 0 ∃𝛿 = 𝛿𝜖,𝛼 > 0 s.t. ∃ codes w.
– Rate ≥ 1 − 𝛼
– Distance ≥ 𝛿
– Locality = 𝑛𝜖
• Promised:
– Locality 𝑛𝑜(1)
– Singleton bound [What if you need higher
distance?]
– Zyablov bound [What if you want a binary code?]
April 9, 2016
Skoltech: Locality in Coding Theory
24 of 28
Alon-Luby Transformation
• Key ingredient in [Meir14], [Kopparty et al.’15]
Short MDS
Expander-based
Interleaving
Rate → 1
Code
Message
April 9, 2016
Encoding
Skoltech: Locality in Coding Theory
25 of 28
Alon-Luby Transformation
• Key ingredient in [Meir14], [Kopparty et al.’15]
Short MDS
Expander-based
Interleaving
Rate → 1
Code
Message
Encoding
Rate/Distance of final code ~ Rate of MDS
[Meir] Locality ~ Locality of Rate 1 code Proof = Picture
April 9, 2016
Skoltech: Locality in Coding Theory
26 of 28
Subpolynomial Locality
• Apply previous transform, with initial code of
Rate 1 − 𝑜(1) and locality 𝑛𝑜(1) !
– [e.g., multiplicity codes with 𝑚 = 𝜔(1)]
• Singleton bound
• Zyablov bound?
– Concatenation [Forney’66]
April 9, 2016
Skoltech: Locality in Coding Theory
27 of 28
Part 4: Locally Testable Codes
(after break)
April 9, 2016
Skoltech: Locality in Coding Theory
28 of 28