Bounds
Cyclic codes
Examples of codes
Introduction to coding theory II
Pierre Arnoux
Manila, July 2013
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
I
The Singleton bound for linear codes
I
For a linear [n, k, d] code :
I
k +d ≤n+1
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
I
The Hamming Bound
I
for an (n, M, d)-code with t =
P
q n ≥ M tj=0 (q − 1)j nj .
I
I
I
I
I
d−1 2
In the linear case of a [n, k, 2t + 1]-code :
P
q n ≥ q k tj=0 (q − 1)j nj .
in the binary case :
P
2n ≥ 2k tj=0 nj .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
I
I
It can
h be rewritten:
P
i
t
j n
k ≤ n − logq
.
j=0 (q − 1) j
I
and in the binary case :
h
P
i
t
n
k ≤ n − log2
.
j=0 j
I
How do these bounds compare?
I
It depends on q
I
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 2, n = 100
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 2, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 3, n = 100
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 3, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 4, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 81, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Upper bounds
Cas q = 1024, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
I
These are upper bounds.
I
It is most interesting to have lower bounds,
I
To see what can be expected.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Gilbert-Varshamov bound
I
I
There is a lower bound :
Theorem (Gilbert-Varshamov bound)
If q n−k+1 >
over Fq
Pd−1
j=0
(q − 1)j
n
j
then there exists a [n, k, d] code
I
Proof by induction on k.
I
If k = 1, the inequality becomes q n >
I
It is true for d ≤ n.
I
The code generated by (1, 1, . . . , 1) is a [n, 1, n] code which
satisfies the condition.
Pierre Arnoux
Pd−1
j=0
(q − 1)j
Introduction to coding theory II
n
j
.
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Gilbert-Varshamov bound
I
Theorem (Gilbert-Varshamov bound)
If q n−k+1 >
over Fq
Pd−1
j=0
(q − 1)j
n
j
then there exists a [n, k, d] code
I
Suppose by induction that the inequality holds and there is a
[n, k − 1, d] code C .
I
By the inequality, the spheres of radius d − 1 centered in C do
not cover the space; there is some word W which is at
distance at least d of the words of C .
I
Let C 0 be the vector space generated by C and W .
I
Let αW + c and βW + c 0 two distinct words of C 0 ; the
distance d(αW + c, βW + c 0 ) is the weight of
(α − β)W + c − c 0 .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Gilbert-Varshamov bound
I
Theorem (Gilbert-Varshamov bound)
P
j n then there exists a [n, k, d] code
If q n−k+1 > d−1
(q
−
1)
j=0
j
over Fq
I Let αW + c and βW + c 0 two distinct words of C 0 ; the
distance d(αW + c, βW + c 0 ) is the weight of
(α − β)W + c − c 0 .
I If α − β = 0, it is the weight of a nonzero element of C ,
hence at least d.
I if α − β = γ 6= 0, it is the weight of W + γ −1 (c − c 0 ), that is
the distance d(W , −γ −1 (c − c 0 ).
I But C is a linear code; hence −γ −1 (c − c 0 ) ∈ C , and this
distance is at least d by definition of W .
I Hence C 0 is a [n, k, d] code : the theorem is proved.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Gilbert-Varshamov bound
I
This bound can be rewritten:
I
I
For given q, n, d, a lower bound for the possible dimension k is
h
P
i
d−1
j n
k ≥ n + 1 − logq
(q
−
1)
j=0
j
I
How does this compare to the previous upper bounds?
I
Here are the curves.
I
red for Singleton, blue for Hamming, orange for
Gilbert-Varshamov .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
Cas q = 2, n = 100
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
Cas q = 2, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
Cas q = 3, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
Cas q = 4, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
Cas q = 81, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
Cas q = 1024, n = 1000
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Upper Bounds
Lower bound
Bounds
I
We have given upper and lower bounds.
I
Two problems remain :
I
Increase the quality of the bounds;
I
Many other bounds are known.
I
Realize effectively the codes;
I
The proof of Gilbert Varshamov is not really constructive.
I
We will add other structures to our codes.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
The binary Hamming codes (revisited)
I
Hamming
code for r = 2 has parity check matrix

0
1.
1
And generating matrix 1 1 1 .
I
It is a [3, 1, 3]-code : the triple code.
I
It is a perfect and MDS code, with rate 13 .
I
The first

1

P= 0
1
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
The binary Hamming codes
I
The second binary Hamming code for r = 3 can be obtained
by parity check matrix


1 0 0
0 1 0 


0 0 1 



1
0
1
P=


1 1 1 


1 1 0 
0 1 1
.
I

1
0
And generating matrix G = 
0
0
Pierre Arnoux
0
1
0
0
0
0
1
0
0
0
0
1
1
1
1
0
0
1
1
1
Introduction to coding theory II

1
1
.
0
1
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
The binary Hamming codes
I

1 0 0
0 1 0
G =
0 0 1
0 0 0
It has minimum
I
This is a [7, 4, 3] perfect code.
I
The transmission rate is 47 , and it corrects one error
I
This is much better than the triple code.
I
It has another interesting property :
I
It is cyclic
I
0 1 0
0 1 1
0 1 1
1 0 1
distance 3
Pierre Arnoux

1
1
.
0
1
by direct inspection.
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
The binary Hamming codes
I

1 0 0 0 1 0
0 1 0 0 1 1
G =
0 0 1 0 1 1
0 0 0 1 0 1
If (V1 , . . . , V7 ) ∈ C , then
I
Let V 1 , V 2 , V 3 , V 4 be the rows of G , and S the cyclic shift.
I
One can check that SV 1 = V 1 + V 2 , S 2 V 1 = V 2 + V 3 ...
I
We will give a more conceptual proof by adding more
structure.
I
We will make the set of messages into a ring.
I

1
1
.
0
1
(V7 , V1 , . . . , V6 ) ∈ C .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
Cyclic codes
I
We say that a code is cyclic if, for any
(V0 , V1 , . . . , Vn−1 ) ∈ C , we have (Vn−1 , V0 , V1 , . . . Vn−2 ) ∈ C .
I
We would like to find a structure which can use this property.
I
Idea : To the
(V0 , V1 , . . . , Vn−1 ), we associate the
Pword
n−1
polynomial j=0 Vj X j .
I
Multiplication by X shifts the coefficient one step to the right,
but we want to send the last coefficient in the first position.
I
Second idea: quotient by X n − 1, so that X n = 1 in the ring.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
Cyclic codes and quotient ring
I
Consider the ring Fq [X ].
I
The multiples of X n − 1 form an ideal < X n − 1 > of Fq [X ].
I
We can define the quotient ring Fq [X ]/ < X n − 1 >.
I
Each class modulo X n − 1 contains a unique polynomial of
degree < n,
I
Hence the ring Fq [X ]/ < X n − 1 > is in bijection with the set
of polynomials of degree < n .
I
We can identify Fnq with Fq [X ]/ < X n − 1 >.
P
Pn−2
j
j+1 .
In this ring, we have X . n−1
j=0 cj X = cn−1 +
j=0 cj X
I
I
Multiplication by X amounts to a cyclic permutation of
coefficients.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
Cyclic codes as ideals
I
Let C be a code, considered as a subset of
Fq [X ]/ < X n − 1 >.
I
C is cyclic if and only if X .C = C .
I
This implies that, for j ∈ N, X j .C = C .
I
By linearity, for any a ∈ Fq and j ∈ N, aX j .C = C .
I
Again by linearity, for any polynomial
P(X ) ∈ Fq (X )/ < X n − 1 >, we have P(X ).C = C .
I
Hence, any cyclic code corresponds to an ideal
I ⊂ Fq [X ]/ < X n − 1 >.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
generating polynomial
I
Let π : Fq [X ] → Fq [X ]/ < X n − 1 > be the canonical
projection.
I
π gives a bijection between the ideals of Fq [X ] containing
X n − 1 and the ideals of Fq [X ]/ < X n − 1 >.
I
Hence, C corresponds to a unique ideal Ĩ ∈ Fq [X ].
I
But Fq [X ] is a principal ring: there exists a unique monic
polynomial g (X ) such that Ĩ =< g (X ) >.
I
g is a divisor of X n − 1, since X n − 1 ∈ Ĩ .
I
Suppose that the degree of g is r . Then any element f ∈ C
can be written uniquely f (X ) = q(X )g (X ), with
deg q < n − r .
I
C is a code of dimension k = n − r .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
generating matrix of a cyclic code
I
The polynomial X i g (X ), for 0 ≤ i < n − r , give obviously a
basis of the code.
I
This give us a generating matrix.
P
If g (X ) = rj=0 gj X j , a generating matrix is

g0 g1 g2 . . . gr
0
...
 0 g0 g1 . . . gr −1 gr
...

0
0
g
.
.
.
g
g
...
G =
0
r
−2
r
−1

. . . . . . . . . . . . . . .
... ...
0
0
0 ...
0
0
...
I
Pierre Arnoux
given by

0
0

0
.
. . .
gr
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
Parity check matrix of a cyclic code
I
I
I
I
I
I
There is a unique polynomial h(X ) such that
g (X )h(X ) = X n − 1 .
Let f (X ) = q(X )g (X ) as above.
We have f (X )h(X ) = q(X )g (X )h(X ) = q(X ).(X n − 1)
Since deg q < n − r , this polynomial has all coefficients of
degree j with n − r ≤ j < n equal to 0.
P
j
This give us a parity check matrix; if h(X ) = n−r
j=0 hj X , then


hn−r
0
... 0
hn−r −1 hn−r . . . 0 


 ...

.
.
.
.
.
.
.
.
.
.
P=
 h0
h1 . . . hk 


 ...
. . . . . . . . .
0
0
...
Pierre Arnoux
h0
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
The binary Hamming codes (again)
I
Let q = 2, g (X ) = X 3 + X 2 + 1.
I
One checks that g (X ).(X 4 + X 3 + X 2 + 1) = X 7 + 1
I
The multiples of g form an ideal of

1 0
0 1
a generating matrix is G = 
0 0
0 0
After
row reduction, we recover
 :
1 0 0 0 1 0 1
0 1 0 0 1 1 1

G =
0 0 1 0 1 1 0.
0 0 0 1 0 1 1
This is Hamming code.
I
I
I
Pierre Arnoux
dimension 4.
1 1 0 0
0 1 1 0
1 0 1 1
0 1 0 1
Introduction to coding theory II

0
0
.
0
1
Bounds
Cyclic codes
Examples of codes
Hamming (bis)
Cyclic codes
Hamming code and F8
The binary Hamming codes (again)
I
Remark that the set of syndromes is isomorphic to
F2 [X ]/ < X 3 + X 2 + 1 >= F8 .
I
Compute: X , X 2 , X 3 = X 2 + 1, X 4 = X 2 + X + 1, X 5 =
X + 1, X 6 = X 2 + X , X 7 = 1.
I
Hence X is a primitive element of the multiplicative group of
F8 , and any class modulo g contains a unique element of
weight 1 : the coset leader.
I
This can be generalized.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
Hamming codes on Fq
I
Consider a maximal set of pairwise independent vectors on Frq .
I
We can find
I
It is the size of the projective space of dimension r − 1 over
Fq .
I
Build a matrix P with these vectors as row vectors.
I
This is the parity check matrix of the Hamming code of length
q r −1
q r −1
q−1 and dimension q−1 − r .
I
Any two rows of P are independent, but we can find 3
dependent rows.
I
I
q r −1
q−1
such vectors.
r
r
−1 q −1
Hence the minimum distance is 3: this is a [ qq−1
, q−1 − r , 3]
code.
This is a perfect (but not MDS) code.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
Some Hamming codes are cyclic codes
I
Fix q a power of prime, and r > 1 prime with q − 1.
I
Define n =
I
q r −1
q−1 .
P
P −1 j
r −2
j +r
Identity n = rj=0
q = (q − 1)
(r
−
1
−
j)q
j=0
shows that n is prime with q − 1.
I
Let α be a primitive nth root of unity in Fqr .
I
To get such a root, take a generator β of the multiplicative
group and define α = β q−1
I
For 1 ≤ j ≤ n − 1, j(q − 1) is not a multiple of n, so that
αj(q−1) 6= 1.
I
Hence for 1 ≤ j ≤ n − 1, αj ∈
/ Fq .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
Some Hamming codes are cyclic codes
I
Since Fqr is a vector space over Fq , we can consider all αj as
vectors in Frq .
I
Suppose that, for 0 ≤ j < k ≤ n − 1, αj and αk are
dependent.
I
Then we can find c ∈ Fq such that αk = cαj .
I
Then αk−j = c ∈ Fq , which contradicts the previous result.
I
Hence the powers of α give n independent vectors.
I
Consider the corresponding matrix.
I
With rows given by 1, α, α2 , . . . , αn−1 .
I
it is a Hamming matrix.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
Some Hamming codes are cyclic codes
I
I
Let V = (V0 , V1 , . . . , Vn−1 ) be a word of the corresponding
code.
P −1
Then V .P = 0 can be interpreted as rj=0
Vj αj = 0.
I
The code words are the polynomials in Fq [X ] which have α as
a root.
I
They are multiples of the minimal polynomial of α.
I
The corresponding code is cyclic.
I
We have proved:
I
If r > 1 is prime with q − 1, and n =
Hamming code is a cyclic code.
Pierre Arnoux
q r −1
q−1 ,
the [n, n − r , 3]
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
Remark on cyclic codes
I
C cyclic code of length n on Fq .
I
Let g be the generating polynomial of C of degree r .
I
g is uniquely defined ; it is the unique monic polynomial of
smallest degree in C .
I
g has roots in an extension Fqk = Fkq .
I
f is in the code if and only if the roots of g are roots of f .
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
BCH Codes
I
Fix q a power of prime, and m ≥ 1.
I
Define n = q m − 1 and fix d < n.
I
Let α be a primitive root of Fqm , seen as a vector space Fm
q of
dimension n over Fq .
I
The code is the set of polynomials f of degree < n such that
f (α) = f (α2 ) = . . . = f (αd−1 ) = 0 .
I
It is a cyclic code, generated by the l.c.m. g of the minimal
polynomials of the αj , 1 ≤ j < n.
I
deg(g ) ≤ (d − 1)m .
I
The code has length n, dimension at least n − (d − 1)m.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
BCH codes
I
I
I
I
I
I
Suppose that the code contains an element f of weight < d.
P
kj
f = d−1
j=1 aj X .
We have f (αi ) = 0, 1 ≤ i < d.
This family of equalities can be written:

  a  0
1
αk1
αk2
...
αkd−1
 a2  0
k
2
k
2
2
d−1
 (αk1 )2

(α )
. . . (α
) 
  

..  =  .. 
 ...

 .  .
(αk1 )d−1 (αk2 )d−1 . . . (αkd−1 )d−1
0
ad−1
.
The matrix is a Vandermonde matrix; since all the αj are
distinct, it is invertible.
We have proved that BCH codes have minimal distance at
least d.
Pierre Arnoux
Introduction to coding theory II
Bounds
Cyclic codes
Examples of codes
General Hamming codes
BCH codes
Reed-Solomon
Reed-Solomon
I
A special case of BCH codes.
I
Fix q a power of prime, and n = q − 1.
I
Fix d < n.
I
Let α be a generator of F∗q .
I
Let C be the code of polynomials f of degree < n such that
f (α) = f (α2 ) = . . . = f (αd−1 ) = 0.
Q
j
Generating polynomial g (X ) = d−1
j=1 (X − α ).
I
I
This code has length n = q − 1, dimension
n − (d − 1) = q − d, and minimal distance at least d.
I
By Singleton bound, the minimal distance is d; it is a
[q − 1, q − d, d]-code, and MDS code.
Pierre Arnoux
Introduction to coding theory II