Introduction
Block codes
Linear Codes
Introduction to coding theory I
Pierre Arnoux
Manila, July 2013
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
References
F.J. MacWilliams and N.J.A. Sloane The theory of error-correcting
codes (North Holland, 1977)
J.H. van Lint Introduction to coding theory (Springer GTM, 1998)
Michael Tsfasman, Serge Vladut and Dmitry Nogin Algebraic
Geometry Codes, Basic Notions (AMS 2007)
Richard Blahut Algebraic codes on Lines, Planes and Curves
(Cambridge University Press, 2008)
Henning Stichtenoth Algebraic Function Fields and Codes
(Springer GTM, 2008)
Harald Niederreiter and Chaoping Xing Algebraic geometry in
Coding Theory and Cryptography (Princeton University Press,
2009)
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Two remarks
The problem
I
Sending a message.
I
Using a finite alphabet Fq with q letters.
I
There might be some errors.
I
We want to detect the errors.
I
And if possible to correct them.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Two remarks
The idea
I
Allow only some messages.
I
Equivalently, add some redundant letters.
I
Check letters.
I
Example: repeat each letter.
I
Detects one error.
I
This is naburally used in langrages
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Two remarks
Binary codes
I
An important basic case.
I
Binary codes : q = 2, Fq = {0, 1}.
I
Common in practice.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Two remarks
The basic examples
I
Binary Repetition code (parity code) :
I
only allowed message is 00 or 11
I
This detects one error.
I
No correction.
I
Triple repetition code :
I
only allowed message is 000 or 111.
I
This detects two errors, or corrects one error (but not both).
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Two remarks
Using mathematics
I
Codes have strong constraints.
I
Contradiction (or trade-off) between efficiency and safety .
I
We impose mathematical structures to achieve best
compromise.
I
These structures are not inherent to messages.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Two remarks
On examples
I
Coding theory started around 1948.
I
Many progresses in 65 years.
I
Most of them are linked to specific codes.
I
It is important to know examples.
I
Parity codes, Hamming codes, Golay Codes,
I
Reed-Solomon codes, BCH codes...
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Block Codes
I
We suppose that all messages have length n.
I
There are q n messages.
I
block code of length n : a set C ⊂ Fqn .
I
These are the allowed messages.
I
The first two parameters of the code C :
I
The number q of symbols (q = 2: binary case)
I
The length n of the block
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Hamming distance
I
Geometric structure on Fqn : distance.
I
U = U1 . . . Un , V = V1 . . . Vn two messages.
I
Hamming distance : d(U, V ) = #{i|Ui 6= Vi }
I
It is a distance in the geometric sense :
I
It satisfies the triangle inequality.
I
One can define balls
I
B(U, r ) = {V ∈ Fqn |d(U, V ) ≤ r }
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Hamming distance
I
Example : n = 11, U = COMPUTATION , V =
COMPETITION
I
d(U, V ) = 2.
I
W =COMPETATION
I
d(U, W ) = d(V , W ) = 1
I
In that case, one can detect, not correct the error
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Minimal distance
I
Code C ⊂ Fqn .
I
The minimal distance of C is
min{d(U, V )|U, V ∈ C ; U 6= V }.
I
If C is a code of length n, cardinal M and minimal distance d,
we say that C is an (n, M, d) code.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Minimal distance and error detection
I
C (n, M, d) code.
I
Word U is received as V , with k > 0 errors
I
d(U, V ) = k
I
If k < d, V ∈
/ C.
I
The code can detect d − 1 errors
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Minimal distance and error correction
I
C code de distance minimale d.
t = d−1
2 .
I
U, V ∈ C
I
B(U, t) ∩ B(V , t) = ∅.
I
for any word, there is at most one element of C at distance t
I
The code can correct t errors
I
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
I
q = 2.
I
n = 2.
I
C = {00, 11}
I
This is a (2, 2, 2) code (parity code)
I
It can detect one error
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
01
11
00
10
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
I
q = 2.
I
n = 3.
I
C = {000, 110, 101, 011}
I
This is a (3, 4, 2) code (parity code)
I
It can detect one error
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
011
111
001
010101
110
000
100
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
I
q = 2.
I
n = 3.
I
C = {000, 111}
I
This is a (3, 2, 3) code
I
It can detect two errors
I
Or correct one error.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
011
111
001
010101
110
000
100
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Some examples
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Trivial bounds
I
Not everything is possible :
I
We must have M ≤ q n .
I
And d ≤ n.
I
If M is large, d cannot be large :
I
Sphere packing problem.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Hamming Bound
I
Let U ∈ Fqn
I
Then the number of words at distance j of U is (q − 1)j
#{V |d(U, V ) = j} = (q − 1)j nj
I
I
I
I
I
I
Hence the size of the ball of radius t :
P
#B(U, t) = tj=0 (q − 1)j nj
For C a (n, M, d)-code, with t = d−1
2
We have Hamming bound:
P
q n ≥ M tj=0 (q − 1)j nj
Pierre Arnoux
Introduction to coding theory I
n
j
.
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Perfect codes
I
If we have the equality:
P
q n = M tj=0 (q − 1)j nj
I
C is called a perfect code
I
Every word is at distanceat most t of a unique word.
I
This is optimal,
I
but rare.
I
The triple code is a (trivial) perfect code.
I
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Perfect codes
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Hamming Distance
Minimal distance of a code
Geometric bounds
Two other invariants
I
Packing radius of C :
I
Largest r s.t. if U 6= V than B(U, r ) ∩ B(V , r ) = ∅
I
Every word is at distance r of at most one word of C .
I
We have d > 2r
I
Covering radius of C :
I
Smallest r s.t. if Fqn ⊂ ∪U∈C B(U, r )
I
Every word is at distance r of at least one word of C .
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Practical problem
I
How can we detect if U ∈ C , and correct it if necessary?
I
Storing C : problem of size.
I
Recognizing C : problem of time.
I
We need more structure!
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Linear code
I
We suppose that q = p r is a prime power.
I
We consider Fq as a finite field.
I
Fqn is an n-dimensional vector space on Fq .
I
A linear code of length n is a vector subspace of Fqn
I
A linear code of length n, dimension k and minimal distance d
is called a [n, k, d]-code.
I
A [n, k, d]-code is a (n, q k , d)-code.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Generating matrix
I
A linear code C is completely given by a basis.
I
If C has dimension k, it has M = q k elements.
I
Generating matrix : matrix G with k rows and n columns
whose rows form a basis of C .
I
This is a huge reduction in storage!
I
Up to a change of columns, we can always suppose that the
generating matrix is in standard form :
I
G = (Ik H) where H is a k × (n − k) matrix.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Generating matrix
I
Example:
Parity code of length 3: generating matrix
1 0 1
0 1 1
For a matrix in standard form (Ik H):
I
The first k bits are the information bits
I
The last n − k bits are the control bits
I
I k
n
I It
is the rate of transmission of the code.
is less than 1; it should be as close to 1 as possible.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Generating matrix : encoding the message
I
Let V be a message of length k, and G a generating matrix.
I
V .G is an element of C .
I
If G is in standard form, then V .G = (V , V .H)
I
This is a simple and efficient method to generate the coded
messages.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Dual code
Pn
I
Natural scalar product on Fqn : U.V =
I
The dual code of C is the set of vectors orthogonal to C .
I
These vectors can be interpreted as linear forms on Fqn .
I
By general theorem, dim C + dim C ⊥ = n
I
If C is a [n, k]-code, C⊥ is a [n, n − k]-code.
I
Their intersection is not always 0
I
There are self-dual codes, equal to their dual :
C = {(0, 0), (1, 1)}
Pierre Arnoux
i=1 Ui Vi .
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Parity check matrix
I
Let P be the transpose of a generating matrix for C ⊥ .
I
The columns of P form a basis of C ⊥ .
I
For every V ∈ C , V .P = 0
I
This is a characterization of C .
I
P is called the parity-check matrix.
I
This is a very efficient way to check if V ∈ C .
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Parity check matrix
I
If G = (Ik H)
I
Any element of C is written V = (U, U.H) with U of length k.
−H
Define P =
(no sign if q = 2n )
In−k
−H
V .P = (U, U.H).
= −U.H + U.H = 0
In−k
P is a parity-check matrix for C .
I
I
I
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Parity check matrix : examples
I
I
I
I
I
I
The [3, 2, 2] parity code
1 0 1
Generating matrix
0 1 1
 
−1
Parity-check matrix −1
1
The [3, 1, 3] tripling code
Generating matrix 1 1 1


−1 −1
0
Parity-check matrix  1
0
1
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
A remark
I
Any subspace of dimension k of Fqn can be expressed in two
ways:
I
Parametric equation
I
Image of a linear map G : Fqk → Fqn
I
Cartesian equation
I
Kernel of a linear map P : Fqn → Fqn−k
I
This is what we have done here.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Weights
I
The weight of an element V ∈ Fqn is the number of nonzero
coordinates.
I
It is also the Hamming distance d(V , 0) to the zero vector.
I
If U, V ∈ C , then d(U, V ) = d(U − V , 0) is the weight of
U − V.
I
The minimal distance of C is the smallest weight of a nonzero
element of C .
I
A simple way to compute the minimal distance!
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Parity check matrix and minimum distance
I
Let P be the parity check matrix.
I
Any nonzero element V ∈ C gives a relation between the rows
of P, since V .P = 0.
I
The minimum weight is the smallest number of linearly
dependent rows of P.
I
This gives another simple way to compute the minimal
distance!
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
The Singleton bound
I
Let C be a [n, k, d] code, with generating matrix in standard
form.
I
Any row of G has at most 1 + (n − k) nonzero elements.
I
Hence d ≤ n − k + 1
I
We obtain the Singleton bound :
I
d + k ≤ n + 1.
I
Any code such that d + k = n + 1 is called a Maximal
Distance separable code
I
or MDS-code.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Syndrome and coset
I
Let C be a code with parity check matrix P.
I
Suppose that V is received as W = V + e, where e is an error
vector.
I
S = W .P is the syndrome of W ; it is nonzero iff W ∈
/ C.
I
S = W .P = e.P
I
Each syndrome s completely defines a translate or coset of
the code C .
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Coset leader
I
For each coset, choose a vector of minimal weight in the coset.
I
It is called coset leader
I
In a code with minimal distance d = 2t + 1, any vector e of
weight ≤ t is a coset leader.
I
Since if e 0 is another vector of weight ≤ t in the same coset,
e 0 − e is a nonzero element of weight ≤ 2t of C .
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
Syndrome decoding
I
For a received message W , compute its syndrome S = W .P
I
This defines a coset of C
I
Let e be the coset leader.
I
Decode the message as W − e
I
If C is a [n, k, d]-code, this gives the original message if the
number of errors is at most [ d−1
2 ].
I
If we have a good way to compute the coset leader, this is
efficient.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
The binary Hamming codes: basic presentation
I
Consider a binary word of length 2r − 1.
I
There are 2r − 1 possible errors.
I
With r check digits, we have 2r possible syndromes.
I
If the syndrome is not 0, a good organisation might allow to
find the error bit.
I
Define the matrix P whose rows are the binary expansions of
integers 1 . . . 2r − 1
I
This is a matrix with r columns and 2r − 1 rows.
I
By construction, the syndrome of a vector of weight 1 in
position j is the binary expansion of j.
Pierre Arnoux
Introduction to coding theory I
Introduction
Block codes
Linear Codes
Basic notions
Weights and bounds
Syndrome decoding
Hamming codes
The binary Hamming codes
I
The binary Hamming code of order r is the
[2r − 1, 2r − r − 1, 3] code given by the r × 2r − 1 parity
matrix whose rows are all possible nonzero vectors of length r .
I
Since any two rows of P are independent, but there are 3
dependent rows, the minimal distance of the code is 3.
I
The size of a ball or radius 1 is 1 + 2r − 1 = 2r .
I
There are 22
r −r −1
r
2r .22 −r −1
words in the code.
r
I
Since
= 22 −1 , the balls of radius 1 around the
vectors of the code fill the space :
I
Hamming codes are perfect codes.
Pierre Arnoux
Introduction to coding theory I