Review of modern
noise proof coding methods
D. Sc.
Valeri V. Zolotarev
The large volume of transmitting
data demands to provide
their high veracity

One of major ways for transmission error
probability decrease in noisy digital
channels is usage of noise proof coding
methods
V. Zolotarev - Review of modern coding methods
2
Principles of noise proof coding

The information is broken into blocks, for
example, binary digits, to which one the
check bits being by a function from an
information part of the transmitting data
are added.
 The relative part of initial information
characters in such enlarged block is called
as code rate R.
V. Zolotarev - Review of modern coding methods
3
The main concepts
of information theory
 Channel
capacity С  characterizes
a
maximum
mean
information quantity, which one can be
transferred to the receiver during the
period of one usage of a channel.
 С - function of a channel noise level,
i. e. of mean transmission error
probability for binary digits.
V. Zolotarev - Review of modern coding methods
4
The main limitation
in information theory for coding

The condition should be always satisfied:
R<C !


In this case there are coders, which one can
ensure a digital transmission with an arbitrary
small probability of an error, if the block length
will be great enough.

.
V. Zolotarev - Review of modern coding methods
5
How to fulfill the indicated condition
in communication engineering?
Is it difficult or not?

1. The introducing

 of redundancy
conforming to a given
value of code rate R is
very simply.

2. For given error
probabilities of
transmitting binary
digits in Gaussian
channel its capacity
3. So R<C - C also is easily
understandable for calculated

the specialists
condition
V. Zolotarev - Review of modern coding methods
6
The elementary encoder for a block
code with 2 correcting errors!
It is way to form redundancy (code rate):
V. Zolotarev - Review of modern coding methods
R=1/2
7
Whenever possible - it is else easier!!!
An example of the encoder for a convolutional
code with the same code rate R=1/2.
V. Zolotarev - Review of modern coding methods
8
Limit possibilities of coding
Interconnection between channel capacity C
and computational rate R1
for BSC with channel error probability Po
1,00
Capacity C and
computational rate R1
0,90
0,80
0,70
0,60
C
0,50
0,40
0,30
R1
0,20
0,10
0,00
0,000
0,050
0,100
0,150
0,200
0,250
0,300
0,350
0,400
0,450
0,500
Po - channel error probability
V. Zolotarev - Review of modern coding methods
9
What quality of codes is main?

- Code
distance d !
 It
determines minimum number of
symbol positions, in which the code
words
(permissible
data)
are
different.
 For example, in parity checking codes
all permissible words - are only ones
with an even number of «ones».
So its code distance is d=2 !
V. Zolotarev - Review of modern coding methods
10
What for it is necessary to take
codes with large d values?

The more d, then the greater number of
errors appeared in the transmitted code
block by, can be corrected.

In this case portion of blocks grows,
which one after decoding can be error-free.

And then what maximum d values are
possible?
V. Zolotarev - Review of modern coding methods
11
Limits of correcting properties
for two code classes
Code rate R
Interconnection between C and R1
for different values of d/n ratio
1,00
0,90
There are no codes !
0,80
0,70
0,60
0,50
d free
d min
0,40
0,30
Codes exist !
0,20
0,10
R block
0,00
0,000 0,050 0,100 0,150 0,200 0,250 0,300 0,350 0,400 0,450 0,500
R block
R conv
Code distance d to code length n ratio: d/n
V. Zolotarev - Review of modern coding methods
12
One of main questions:
// What may the code length be?
 As
at R<C the theory
guarantees good outcomes
of the coded data transmission,
let's see, as far as lengthy
should be the code block in
different cases.
V. Zolotarev - Review of modern coding methods
13
The lower estimations of error probabilities of optimum block
code decoding with R=1/2 in BSC. Even the codes with length
n=1000 are ineffective at channel error probability Po > 0.07.
But the theory affirms, that it is possible to work successfully
at Po < 0.11, in accordance to main condition C > 1/2.
And it is true for total searching methods!
V. Zolotarev - Review of modern coding methods
14
The main «jokes» of the Nature

1. Almost all codes are "good". If decoder is
optimal then resulting error probability will
be close to the best ones.
 2. Almost all codes can be decoded only by
total searching methods. For a code length
n=1000 exhaustive search at R=1/2 requires
to look through 2500(!!!) versions of the
possible code words. But
it exceeds
number of atoms in the Universe!
 So
what must we do?
PROBLEM!!!
V. Zolotarev - Review of modern coding methods
15
The Main Problem
of the noise proof coding theory



1. To find and to investigate simple non
exhaustive search decoding methods in noisy
channel.
2. To ensure such decoding quality with these
methods, that they were more close to efficiency
of optimal procedures.
3. To remember needs and conditions of coding
usage in communication systems.
V. Zolotarev - Review of modern coding methods
16
Threshold decoders: everything is simple!
Let's pay attention: It is truly the elementary
errors correcting scheme!
V. Zolotarev - Review of modern coding methods
17
But
TD efficiency - is paltry!
It is extremely far from Ро=0.11.
V. Zolotarev - Review of modern coding methods
18
Multithreshold decoders (MTD)
for Gaussian channels
They are designed and deeply investigated
during last 30 years multithreshold
decoders very poorly distinguished from
customary extremely simple classic
threshold
procedures,
offered
by
J.L.Massey.
The main property MTD
- at each change of symbols new decoder
decision becomes more close to the
optimum one!
V. Zolotarev - Review of modern coding methods
19
The main consequence
from MTD properties




If MTD for a long time changes characters
of the received data, it can achieve the
solution of the optimum decoder (OD) at
linear complexity of decoding.
Usually solutions OD - are the outcomes of
exponential growing with code length
exhaustive search .....,
but here we get linear complexity?!!
V. Zolotarev - Review of modern coding methods
20
It is multithreshold decoder!!!
It is a view of
block MTD. The
new register
contains a
û
difference between
the MTD solutions
and values of
information bits of v̂
a channel.
0
1
2
3
4
5
6
7
8
9 10 11 12
0
1
2
3
4
5
6
7
8
9 10 11 12
12 11 10 9
8
7
6
5
4
3
2
1
0
T
Why?
Декодер блокового СОК с R = 1/2, d = 5 и n = 26
V. Zolotarev - Review of modern coding methods
21
This is convolutional MTD
with R=1/2, d=5 and 3 iterations
û
vˆ
0
1
2
3
4
5
6
0
1
2
3
4
5
6
0
1
2
3
4
5
6
0
1
2
3
4
5
6
0
1
2
3
4
5
6
0
1
2
3
4
5
6
6
5
4
3
2
1
0
6
5
4
3
2
1
0
6
5
4
3
2
1
0
T1
T2
T3
Рис. 1. Многопороговый декодер сверточного СОК с R=1/2, d=5 и nA=14
V. Zolotarev - Review of modern coding methods
22
The resolved MTD problems

1. The very complicated problem of
an error propagation effect
(EP) estimation in TD is completely resolved
 2. The codes with minimum EP were successfully constructed !
 3. Four generations of MTD coding equipment have been built.
 4. Most important: the minimum possible complexity of
decoding, referenced for customary TD is saved.
 5. Consequent. MTD works at high noise levels almost as OD.
 6. TOTAL.
Creation of the effective
decoder near channel capacity C

- generally resolved problem.
V. Zolotarev - Review of modern coding methods
23
The estimations of convolutional code
error probability decoding for Viterbi
algorithm and MTD in BSC with R=1/2.
Decoder's error probabilities in BSC
MTD1
MTD2
MTD3
av7h
av11h
av15h
n1000
n=10000
n=3000
G4
G6
G8
Bit error probability, Pb(e)
1,0E-02
1,0E-03
1,0E-04
4
1,0E-05
СBSC
1,0E-08
-1.5 -1.0
G=4
1
n1000
n10000-n3000
G=8
0.5
0.0 0.5 1.0 1.5
1,0E-09
0,1170
7
11
C
C
1,0E-06
1,0E-07
15
0,1038
0,0909
0,0787
0,0671
0,0563
0,0464
3
2
G=6
2.0
2.0
2.5
3.0 Es/No
0,0375
0,0297
0,0229
Po
Signal-to-noise ratio in a Guassian channel, dB
and channel probability in BSC
V. Zolotarev - Review of modern coding methods
24
And what is necessary for
communication engineering?



“The energy decrease in communication
channel at 1 dB gives an economic efficiency
$1’000’000 ,” - E.R.Berlecamp, IEEE, 1980,
vol.68, №5.
Now at enormous growth of communication
network cost the price of signal power
decrease has increased (!!!) multiply.
But how to fasten probabilistic channel
parameters to its signal energy?
V. Zolotarev - Review of modern coding methods
25
The coding considerably reduces signal
power in transmission channel!

The value of a decrease is called code gain (CG)
G = 10*Lg(R*d) dB


The signalmen for a long time know how to
change the receiver for increase code gain.

And where are limits of signal power decrease?

V. Zolotarev - Review of modern coding methods
26
The “soft”
modem estimating
reliability of a signal reception
instead of "hard", which one only
makes a decision about value of
received bit, allows to diminish
signal power approximately at 2 dB.
Distribution of voltage output of a binary signal in the modem
Распределение выходного напряжения двоичного сигнала в модеме
0,18
0,16
1
0
0,14
« - »
0,1
Ноль
0,08
Один
« + »
Значения
0,12
0,06
0,04
0,02
№
п/п
0
1
2
3
4
5
6
7
0
8
9
10
11
12
13
14
15
V. Zolotarev
- Review of modern coding methods
Номера областей
16
17
27
The minimally possible ratio of energy per
bit of the transmitted information to a noise
power density Eb/No in binary channel for
different code rate R can be submitted for
“hard” and “soft” modems so:
7
6
E b /N 0 , dB
5
4
3
'hard' М=2
2
1
"soft" М=16
0
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1
-1
V. Zolotarev - Review of modern coding methods
R -
code rate
28
Error probabilities of the main base decoding algorithms
in Gaussian channel with "soft" demodulation
av7
av11
av15
м11
m7
Bit error rate, Pb(e)
1,0E-02
11
1,0E-03
1,0E-04
1,0E-05
C
C
м7
15
1,0E-06
m11
7
G=4
m9
G=6
-1.5 -1.0 -0.5 0.0 0.5 1.0
1.5 2.0
1,0E-07
G=8
m9
э4
э6
э8
2.5 3.0 Es/No
1,0E-08
0,117 0,104 0,091 0,079 0,067 0,056 0,046 0,038 0,030 0,023
Po
Signal to noise ratio in Gausssian channel, dB and Po
V. Zolotarev - Review of modern coding methods
29
Concatenation
it’s
the
best!
In this case the coding implements two and more codes, which ones in the
receiver are decoded in the return order and at definite interplay of decoders.
On the chart - best known outcomes on efficiency in Gaussian channel:
Viterbi algorithm (VAk), MTD usual and cascaded (MTDK), VA+RS-code,
best of turbo (T1 and T2), and woven code (W1) too.
Decoder bit error probability
as function of bit energy to noise power density ratio
1,0E-02
VA20
VA15
Bit error rate
1,0E-03
VA-RS
VA7
T1
1,0E-04
W1
VA11
MTDK
1,0E-05
MTD
T2
C=1/2
1,0E-06
0,0
0,5
1,0
1,5
3,0
V. Zolotarev
- Review 2,0
of modern 2,5
coding methods
Eb/No, dB
3,5
304,0
BUT! MTD in 100 times more simple!!!
Decoder operation number comparision
100000
Operation number
10000
1000
MTD-S
MTD-a
MTD-b
turbo
100
10
1
0
0,25
0,5
0,75
1
1,25
1,5
1,75
2
2,25
2,5
V. Zolotarev - Review of modern coding methods
Channel bit energy/noise density ratio, Eb/No, dB
31
What shall we use ?
- Most simple and effective methods !!!
History of achivements - increase in code gain
10
9
Code gain, dB
8
7
Turbo
6
MTD
5
4
3
TD
MTD-K
CC: АВ+RS
VA
CC: turbo?
MTD?
…….??
2
1
0
1965
1970
1975
1980
1985
1990
1995
2000
2005
2010
Years
V. Zolotarev - Review of modern coding methods
32
E-mail: [email protected]
Work tel. +7 - 095-333-23-56,
+7 - 095-261-54-44
моb.: +7-916-518-86-28
www.mtdbest.iki.rssi.ru
05.09.2003
V. Zolotarev - Review of modern coding methods
33