Part 1:
Overview of Low Density
Parity Check(LDPC) codes
Low density parity check codes
R. G. Gallager, “Low-Density Parity Check Codes,” in 1962,
Simple parity-check code specified by a parity-check matrix or Tanner graph.
An ‘optimal’ LDPC can get within ~.005 db of channel capacity
Low density parity check codes
R. G. Gallager, “Low-Density Parity Check Codes,” in 1962,
Simple parity-check code specified by a parity-check matrix or Tanner graph.
An ‘optimal’ LDPC can get within ~.005 db of channel capacity
Low density parity check codes
R. G. Gallager, “Low-Density Parity Check Codes,” in 1962,
Simple parity-check code specified by a parity-check matrix or Tanner graph.
An ‘optimal’ LDPC can get within ~.005 db of channel capacity
Low density parity check codes
R. G. Gallager, “Low-Density Parity Check Codes,” in 1962,
Simple parity-check code specified by a parity-check matrix or Tanner graph.
An ‘optimal’ LDPC code can get within ~.005 db of channel capacity
Low density parity check codes
• Low density parity check
- H matrix has a large number of columns (n> 1000 or 10,000)
- Number of 1’s in H is small ( <<1%)
- H is constructed pseudorandomly subject to some constraints
Low density parity check codes
• Low density parity check
- H is constructed pseudorandomly subject to some constraints:
- fixed number of rows and columns - this fixes the rate
- randomly fill H with 1’s
- e.g. fixed number of 1’s per row/column
Encoding (Cont’d)
• Encoding in graph
Degree k
m1,1
m1,2
Degree c
m1,k-1
p1
m2,1
m2,2
m2,c-1
p2
Decoding
• Message passing decoder
– Iterative algorithm
• Amenable to highly parallelized hardware
implementation
Code design
‘Optimal’ LPDC codes can be described
by distributions (x) and (x)
Optimization
• Basic idea
– Optimize the triple ( (x), (x), p(x) )
f ( k ) ( v )  e( k )  0 ( v )
 g( k ) ( v)
x)
ud( kc )
dc  1
v
f ( k ) (u)   ( k )  0 (u)
v1( k )
(k)
dc  1
x)
 (1   ( k ) )h( k ) ( u)
f
(0)

( v)  e
(0)
(0)
(0)
f
(v)

p
(v)

g
(v)
(0)
 ( v)  g ( v) j
0
Optimization recursion
f ( k ) ( v )  e( k )  0 ( v )
 g( k ) ( v)
x)
ud( kc )
f
(k)
dc  1
v
(u)    0 (u)
(k)
v1( k )
(k)
dc  1
x)
 (1   ( k ) )h( k ) ( u)
f
(0)

( v)  e
(0)
(0)
(0)
f
(v)

p
(v)

g
(v)
(0)
j
 ( v)  g ( v)
0
Simulation Results
Simulation Results
2-IID capacity
One encoder/one decoder
Simulation Results
2-IID capacity
One encoder/one decoder
A single LDPC code designed for
one rate and then punctured for a
range of rates can
be optimal for “all rates”
Part 2:
Details of Low Density Parity
Check(LDPC) codes: binary
symmetric channels
Details: Encoding
rn

• Encoder is derived from the the parity check matrix H
• Row reduction of H into systematic form -- get G from this
• If H is sparse then with high probability G will be dense
• Not addressed here, but this continues to be a topic of great
concern
Details: Decoding
rn

n
r
 (r1,r2 ,..., rn ) can be either hard
• the channel output
or soft information
• Use the properties of the graph to decode
• Decoding will
 be done in an iterative way: iterate between
variable (bit) nodes and checks nodes
Low density parity check codes
r n  (r1,r2 ,..., rn )
Consider the MAP rule discussed in the context of

convolutional
codes
If p(x j  0 | r n )  p(x j  1| r n ) choose x j  0
If p(x j  1| r n )  p(x j  0 | r n ) choose x j  1
Low density parity check codes
rn

cn  (c1, c2 ,...,cn )
Transmit
r n  (r1,r2 ,..., rn )
Receive
Bit flipping decoder
rn

0
0
0
•••
cn  (c1, c2 ,...,cn )
0
0
1
0
•••
r n  (r1,r2 ,..., rn )
Receive
1
Motivating example
• Hard decoder: bit flipping decoder
Bit
(variable)
nodes
Check
nodes
Motivating example
• All 0’s codeword is sent
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Bit
(variable)
nodes
Check
nodes
Example 1: single error
Assume all 0’s codeword and a single error
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 1: Check node: Identify which parity checks are in error
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 1: Parity node check:
Identify which parity checks are in error
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 1: Parity node check:
Identify bits that are connected to those checks
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
For each bit node that is potentially in error identify number of unsatisfied checks for that bit node
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
For each bit node that is potentially in error identify number of unsatisfied checks for that bit node
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
For each bit node that is potentially in error identify number of unsatisfied checks for that bit node
Number of unsatisfied checks for this bit
0 2 0 1 1 1 0 0 0 0 1 0 1 1 0 0
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
For each bit node that is potentially in error identify number of unsatisfied checks for that bit node
Number of unsatisfied checks for this bit
0 2 0 1 1 1 0 0 0 0 1 0 1 1 0 0
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
For each bit node that is potentially in error identify number of unsatisfied checks for that bit node
Number of unsatisfied checks for all bits
0 2 0 1 1 1 0 0 0 0 1 0 1 1 0 0
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
Flip the bits with the most unsatisfied checks
0 2 0 1 1 1 0 0 0 0 1 0 1 1 0 0
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
Flip the bits with the most unsatisfied checks
0 2 0 1 1 1 0 0 0 0 1 0 1 1 0 0
Rec’d
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 2: Bit node check:
Flip the bits with the most unsatisfied checks
0 2 0 1 1 1 0 0 0 0 1 0 1 1 0 0
Rec’d
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 3: parity node check:
Check if all parities are satisfied
Rec’d
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 3: parity node check:
Check if all parities are satisfied
Rec’d
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 1: single error
Step 3: parity node check:
Check if all parities are satisfied
Done!
Rec’d
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 2: Double error
Iteration #1
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Example 2: Double error
Iteration #1
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Example 2: Double error
Iteration #1
2 0 1 1 0 1 1 1 0 2 1 1 0 1 1 2
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Example 2: Double error
Iteration #1
2 0 1 1 0 1 1 1 0 2 1 1 0 1 1 2
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
Example 2: Double error
Iteration #2
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
Example 2: Double error
Iteration #2
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
Example 2: Double error
Iteration #2
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
Example 2: Double error
Iteration #2
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
Example 2: Double error
Iteration #2
1 0 0 0 0 1 1 0 0 2 0 1 0 1 0 1
0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
Example 2: Double error
Iteration #2
1 0 0 0 0 1 1 0 0 2 0 1 0 1 0 1
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example 2: Double error
Iteration #2
Done
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0