Improved Sum-Min Decoding for Irregular LDPC Codes
Gottfried Lechner & Jossy Sayir
Telecommunications Research Center Vienna (ftw.), Donaucitystr. 1, 1220 Vienna, Austria, {lechner|sayir}@ftw.at
Abstract
An analytical approach to determining the optimal post-processing function for the minimum operation in the SumMin algorithm, previously derived for regular Low-Density Parity-Check (LDPC) codes, is extended to irregular
LDPC codes. The optimal post-processing expression for the irregular case varies from one check node to another
as well as from one iteration to the next. For practical use, it is necessary to approximate this optimal function.
Unlike for the regular case, where one could use a unique post-processing function throughout the decoding process
without a loss of bit error performance, for irregular codes it is crucial to vary the post-processing from one iteration
to the next in order to achieve good performance. Using this approach, a bit error performance of 0.15dB from the
Sum-Product decoder was demonstrated, corresponding to a 1dB improvement compared to the Sum-Min algorithm
without post-processing.
1
Introduction
The Sum-Product Algorithm for decoding LDPC codes
is hard to implement in practice, since it requires
nonlinear functions and multiplications. The algorithm
can be simplified by approximating the calculation at
the check nodes using a simple minimum operation, resulting in the well-known Sum-Min algorithm (SMA).
While the Sum-Min algorithm is less complex to implement, it requires approximately an additional 0.5dB
of signal-to-noise ratio Eb /N0 to achieve the same bit
error rate as the Sum-Product algorithm, when using a
regular LDPC code for transmission over an additive
white Gaussian noise (AWGN) channel with binary
input. For irregular codes, the loss in performance can
be up to 1.0dB.
It has been observed by several authors that the
performance of the Sum-Min Algorithm can be improved by “post-processing” the minimum. For example, the minimum can be multiplied by a scaling
factor, or offset by a constant term. These approaches,
known as normalized belief-propagation, offset beliefpropagation [1], [2] or applying an extrinsic scaling
factor [3], have been developed heuristically. The optimal parameter, whether scaling factor or offset, is determined by simulation or by density evolution. In [4], we
introduced an analytical framework for determining the
optimal post-processing function, and found that it is
neither a linear (i.e., a scaling factor) nor an affine (i.e.,
an offset) function but a general S-shaped non-linear
function. This optimal post-processing function can be
approximated by a linear or affine function, in each
case resulting in the same values obtained heuristically
in the papers cited above.
This work was supported by the Kplus program.
In the present paper, we extend our analytical approach and derive the optimal post-processing function
for irregular LDPC codes. Unlike for regular codes,
where it was possible to implement the optimal postprocessing function efficiently using a look-up table,
for irregular codes the optimal post-processing function
is different for every node and for every iteration.
Therefore, it is imperative to find easily implementable
approximations of the optimal post-processing function.
We begin in Section 2 by reviewing the approach derived in [4] and introduce our analytical framework for
optimal post-processing. In Section 3, we present the
optimal post-processing function for irregular codes and
discuss possible simplifications. Finally, in Section 4,
we show simulation results and discuss the cost and
benefit of the approximation.
2
Post-Processing
In the Sum-Product algorithm, the message computed
for each outgoing edge of a check node is an aposteriori probability distribution (i.e., a log-likelihood
ratio) of a code digit, given the incoming messages to
the check node, which are interpreted as independent
observations of the other code digits involved in the corresponding parity-check equation. This interpretation is
asymptotically accurate in the codeword length when
using random interleavers, a fact often referred to as
the independence assumption.
When the check-node operation is replaced by a minimization, the resulting message sent along an outgoing
edge is not a true log-likelihood ratio, even though
its value may be closely correlated to the true loglikelihood ratio. In [4], we demonstrated that most of
the performance loss of the Sum-Min algorithm is due
to the misinterpretation of the check node outputs by
and φ+ (ζ), φ− (ζ) are defined as
Z +∞
p(y)dy
φ+ (ζ) =
the variable nodes. The EXIT curve for a check node in
the Sum-Min algorithm differs only very slightly from
the corresponding EXIT curve for the Sum-Product
algorithm, whereas the actual performance trajectory
of a simulated Sum-Min decoder was shown to miss
the EXIT curves by a wide margin. This signals that
the minimum contains almost all of the information
originally contained in the input messages to the check
nodes, but that the information is in a form that is not
suitable for use by the variable nodes.
EXIT functions plot the mutual information between
the outgoing message and the code digit to be estimated. As such, they are invariant under bijective
mappings of the outgoing messages. In other words,
the EXIT chart is not fooled by the misrepresentation
of information in the minimum and predicts that there
exists a one-to-one transformation of the minimum that
would allow the Sum-Min algorithm to perform almost
as well as the Sum-Product algorithm. This one-to-one
transformation of the minimum is what we call postprocessing.
The aim of post-processing is to convert the information in the minimum into a form that will be interpreted
correctly by the variable nodes. The variable nodes
expect the messages from the check nodes to contain
log-likelihood ratios. Therefore, we need to convert the
minimum to a log-likelihood ratio. We can do this
by reinterpreting the minimum as an observation of
the code digit to be estimated, and computing the aposteriori probability distribution (i.e., log-likelihood
ratio) given the minimum. If Xn is the code digit to
be estimated and LSMA
mn is the minimum computed by
the Sum-Min algorithm without post-processing, then
we want to calculate
p Xn = 0|LSMA
mn
Lmn = log
,
(1)
p (Xn = 1|LSMA
mn )
+|ζ|
φ− (ζ) =
Ψ+ (ζ; dc , p(·))
,
Ψ− (ζ; dc , p(·))
−|ζ|
p(y)dy.
−∞
The function depends on the check node degree dc and
on the probability density function p(.) of the incoming
messages. If we assume a Gaussian distribution for
the incoming messages, the expression depends on
the variance of the Gaussian, or, equivalently, on the
a-priori mutual information at each iteration. It was
shown in [4] that, while the optimal function does
vary from iteration to iteration, it is sufficient to select
the post-processing function at the narrowest point
in the EXIT chart to guarantee the optimal bit error
performance of the decoder. Varying the function from
one iteration to the next would only serve to reduce the
number of iterations, but not to improve the bit error
performance. An example of a post-processing function
for a regular (3,6) code is shown in Figure 1, along
with its approximations by a scaling factor or an offset
as in [1], [2], [3]. It is important to note that for the
regular case, both the linear and offset approximations
yield almost optimal performance and there is not much
to be gained in implementing the optimal non-linear
post-processing function. On the other hand, since it
is sufficient to use a unique post-processing function
throughout the iterations without loss in bit error performance, the optimal function can be implemented
as a look-up table without a significant increase in
complexity as compared to its approximations.
5
4
and transmit this log-likelihood ratio instead of the
minimum on the outgoing edge of the check node.
This expression, when evaluated analytically, gives a
one-to-one mapping of the minimum to a log likelihood
ratio. We consider this mapping to be the optimal
post-processing operation, meaning that the resulting
decoding performance with post-processing is exactly
as predicted by the EXIT curves. In other words the
decoder makes full use of the information contained
in the minimum. The resulting expression for regular
LDPC codes, which was derived in [4, Equation 3], is
Lmn = f (ζ; dc , p(·)) = log
Z
3
2
L
1
0
−1
−2
−3
(2)
−4
where
Ψ+ (ζ; dc , p(·))
Ψ− (ζ; dc , p(·))
= [p(ζ) + p(−ζ)] [φ+ (ζ) + φ− (ζ)]
dc −2
+ [p(ζ) − p(−ζ)] [φ+ (ζ) − φ− (ζ)]
dc −2
= [p(ζ) + p(−ζ)] [φ+ (ζ)
−5
−5
−4
−3
−2
−1
0
1
2
3
4
5
SMA
L
Fig. 1. Post-processing function L = f (LSMA ; dc = 6, IAc =
0.5)
(solid), linear approximation (dashed) and offset approximation
d −2
+ φ− (ζ)] c
(dashed-dotted).
− [p(ζ) − p(−ζ)] [φ+ (ζ) − φ− (ζ)]
dc −2
,
3
Extension to irregular codes
Irregular LDPC codes can be designed to achieve a
target bit error probability at rates much closer to
capacity than regular codes, provided that the code
length is large and that decoding is performed using the
Sum-Product algorithm. However, irregular codes are
more sensitive to the effects of replacing the check-node
operation by a minimum, so much so that the loss of
performance when using the Sum-Min approximation
outweighs the benefits of using irregular codes.
Again, as for regular codes, the EXIT chart of an
irregular code does not predict a significant loss of
performance for the Sum-Min approximation. Indeed,
since irregular codes are often designed to be checkregular (i.e., the degree of all check nodes is the same),
the EXIT functions for the check nodes for both algorithms are identical to the curves presented in [4] for
regular codes, and the irregular codes only distinguish
themselves by a different variable node EXIT function.
This indicates that there is again a one-to-one mapping
that can be applied to the minimum such that the
performance loss of the Sum-Min approximation is
negligeable, as predicted by the EXIT chart.
So far the good news: for irregular as for regular
LDPC codes, post-processing can help the Sum-Min
algorithm achieve a bit-error performance only slightly
worse than for the Sum-Product algorithm. There appears to be no difference in theory between the irregular
and the regular case, except that we stand to gain much
more by applying post-processing for irregular LDPC
codes. Now for the bad news: as we will see, there are a
two fundamental differences between the optimal postprocessing functions in the regular and in the irregular
cases, so that optimal post-processing is not applicable
in practice for irregular codes, and we will be forced
to use approximations of the optimal post-processing
functions.
The two fundamental differences are as follows:
• the model we used to derive the post-processing
function for regular LDPC codes, whereby incoming messages were all assumed to have the
same density, is not valid anymore. In the irregular
case, incoming messages to a check node originate
at variable nodes of different degrees, so that
each message will potentially have a different
probability density function. If we use a Gaussian
approximation for the density, ideally, we should
assume a different variance for each incoming
message according to which type of variable node
it originates in;
• the argument used in the regular case, whereby
it is possible to use one post-processing function
throughout the iterations without a loss in bit error performance, by selecting the post-processing
function corresponding to the narrowest point in
the EXIT chart, does not apply to irregular LDPC
codes. Indeed, irregular LDPC codes are designed
so that the EXIT functions match each other as
closely as possible, resulting in a long narrow
tunnel between the EXIT curves instead of a single
narrowest position. Therefore, it is crucial to apply post-processing optimally (or near-optimally)
throughout the iterations, because the decoding
algorithm is liable to get stuck almost at any point
in the iterative decoding process.
We will now proceed to address each of these two
differences between the regular and the irregular case.
3.1 Optimal Post-Processing for Unequal
Message Densities
The optimal post-processing function for regular LDPC
codes, stated in Equation 2 and derived in [4], is valid
for any message density function, but it specifically assumes that all incoming messages to a check node have
the same density. For irregular LDPC codes, this is not
a valid assumption anymore. All incoming messages
to a check node may originate in variable nodes of
different degrees and posess different message density
functions. If we wish to determine the a-posteriori loglikelihood ratio Lmn given the minimum reinterpreted
as an observation, as specified in Equation 1, for a
check node with unequal incoming message densities,
we obtain the same expression as in Equation 2, but
with a modified expression for the functions Ψ+ and
Ψ− . Let p(·) be the vector of the probability densitiy
functions of all incoming messages to a check node,
and pi (·) be the density function of the i-th incoming
message, then we obtain
Ψ± (ζ; dc , p(·)) =

dX
dY
c −1
c −1
[pi (ζ) + pi (−ζ)]
[φi+ (ζ) + φi− (ζ)]
i=1
± [pi (ζ) − pi (−ζ)]
j=1;j6=i
dY
c −1
j=1;j6=i

[φi+ (ζ) − φi− (ζ)]
and the functions φi+ and φi− are the corresponding
integrals as defined for regular codes.
Calculating the exact optimal post-processing function for a check node with unequal input message
density functions, even assuming Gaussian densities, is
prohibitively complicated. It includes a multiplication
of every combination of cross-products of distinct densities. In addition, it requires us to be able to determine
the density (or the variance if we assume a Gaussian
density) individually for each incoming message in
function of the degree of the variable node that it
originates from. Finally, this operation is likely to be
different for every check node, as the composition
of degrees of variable nodes that each check node is
connected to will not be the same for every check node.
We conclude that, although knowing the optimal
post-processing function may help us design practical
post-processing strategies to improve the performance
of the Sum-Min algorithm, it is not practical to implement optimal post-processing as is. This was different
for regular codes, where all incoming message densities
are the same and the expression above simplifies, so that
we can compute the optimal post-processing function
once and for all off-line and compute a look-up table
that can easily be used in practice.
Example 1: Assume an irregular LDPC code with
variable node degree distribution λ(x) = 0.25x +
0.75x9 and check node degree dc = 10. If we compute
the post-processing function for one output of the check
node, g inputs are connected to a variable node of
degree 2, and dc − 1 − g inputs are connected to a
degree 10 variable node. The post-processing functions
assuming a Gaussian message distribution are shown
in Figure 2, in function of the parameter g and for
IAc = 0.5 .
5
4
3
2
g=0
1
L
g=9
0
every check node, but depends on the variance of
the incoming messages, or, equivalently, on the apriori mutual information. Therefore, it is necessary
to compute a new post-processing function for every
iteration. The argument used for regular LDPC codes,
that the decoding trajectory only needs to attain the
EXIT curves at the narrowest point in the EXIT chart
if bit error performance is all we are interested in,
does not extend to irregular LDPC codes. For irregular
codes that are designed by optimizing the threshold,
the EXIT curves are separated by a narrow tunnel, and
failure to touch the EXIT curves at any point in that
tunnel would result in the decoder getting stuck and the
threshold would thus be affected. Therefore, whether
we use non-linear post-processing functions or linear
or offset approximations, it is imperative that we vary
the post-processing from one iteration to the next.
For the linear or offset approximation, this results in a
sequence of scaling factors or offset terms, respectively.
This sequence can be found by predicting the trajectory
in the EXIT chart and computing a correction term for
every iteration. The correction term can be predicted
by simulation or by density evolution as in [1], [2],
[3], or by direct approximation of the non-linear postprocessing function found by our method, using the
expressions
Z ∞h
f (LSMA ; dc , IAc ) −
α(dc , IAc ) = argmin
α0
−1
0
αL
−2
−4
Fig. 2.
i2
· q(LSMA ; IAc )dLSMA .
for the optimal linear factor, or
Z ∞h
β(dc , IAc ) = argmin
f (LSMA ; dc , IAc ) −
−3
−5
−5
−∞
SMA
β0
−4
−3
−2
−1
0
LSMA
1
2
3
4
5
Post-processing function for Example 1.
For practical use, we are constrained to compute
an average incoming message density and calculate
the optimal post-processing function using the same
expression as for regular codes. It is not possible to
measure the loss of performance due to this simplification, because we cannot even implement optimal postprocessing in an offline simulation due to its prohibitive
complexity. However, it is important to keep in mind
that any failure to achieve a performance predicted by
EXIT analysis of Sum-Min decoding of irregular LDPC
codes may be due in part to this simplification.
3.2 Varying the Post-Processing Function
After simplifying optimal post-processing by assuming
equal message densities and further simplifying by
assuming Gaussian message densities, if we restrict
ourselves to check-regular LDPC codes, we are left
with a post-processing function that is identical for
sgn
−∞
LSMA
mn
· max LSMA
mn − β, 0
q(LSMA ; IAc )dLSMA .
i2
·
for the optimal offset term, where f (.) is the nonlinear post-processing function and q(.) is the density
function of the minimum before post-processing. The
two approximations of the non-linear post-processing
functions proposed here are weighted minimum mean
squared error approximations. Post-processing is then
performed using
Lmn ≈ α(dc , IAc )LSMA
mn
for linear post-processing, and
Lmn ≈ sgn LSMA
· max |LSMA
mn
mn | − β, 0 .
(3)
(4)
for affine post-processing.
Note that we opted for a multiplicative linear factor,
whereas in [1], [4], the minimum is divided by a
constant factor. Apart from this distinction, we obtain
the same optimal factor or offset as in the papers cited
above, using our method of approximating the optimal
non-linear post-processing function.
1
0.9
0.8
0.7
α
Since the minimum computed in the Sum-Min algorithm becomes close to the check node outputs
computed by the Sum-Product algorithm for large messages, i.e. for large IAc , it can be observed that the
values for α and β go to 1.0 or 0.0, respectively, as the
number of iterations grows. This is consistent with the
observations made in [5], where the authors proposed
to set the offset to 0.0 after performing a fixed number
of iterations, i.e. at high IAc .
0.6
0.5
1
variable nodes
check nodes
0.4
0.9
0
0.8
0.7
Fig. 4.
0.6
10
20
30
40
50
iteration
60
70
90
100
Sequence of scaling factors for the irregular code.
0
10
SPA
SMA
scaled SMA
Ac
Ev
I ,I
80
0.5
−1
10
0.4
0.3
−2
bit error rate
10
0.2
0.1
−3
10
0
0
0.1
0.2
0.3
0.4
0.5
IAv, IEc
0.6
0.7
0.8
0.9
1
−4
10
Fig. 3.
EXIT chart of the irregular LDPC code.
0.5
1
1.5
Eb/N0 [dB]
4
Simulation results
To verify our derivations, we performed bit error rate
simulations of regular and irregular LDPC codes with
the Sum-Product, the Sum-Min, and the Sum-Min
algorithm with linear post-processing. Both codes have
a rate of R = 0.5. The regular code has dv = 3 and
dc = 6, and the irregular LDPC code has check node
degree dc = 9 and a variable node degree distribution
(edge perspective) as shown in the following table:
degree
2
3
5
6
7
8
9
10
20
30
λ
0.212364
0.198528
0.008380
0.074687
0.014241
0.166518
0.009124
0.020019
0.000251
0.295888
The EXIT chart of the irregular code is shown in
Figure 3. The block length was set to N = 105 in
order to avoid effects that are caused by cycles in
the decoding graph. The code was constructed using
the progressive edge growth algorithm [6], [7]. All
Fig. 5.
Bit error rates for the regular LDPC code.
simulations were performed using binary phase shift
keying over an AWGN channel, and the decoder was
allowed to perform up to 100 iterations.
Simulations for the regular LDPC code are shown
in Figure 5. The Sum-Min algorithm with linear postprocessing uses a fixed scaling factor of α = 0.80
for every iteration. It can be seen that almost all of
the performance loss can be recovered, and the gap to
the Sum-Product algorithm is approximately 0.05dB.
Note that the thresholds computed via EXIT charts
for the Sum-Product algorithm and for the Sum-Min
algorithms are 1.13 dB and 1.19 dB, respectively, so
that any further improvement of the Sum-Min algorithm by using the optimal non-linear post-processing
function instead of linear post-processing is likely to be
negligeable.
For the irregular LDPC code, simulation results are
shown in Figure 6. The improved Sum-Min algorithm
uses a sequence of scaling factors, shown in Figure 4.
We observe that most of the loss in performance of the
Sum-Min algorithm is recovered using variable linear
post-processing, and the resulting gap to the SumProduct algorithm is approximately 0.15dB. This is in
2
0
10
SPA
SMA
variable scaling
fixed scaling α=0.66
fixed scaling α=0.72
−1
10
−2
bit error rate
10
−3
10
−4
10
0.5
1
1.5
Eb/N0 [dB]
Fig. 6.
Bit error rates for the irregular LDPC code.
stark contrast to the Sum-Min algorithm without postprocessing, where a quick comparison of Figures 5
and 6 show that the peformance for the irregular code is
even worse than the performance for the regular code,
despite the much better threshold of the irregular code.
This last comparison shows how crucial post-processing
can be if one wishes to use irregular LDPC codes in
practice.
In order to show the importance of varying the
scaling factor throughout the iterations, we compared
the performance with a decoder that uses a constant
scaling factor. The results are shown in Figure 6 for
α = 0.66 and α = 0.72. It can be seen that for the
lower value of α the waterfall region is close to that of
variable scaling. However, the scaling is far to small
for high values of IAc and the decoder is not able
to converge to low bit error rates, resulting in a high
error floor. For the higher value of α this error floor is
lower but the price to pay is a larger threshold. Using
varying values of α combines both good threshold and
no observable error floor.
5
Conclusion
We discussed how the Sum-Min algorithm for decoding
LDPC codes can be improved by means of postprocessing, by re-interpreting the minimum as an observation and computing the a-posteriori log-likelihood
ratio of the code digits given this observation. We stated
the optimal post-processing expression for irregular
LDPC codes and concluded that, unlike for regular
codes, optimal post-processing is not practically feasible for irregular codes. We opted for simplifying the
problem by assuming equal densities for all incoming
messages to a check node, but showed that it would
be over-simplifying to also assume the same density
at every iteration, as we had done in the regular case.
We proceeded to implement linear post-processing with
a variable scaling factor and were able to measure
a decoding performance for the Sum-Min algorithm
2
within 0.15dB of the performance for the Sum-Product
algorithm.
Several methods have been proposed recently to
improve the performance of the Sum-Min algorithm
for irregular codes, notably by Fossorier in [8]. What
the present paper shows is that optimal post-processing
is not practical. Therefore, any engineering technique
that approaches optimal post-processing using practical
tools is a valid candidate for improving the Sum-Min algorithm in practice. In particular, the method proposed
in [8], that combines linear post and pre-processing
of the minimum, can be a valid approach, as is our
approach of using variable linear post-processing factors. This despite the fact that in theory, it is sufficient
to use post-processing to achieve optimal performance,
and pre-processing would not be necessary if one were
able to implement optimal post-processing.
References
[1] Jinghu Chen and Marc P. C. Fossorier, “Near optimum universal
belief propagation based decoding of low-density parity check
codes,” IEEE Transactions on Communications, vol. 50, no. 3,
pp. 406–414, March 2002.
[2] Jinghu Chen and Marc P. C. Fossorier, “Density evolution for
two improved BP-based decoding algorithms of LDPC codes,”
IEEE Communications Letters, vol. 6, no. 5, pp. 208–210, May
2002.
[3] Kienle F. and Wehn N., “Joint graph-decoder design of
IRA codes on scalable architectures,” in Proc. ICASSP 2004,
IEEE International Conference on Acoustics, Speech and Signal
Processing, Montreal, Canada, 2004.
[4] G. Lechner and J. Sayir, “Improved sum-min decoding of
LDPC codes,” in Proc. ISITA 2004, International Symposium
on Information Theory and its Applications, Parma, Italy, 2004.
[5] J. Chen, R.M. Tanner, C. Jones, and Y. Li, “Improved min-sum
decoding algorithms for irregular LDPC codes,” in Proc. 2005
IEEE International Symposium on Information Theory, Adelaide,
Australia, 2005, pp. 449–453.
[6] Xiao-Yu Hu, E. Eleftheriou, and D.M. Arnold, “Regular and
irregular progressive edge-growth tanner graphs,” IEEE Transactions on Information Theory, vol. 51, no. 1, pp. 386–398, Jan
2005.
[7] “Software for PEG code construction,” available at
http://www.inference.phy.cam.ac.uk.
[8] J. Zhang, M. Fossorier, D. Gu, and J. Zhang, “Improved min-sum
decoding of LDPC codes using 2-dimensional normalization,”
in Proc. Globecom 2005, IEEE Global Telecommunications
Conference, St. Louis, Missouri, USA, 2005.