Factor Graphs and the Sum-product Algorithm
- A New Tool for Communication Receiver
Design
Mark Flanagan, UCD
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Overview
A Unification of Algorithms
Marginalization of Functions
Communications Context
A General Method for Communication Receiver Design
Decoding of Convolutional Codes
Turbo Decoding
LDPC Decoding
Turbo Equalization
Current Work - Turbo Equalization of a “Difficult” Channel
Conclusions
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
A Unified Framework for Many Famous Algorithms
The Viterbi Algorithm
Codeword-error-optimum decoding of block and
convolutional codes
The BCJR Algorithm
Bit-error-optimum decoding of block and convolutional
codes
Turbo Decoding
LDPC Decoding
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
A Unified Framework for Many Famous Algorithms
Turbo Equalization
“Iterative Receivers”
FFT Algorithms
Kalman Filter
Weight Distribution Algorithms for Block Codes
“Belief Propagation” in Bayesian Networks
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Marginalization of a function
Suppose we have a function
g (x1 , x2 , . . . xn )
Marginalization or summary for variable xi is
X
gi (xi ) =
g (x1 , . . . xn )
∼{xi }
Example: say n = 4; then
XXX
g (x1 , x2 , x3 , x4 )
g1 (x1 ) =
x2
Mark Flanagan, UCD
x3
x4
Factor Graphs and the Sum-product Algorithm - A New Tool f
Marginalization via Factor Graph
Suppose we factorize g, e.g.
g (x1 , x2 , x3 , x4 ) = f1 (x1 ) f2 (x2 , x3 , x4 ) f3 (x1 , x3 , x4 )
A factor graph for g is a graphical representation of this
factorization
f1
x1
f2
x2
f3
x3
x4
“Sum-product” algorithm - proceeds via “message passing”
on the graph
If the factor graph is a tree =⇒ Sum-Product algorithm
computes exact marginal functions
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Marginalization - Communications Context
Suppose:
u are information bits
x are other variables associated with the communication
system
r are observations made by the receiver
Then:
X
P (u, x|r) = P (uk |r)
∼{uk }
The a posteriori probability (APP)
Leads to the BER-optimum decision rule
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
A General Method For Communication System Design
Define a probability function which includes information bits
g (u, x) = P (u, x|r)
Factorize this probability function
Draw the corresponding factor graph
Decide on a message passing schedule
Translate into the corresponding iterative receiver algorithm
Execute this algorithm on the graph
Estimate marginals P (uk |r) when the algorithm is deemed
to have converged
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Convolutional Code Decoding
A recursive systematic convolutional (RSC) encoder
uk
uk
xk
Factor graph...
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Factor Graph
P1
P2
P3
u1
u2
s1
x1
Q1
Q2
P6
u4
T3
s2
x2
P5
u3
T2
T1
s0
P4
u5
T4
s3
x3
Q3
Q4
u6
s5
x5
Q5
u7
T6
T5
s4
x4
P7
T7
s6
x6
Q6
s7
x7
Q7
Factor Graph is a tree =⇒ we can compute exact APPs
BER-optimum receiver
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Turbo Encoding
Turbo code - a parallel concatenation of two codes
RSC codes used
Separated by a pseudorandom interleaver
u
RC ENCODER
R1
a
MUX
Π
RC ENCODER
R2
u0
b
Factor graph...
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Turbo Decoding - Factor Graph
PA(1)
PA(3)
PA(2)
a1
a3
a2
T2
T1
s0
PU(1)
a4
T3
s1
u1
PA(4)
s2
u2
T4
s3
u3
PU(2)
PU(3)
s4
u4
PU(4)
Π
T1
T2
s00
s01
b1
PB(1)
T3
s02
b2
PB(2)
Mark Flanagan, UCD
T4
s03
b3
PB(3)
s04
b4
PB(4)
Factor Graphs and the Sum-product Algorithm - A New Tool f
Turbo Decoder Schematic
{ûk }
DEC
n
o
(u)
λk
SISO
MODULE 1
o
n
(a)
λk
n
Π
(21e)
λk
o
Π−1
n
o
(12e)
λk
Π
SISO
MODULE 2
o
n
(b)
λk
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
LDPC Encoding
LDPC code - simply a block code with:
Large block length
A sparse distribution of ones in its parity-check matrix
LDPC encoder is simply a matrix multiplication
c = uG
Systematic code assumed
Factor graph...
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
LDPC Decoding - Factor Graph
χ1
c1
P1
χ2
c2
P2
c3
P3
Mark Flanagan, UCD
χ3
c4
P4
c5
P5
c6
P6
c7
P7
Factor Graphs and the Sum-product Algorithm - A New Tool f
Turbo Equalization
Coded transmission over a frequency selective fading
channel
A frequency selective fading channel
Individual taps are changing quite quickly
Channel is not known but must be estimated
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Turbo Equalization of a Difficult Channel
Transmitter for a turbo equalization scheme: Example
uk
LDPC
ENCODER
ck
∏C
MODULATOR
INSERT
PILOT
SYMBOLS
xk
Receiver: iterative exchange of information between
equalizer and decoder
Can we use the factor graph based receiver to help
channel estimation?
Factor graph...
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Factor Graph
Assuming perfect channel knowledge
χ1
χ2
χ3
J1
c2
c1
c3
c4
c5
c6
c7
J2
q1
J3
q2
q3
INTERLEAVER
T1
s000
T4
s002
z1
P1
T3
T2
s001
s003
z2
P2
z3
P3
T5
s004
z4
P4
s006
z5
P5
Mark Flanagan, UCD
T7
T6
s005
z6
P6
s008
z7
P7
T9
T8
s007
z8
P8
T10
s009
s0010
z9
P9
z10
P10
Factor Graphs and the Sum-product Algorithm - A New Tool f
Receiver Structure
rk
h, σ 2
EQUALIZER
SISO
λ e (xk )
REMOVE
PILOT
INFO
−1
∏C
INSERT
PILOT
INFO
∏C
Mark Flanagan, UCD
LDPC
DECODER
SISO
DEC
λ (uk )
ûk
Factor Graphs and the Sum-product Algorithm - A New Tool f
Receiver Structure
rk
CHANNEL
ESTIMATOR
ĥ, σ̂ 2
EQUALIZER
SISO
λ e (xk )
REMOVE
PILOT
INFO
−1
∏C
INSERT
PILOT
INFO
∏C
Mark Flanagan, UCD
LDPC
DECODER
SISO
DEC
λ (uk )
ûk
Factor Graphs and the Sum-product Algorithm - A New Tool f
Receiver Structure
rk
x̂k
CHANNEL
ESTIMATOR
ĥ, σ̂ 2
EQUALIZER
SISO
INSERT
PILOT
SYMBOLS
NONLINEAR
FUNCTION
∏C
λ (ck )
λ e (xk )
REMOVE
PILOT
INFO
−1
∏C
INSERT
PILOT
INFO
∏C
Mark Flanagan, UCD
LDPC
DECODER
SISO
DEC
λ (uk )
ûk
Factor Graphs and the Sum-product Algorithm - A New Tool f
BER Performance Results
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
MSE Performance Results (Channel Estimator)
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
Conclusions
A unifying framework for many seemingly different
problems in communications and signal processing
Useful means of designing communications receivers in
general situations
Interesting interplay between “message passing” methods
and “traditional signal processing” methods
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f
The End
Thank you for your attention
Any questions?
Mark Flanagan, UCD
Factor Graphs and the Sum-product Algorithm - A New Tool f