Principles of Communications
Chapter 7: Optimal Receivers
Selected from Chapter 8.1-8.3, 8.4.6, 8.5.3 of
Fundamentals of Communications Systems, Pearson
Prentice Hall 2005, by Proakis & Salehi
Meixia Tao @ SJTU
Topics to be Covered
Source
A/D
converter
Source
encoder
Channel
encoder
Absent if
source is
digital
User
D/A
converter
 Detection theory
Modulator
Noise
Source
decoder
Channel
decoder
Channel
Detector
 Decision regions
 Optimal receiver structure  Error probability analysis
 Matched filter
Meixia Tao @ SJTU
2
Example
 Alice tells her Dad that she wants either strawberry
Alice
or blueberry.
Bob hears
“I want blawberry
”
Alice says
“I want strawberry
”
Dad
Channel
Bob’s question: which berry does Alice want?
Is strawberry
more possible?
Why?
Strawberry or Blueberry?
Meixia Tao @ SJTU
3
Statistical Decision Theory
 Demodulation and decoding of signals in digital communications is
Alice
Bob
directly related to Statistical
“I want decision theory
“I want
blawberry”
” are given a finite
 In the general setting, we
set of possible hypotheses
about an experiment, along with observations related statistically to
the various hypotheses
 The theory provides rules for Channel
making the best possible decision
(according
to some
criterion)
about which
hypothesis
is
Alice
can askperformance
either strawberry
or blueberry
from Bob
.
likely to be true
 In digital communications, hypotheses are the possible messages and
observations are the output of a channel
 Based on the observed values of the channel output, we are interested
in the best decision making rule in the sense of minimizing the
probability of error
Meixia Tao @ SJTU
4
Detection Theory
 Given M possible hypotheses Hi (signal mi) with probability
,
 Pi represents the prior knowledge concerning the probability
of the signal mi – Prior Probability
 The observation is some collection of N real values,
denoted by
with conditional pdf
-- conditional pdf of observation
given the signal mi
 Goal: Find the best decision-making algorithm in the sense
of minimizing the probability of decision error.
Message
Channel
Meixia Tao @ SJTU
Decision
5
Observation Space
 In general, can be regarded as a point in some
observation space
 Each hypothesis Hi is associated with a decision region Di:
 The decision will be in favor of Hi if
is in Di
 Error occurs when a decision is made in favor of another
when the signals
falls outside the decision region Di
D1
D2
D3
D4
Decision Space M=4
Meixia Tao @ SJTU
6
MAP Decision Criterion
 Consider a decision rule based on the computation of the
posterior probabilities defined as
 Known as a posterior since the decision is made after (or given) the
observation
 Different from the a prior where some information about the
decision is known in advance of the observation
Meixia Tao @ SJTU
7
MAP Decision Criterion (cont’d)
 By Bayes’ Rule:
 Since our criterion is to minimize the probability of detection
error given , we deduce that the optimum decision rule is to
choose
if and only if
is maximum for
.
 Equivalently,
Choose
if and only if
 This decision rule is known as maximum a posterior or MAP
decision criterion
Meixia Tao @ SJTU
8
ML Decision Criterion
 If
, i.e. the signals
finding the signal that maximizes
the signal that maximizes
are equiprobable,
is equivalent to finding
 The conditional pdf
is usually called the likelihood
function. The decision criterion based on the maximum of
is called the Maximum-Likelihood (ML) criterion.
 ML decision rule:
Choose
if and only if
 In any digital communication systems, the decision task
ultimately reverts to one of these rules
Meixia Tao @ SJTU
9
Topics to be Covered
Source
A/D
converter
Source
encoder
Channel
encoder
Absent if
source is
digital
User
D/A
converter
 Detection theory
Modulator
Noise
Source
decoder
Channel
decoder
Channel
Detector
 Decision regions
 Optimal receiver structure  Error probability analysis
 Matched filter
Meixia Tao @ SJTU
10
Optimal Receiver in AWGN Channel
 Transmitter transmits a sequence of symbols or messages from a set
of M symbols
with prior probabilities
 The symbols are represented by finite energy waveforms
, defined in the interval
 The channel is assumed to corrupt the signal by additive white
Gaussian noise (AWGN)
Channel
Transmitter
Σ
Receiver
Meixia Tao @ SJTU
11
Signal Space Representation
 The signal space of {s1(t), s2(t), …, sM(t)} is
assumed to be of dimension N (N ≤ M)

for k = 1, …, N will denote an orthonormal
basis function
 Then each transmitted signal waveform can be
represented as
where
Meixia Tao @ SJTU
12
 Note that the noise nw(t) can be written as
where
Projection of nw(t) on the N-dim space
orthogonal to the space, falls outside the signal
space spanned by
 The received signal can thus be represented as
where
Projection of r(t) on N-dim signal space
Meixia Tao @ SJTU
13
Graphical Illustration
 In vector forms, we have
Received signal point
Observation
vector
Noise vector
Message point
Signal vector
Meixia Tao @ SJTU
14
Receiver Structure
 Subdivide the receiver into two parts
 Signal demodulator: to convert the received waveform r(t)
into an N-dim vector
 Detector: to decide which of the M possible signal waveforms
was transmitted based on observation of the vector
receiver
r(t)
Signal
demodulator
Detector
 Two realizations of the signal demodulator
 Correlation-Type demodulator
 Matched-Filter-Type demodulator
Meixia Tao @ SJTU
15
Topics to be Covered
Source
A/D
converter
Source
encoder
Channel
encoder
Absent if
source is
digital
User
D/A
converter
 Detection theory
Modulator
Noise
Source
decoder
Channel
decoder
Channel
Detector
 Decision regions
 Optimal receiver structure  Error probability analysis
 Matched filter
Meixia Tao @ SJTU
16
What is Matched Filter (匹配滤波器)?
 The matched filter (MF) is the optimal linear filter for
maximizing the output SNR.
 Derivation of the MF
∞
 Input signal component si ( t ) ↔ A ( f ) = ∫−∞ si ( t ) e− jωt dt
 Input noise component ni ( t ) with PSD Sn ( f ) = N 0 / 2
i
so ( t )
 Output signal component =
 Sample at
∫
=∫
∞
s ( t − τ ) h (τ ) dτ
−∞ i
∞
−∞
A ( f ) H ( f ) e jωt df
Meixia Tao @ SJTU
17
Output SNR
 At the sampling instance
,
∞
so ( t0 ) = ∫ A ( f ) H ( f ) e jωt0 df
−∞
 Average power of the output noise is
N0
2
=
N E=
{no2 ( t )}
∫
∞
H ( f ) df
2
−∞
 Output SNR
 A ( f ) H ( f ) e df 
s ( t0 )
 ∫−∞

=
d =
2
N0 ∞
E{no2 ( t )}
H ( f ) df
∫
−∞
2
∞
2
o
jωt0
2
Find H( f ) that can maximize d
Meixia Tao @ SJTU
18
Maximum Output SNR
 Schwarz’s inequality:
∫
∞
∞
F ( x ) dx ∫ Q ( x ) dx ≥
2
−∞
2
−∞
∫
∞
−∞
F
*
( x ) Q ( x ) dx
2
equality holds when F ( x ) = CQ ( x )
 Let
F * ( x ) = A ( f ) e jωt0
, then
Q( f ) = H ( f )
d≤
∫
∞
−∞
E : signal energy
A ( f ) df ∫
2
∞
−∞
N0
2
∫
∞
−∞
H ( f ) df
2
H ( f ) df
2
∞
A ( f ) df 2 E
∫
−∞
==
N0
N0
2
Meixia Tao @ SJTU
2
19
Solution of Matched Filter
 When the max output SNR
H m ( f ) = A* ( f ) e − jωt0
hm ( t ) = ∫s ( t0 − t )
*
i
is achieved, we have
∞
hm ( t ) = ∫ H m ( f ) e jωt df
−∞
∞
= ∫ A* ( f ) e
−∞
− jω ( t0 −t )
df
= si* ( t0 − t )
 Transfer function: complex conjugate of the input signal
spectrum
 Impulse response: time-reversal and delayed version of
the input signal s(t )
Meixia Tao @ SJTU
20
Properties of MF (1)
 Choice of t0 versus the causality (因果性）
0
0
T
0
Not implementable
Not implementable
 si ( t0 − t ) 0 ≤ t < t0
hm ( t ) = 
otherwise
 0
where t0 ≥ T
Meixia Tao @ SJTU
21
Properties of MF (2)
 Equivalent form – Correlator
 Let si ( t ) be within
=
y ( t ) x(=
t ) * hm ( t ) x(t ) * si (T − t )
=
∫
T
0
y (t )
x(t )
t =T
y (T )
MF
x(τ ) si (T − t + τ ) dτ
 Observe at sampling time t = T
=
y (T )
x(τ ) s (τ ) dτ ∫ x ( t ) s ( t ) dt
∫=
T
0
T
i
0
i
x(t )
Correlation
integration
(相关积分）
si (t )
Meixia Tao @ SJTU
y (T )
correlator
22
Correlation Integration
 Correlation function
∞
∞
−∞
−∞
R12 (τ ) =∫ s1 ( t ) s2 ( t + τ ) dt =∫ s1 ( t − τ ) s2 ( t ) dt =R21 (−τ )
∞
=
R (τ ) ∫ s ( t ) s ( t + τ ) dt
 Autocorrelation function
−∞

R (τ=
) R ( −τ )

R ( 0 ) ≥ R (τ )
 R ( 0)
=

∞
2
=
s
∫ ( t ) dt E
−∞
R ( 0)
R (τ ) ↔ A ( f ) =
2
∞
s ( t ) dt
∫=
−∞
2
∫
∞
−∞
A ( f ) df
Meixia Tao @ SJTU
2
23
Properties of MF (3)
 MF output is the autocorrelation function of input signal
=
so ( t )
=
∫
∫
∞
∞
s ( t − u ) hm ( u ) du = ∫ si ( t − u ) si ( t0 − u ) du
−∞ i
∞
−∞
s ( µ ) si [ µ + t − t0 ] d=
µ Rs0 (t − t0 )
−∞ i
t = t0
 The peak value of s0 (t ) happens
=
so ( t0 )
∞
2
=
s
∫ i ( µ ) du E
−∞
 s0 (t ) is symmetric at t = t0
=
Ao ( f ) A=
( f ) Hm ( f )
A ( f ) e − jωt0
2
Meixia Tao @ SJTU
24
Properties of MF (4)
 MF output noise
 The statistical autocorrelation of
autocorrelation of
N0
=
+
τ
Rno (τ ) E=
n
t
n
t
{ o ( ) o ( )} 2
N0 ∞
si ( t ) si ( t − τ ) dt
=
∫
−∞
2
∫
∞
−∞
depends on the
hm ( u ) hm ( u + τ ) du
 Average power
N0 ∞ 2
E {n=
si ( µ ) du Time domain
( t )} R=
no ( 0 )
∫
−∞
2
2
2
N0 ∞
N0 ∞
= =
A ( f ) df
H m ( f ) df Frequency domain
∫
∫
−∞
−∞
2
2
N
= 0E
2
2
o
Meixia Tao @ SJTU
25
Example: MF for a rectangular pulse
 Consider a rectangular pulse
A
 The impulse response of a filter
matched to
is also a
rectangular pulse
0
2
( SNR )o =
N0
∫
T
0
t
0
T
t
0
T
2T
A
 The output of the matched filter
is
 The output SNR is
T
A2T
2 A2T
s (t )dt =
N0
2
Meixia Tao @ SJTU
26
t
What if the noise is Colored?
 Basic idea: preprocess the combined signal and noise
such that the non-white noise becomes white noise Whitening Process
x=
( t ) si ( t ) + n ( t ) where n(t) is colored noise with PSD Sn ( f )
′ ( t ) s′ ( t ) + n′ ( t )
x=
Choose H1( f ) so that
is white, i.e.
=
S n′ ( f ) H
=
C
1 ( f ) Sn ( f )
2
Meixia Tao @ SJTU
27
H1( f ) , H2( f )
C

: H1 ( f ) =

should match with
2
Sn ( f )
A′ ( f ) = H1 ( f ) A ( f )
′∗ ( f ) e − j 2π ft0 H1∗ ( f ) A∗ ( f ) e − j 2π ft0
=
H 2 ( f ) A=
 Overall transfer function of the cascaded system:
H ( f ) = H1 ( f ) ⋅ H 2 ( f ) = H1 ( f ) H1∗ ( f ) A∗ ( f ) e − j 2π ft0
= H1 ( f ) A∗ ( f ) e − j 2π ft0
2
A ( f ) − j 2π ft0
H(f )=
e
Sn ( f )
∗
MF for colored
noise
Meixia Tao @ SJTU
28
Update
 We have discussed what is matched filter
 Let us now come back to the optimal receiver
structure
receiver
r(t)
Signal
demodulator
Detector
 Two realizations of the signal demodulator
 Correlation-Type demodulator
 Matched-Filter-Type demodulator
Meixia Tao @ SJTU
29
Correlation Type Demodulator
 The received signal r(t) is passed through a parallel
bank of N cross correlators which basically compute
the projection of r(t) onto the N basis functions
To detector
Meixia Tao @ SJTU
30
Matched-Filter Type Demodulator
 Alternatively, we may apply the received signal r(t) to
a bank of N matched filters and sample the output of
filters at t = T. The impulse responses of the filters are
To detector
Sample at t=T
Meixia Tao @ SJTU
31
Update
 We have demonstrated that
 for a signal transmitted over an
AWGN channel, either a
correlation type demodulator or a
matched filter type demodulator
produces the vector
which contains all the necessary
information in r(t)
 Now, we will discuss
 the design of a signal detector that makes a decision of the
transmitted signal in each signal interval based on the
observation of , such that the probability of making an
error is minimized (or correct probability is maximized)
Meixia Tao @ SJTU
32
Decision Rules
Recall that
 MAP decision rule:
choose
if and only if
 ML decision rule
choose
if and only if
In order to apply the MAP or ML rules, we need to evaluate
the likelihood function
Meixia Tao @ SJTU
33
Distribution of the Noise Vector
 Since nw(t) is a Gaussian random process,

is a Gaussian random variable
(from definition)
 Mean:
,
 Correlation between nj and nk
PSD of
is
Meixia Tao @ SJTU
34
 Using the property of a delta function
we have:
 Therefore, nj and nk (
random variables
) are uncorrelated Gaussian
 They are independent with zero-mean and variance N0/2
 The joint pdf of
Meixia Tao @ SJTU
35
Likelihood Function
 If mk is transmitted,
with

Transmitted signal values in each dimension represent the
mean values for each received signal

 Conditional pdf of the random variables
Meixia Tao @ SJTU
36
Log-Likelihood Function
 To simplify the computation, we take the natural logarithm
of
, which is a monotonic function. Thus
 Let

is the Euclidean distance between and in the Ndim signal space. It is also called distance metrics
Meixia Tao @ SJTU
37
Optimum Detector
 MAP rule:
 ML rule:
ML detector chooses
iff received vector
in terms of Euclidean distance than to any other
is closer to
for i ≠ k
Minimum distance detection
(will discuss more in decision region)
Meixia Tao @ SJTU
38
Optimal Receiver Structure
 From previous expression we can develop a receiver
structure using the following derivation
in which
= signal energy
= correlation between the received signal
vector and the transmitted signal vector
= common to all M decisions and hence can be ignored
Meixia Tao @ SJTU
39
 The new decision function becomes
 Now we are ready draw the implementation diagram of
MAP receiver (signal demodulator + detector)
Meixia Tao @ SJTU
40
MAP Receiver Structure
Method 1 (Signal Demodulator + Detector)
+
Compute
+
Comparator
(select the
largest)
+
This part can also be implemented
using matched filters
Meixia Tao @ SJTU
41
MAP Receiver Structure
Method 2 (Integrated demodulator and detector)
+
+
Comparator
(select the
largest)
+
This part can also be implemented
using matched filters
Meixia Tao @ SJTU
42
Method 1 vs. Method 2
 Both receivers perform identically
 Choice depends on circumstances
 For instance, if N < M and
are easier to generate
than
, then the choice is obvious
Meixia Tao @ SJTU
43
Example: optimal receiver design
 Consider the signal set
Meixia Tao @ SJTU
44
Example (cont’d)
 Suppose we use the following basis functions
 Since the energy is the same for all four signals, we can
drop the energy term from
Meixia Tao @ SJTU
45
Example (cont’d)
 Method 1
Compute
+
+
Choose
the
Largest
+
Meixia Tao @ SJTU
46
Example (cont’d)
 Method 2
+
Chose
the
Largest
+
Meixia Tao @ SJTU
47
Exercise
In an additive white Gaussian noise channel with a
noise power-spectral density of N0/2, two
equiprobable messages are transmitted by
 At
s1 (t ) =  T
 0
0≤t≤T
otherwise
At

A−
0≤t≤T
s2 (t ) = 
T
otherwise
 0
 Determine the structure of the optimal receiver.
Meixia Tao @ SJTU
48
Notes on Optimal Receiver Design
 The receiver is general for any signal forms
 Simplifications are possible under certain
scenarios
Meixia Tao @ SJTU
49
Topics to be Covered
Source
A/D
converter
Source
encoder
Channel
encoder
Absent if
source is
digital
User
D/A
converter
 Detection theory
Modulator
Noise
Source
decoder
Channel
decoder
Channel
Detector
 Decision regions
 Optimal receiver structure  Error probability analysis
 Matched filter
Meixia Tao @ SJTU
50
Graphical Interpretation of Decision
Regions
 Signal space can be divided into M disjoint decision
regions R1 R2, …, RM.
If
decide mk was transmitted
Select decision regions so that Pe is minimized
 Recall that the optimal receiver sets
iff
is minimized
 For simplicity, if one assumes pk = 1/M, for all k, then the
optimal receiver sets
iff
is minimized
Meixia Tao @ SJTU
51
Decision Regions
 Geometrically, this means
 Take projection of r(t) in the signal space (i.e. ). Then,
decision is made in favor of signal that is the closest to
in the sense of minimum Euclidean distance
 And those observation vectors
for all
with
should be assigned to decision region Rk
Meixia Tao @ SJTU
52
Example: Binary Case
 Consider binary data transmission over AWGN channel
with PSD Sn(f) = N0/2 using
 Assume P(m1) ≠ P(m2)
 Determine the optimal receiver (and optimal decision
regions)
Meixia Tao @ SJTU
53
Solution
 Optimal decision making
Choose m1
<
>
Choose m2
 Let
and
 Equivalently,
Choose m1
Choose m2
R1:
and
Constant c
R2:
Meixia Tao @ SJTU
54
Solution (cont’d)
 Now consider the example with
boundary
R2
on the decision
R1
r
s2
0
s1
Meixia Tao @ SJTU
55
Determining the Optimum Decision
Regions
 In general, boundaries of decision regions are
perpendicular bisectors of the lines joining the original
transmitted signals
 Example: three equiprobable 2-dim signals
R2
R1
s1
s2
R3
s3
Meixia Tao @ SJTU
56
Example: Decision Region for QPSK
 Assume all signals are equally likely
 All 4 signals could be written as the linear combination of
two basis functions
 Constellations of 4 signals
R2
s1=(1,0)
s2=(0,1)
s2
R3
R1
s3
s1
s3=(-1,0)
s4=(0,-1)
R4
s4
Meixia Tao @ SJTU
57
Exercise
Three equally probable messages m1, m2, and m3 are to be transmitted over an AWGN channel
with noise power-spectral density N 0 / 2 . The messages are
1
s1 (t ) = 
0

1


s2 (t ) = − s3 (t ) = − 1

0


0≤t ≤T
otherwise
0≤t ≤
T
2
T
≤t ≤T
2
otherwise
1. What is the dimensionality of the signal space ?
2. Find an appropriate basis for the signal space (Hint: You can find the basis without using the
Gram-Schmidt procedure ).
3. Draw the signal constellation for this problem.
4. Sketch the optimal decision regions R1, R2, and R3.
Meixia Tao @ SJTU
58
Notes on Decision Regions
 Boundaries are perpendicular to a line drawn between two
signal points
 If signals are equiprobable, decision boundaries lie exactly
halfway in between signal points
 If signal probabilities are unequal, the region of the less
probable signal will shrink
Meixia Tao @ SJTU
59
Topics to be Covered
Source
A/D
converter
Source
encoder
Channel
encoder
Absent if
source is
digital
User
D/A
converter
 Detection theory
Modulator
Noise
Source
decoder
Channel
decoder
Channel
Detector
 Decision regions
 Optimal receiver structure  Error probability analysis
 Matched filter
Meixia Tao @ SJTU
60
Probability of Error using Decision
Regions
 Suppose
is transmitted and
 Correct decision is made when
is received
with probability
 Averaging over all possible transmitted symbols, we obtain
the average probability of making correct decision
 Average probability of error
Meixia Tao @ SJTU
61
Example: Pe analysis
 Now consider our example with binary data transmission
•Given m1 is transmitted, then
R2
s2
R1
0
s1
•Since n is Gaussian with zero
mean and variance
Meixia Tao @ SJTU
62
 Likewise
 Thus,
where
and
Meixia Tao @ SJTU
63
Example: Pe analysis (cont’d)
 Note that when
Meixia Tao @ SJTU
64
Example: Pe analysis (cont’d)
 This example demonstrates an interesting fact:
 When optimal receiver is used, Pe does not depend upon the
specific waveform used
 Pe depends only on their geometrical representation in signal
space
 In particular, Pe depends on signal waveforms only through
their energies (distance)
Meixia Tao @ SJTU
65
Exercise
Three equally probable messages m1, m2, and m3 are to be transmitted over an AWGN channel
with noise power-spectral density N 0 / 2 . The messages are
1
s1 (t ) = 
0

1


s2 (t ) = − s3 (t ) = − 1

0


0≤t ≤T
otherwise
0≤t ≤
T
2
T
≤t ≤T
2
otherwise
1. What is the dimensionality of the signal space ?
2. Find an appropriate basis for the signal space (Hint: You can find the basis without using the
Gram-Schmidt procedure ).
3. Draw the signal constellation for this problem.
4. Sketch the optimal decision regions R1, R2, and R3.
5. Which of the three messages is more vulnerable to errors and why ? In other words, which of
p ( Error | mi transmitted ), i = 1, 2, 3 is larger ?
Meixia Tao @ SJTU
66
General Expression for Pe
 Average probability of symbol error
Likelihood function
 Since
N-dim integration
 Thus we rewrite Pe in terms of likelihood functions,
assuming that symbols are equally likely to be sent
Meixia Tao @ SJTU
67
Union Bound
 Multi-dimension integrals are quite difficult to evaluate
 To overcome this difficulty, we resort to the use of bounds
 Now we develop a simple and yet useful upper bound for
Pe, called union bound, as an approximation to the average
probability of symbol error
Meixia Tao @ SJTU
68
Key Approximation
 Let
denote the event that
in the signal space when
(
is closer to
) is sent
 Conditional probability of symbol error when
than to
is sent
 But
Meixia Tao @ SJTU
69
Key Approximation Example
R2
s2
R3 s
3
R1
s1
R4
s4
s2
s3
s1
s1
s1
s4
Meixia Tao @ SJTU
70
Pair-wise Error Probability
 Define the pair-wise (or component-wise) error probability
as
 It is equivalent to the probability of deciding in favor of
when was sent in a simplified binary system that
involves the use of two equally likely messages and
 Then

is the Euclidean distance between
Meixia Tao @ SJTU
and
71
Union Bound
 Conditional error probability
 Finally, with M equally likely messages, the average
probability of symbol error is upper bounded by
The most general
formulation of union
bound
Meixia Tao @ SJTU
72
Union Bound (cont’d)
 Let
denote the minimum distance, i.e.
 Since Q( .) is a monotone decreasing function
 Consequently, we may simplify the union bound as
Simplified form of
union bound
Meixia Tao @ SJTU
73
Question
What is the design criterion of a
good signal set?
Meixia Tao @ SJTU
74