Lecture 28∗
Agenda for the lecture
• Random coding achievability for additive Gaussian noise (AGN) channel
28.1
Random coding proof
We use Han-Verdú’s approach for showing the achievability of the rate
1
P
log 1 + 2 .
2
σ
The proof is exactly the same as that for the discrete case, except the following:
• We need to handle continuous distributions (though only with densities);
• the codewords selected must satisfy the average power constraint.
Before we describe our encoders and decoders, we first describe our large probability typical
set. Let X1 , ..., Xn be i.i.d. with common distribution N (0, P − η). Let Y1 , ..., Yn be the
output of an AGN channel when inputs are X1 , ..., Xn , i.e., Yi = Xi + Zi where Zi ∼
N (0, σ 2 ) and Z1 , ..., Zn are independent. Thus, {(Xi , Yi )}ni=1 are i.i.d.. The information
∗
c
Himanshu
Tyagi. Feel free to use with acknowledgement.
1
density of Xi , Yi is given by
iPXi Yi (x, y) = log
fXi Yi (x, y)
,
fXi (x)fYi (y)
where fU denotes the density of a random variable U . Note that this definition looks
slightly different from the one used for the discrete case. However, it is possible to give a
general definition which includes both these definitions as special cases, but it is beyond the
scope of this course. With abuse, we simply denote the random value of the information
density above by i(Xi , Yi ). For our distribution of (Xi , Yi ), i(Xi , Yi ) can be expressed as
P −η
(Yi − Xi )2
Yi2
1
i(Xi , Yi ) = log 1 +
−
+
,
2
σ2
2σ 2
2(P − η + σ)2
since Yi ∼ N (0, P − η + σ). Thus,
1
P −η
E [i(Xi , Yi )] = log 1 +
.
2
σ2
Denoting by Ui = i(Xi , Yi ), the random variables (U1 , ..., Un ) are i.i.d. and therefore by
Chebyshev’s inequality
P
n
X
r
Ui ≤ nE [U1 ] −
i=1
nVar [U1 ]
!
≤ .
It can be seen that Var [U1 ] < ∞. Note that
n
X
i(Xi , Yi ) =
i=1
n
X
i=1
Qn
fXY (Xi , Yi )
fXY (Xi , Yi )
= log Qni=1
.
log
fX (Xi )fY (Yi )
i=1 fX (Xi )fYi (Yi )
Denoting by T the set
T =
n
fX n Y n (x, y)
P −η
(x, y) ∈ R × R : log
≥ log 1 +
− nη
fX n (x)fY n (y)
2
σ2
n
n
2
(1)
has probability
PX n Y n (T ) ≥ 1 − .
28.2
Random code construction and analysis
We now describe out construction.
28.2.1
Random codebook and decoder
Codebook generation. The codewords Xm , 1 ≤ m ≤ M , are independent and identically
distributed. Each codeword is generated as follows: Xm = (Xm1 , ..., Xmn ) are generated
i.i.d. with common distribution N (0, P − η). If the codeword Xm satisfies the average
power power constraints
n
X
Xmi ≤ nP,
i=1
we keep it; else we replace it with the all 0 codeword. We denote this random codebook
by C.
Decoder. As we have been doing, we will use the template of Figure 2 of Lecture 7.
Thus, we only need to define Bm , which we define as follows:
Bm (Y n ) = 1((Xm , Y n ) ∈ T ),
where T is as in (1).
While the way we presented our encoder and decoder is closer to how we want our codes
to look, an equivalent formulation is more suited for analysis. In this alternative form we
do not ignore a codeword if it violates the average power constraint. Instead, we include
this clause in the definition of Bm as follows:
Bm (Y n ) = 1((Xm , Y n ) ∈ T and
n
X
i=1
3
Xmi ≤ nP ).
Of course, this is just a theoretical tool to look for good codebooks. When we find one, we
will not use the codewords which violate the power constraints. This will not cause any
problem since anyway their corresponding Bm ’s were always declaring a 0. So, in effect,
these codewords were never received.
Error analysis. We now bound the expected average error. We will use the alternative
form described above. As we saw in lecture 25,26,
E [(C)] ≤ +P (B1 = 0|1 sent) + M P (B2 = 1|1 sent) .
We will handle each term separately. For the first term,
P (B1 = 0|1 sent) = E [P (B1 = 0|1 sent, X1 )]
n
n
c
≤ E [W ((X1 , Y) ∈ T |X1 )] + P
n
=P
XnY n
c
(T ) + P
1X
X1i > P
n
1X
X1i > P
n
i=1
!
!
,
i=1
where the inequality holds by the definition of B1 and the second equality holds since
PY n |X n = W n , i.e., the conditional probability coming from the AGN channel. The first
term on the right-side above is bounded above by , as we saw in the previous section. For
the second term, since X11 , ..., X1n are i.i.d. N (0, P − η), by Chebyshev’s inequality
n
P
p
1X
X1i > E [X11 ] + Var [X11 ]nδ
n
i=1
Thus, for all n sufficiently large,
n
P
1X
X1i > P
n
i=1
4
!
≤ δ.
!
≤ δ.
Thus,
P (B1 = 0|1 sent) ≤ + δ,
for all n sufficiently large.
We now move to the second term,
P (B2 = 1|1 sent) = E [P (B2 = 1|1 sent, X1 , X2 )]
≤ E [W n ((X2 , Y) ∈ T |X1 )]
= E [E [W n ((X2 , Y) ∈ T |X1 )|X2 ]]
= E [E [(PW )n ((X2 , Y) ∈ T )|X2 ]]
= E [(PW )n ((X2 , Y) ∈ T )] ,
where the first inequality is obtained by ignoring the second condition in the definition of
B2 , the third equality uses the independence of X1 and X2 and the fact that the expected
probability at the output is (PW )n .
We now use the definition of T . Note that PW is N (0, P − η + σ 2 ). Thus, for every
(x2 , y) ∈ T ,
fY n (y) ≤ 2−λ fY n |X n (y|x2 ),
where λ =
n
2
log 1 +
P −η
σ2
− nη. Thus,
E [(PW )n ((X2 , Y) ∈ T )] ≤ 2−λ E [W n ((X2 , Y) ∈ T |X2 )] ≤ 2−λ .
Thus,
E [(C)] ≤ + δ + M 2−λ .
The rest of the proof is as in lecture 26. We can show that a rate
1
2
log 1 +
P −η
σ2
( + δ + 2−η )-achievable. Since , δ, and η are arbitrary, the proof is complete.
5
− 2η is