Characterizing Source-Channel Diversity Approaches
Beyond the Distortion Exponent ∗
Brian Dunn and J. Nicholas Laneman
University of Notre Dame
Dept. of Electrical Engineering
Notre Dame, IN 46556
{bdunn2, jnl}@nd.edu
Abstract
In this paper we analytically characterize the performance of several sourcechannel diversity approaches used to communicate over single and parallel quasistatic fading channels. An approximation is developed that provides a more accurate characterization than considering only the so-called “distortion exponent”.
Specifically, we consider transmitting i.i.d. samples of a Gaussian source over quasistatic Rayleigh fading channels with receive, but not transmit, channel state information (CSI). The objective is to to minimize the mean-square distortion averaged
over channel realizations.
For a single channel, we obtain approximations for uncoded analog transmission,
single and dual-layer digital schemes, and derive a lower bound on the expected
distortion based upon CSI at the transmitter. For parallel channels, in addition to
extending the above approaches, we develop a hybrid digital-analog scheme as a
simple form of multiple descriptions and show that it offers good performance for
a wide range of channel bandwidths.
1
Introduction
In recent years it has become of increasing interest to communicate multimedia information over wireless links. Example applications include sensor data over a wireless sensor
network, voice over a cellular network, digital radio broadcasts, or audio to wireless speakers. Over the past fifty years, fundamental techniques in digital communications have
relied upon concepts outlined by Shannon in his landmark paper [8]. Unfortunately, for
theoretically optimal performance these techniques require large delays and impractical
complexity, and therefore do not explicitly apply to non-ergodic wireless link models that
result from limited channel variations and practical delay constraints. For these scenarios, the source-channel diversity framework of [5] allows for evaluation and comparison
of various communication architectures. For a thorough review of related literature, the
reader is referred to [5].
At a high level, source-channel diversity models scenarios in which ergodic sources are
communicated over non-ergodic channels. Instead of complete joint design of source and
This work has been supported in part by the National Science Foundation through contract ECS0329766.
∗
channel coding, source coding blocks and channel coding blocks are combined in various
ways, with rate, power, and bandwidth optimizations appropriate to each particular
combination. Asymptotic performance in terms of rate-distortion regions are used to
characterize source coding blocks, and capacity-versus-outage, or outage probability [6],
is used to characterize performance of channel coding blocks.
For a variety of approaches based upon single and multiple description source coding
over parallel channels, [5] shows that end-to-end average distortion behaves for large SNR
·
as1 E[D] = SNR−∆(L) , where SNR is the average channel signal-to-noise ratio, ∆(L) is the
so-called “distortion exponent”, and L is the number of channel uses per source sample.
In this paper, we refine the performance analysis to be of the form
E[D] ≈ C(L) · log(L · SNR)p · SNR−∆(L) ,
(1)
where p ∈ {0, 1, 2} and C(L) is a coefficient independent of SNR, and analyze new
approaches. Specifically, Section 2 summarizes approximations of the form (1) for a single
quasi-static fading channel with the following approaches: uncoded analog transmission,
a digital system designed using separation, and a superposition successive refinement
scheme. These approaches are compared to a lower bound based upon transmit channel
state information (CSI). We note that the distortion exponents presented in Section 2
have been found concurrently in [4]. Section 3 extends these results to parallel quasistatic fading channels and introduces a hybrid digital analog (HDA) scheme that performs
well compared to the approaches considered.
2
Communication over a Single Channel
We consider transmitting i.i.d. samples of a unit-variance Gaussian source over a single
quasi-static Rayleigh fading channel. We assume a delay constraint prevents us from
increasing the block length sufficiently to average over variations in the channel, but the
block length is large enough to approach the rate-distortion function of the source. For
regimes in the latter cannot be achieved, our results serve a lower bounds on average
distortion.
For a single block, the quasi-static fading channel is modeled as
y (n) = a · x(n) + z(n),
(2)
where the fading coefficient a is a circularly symmetric complex Gaussian random variable (RV) that remains constant over the entire block and is chosen independently between blocks. The channel inputs and outputs are xi and yi , respectively, and wi is i.i.d.
zero-mean, unit-variance circularly symmetric complex additive white Gaussian noise
(AWGN). The encoder maps K real source samples to N real channel inputs, or equivalently, to N/2 complex channel inputs. We characterize the relation between the channel
and source bandwidths through the bandwidth expansion ratio L := N/K.2
1
·
Throughout the paper, the notations f (x) = xν and f (x) ≈ g(x) correspond to the limiting operations log f (x)/ log x → ν and f (x)/g(x) → 1, respectively, under the appropriate limit with respect to
x.
2
In [5] the encoder maps K real source samples into to Nc = N/2 complex channel uses. Thus, the
bandwidth expansion ratio ω = Nc /K in [5] satisfies ω = L/2.
2.1
A Lower Bound
We derive a lower bound on the achievable distortion by considering the case in which
the transmitter has complete knowledge of the realized fading coefficient. A system that
does not have channel state information at the transmitter can do no better than one
that does, allowing us to compute a simple lower bound. Conditioned on knowledge of
a, the channel becomes an AWGN channel with known SNR. Here separation is optimal
if the source and channel code can both be adapted. This allows the transmitter to find
the realized mutual information, I (x; y ) = 12 log (1 + |a|2 SNR), and hence the maximal
rate at which the channel realization can support reliable communication. For a specific
a, the expected distortion can be found by evaluating the distortion rate function of the
source at a rate equal to the realized mutual information. Averaging over the exponential
distribution of |a|2 yields
Z
e1/SNR ∞ e−t/SNR
E[D] =
dt.
(3)
SNR 1
tL
For asymptotically high SNR, (3) can be approximated as
(
log(SNR) · SNR−1 , for L = 1
E[D] ≈
.
1
SNR−1 ,
for L > 1
L−1
2.2
(4)
Uncoded Transmission
It is well known that, for L = 1, uncoded transmission over an AWGN channel is optimal
for all values of SNR. Since uncoded transmission does not require knowledge of the
channel’s effective SNR at the transmitter, it remains optimal for the quasi-static fading
channel with L = 1. For L < 1, only the first N source samples will be transmitted, and
thus for asymptotically high SNR, the expected distortion will approach σs2 · (1 −L). This
implies for L < 1 the expected distortion can be approximated as E[D] ≈ σs2 · (1 − L).
We assume for L > 1 the extra bandwidth of N − K channel uses is not used, but
the additional power available is used for the first K channel uses, to maintain the
appropriate average power. Performing minimum mean-square error (MMSE) estimation
at the receiver results in
1
1 1/SNR
e
E1
.
(5)
E[D] =
SNR
SNR
For asymptotically high SNR, (5) can be approximated as
E[D] ≈
1
log(L · SNR) · SNR−1 .
L
(6)
We see that for L ≥ 1, uncoded transmission achieves the optimal distortion exponent
found in Section 2.1 of ∆(L) = 1. In the special case of L = 1 the expected distortion of
uncoded transmission is optimal for all SNR, however, this is not the case for L 6= 1.
2.3
Rate-Optimized Digital
We analyze the performance of standard (separate source and channel coding) digital
transmission over the quasi-static Rayleigh fading channel given by (2). Without knowledge of the fading coefficients at the transmitter, there is a non-zero probability the
realized channel mutual information will not support reliable communication at a target
rate R. This probability of outage can be computed as follows:
2R/L
e
−1
Pout (R, SNR) := Pr [I (x; y ) < R/L] = 1 − exp −
,
(7)
SNR
where R is the source rate in nats per real source sample. Using the total probability
law the expected distortion is found to be
2R/L
2R/L
e
−1
e
−1
−2R
E[D] = 1 − exp −
+e
· exp −
.
(8)
SNR
SNR
Note that the probability of outage, and hence the expected distortion, is a function
of both R and SNR. For asymptotically high SNR, the optimal rate can be approximated
as r0 + r log(SNR), and the expected distortion can be approximated as
L
E[D] ≈ (L + 1)(L · SNR)− L+1 .
(9)
From a higher level, the sub-optimality of separate source and channel coding is due to
the inherent nature of the separation. The source coder is designed under the assumption
that it’s output will be available to the source decoder with no errors, a condition that
may be impossible to meet for certain channels. When errors are present in the decoded
bit stream the source decoder may fail completely, resulting in a mean-square error equal
to the source variance. One possible solution may be to design encoding schemes that
degrade more gracefully as the quality of the channel decreases.
2.4
Superposition Successive Refinement Coding
In order to partially combat the on-off nature of rate-optimized digital transmission
that results in suboptimal performance, we consider a layered scheme using successive
refinement source coding [2]. We consider a dual-layer successive refinement code, where
the refinement layer is superimposed on the base layer and power allocation between the
layers is performed to minimize the expected distortion. The transmitted signal is the
sum of the two layers,
x = α · SNR · xB + (1 − α) · SNR · xE .
(10)
The decoding is performed as follows: The receiver first attempts to decode the
base layer xB treating the enhancement layer xE as additive noise. If the base layer
is successfully decoded, the receiver subtracts its estimate of the transmitted codeword
from the received signal and attempts to decode the enhancement layer. The average
distortion as a function of α, RB , and RE can be expressed as
E[D] = Pr[Bout ] + e−2RB · Pr[Bout ] Pr[Eout |Bout ] + e−2(RB +RE ) · Pr[Bout ] Pr[Eout |Bout ], (11)
where Bout and Eout denote the events of a base layer and enhancement layer outage,
and RB and RE denote the base and enhancement layer rates in nats per source sample,
respectively. The probability of the outage events are computed as
e2RB /L − 1
Pr[Bout ] = 1 − exp −
, and
(12)
SNR [1 − (1 − α)e2RB /L ]
e2RB /L − 1
e2RE /L − 1
Pr Eout |Bout = 1 − exp
−
.
(13)
SNR [1 − (1 − α)e2RB /L ] (1 − α)SNR
By performing the optimization over RB , RE , and α numerically, we observe that
for high SNR the optimal rates can be approximated as RB ≈ r0B + rB · log(SNR) and
RE ≈ r0E + rE · log(SNR). Also, the optimal α behaves as α ≈ 1 − SNR−α̂ , where the
constant α̂ determines the exponential rate at which more power is allocated to the base
layer. Analysis for asymptotically high SNR and minimization over r0B , rB , r0E , rE , and
α̂ yields the approximation for expected distortion
− L(L+2)
2
E[D] ≈ L
(L+1)
L+2
· (L + 1) L+1 · SNR
−
L(L+1)
L2 +L+1
,
(14)
where the distortion exponent given in (14) was also found in [4], and generalized to any
number of layers.
3
Communication over Parallel Channels
An attractive means to combat multipath fading is the use of multiple antennas, frequency
bands, or time slots to provide the user access to parallel channels. In this section we
present and analyze the performance of several methods to transmit an i.i.d. Gaussian
source over two independent Rayleigh block-fading AWGN channels. As in Section 2,
we assume a delay constraint prevents us from increasing the block length sufficiently to
average over variations in the channel, but the block length is large enough to reasonably
approximate the source coding distortion as the distortion-rate function of the source.
The corresponding channel model becomes
y1 (n) =a1 · x1 (n) + z1 (n)
y2 (n) =a2 · x2 (n) + z2 (n),
(15a)
(15b)
where a1 and a2 remain constant over the entire block. Furthermore, the encoder maps
K real source samples to N pairs of real channel inputs, or N/2 pairs of complex channel
inputs. As for a single channel, the bandwidth expansion ratio is denoted as L := N/K.
In this section we derive a lower bound on the mean-square error for communication
of an i.i.d. Gaussian source over parallel channels, and the corresponding upper bound on
the distortion exponent. We also analyze the performance of simple analog and digital
schemes, followed by successive refinement source coding over parallel channels. Finally,
we introduce a hybrid digital-analog scheme and evaluate its performance.
3.1
A Lower Bound
As for a single channel, we derive a lower bound on the expected distortion by assuming
the realized fading coefficients are known a priori at the transmitter. This allows the
transmitter to compute the mutual information between the inputs and outputs of the
composite channel, and determine the maximum rate at which reliable communication
can be supported. The expected distortion is found to be
Z ∞ −t/SNR 2
e2/SNR
e
E[D] =
dt ,
(16)
2
tL
SNR
1
which can be approximated for high SNR as
(
(log SNR)2 · SNR−2 ,
E[D] ≈
1
SNR−2 ,
(L−1)2
for L = 1
.
for L > 1
(17)
Although the distortion exponent derived for a single channel with perfect CSIT is achievable for L > 1 using uncoded transmission, this is not the case for parallel channels. A
scheme that can achieve the distortion exponent given in (16) without CSIT likely requires some form of joint source-channel coding. Next, we analyze several practical
schemes that do not require CSIT and compare their performance to the lower bound
derived in this section.
3.2
Uncoded Repetition
The natural extension of uncoded transmission considered in Section 2.2 to parallel channels consists of simply transmitting the source uncoded on each component channel.
Because s and y are jointly Gaussian RVs, the minimum mean-square error (MMSE)
estimate of s coincides with the linear least squares (LLS) estimate, for which closed
form expressions for the estimate and the distortion exist. The expected distortion using
vector LLS estimation is
1
−1
−2 1/SNR
,
(18)
E[D] = SNR − SNR e
E1
SNR
1
≈
· SNR−1 .
(19)
L
Note that (19) differs from (6) only in the absence of the log(L · SNR) term. This implies
uncoded transmission over a single channel and parallel channels have the same distortion
exponent.
3.3
Multi-Rate Digital Transmission
Perhaps the simplest form of digital transmission over parallel channels consists of performing separate source and channel coding and selection combining at the destination.
We allow the rates of each description, R1 and R2 , to be chosen independently, a condition slightly more general than that considered in [5] where the rates are constrained to
be equal. Without loss of generality we assume the (possibly) higher rate code is sent
over channel 1, i.e. R1 ≥ R2 .
Using techniques analogous to the outage probability analysis of Section 2.3, we find
the expected distortion and optimize the rate. Interestingly, numerical optimization of
R1 and R2 reveals that a lower average distortion is achieved for R1 6= R2 . In the limit
of high SNR, the expected distortion can be approximated as
2L+1
−
E[D] ≈ (L + 1) L+1 · (L · SNR)
L(2L+1)
(L+1)2
.
(20)
As previously mentioned, a simpler version of digital transmission with selection combining [5] constrains the rates to be equal. In the limit of high SNR, this digital repetition
scheme can be approximated as
2
L+2
2L
2
L
E[D] ≈ 1 +
· SNR− L+2 .
L
2
(21)
In [5] several other schemes were also analyzed in this context, such as optimal channel
coding diversity, which was shown to have a distortion exponent of ∆(L) = 2L/(L + 1).
3.4
Parallel Successive Refinement
A simple extension of the successive refinement scheme considered in Section 2.4 to parallel channels consists of transmitting the base and enhancement layers each on their
own independent channel. The encoder generates a dual-layer successive refinement code
consisting of a base layer description, xB , at rate RB nats per source sample and an enhancement layer description, xE , at rate RE nats per source sample. We also considered
power allocation between each component channel and found the gains in performance
are negligible.
We compute the expected distortion for a given RB and RE , and optimize numerically.
Just as for the successive refinement scheme considered for a single channel, the optimal
rates can be approximated as RB ≈ r0B + rB · log(SNR) and RE ≈ r0E + rE · log(SNR)
for high SNR. Minimizing over r0B , r0E , rB , and rE leads to the approximation
L+2
− L(L+2)
2
E[D] ≈ (L + 1) L+1 · (L · SNR)
3.5
(L+1)
.
(22)
Hybrid Digital-Analog Transmission
We consider a scheme that offers good performance for all values of L, and has the same
distortion exponent as optimal channel coding diversity [5]. The technique consists of
transmitting both an analog and a digital version of the source, one on each component
channel, and is therefore a hybrid digital-analog (HDA) scheme, with similarities to that
presented in [7]. On the first channel we transmit the source uncoded; this is optimal
in terms of its distortion exponent. The distortion on this channel will be independent
of the source, because it is solely caused by additive noise. On the second channel we
perform rate-optimized single-layer separate source and channel coding. Note that this
scheme can be thought of as a form of multiple descriptions coding [1] in the sense that
we are providing complete, yet complementary, description of the source on each channel.
We assume the receiver has perfect knowledge of the fading coefficients and can determine the reliability of decoding the digital transmission. If reliable decoding is possible,
the receiver decodes the digital transmission to form an estimate of the source ŝ2 and
linearly combines it with the noisy uncoded and scaled version ŝ1 . If we are unable to
reliably decode the digital transmission, the final source reconstruction is the MMSE
estimate of s given the noisy uncoded description. Using the LLS combiner results in an
average distortion
2R/L
1
1
1
e
−1
exp
E1
E[D] = 1 − exp −
SNR
SNR
SNR
SNR
2R/L
2R 2R e
−1 1
e
e
+ exp −
exp
E1
.
SNR
SNR
SNR
SNR
As in other digital systems studied, the numerically optimized rate grows linearly with
log(SNR) for asymptotically high SNR, and can be approximated as R ≈ r0 + r · SNR.
The final high SNR approximation is
1−L
2L
E[D] ≈ L 1+L log(L · SNR) · SNR− L+1 .
(23)
In the limit of high SNR the digital distortion can be modeled as additive Gaussian
noise independent of the source, with variance equal to the distortion-rate function of
the source [3]. In this case, the MMSE estimate of s is a linear function of ỹ1 and ỹ2 .
2
0.9
1.8
0.8
1.6
0.7
1.4
0.6
1.2
∆(L)
∆(L)
1
0.5
1
0.4
0.8
0.3
0.6
0.2
0.4
0.1
0.2
0
−10
−8
−6
−4
−2
0
2
4
Bandwidth Expansion Ratio, L (dB)
6
(a) Single Channel
8
10
0
−10
−8
−6
−4
−2
0
2
4
Bandwidth Expansion Ratio, L (dB)
6
8
10
(b) Parallel Channels
Figure 1: Distortion exponents ∆(L) as a function of the bandwidth expansion ratio, L,
in dB for a single channel (a) and parallel channels (b). At L = −4 dB from top to bottom
the curves correspond to: (a) the CSIT upper bound, two-layer successive refinement,
single-layer separate source and channel coding, and uncoded transmission, (b) the CSIT
upper bound, hybrid digital-analog transmission, multi-rate digital transmission, analog
repetition, and parallel successive refinement.
This means for asymptotically high SNR, the LLS estimate will converge to the MMSE
estimate. Since (23) is a characterization of how the system behaves at asymptotically
high SNR, it is also valid for HDA transmission utilizing a significantly more complicated
MMSE combiner.
In addition to good performance at high SNR, the HDA scheme has the desirable
property that there is never a complete outage event. This means it may have benefits
in terms of the higher order moments of the distortion. The uncoded version of the
source ensures that it is always possible to provide an estimate of the source that is
better than the naive choice of reconstructing to the source mean. A consequence of this
attribute is that the distortion has a more desirable PDF than one where a complete
outage event occurs with non-zero probability. Although this distinction is not apparent
by comparing the first moment of the distortion, for practical systems, a higher order
moment performance criterion may be of importance.
4
4.1
Discussion
System Comparison
Figure 1 shows the distortion exponents as a function of the bandwidth expansion ratio.
The average distortion for each scheme with L = 1 and L = 3 are shown in Figure 2 for
a single channel and Figure 3 for parallel channels.
4.2
Summary
This paper considers transmitting an i.i.d. Gaussian source over an AWGN channel with
Rayleigh block-fading. We introduced a more accurate characterization of a scheme’s
0
0
−2
−5
−4
−10
−6
−15
E[D] (dB)
E[D] (dB)
−8
−10
−12
−20
−25
−14
−30
−16
−35
−18
−20
0
5
10
15
20
25
30
35
−40
40
0
5
10
15
SNR (dB)
20
25
30
35
40
SNR (dB)
(a) L = 1
(b) L = 3
Figure 2: Average distortion (solid lines) and their corresponding approximations
(dashed) on a single quasi-static Rayleigh fading AWGN channel with bandwidth expansion ratio (a) L = 1 and (b) L = 3. The lower bound (Section 2.1) on expected distortion
is shown with (+), uncoded transmission (Section 2.2) with (), optimal separate source
and channel coding (Section 2.3) with (◦), and successive refinement (Section 2.4) with
(⋄).
0
0
−5
−5
−10
−10
−15
−20
E[D] (dB)
E[D] (dB)
−15
−20
−25
−30
−25
−35
−30
−40
−35
−40
−45
0
5
10
15
20
SNR (dB)
(a) L = 1
25
30
35
40
−50
0
5
10
15
20
SNR (dB)
25
30
35
40
(b) L = 3
Figure 3: Average distortion (solid lines) and their corresponding approximations
(dashed) on parallel quasi-static Rayleigh fading AWGN channels with bandwidth expansion ratio (a) L = 1 and (b) L = 3. The lower bound (Section 2.1) on distortion is
shown with (+), analog repetition (Section 3.2) with (), multi-rate digital (Section 3.3)
with (◦), parallel successive refinement (Section 3.4) with (⋄), and hybrid digital-analog
(Section 3.5) with (△).
average distortion of the form E[D] ≈ C(L) · log(L · SNR)p · SNR−∆(L) . We showed that
the distortion exponent, although always a relevant metric, does not always provide an
accurate characterization of expected distortion. In particular, the additional parameters
C(L) and p become very important for large L. One example that illustrates this notion
is the fact that uncoded transmission over a channel with L > 1 is optimal in terms of the
distortion exponent, but there is a significant gap between E[D] for uncoded transmission
and the CSIT lower bound.
Power allocation was also considered as a means to decrease distortion for schemes
that operate with multiple layers or coding rates simultaneously. Although power allocation improved performance for most schemes, the improvement was often negligible
at high SNR and may not outweigh the added system complexity and cost. The one
exception to this was the superposition successive refinement coding scheme for a single
channel considered in Section 2.4. Because the enhancement layer was treated as additive noise, it was essential that as SNR increased, more power was allocated to the base
layer.
For parallel channels, we introduced a hybrid digital-analog scheme and found that
it has the same distortion exponent as optimal channel coding [5]. Its performance,
however, still falls well below the lower bound derived in Section 3.1.
References
[1] Abbas A. El Gamal and Thomas M. Cover. Achievable Rates for Multiple Descriptions. IEEE Trans. Inform. Theory, 28(6):851–857, November 1982.
[2] W. H. R. Equitz and Thomas M. Cover. Successive refinement of information. IEEE
Trans. Inform. Theory, 37(2):269–275, March 1991.
[3] Allen Gersho and Robert M. Gray. Vector Quantization and Signal Compression.
Kluwer Academic Publishers, Boston, MA, 1992.
[4] Deniz Gunduz and Elza Erkip. Source and Channel Coding for Quasi-Static Fading
Channels. In Proc. Asilomar Conf. Signals, Systems, and Computers, Pacific Grove,
CA, October 2005. To appear.
[5] J. Nicholas Laneman, Emin Martinian, Gregory W. Wornell, and John G. Apostolopoulos. Source-Channel Diversity Approaches for Multimedia Communication.
IEEE Trans. Inform. Theory, October 2005. To appear.
[6] Lawrence H. Ozarow, Shlomo Shamai (Shitz), and Aaron D. Wyner. Information Theoretic Considerations for Cellular Mobile Radio. IEEE Trans. Vehic. Tech., 43(5):359–
378, May 1994.
[7] Shlomo Shamai (Shitz), Sergio Verdú, and Ram Zamir.
Systematic Lossy
Source/Channel Coding. IEEE Trans. Inform. Theory, 44(2):564–579, March 1998.
[8] Claude E. Shannon. A Mathematical Theory of Communication. Bell Syst. Tech. J.,
27:379–423 and 623–656, July,October 1948.