Information Theory of Wireless Networks:
A Deterministic Approach
David Tse
Wireless Foundations
U.C. Berkeley
CISS 2008
March 21, 2008
Joint work with Salman Avestimehr, Guy Bresler,
Suhas Diggavi, Abhay Parekh.
The Holy Grail
• Shannon’s information theory provides the basis for
all modern-day communication systems.
• His original theory was point-to-point.
• After 60 years we are still very far away from
generalizing the theory to networks.
• We propose an approach to make progress in the
context of wireless networks.
Modeling the Wireless Medium
• broadcast
• superposition
• high dynamic range in channel strengths between
different nodes
• Basic model: additive Gaussian channel:
Gaussian Network Capacity: What We Know
Tx
Rx
point-to-point (Shannon 48)
C = log2 (1 + SNR)
Tx 1
Rx1
Rx
Tx 2
multiple-access
(Alshwede, Liao 70’s)
Tx
Rx 2
broadcast
(Cover, Bergmans 70’s)
What We Don’t Know
Unfortunately we don’t know the capacity of most
other Gaussian networks.
Tx 1
Rx 1
Tx 2
Rx 2
Interference
(Best known achievable region: Han & Kobayashi 81)
Relay
S
relay
D
(Best known achievable region: El Gamal & Cover 79)
30 Years Have Gone by…..
We are still stuck.
How to make progress?
It’s the model.
• Shannon focused on noise in point-to-point
communication.
• But many wireless networks are interference rather
than noise-limited.
• We propose a deterministic channel model
emphasizing interaction between users’ signals
rather than on background noise.
• Far more analytically tractable and can be used to
determine approximate Gaussian capacity
Agenda
Warmup:
• point-to-point channel
• multiple access channel
• broadcast channel
The meat:
• relay networks (Avestimehr, Diggavi & T. 07)
• interference channels (Bresler &T. 08, Bresler,Parekh & T. 08)
Example 1: Point-to-Point Link
Gaussian
Cawgn (SNR) =
1
2
Deterministic
log(1 + SNR)
Transmit a real number
Least significant bits are
truncated at noise level.
If
p
SNR = 2n we have
Cdet (n) = n
n / SNR on the dB scale
Example 2: Multiple Access
Gaussian
SNR1 ¸ SNR2
SNR1
Deterministic
user 2
SNR2
user 1 sends cloud centers, user 2
sends clouds.
mod 2
addition
user 1
Comparing Multiple Access Capacity Regions
Gaussian
Deterministic
SNR1 ¸ SNR2
SNR1
user 2
SNR2
mod 2
addition
user 1
R2
R2
R1 + R2 =
n2
logSNR2
log(1 + SNR1 + SNR2 )
¼ logSNR1
accurate to within
(3; 2)
1 bit per user
logSNR1
R1
n 1 R1
Example 3: Broadcast
Gaussian
Deterministic
n1 = 5
user 1
SNR1 ¸ SNR2
SNR1
SNR2
n2 = 2
log(1 +SNR2 )


n2
R2
To within 1 bit

R1
n1

log(1 +SNR1)
user 2
Agenda
Warmup:
• point-to-point channel
• multiple access channel
• broadcast channel
The meat:
• relay networks
• interference channels
History
• The (single) relay channel was first proposed by Van
der Meulen in 1971.
• Cover and El Gamal (1979) provided a whole array of
achievable strategies.
• Recent generalization of these techniques to more
than 1 relay.
• Do not know how far they are from optimal
¹ cut set
• General upper bound: cutset bound C
The Relay Channel
Gaussian
hSR
R
hSD
S
Deterministic
hRD
D
nSR
nRD
Theorem (Avestimehr et al 07)
Gap from cutset bound is at most 1 bit.
nSD
¹ cut set = min (max(nSD ; nSR ); max(nSD ; nRD ))
C
| hRD |2
| hSD |2
gap
| hSR |2
| hSD |2
On average it is much less than 1-bit
Decode-Forward is near optimal
¡
¢
= nSD + min (nSR ¡ nSD ) + ; (nR D ¡ nSD ) +
Cutset bound is achievable.
Decode-Forward is optimal
Generalization to Relay Networks
• Can the cutset bound be achievable in the
deterministic model?
• Can one always achieve to within a contant gap of
the cutset bound in the Gaussian case?
General Relay Networks
Sc
S
Main Theorem:
Cutset bound is achievable for deterministic networks.
¹ cut set = min rank(G S;S c )
Cr elay = C
S
(Avestimehr, Diggavi & T. 07)
Main Theorem
Cr elay = min rank(G S;S c )
S
Theorem generalizes to arbitrary linear MIMO
channels on finite fields.
In the case of wireline graph, rank G S;S c is the
number of links crossing the cut.
Our theorem is a generalization of Ford-Fulkerson
max-flow min-cut theorem.
Connections to Network Coding
• Achievability: random linear coding at relays
• Proof style: similar to Ahlswede et al 2000 for wireline
networks.
• Technical innovation: dealing with “inter-symbol
interference” between signals arriving along paths of
different lengths.
Back to Gaussian Relay Networks
Approximation Theorem:
There is a scheme that achieves within a constant
gap to the cutset bound, independent of the SNR’s of
the links.
(Avestimehr, Diggavi and T. 2008)
Agenda
Warmup:
• point-to-point channel
• multiple access channel
• broadcast channel
The meat:
• relay networks
• interference channels
Interference
• So far we have looked at single source, single
destination networks.
• All the signals received is useful.
• With multiple sources and multiple destinations,
interference is the central phenomenon.
• Simplest interference network is the two-user
interference channel.
Two-User Gaussian Interference Channel
message m1
want m1
message m2
want m2
• Capacity region unknown
• Best known achievable region: Han & Kobayashi 81.
Gaussian to Deterministic Interference Channel
Gaussian
Deterministic
n
In symmetric case, channel
described by two parameters:
SNR, INR
n $ log2 SNR: m $ log2 INR
Capacity can be computed using
a result by El Gamal and Costa 82.
m
Symmetric Deterministic Capacity
r=
1
1/2
® =
=
®=
r =
2
3
1
3
®=
r =
2
3
2
3
log INR
log SNR
m
n
Back to Gaussian
• Theorem:
Constant gap between capacity regions of the twouser deterministic and Gaussian interference
channels.
(Bresler & T. 08)
• A deeper view of earlier 1-bit gap result on two-user
Gaussian interference channel (Etkin,T. & Wang 06).
• Bounds further sharpened to get exact results in the
low-interference regime ( < 1/3)
(Shang et al 07,Annaprueddy&Veeravalli08,Motahari&Khandani07)
Extension:
Many-to-One Interference Channel
Gaussian
Deterministic
Deterministic capacity can be computed exactly .
Gaussian capacity to within constant gap, using structured
codes and interference alignment.
(Bresler, Parekh & T. 07)
Example
Tx0
Rx0
Tx1
Rx1
Tx2
Rx2
• Interference from users 1 and 2 is aligned at the MSB
at user 0’s receiver in the deterministic channel.
• How can we mimic it for the Gaussian channel ?
Gaussian
Lattice
codes
Han-Kobayashi
can achieveNot
constant
Optimal
gap
• Suppose users 1 and 2 use a random
Gaussian codebook:
Random Code
Sum of Two Random Codebooks
Lattice Code for Users 1 and 2
Interference from users 1 and 2 fills the space: no
room for user 0. User 0 Code
Tx0
Rx0
Tx1
Rx1
Tx2
Rx2
Interference Channels: Recap
• In two-user case, we showed that an existing strategy
can achieve within 1 bit to optimality.
• In many-to-one case, we showed that a new strategy
can do much better.
• General K-user interference channel still open.
Evolution of Ideas
• deterministic network capacity in 1980’s:
– broadcast channels (Marton 78, Pinsker 79)
– 2-user interference channel (El Gamal & Costa 82)
– single-relay channel (El Gamal & Aref 82)
– relay networks with broadcast but no interference (Aref 79)
• inspired by network coding in early 2000’s:
– finite-field model with erasures (Gupta et al 06)
but connection to Gaussian networks missing.
• 2-user Gaussian interference channel capacity to within 1 bit
(Etkin, T & Wang 06)
• Linear deterministic model (Avestimehr, Diggavi & T 07) and
applied to relay networks.
Parting Words
• Main message:
If something can’t be computed exactly, approximate.
• Similar evolution has happened in other fields:
– fluid and heavy-traffic approximation in queueing networks
– approximation algorithms in CS theory
• Approximation should be good in engineering-relevant
regimes.