A graph-based framework for
transmission of correlated information
sources over multiuser channels
S. Sandeep Pradhan
University of Michigan,
Ann Arbor, MI
Acknowledgements
• Suhan Choi
• Kannan Ramchandran
• David Neuhoff
Outline
•
•
•
•
•
Introduction
Motivation
Problem Formulation
Main Result
Conclusions
Multiuser Communication
Multiuser Communication
Many-To-One
Communications
One-To-Many
Communications
• Practical Applications
– Sensor Networks
– Wireless Cellular Systems, Wireless LAN
– Broadcasting Systems
Motivation (1)
Near lossless transmission of correlated sources over multiuser
channels:
Encoder/
Encoders
Channel
Decoder/
Decoders
Source: Discrete Memoryless Vector
Channel: Discrete Memoryless (without feedback)
Motivation (2): Example
Encoder
Channel
Decoder
Encoder
S:
temparature readings in Ann Arbor
T:
temparature readings in Detroit
Channel: wireless channel to Lansing.
Motivation (3):Point-to-point Communication
Near lossless transmission of a source over a channel:
Channel
Decoder
Encoder
Separation Approach: [Shannon 1959]
Channel
Source
Encoder
Channel
Encoder
Channel
Decoder
Source
Decoder
Reliable transmission  Entropy of source < Capacity of channel
Motivation (4)
• Separation Approach: source coding+channel coding
• Source Coding (compression): Removal of redundancy
• Example: Distributed source coding.
• Channel Coding: Structured reintroduction of redundancy
• Example: CDMA (uplink) with multiuser detection.
• This approach is modular.
• Source coding and channel coding optimization can be done
separately.
• The Alternative: Joint source-channel coding.
Motivation (5): Example
Source
Encoder
Source
Encoder
Channel
Encoder
Channel
Encoder
C
H
A
N
N
E
L
Channel
Decoder
Source
Decoder
Motivation (5): Example
Indexes
Source
Encoder
Source
Encoder
Channel
Encoder
Channel
Encoder
Indexes
C
H
A
N
N
E
L
Channel
Decoder
Source
Decoder
Indexes
Motivation (6)
• Indexes (bits) at multiple channel
encoders are independent.
• Distributed information is represented as
multiple independent bit streams.
• Unfortunately this scheme is not optimal
Motivation (7): Example [Cover, El Gamal, Salehi, 1980]
p (u, v)
S
0
1
0
1/3
1/3
1
0
1/3
T
R2
H (S2 )
H ( S2 | S1 )
0
H ( S1 | S2 )
H (S1 )
R1
Motivation (8)
• Essence: conventional separation-approach is not
optimal for multiuser communication. This approach is
modular but not optimal.
• Shannon showed that separation-approach is optimal for
point-to-point communication.
• We have built the telephone-network and the Internet
using this principle.
• Why does it work in point-to-point case and not in
multiuser case?
• In other words how can we inject modularity in multiuser
communication without losing optimality?
Motivation (9)
Q: What makes separation work in point-to-point setting?
A: Typicality.
Non-typical set
Motivation (10): Example
• Bernoulli source with Pr(S=1)=0.2.
• Typical sequences are binary sequences
with fraction of “heads” nearly equal to 0.2.
• If you toss a biased coin (bias=0.2) many
many times, you will most likely see a
sequence which is typical.
Motivation (11)
Motivation (12):
Motivation (13)
• Not all pairs of S-typical and T-typical
sequences are jointly typical.
• Because H(S,T)<H(S)+H(T).
Motivation (14): Joint typicality can be captured by a graph
n
Typicality Graph
Graph
n
1
1
1
1
2
2
2
2
Nearly Semi-regular
Bipartite Graph
Motivation (15)
• Could nearly semi-regular bipartite graphs
be used as discrete interface for
multiterminal communication?
Graph-based separation Approach ?
Source
Encoder
Source
Encoder
Channel
Encoder
Channel
Encoder
C
H
A
N
N
E
L
Channel
Decoder
Source
Decoder
Graph-based separation Approach ?
Edges of
A graph
Source
Encoder
Source
Encoder
Channel
Encoder
Channel
Encoder
C
H
A
N
N
E
L
Channel
Decoder
Source
Decoder
Related Work: [Slepian, Wolf, 73, BSTJ], [Ahlswede, Han, 83, IT]
Big Picture
• Extended source coding: Structured way
to retain redundancy in the source
representation.
• Extended channel coding: Structured way
to reintroduce redundancy into this
representation.
Definitions: Bipartite Graphs
1
A
B
2
C
Definition: Nearly Semi-Regular Bipartite Graphs
1
1
2
2
3
3
4
5
4
6
Equivalence Classes of Graphs
• Consider
•
can be partitioned into equivalence classes
• Two graphs belong to the same classes if one can
be obtained from the other by relabeling the vertices.
Examples
Two graphs that belong to the same equivalence class
1
1
1
1
2
2
2
2
3
3
3
3
Two graphs that belong to different equivalence classes
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
Today:
• A characterization of the set of nearly semi-regular graphs
whose edges can be transmitted over a multiple-access
channel.
Multiple-Access Channel
Channel
Encoder
Channel
Encoder
C
H
A
N
N
E
L
Channel
Decoder
• Input Alphabets :
• Output Alphabet :
• Stationary Discrete Memoryless Channel without
feedback
• An ordered tuple:
Multiple-Access Channel
• This channel was introduced in 1971 by Ahlswede & Liao.
• The capacity region is known.
• Literature on this is too exhaustive to list here.
Multiple-Access Channel Capacity
[Ahlswede, Liao, 1971]
Problem Formulation: Transmission System
Example
100100000
1
1
010101010
010100010
2
2
100000101
010100010
3
3
100010100
(2,2)
(1,1)
(2,3)
(3,3)
(1,2)
(3,1)
In other words
The messages have the distribution:
Definition of Achievable Rates
Remark on Achievable Rates:
• Find a sequence of nearly semi-regular graphs
– The number of vertices & the degrees are increasing exponentially
with given rates
– Edges from these graphs are reliably transmitted
Rates are achievable
• Definition: Rate region
– The set of all achievable tuple of rates
• Goal: Find the rate region
• Note the distribution of the message pair is changing with
blocklength n.
Main Result
Remark on Theorem 1
Sketch of the Proof of Theorem 1 (1)
Sketch of the Proof of Theorem 1 (2)
Sketch of the Proof of Theorem 1 (3)
Gaussian Example
Z
X1
X2

Y
Gaussian Example Contd.
Source Coding Module
• Similarly a problem formulation for representing a pair of
correlated sources into nearly semi-regular bipartite
graphs can be done.
• One can then obtain a characterization of a set
of
nearly semi-regular bipartite graphs which can reliably
represent the source pair.
Edges
of a graph
Edges
of a graph
Channel
Source
Encoder
Channel
Encoder
Channel
Decoder
Source
Decoder
Transmission of sources over channels
• Given a source-pair and a multiple-access channel.
• What if
• Q: Does it mean that we can reliably transmit the
pair over the multiple-access channel?
• A: Not in general.
• Because the graph for the source and that for the
channel may belong to different equivalence
classes.
Conclusions
• A graph-based framework for transmission
of correlated sources over multiple-access
channels.
• A characterization of a set of nearly semiregular bipartite graphs whose edges can
be transmitted over a multiple-access
channel.