Tighter Cut-Based
Bounds for k-pairs
Communication Problems
Nick Harvey
Robert Kleinberg
Overview

Definitions
 Sparsity
and Meagerness Bounds
Show these bounds very loose
 Define Informational Meagerness

 Based

on Informational Dominance
Show that it can be slightly loose
k-pairs Communication Problem
S(1)
M1
M2
S(2)
M1⊕M2
T(2)
T(1)
Concurrent Rate
Source i desires communication rate di.
 Rate r is achievable if rate vector
[ rd1, rd2, …, rdk ] is achievable
 Rate region interval of R+

Def: “Network coding rate” (or NCR)
:= sup { r : r is achievable }
k-pairs Communication Problem
S(1)
M1
M2
M1⊕M2
T(2)
S(2)
d1 = d2 = 1
ce = 1 eE
Rate 1 achievable
T(1)
Upper bounds on rate
[Classical]: Sparsity bound for multicommodity flows
[CT91]: General bound for multi-commodity information
networks
[B02]: Application of CT91 to directed network coding
instances; equivalent to sparsity.
[KS03]: Bound for undirected networks with arbitrary
two-way channels
[HKL04]: Meagerness
[SYC03], [HKL05]: LP bound
[KS05]: Bound based on iterative d-separation
Vertex-Sparsity
Def: For U  V,
VS (U) :=
Capacity of edges crossing between U and U
Demand of commodities separated by U
 VS (G) := minUV
VS (U)
Claim: NCR  VS (G)
Edge-Sparsity

Def: For A  E,
ES (A) :=
Capacity of edges in A
Demand of commodities separated in G\A
 ES (G) = minAE
ES (A)
Claim: Max-Flow  ES (G)
But: Sometimes NCR > ES (G)
NCR > Edge-Sparsity
S(1)
S(2)
e
T(2)

Cut {e} separates S(1) and S(2)
 ES ({e}) = 1/2

But rate 1 achievable!
T(1)
Meagerness

Def: For A  E and P  [k],
A isolates P if for all i,j  P,
S(i) and T(j) disconnected in G\A.
M (A) := minP isolated by A
 M (G) := minAE
Capacity of edges in A
Demand of commodities in P
M (A)
Claim: NCR  M (G)
Meagerness & Vtx-Sparsity are weak
S(n)
fn-1
gn
S(n-1)
f3
gn-1
S(3)
f2
g3
S(2)
f1
g2
S(1)
g1
Gn :=
T(n)

hn-1
T(n-1)
h3
T(3)
h2
Thm: M (Gn) = VS (Gn) = (1),
but NCR  1/n.
T(2)
h1
T(1)
A Proof Tool
Def: Let A,B  E. B is downstream of A
if B disconnected from sources in G\A.
Notation: A  B.
Claim: If A  B then H(A)  H(A,B).
Pf: Because S  A  B form Markov chain.
Lemma: NCR  1/n
S(n)
fn-1
gn
S(n-1)
gn-1
f3
S(3) f2
g3
S(2) f1
g2
S(1)
g1
Gn :=
T(n)
hn-1
T(n-1)
Proof:
{gn}  {gn,T(1),h1}
h3 T(3) h2
T(2) h1
T(1)
Lemma: NCR  1/n
S(n)
fn-1
gn
S(n-1)
gn-1
f3
S(3) f2
g3
S(2) f1
g2
S(1)
g1
Gn :=
T(n)
hn-1
T(n-1)
h3 T(3) h2
T(2) h1
Proof:
{gn}  {gn,T(1),h1}  {S(1),f1,g1,h1}
T(1)
Lemma: NCR  1/n
S(n)
fn-1
gn
S(n-1)
gn-1
f3
S(3) f2
g3
S(2) f1
g2
S(1)
g1
Gn :=
T(n)
hn-1
T(n-1)
h3 T(3) h2
T(2) h1
Proof:
{gn}  {gn,T(1),h1}  {S(1),f1,g1,h1}
 {S(1),f1,T(2),h2}
T(1)
Lemma: NCR  1/n
S(n)
fn-1
gn
S(n-1)
gn-1
f3
S(3) f2
g3
S(2) f1
g2
S(1)
g1
Gn :=
T(n)
hn-1
T(n-1)
h3 T(3) h2
T(2) h1
Proof:
{gn}  {gn,T(1),h1}  {S(1),f1,g1,h1}
 {S(1),f1,T(2),h2}  {S(1),S(2),f2,g2,h2}
T(1)
Lemma: NCR  1/n
S(n)
fn-1
gn
S(n-1)
gn-1
f3
S(3) f2
g3
S(2) f1
g2
S(1)
g1
Gn :=
T(n)
hn-1
T(n-1)
h3 T(3) h2
T(2) h1
Proof:
{gn}  {gn,T(1),h1}  {S(1),f1,g1,h1}
 {S(1),f1,T(2),h2}  {S(1),S(2),f2,g2,h2}
 {S(1),S(2),f2,T(3),h3}
T(1)
Lemma: NCR  1/n
S(n)
fn-1
gn
S(n-1)
gn-1
f3
S(3) f2
g3
S(2) f1
g2
S(1)
g1
Gn :=
T(n)
hn-1
T(n-1)
h3 T(3) h2
T(2) h1
Proof:
{gn}  …  {S(1),S(2),…,S(n)}
Thus 1  H(gn)  H(S(1),…,S(n)) = n∙r
So 1/n  r
T(1)
Towards a stronger bound

Our focus: cut-based bounds
 Given A 
E, we want to infer that
H(A)  H(A,P) where P{S(1),…,S(k)}
Meagerness uses Markovicity:
(sources in P)  A  (sinks in P)
 Markovicity sometimes not enough…

Informational Dominance
Def: A dominates B if information in A
determines information in B
in every network coding solution.
Denoted A i B.
 Trivially implies H(A)  H(A,B)
 How to determine if A dominates B?
 [HKL05]
give combinatorial characterization
and efficient algorithm to test if A dominates B.
Informational Meagerness
Def: For A  E and P  {S(1),…,S(k)},
A informationally isolates P if
AP i P.
Capacity of edges in A
 iM (A) = minP
Demand of commodities in P

for P informationally isolated by A
 iM (G) = minA  E
iM (A)
Claim: NCR  iM (G).
iMeagerness Example
s1
s2
t1
t2


“Obviously” NCR = 1.
But no two edges disconnect t1 and t2 from both
sources!
iMeagerness Example
s1
s2
t1
Cut A
t2

After removing A, still a path from s2 to t1!
Informational Dominance Example
s1
Cut A
s2
t1
t2
Our characterization shows A i {t1,t2}
 H(A)  H(t1,t2) and iM (G) = 1

A bad example: Hn
Thm: iMeagerness gap of Hn is (log |V|)
s(00)
H2
s(0)
q(00)
r(00)
t(00)
t(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
t(01)
t(10)
t(ε)
s(1)
s(11)
q(11)
r(11)
t(1)
t(11)
Capacity 2-n
s(00)
s(0)
s(ε)
s(01)
s(10)
s(1)
s(11)
Tn = Binary tree of depth n
Source S(i) iTn
s(00)
s(0)
s(ε)
s(01)
s(10)
s(1)
s(11)
Tn = Binary tree of depth n
Source S(i) iTn
Sink T(i) iTn
t(00)
t(0)
t(01)
t(10)
t(ε)
t(1)
t(11)
s(00)
s(0)
s(ε)
s(01)
s(10)
s(1)
s(11)
Nodes q(i) and r(i) for every leaf i of Tn
q(00)
r(00)
t(00)
t(0)
q(01)
r(01)
q(10)
r(10)
t(01)
t(10)
t(ε)
q(11)
r(11)
t(1)
t(11)
s(00)
s(0)
q(00)
r(00)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
q(11)
r(11)
Complete bip. graph between sources and q’s
t(00)
t(0)
t(01)
t(10)
t(ε)
t(1)
t(11)
(r(a),t(b)) if b ancestor of a in Tn
s(00)
s(0)
q(00)
r(00)
t(00)
t(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
t(01)
t(10)
t(ε)
s(1)
s(11)
q(11)
r(11)
t(1)
t(11)
(s(a),t(b)) if a and b cousins in Tn
s(00)
s(0)
q(00)
r(00)
t(00)
t(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
t(01)
t(10)
t(ε)
s(1)
s(11)
q(11)
r(11)
t(1)
t(11)
All edges have capacity  except (q(i),r(i))
s(00)
s(0)
q(00)
r(00)
t(00)
t(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
t(01)
t(10)
t(ε)
s(1)
s(11)
q(11)
r(11)
t(1)
t(11)
Capacity 2-n
Demand of source at depth i is 2-i
s(00)
s(0)
q(00)
r(00)
t(00)
t(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
t(01)
t(10)
t(ε)
s(1)
s(11)
q(11)
r(11)
t(1)
t(11)
Capacity 2-n
Properties of Hn
Lemma: iM (Hn) = (1)
Lemma: NCR < 1/n
Corollary: iMeagerness gap is n=(log |V|)
Properties of Hn
Lemma: iM (Hn) = (1)
Lemma: NCR < 1/n
Corollary: iMeagerness gap is n=O(log |V|)
We will prove this
Proof Ingredients

Entropy moneybags
 i.e.,

sets of RVs
Entropy investments
 Buying
sources and edges, putting into moneybag
 Loans may be necessary

Profit
 Via
Downstreamness or Info. Dominance
 Earn new sources or edges for moneybag

Corporate mergers
 Via
Submodularity
 New Investment Opportunities and Debt Consolidation

Debt repayment
Submodularity of Entropy
Claim: Let A and B be sets of RVs.
Then H(A)+H(B)  H(AB)+H(AB)
Pf: Equivalent to I( X; Y | Z )  0.
Lemma: NCR < 1/n
Proof:
 Two entropy moneybags:
 F(a)
= { S(b) : b not an ancestor of a }
 E(a) = F(a)  { (q(b),r(b)) : b is descendant of a }
Entropy Investment
a
s(00)
q(00)
r(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)

Let a be a leaf of Tn

Take a loan and buy E(a).
s(1)
s(11)
q(11)
r(11)
Earning Profit
a
s(00)
q(00)
r(00)
t(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
q(11)
r(11)
Claim: E(a)  T(a)
Pf: Cousin-edges not from ancestors.
Vertex r(00) blocked by E(a).
Earning Profit
a
s(00)
q(00)
r(00)
t(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
q(11)
r(11)
Claim: E(a)  T(a)
Result:
E(a) gives free upgrade to E(a){S(a)}.
Profit = S(a).
aL
s(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
E(aL){S(aL)}
q(00)
r(00)
aR
s(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
q(11)
r(11)
s(1)
s(11)
E(aR){S(aR)}
q(00)
r(00)
q(11)
r(11)
Applying submodularity
s(00)
(E(aL){S(aL)})

(E(aR){S(aR)})
s(0)
q(00)
r(00)
s(00)
(E(aL){S(aL)})

(E(aR){S(aR)})
q(00)
r(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
q(11)
r(11)
s(1)
s(11)
q(11)
r(11)
New Investment
a
(E(aL){S(aL)})

(E(aR){S(aR)})
s(00)
q(00)
r(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
q(11)
r(11)
Union term has more edges
 Can use downstreamness
or informational dominance again!
 (E(aL){S(aL)})  (E(aR){S(aR)}) = E(a)

Debt Consolidation
Intersection term has only sources
 Cannot earn new profit.
Used for later “debt repayment”
 (E(aL){S(aL)})  (E(aR){S(aR)}) = F(a)

a
s(00)
(E(aL){S(aL)})

(E(aR){S(aR)})
q(00)
r(00)
s(0)
s(ε)
s(01)
s(10)
q(01)
r(01)
q(10)
r(10)
s(1)
s(11)
q(11)
r(11)
What have we shown?



Let aL,aR be sibling leaves; a is their parent.
H(E(aL)) + H(E(aR))  H(E(a)) + H(F(a))
Iterate and sum over all nodes in tree
 H ( E (l ))  H ( E (r ))   H ( F (v))
leaf l

nonleaf v
where r is the root.
Note: E(v) = F(v)  {(q(v),r(v))} when v is a leaf
 H ( F (l ))   H (( q (l ), r (l )))
leaf l
leaf l
 H ( E (r ))   H ( F (v))
nonleaf v
Debt Repayment
 H ( F (l ))   H (( q (l ), r (l )))
leaf l
leaf l
 H ( E (r ))   H ( F (v))
nonleaf v
( F (v))
l H ( F (l ))  nonleaf
H
Claim: leaf
v
Pf: Simple counting argument.

 H ((q(l ), r (l )))  H ( E (r ))
leaf l
Finishing up
 H ((q(l ), r (l )))

H ( E (r ))
leaf l

=
 c( q (l ),r (l ))
 H ( S (i))
leaf l
i
=
=
1
  2
 depth ( v )
 n
v
(where α = rate of solution)
 Rate < 1/n