Lower Bounds on the Communication
of Distributed Graph Algorithms:
Progress and Obstacles
Rotem Oshman
ADGA 2013
Overview: Network Models
LOCAL
CONGESTED CLIQUE
CONGEST / general network
ASYNC MESSAGE-PASSING
X
Talk Overview
I. Lower bound techniques
a. CONGEST general networks: reductions from 2party communication complexity
b. Asynchronous message passing: reductions from
multi-party communication complexity
II. Obstacles on proving lower bounds for the
congested clique
Communication Complexity
𝑓 𝑋, 𝑌 = ?
𝑋
𝑌
Example: DISJOINTNESS
DISJ𝑛 :
𝑋 ⊆ {1, … , 𝑛}
𝑋 ∩ 𝑌 = ∅?
Ω(𝑛) bits
needed
𝑌 ⊆ {1, … , 𝑛}
[Kalyanasundaram and Schnitger, Razborov ’92]
Applying 2-Party Communication
Complexity Lower Bounds
Textbook reduction:
Given algorithm 𝐴 for solving task 𝑇…
𝑋
𝑌
Simulate 𝐴
Based
on 𝑋
𝑛 bits
Based
on 𝑌
Solution for 𝑇 ⇨ answer for DISJOINTNESS
Example: Spanning Trees
• Setting: directed, strongly-connected network
• Communication by local broadcast with
bandwidth 𝐵
• UIDs 1, … , 𝑛
• Diameter 2
• Question: how many rounds to find a rooted
spanning tree?
New Problem: PARTITION
• Inputs: 𝑋, 𝑌 ⊆ 1, … , 𝑛 , with the promise
that 𝑋 ∪ 𝑌 = 1, … , 𝑛
𝑋
• Goal:
𝑌
Alice outputs 𝐴 ⊆ 𝑋,
Bob outputs 𝐵 ⊆ 𝑌 such that
𝐴, 𝐵 partition {1, … , 𝑛}.
The PARTITION Problem
• Trivial algorithm:
– Alice sends her input to Bob
– Alice outputs all tasks in her input
– Bob outputs all remaining tasks
• Communication complexity: 𝑛 bits
• Lower bound?
Reduction from DISJ to PARTITION
• Given input 𝑋, 𝑌 for DISJ :
– Notice: 𝑋 ∩ 𝑌 = ∅ iff 𝑋 ∪ 𝑌 = 𝑛
– To test whether 𝑋 ∪ 𝑌 = 𝑛 :
• Try to solve PARTITION on 𝑋, 𝑌 ⇒ 𝐴, 𝐵
• Ensure 𝐴 ⊆ 𝑋, 𝐵 ⊆ 𝑌
• Check if 𝐴, 𝐵 is a partition of 𝑛 : Alice sends Bob
hash(𝐴), Bob compares it to hash(𝐵)
From PARTITION to Spanning Tree
Given a spanning tree algorithm 𝐴…
1
2
a
𝑋 = {1,2,3}
3
4
5
6
b
𝑌 = {2,4,5,6}
From PARTITION to Spanning Tree
Simulating one round of 𝐴 :
1
2
a
Node b’s
message
𝑋 = {1,2,3}
3
4
6
5
b
Node a’s
message
𝑌 = {2,4,5,6}
From PARTITION to Spanning Tree
When 𝐴 outputs a spanning tree:
1
2
a
𝑋 = {1,2,3}
3
4
5
6
b
𝑌 = {2,4,5,6}
From PARTITION to Spanning Tree
• If 𝐴 runs for 𝑡 rounds, we use 2𝐵𝑡 bits
⇒ 𝑡 = Ω 𝑛/𝐵
• One detail: randomness
– Solution: Alice and Bob use public randomness
When Two Players Just Aren’t Enough
• No bottlenecks in the network
When Two Players Just Aren’t Enough
• Too much information revealed
Multi-Player Communication
Complexity
• Communication by shared blackboard ??
• Number-on-forehead
• Number-in-hand
The Message-Passing Model
•
•
•
•
𝑘 players
Private channels
Private 𝑛-bit inputs 𝑋1 , … , 𝑋𝑘
Private randomness
• Goal: compute 𝑓 𝑋1 , … , 𝑋𝑘
• Cost: total communication
The Coordinator Model
• 𝑘 players, one coordinator
• The coordinator has no input
Message-Passing vs. Coordinator
≈
Prior Work on Message-Passing
• For 𝑘 players with 𝑛-bit inputs…
• Phillips, Verbin, Zhang ’12:
– Ω 𝑘𝑛 for bitwise problems (AND/OR, MAJ, …)
• Woodruff, Zhang ‘12, ‘13:
– Ω 𝑘𝑛 for threshold and graph problems
• Braverman, Ellen, O., Pitassi, Vaikuntanathan
‘13: Ω 𝑘𝑛 for DIS J 𝑛,𝑘
Set Disjointness
𝑋2
𝑋1
𝑋3
?
𝑋5
𝑋4
𝑛
𝑘
𝑗
𝑋𝑖
DIS J 𝑛,𝑘 =
𝑖=1 𝑗=1
Notation
• Π : randomized protocol
– Also, the protocol’s transcript
– Π𝑖 : player 𝑖’s view of the transcript
• 𝐶𝐶 Π = worst-case communication of Π
• 𝐶𝐶𝜖 𝑓 =
min
Π with error 𝜖
𝐶𝐶(Π)
in the worst case
Entropy and Mutual Information
• Entropy:
𝐻 𝑋 =−
Pr 𝑋 = 𝑥 ⋅ log Pr 𝑋 = 𝑥
𝑥
• A lossless encoding of 𝑋 requires 𝐻 𝑋 bits
• Conditional entropy:
𝐻 𝑋 𝑌 = 𝔼𝑦 𝐻 𝑋 𝑌 = 𝑦 ≤ 𝐻(𝑋)
Entropy and Mutual Information
• Mutual information:
𝐼 𝑋; 𝑌 = 𝐻 𝑋 − 𝐻(𝑋|𝑌)
• Conditional mutual information:
𝐼 𝑋; 𝑌 𝑍 = 𝔼𝑧 𝐼 𝑋; 𝑌 𝑍 = 𝑧
= 𝐻 𝑋 𝑍 − 𝐻(𝑋|𝑌, 𝑍)
Information Cost for Two Players
[Chakrabarti, Shi, Wirth, Yao ’01], [Bar-Yossef, Jayram, Kumar, Sivakumar ‘04], [Braverman, Rao ‘10], …
Fix a distribution 𝜇,
• External information cost:
𝐼 𝑋,𝑌 ∼𝜇 𝑋𝑌; Π = 𝐻 Π − 𝐻 Π 𝑋𝑌 ≤ |Π|
• Internal information cost:
𝐼 𝑋,𝑌 ∼𝜇 𝑋; Π 𝑌 + 𝐼 𝑋,𝑌 ∼𝜇 (𝑌; Π|X
Extension to the coordinator model:
𝑘
[𝐼 𝑋𝑖 ; Π𝑖 𝑀, 𝑍 + 𝐼(𝑀; Π𝑖 |𝑋𝑖 , 𝑍)]
𝑖=1
Why is Info Complexity Nice?
• Formalizes a natural notion
– Analogous to causality/knowledge
• Admits direct sum theorem:
“The cost of solving 𝑛 independent
copies of problem 𝑃 is 𝑛 times the cost
of solving 𝑃”
Example
𝑛
𝑘
𝑗
𝑋𝑖
DIS J 𝑛,𝑘 =
𝑖=1 𝑗=1
Example (Work in Progress)
• Triangle detection in general congested graphs
• “Is there a triangle” =
𝑣1 ,𝑣2 ,𝑣3 ”is {𝑣1 , 𝑣2 , 𝑣3 } a triangle”
Application of DISJ Lower Bound
• Open problem from Woodruff & Zhang ‘13:
– Hardness of computing the diameter of a graph
• We can show: Ω 𝑛𝑘 bits to distinguish
diameter ≤3 from diameter ∞
• Reduction from DISJ : given 𝑋1 , … , 𝑋𝑘 ,
– Notice: 𝑋1 , … , 𝑋𝑘 disjoint iff 𝑋1 ∪ ⋯ ∪ 𝑋𝑘 = 𝑛
Application of DISJ Lower Bound
3
2
4
𝑝2
1
𝑝1
𝑝3
𝑋3
5
𝑝4
6
• 𝑋1 ∪ ⋯ ∪ 𝑋𝑘 = 𝑛 ⇒ Diameter ≤ 3
• 𝑋1 ∪ ⋯ ∪ 𝑋𝑘 ≠ 𝑛 ⇒ Diameter = ∞
Part II: The Power of the Congested
Clique
CONGESTED CLIQUE
Conversion from Boolean Circuit
• Suppose we have a Boolean circuit 𝐶
– Any type of gate, 𝑛2 inputs
– Fan-in ≤ 2
– Depth = polylog(𝑛), #gates and wires =
𝑛2 polylog 𝑛
• Step 1: reduce the fan-out to ≤ 2
– Convert large fan-out gates to “copying tree”
– Blowup: 𝑂 log 𝑛 depth, 𝑂 1 size
• Step 2: convert to a layered circuit
Conversion from Boolean Circuit
• Now we have a layered circuit 𝐶′ of depth
polylog 𝑛 and size = 𝑛2 polylog 𝑛
– With fan-in and fan-out ≤ 2
• Design a CONGEST protocol:
– Fix partition of inputs 𝑆1 , … , 𝑆𝑛 of size 𝑛 each
– Assign each gate to a random CONGEST node
– Simulate the circuit layer-by-layer
Simulating a Layer
• If node 𝑢 “owns” gate 𝐺 on layer 𝑑, it sends
𝐺’s output to the nodes that need it on layer
𝑑+1
• Size of layer 𝑑 + 1
≤ 2 ⋅ size of layer 𝑑
• What is the load on edge 𝑖, 𝑗 ?
– For each wire from layer 𝑑 to layer 𝑑 + 1,
Pr 𝑤𝑖𝑟𝑒 𝑎𝑠𝑠𝑖𝑔𝑛𝑒𝑑 𝑡𝑜 𝑖, 𝑗 = 1/𝑛2
– At most 𝑛2 polylog 𝑛 wires in total
– By Chernoff, w.h.p. the load is polylog 𝑛
Conversion from Boolean Circuit
• A union-bound finishes the proof
• Corollary: explicit lower bounds in the
congested clique imply explicit lower bounds
on Boolean circuits with polylogarithmic
depth and nearly-linear size.
• Even worse:
– Reasons to believe even Ω(log log 𝑛) bound hard
Conclusion
LOCAL
CONGESTED CLIQUE
CONGEST / general network
ASYNC MESSAGE-PASSING
X