Random Regular Graph &
Generalized De Bruijn Graph
with k-shortest Path Routing
Peyman Faizian, Md Atiqul Mollah, Xin Yuan
Scott Pakin, Michael Lang
Florida State University
Los Alamos National Laboratory
Where it All Started
• Random Regular topologies(Jellyfish) have been proposed for data
centers and HPC clusters
• They are known to have some good topological properties
• They achieve higher performance in compare to Fat Tree under
certain traffic patterns
We are Going to do this the Hard Way
• Derive theoretical bounds for the following key metrics on any DRG
• Diameter
• Average k-shortest path length
• Load balancing property
• Evaluate RRG based on these criteria
• Introduce a near-optimal DRG and compare it to RRG through
simulation
What is a Random Regular Topology
• Random graph between ToR switches
•
•
•
•
Each switch has k ports
r ports connected to other switches
k-r ports connected to processing nodes
r-regular random interconnect
topology(RRG)
[Singla et al; NSDI’12]
What We Know About RRG
• High bisection bandwidth
• High network capacity
• Sufficient short path diversity
• k-shortest path routing is preferred
[Singla et al; NSDI’12]
What is Needed for K-Path Routing
• High bisection bandwidth
• Minimal resource usage
• Low diameter
• Low average k-shortest path length
• Good load balancing
• Even distribution of paths over SD pairs
• Even distribution of load over all network links
What Should We Compare RRG to
• RRG has been compared to one of the best available designs
• Fat Tree
• We want to compare it to the best possible design
• Random Regular Graphs are a special case of Directed Regular Graphs
• We derived the optimal bounds for all discussed properties on Regular Graphs
4 Lemmas. Some Graph Theory!
Diameter
33%
𝐷𝑖𝑎𝑚𝑒𝑡𝑒𝑟 𝑜𝑓 𝐷𝑅𝐺(𝑁, 𝑟) ≥ 𝑙𝑜𝑔𝑟 𝑁 𝑟 − 1 + 1 − 1
66%
Average Shortest Path Length
14%
𝐴𝑣𝑒𝑟𝑎𝑔𝑒 𝑘 𝑠ℎ𝑜𝑟𝑡𝑒𝑠𝑡 𝑝𝑎𝑡ℎ 𝑙𝑒𝑛𝑔𝑡ℎ 𝑓𝑜𝑟 𝐷𝑅𝐺(𝑁, 𝑟) ≥
ℎ−1 𝑗
𝑗=1 𝑗𝑟
+ ℎ𝑅
𝑘(𝑁 − 1)
Average k-Shortest Path Length
𝐴𝑣𝑒𝑟𝑎𝑔𝑒 𝑘 𝑠ℎ𝑜𝑟𝑡𝑒𝑠𝑡 𝑝𝑎𝑡ℎ 𝑙𝑒𝑛𝑔𝑡ℎ 𝑓𝑜𝑟 𝐷𝑅𝐺 𝑁, 𝑟 = 𝐿𝐾(𝑁, 𝑟, 𝑘) ≥
ℎ−1 𝑗
𝑗=1 𝑗𝑟
+ ℎ𝑅
𝑘(𝑁 − 1)
Path Distribution
Network Degree=30, N=901
min 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑎𝑡ℎ𝑠 𝑜𝑓 𝑙𝑒𝑛𝑔𝑡ℎ ≤ 𝐻 ∶
𝑟 + 𝑟2 + ⋯ + 𝑟𝐻
𝑁−1
Load Balancing
• All-to-All communication pattern
• Use K-shortest paths between each SD pair
• Monitor the number of flows passing each link
• Pick the load value on the most loaded link in the network
Load Balancing
𝐿𝐾 𝑁, 𝑟, 𝑘 × (𝑁 − 1)
max 𝑙𝑖𝑛𝑘 𝑙𝑜𝑎𝑑 𝑓𝑜𝑟 𝐷𝑅𝐺 𝑁, 𝑟 = 𝑀𝐿(𝑁, 𝑟) ≥
𝑟
A Quick Recap
• We compared these topological properties of Jellyfish with optimal:
•
•
•
•
Diameter
Average k-shortest path length
Path distribution
Load distribution
• In some of the cases Jellyfish is far from theoretical bounds
• But is there any near optimal r-regular topology?
Generalized de Bruijn Graph
• We proved that GDBG has:
• Near optimal diameter
• Near optimal average
k-shortest path length
• Near optimal path distribution
• Near optimal load distribution
𝐺𝐷𝐵𝐺(6,2)
7 Lemmas. Some More Math!
𝑛𝑜𝑑𝑒 𝑖 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑠 𝑡𝑜 𝑛𝑜𝑑𝑒𝑠:
𝑖 ∗ 𝑑, 𝑖 ∗ 𝑑 + 1, … , 𝑖 + 1 ∗ 𝑑 − 1 𝑚𝑜𝑑 𝑁
GDBG: A Near Optimal r-Regular Topology
GDBG: A Near Optimal r-Regular Topology
There is more…
• GDBG is a near optimal r-regular topology
• It can be used as a simulation benchmark to evaluate other
topologies in this family
• We are going to evaluate RRG
Simulation
• Topology
• Generalized de Bruijn Graph(almost optimal)
• Jellyfish
• Routing
• Hop limited all path routing (GDBG)
• k-shortest path routing (Jellyfish)
• Traffic Pattern
•
•
•
•
Node level random permutation
Node level shift
Switch level random permutation
Switch level shift
• Metric
• Aggregate throughput
Node Level Throughput
N=150, Network Degree=8
Switch Level Throughput
Conclusion
• Derived theoretical bounds for the following key metrics on any DRG
• Average k-shortest path length
• Load balancing property
• RRG is near optimal in terms of average k-shortest path length
• RRG is far from optimal for all other metrics
• GDBG was found near optimal for all metrics
• GDBG was used as a simulation benchmark to evaluate RRG
• Depending on traffic pattern, RRG is not always near optimal
Thanks for your time
Questions