Efficient Round-Trip Time Monitoring
in OpenFlow Networks
Alon Atary, Anat Bremler-Barr
Interdisciplinary Center Herzliya
April 13th 2016
Supported by the ERC grant number 259085 and Neptune Consortium.
1
Background
• RTT – Round Trip Time
– Time to send a packet and receive a response
– The RTT is not constant
• Applications: Bottlenecks and anomalies detection,
traffic engineering
• Existing Tools, such as Ping and Traceroute,
are limited in traditional networks
– Only between L3 entities
– Cannot measure any selected route
– Routing changes
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
2
Goal: RTT Monitoring in OpenFlow
• RTT monitoring for two types of paths:
– All links in the network (L2)
• 𝑠𝑖𝑛𝑔𝑙𝑒 𝑙𝑖𝑛𝑘: 𝑅𝑇𝑇𝑠2 ↔𝑠1
OpenFlow
Controller
– Any selected round trip path (RTP)
• 𝑆𝑦𝑚𝑚𝑒𝑡𝑟𝑖𝑐 𝑅𝑇𝑃: 𝑅𝑇𝑇𝑠2 →𝑠4 →𝑠6 →𝑠4 →𝑠2
• 𝐴𝑠𝑦𝑚𝑚𝑒𝑡𝑟𝑖𝑐 𝑅𝑇𝑃: 𝑅𝑇𝑇𝑠2 →𝑠3 →𝑠4 →𝑠2
1
𝑠1
• OpenFlow
•
𝑠2
2
3
1
4
No support for RTT measurements
𝑠3
+ Full control over network routing
+ Duplication, Tagging
𝑠4
2
3
2
𝑠5
𝑠6
2
𝑠7
Our solution – GRAMI
GRAnular RTT Monitoring Infrastructure
• GRAMI works in two phases:
• Offline: Controller application finds a
location set for k monitoring points
(MPs) and builds an overlay network
• Online: Active probing from the MPs
– The RTT of a single link\RTP is calculated
in the MPs by subtracting the RTT of
different paths
Controller
application
𝑟𝑡𝑡𝑠1 ↔𝑠2= 𝑟𝑡𝑡𝑚1↔𝑠1↔𝑠2− 𝑟𝑡𝑡𝑚1↔𝑠1
𝑚1
𝑚2
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
4
Agenda
• Solution overview
– Considerations
– Workflow
• Solution drilldown
• Related work
• Experimental results
GRAMI considerations
 Compatibility with every OpenFlow network
– No assumptions on topology or size
– No changes to switch software or hardware
– No time synchronization
 Resource efficient
– Minimal number of probe packets and flow entries
• Up to 4 + 2𝑟 flow entries for 𝑟 RTPs and 𝑛 links
 Minimal controller involvement
 Dynamic updates
6
Workflow – offline and online phases
Offline Calculations – Controller Application
Compute
MPs
location set
Single MP
Briefly
Establish
the overlay
network
Derive and
install flow
entries
How many MPs?
Which RTPs?
Controller app
𝑚1
Online RTT Monitoring – MPs
Links and RTPs
Probe
packets
distribution
Briefly
𝑚2
RTT
calculations
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
7
Step 2 offline phase:
Establish the overlay network
1. Sets a singular shortest
path from every switch and
link to its closest MP
– Fixed routing
– Accuracy
• Lower depth = Higher accuracy
𝑚1
2. Every link and RTP is
measured exactly once
– Minimal overhead caused by
probe packets
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
8
Step 2 offline phase:
Establish the overlay network
Two steps for establishment:
1. Set a shallow spanning tree
•
Solid links
2. Add the rest of the links
•
Dashed links
𝑚1
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
9
Important terms
• Parent link: The single ingress
solid link of every switch.
– Singular shortest path to every
switch
𝑚1
• Egress links: links directed out
– Every link is measured exactly
once
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
10
Step 1 online phase:
Overlay network Probe packet distribution
• Four Probe packet types
– Tagging determines the type
Forward
Return
Link
RTP
𝑚1
𝑠1
𝑠2
3
𝑠3
• Packet’s type determines the switch
behavior and actions
𝑠5
𝑠6
1
𝑠4
2
11
𝑠7
Step 1 online phase:
Overlay network Probe packet distribution
• Switches algorithm - Traverse the overlay network:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
IF forward packet:
IF link packet:
Send return link packet to ingress port
IF from parent (solid):
Duplicate and Distribute to egress links
IF Return packet:
Tag if needed
Send packet to parent port.
𝑚1
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
12
Step 1 online phase:
Full overlay network distribution
𝑚1
𝑚
𝑠131 ↔
↔ 𝑠𝑠213
𝑠23 ↔ 𝑠42
𝑠43 ↔ 𝑠64
𝑠36 ↔ 𝑠65
𝑠6 ↔ 𝑠7
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
13
Step 2 online phase:
RTT computations for a single link
• RTT of a single link
𝑟𝑡𝑡𝑠1 ↔𝑠2 = 𝑟𝑡𝑡𝑚1 ↔𝑠1 ↔𝑠2 − 𝑟𝑡𝑡𝑚1 ↔𝑠1
𝑚1
𝑠5
1
𝑠1
𝑠2
2
𝑠3
𝑠4
𝑠6
𝑠7
14
Step 1 online phase:
RTP Probe packet distribution
• Let 𝑆𝑝 to be the first switch in RTP 𝑃
• Switches algorithm - Traverse the overlay network and RTP:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
IF forward packet:
IF link packet:
Send return link packet to ingress port
IF from parent (solid):
Duplicate and Distribute to egress links
Send forward RTP packet to RTPs
IF RTP packet:
IF not 𝑆𝑝 : send over RTP 𝑃
IF 𝑆𝑝 : send return RTP packet to parent
IF Return packet:
Tag if needed
Send packet to parent port.
𝑚1
𝑠1
𝑠2
3
𝑠3
𝑠5
𝑠6
1
𝑠4
2
15
𝑠7
RTT computations for an RTP
• RTT of an RTP
𝑟𝑡𝑡𝑠2→𝑠2 = 𝑟𝑡𝑡𝑚1→𝑠2 →𝑠2 →𝑚1 − 𝑟𝑡𝑡𝑚1↔𝑠2
𝑚1
𝑠1
𝑠2
3
𝑠3
𝑠5
𝑠6
1
𝑠4
2
16
𝑠7
Step 1 online phase:
RTP Probe packet distribution
• Let 𝑆𝑝 to be the first switch in RTP 𝑃
• Switches algorithm - Traverse the overlay network and RTP:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
IF forward packet:
IF link packet:
Send return link packet to ingress port
IF from parent (solid):
Duplicate and Distribute to egress links
Send forward RTP packet to RTPs
IF RTP packet:
IF not 𝑆𝑝 : send over RTP 𝑃
IF 𝑆𝑝 : send return RTP packet to parent
IF Return packet:
Tag if needed
Send packet to parent port.
Two forward for entries on every
switch (Distribute and don’t distribute)
Two RTP flow entries for every switch
for every RTP
Two return for entries on every switch
(Tag and don’t tag)
17
The flow entries
GRAMI – features summary
• Controller is not involved in the measurements
– It gets a summary of the results
• Any number of MPs in any location
• Resource efficient
(n links, r RTPs and k MPs):
– Small number of probe packets
•
𝑚1
Send k, return 𝑛 + 𝑟
– Up to 4 + 2𝑟 flow entries
𝑚2
Overlay network traversal
flow entries
RTP 𝑃1 traversal flow
entries
RTP 𝑃𝑟 traversal flow
entries
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
19
Related work
• RTT measurements in OpenFlow networks
[Phemius et al. 2013], [Van Adrichem et al. 2014], [Yu et
al. 2015], [Agarwal, et al. 2014]
– Overloading the controller
𝑠5
𝑠1
𝑠2
𝑠3
𝑠4
𝑠6
𝑠7
Related work - Shibuya et al. [2014]
• Shibuya et al. – Monitor all links RTT from a single MP
– GRAMI can monitor all links and any RTP from any number
of MPs
– GRAMI is more scalable and resource efficient (flow entries,
probe packets)
GRAMI vs Shibuya for network with n links and r RTPs
MPs
Measuring
Flow entries
Probe packets
GRAMI
Any k
Links and RTPs
4+2r
Send k, return n+r
Shibuya
1
Links only
O(n)
Send n, return n
Evaluation
• Worked on Mininet, Cpqd switches with Ryu
controller
• Real networks topologies (Topology Zoo)
– 7 to 62 switches
• Objectives:
– Overhead
– Response to network changes
22
Results: overhead
• Path length increases → Overhead increases
– Duplication and tagging in Mininet is relatively long
• Mininet(~12𝜇𝑠, ~55𝜇𝑠) , NoviFlow (< 4𝜇𝑠, < 1𝜇𝑠)
Path of length 2
Path of length 4
23
Results: response to network changes
Network overload
Link latency changes,
failure and recovery
24
Summary and conclusions.
• GRAMI
– Efficient and scalable
– Monitoring the entire network from any number of MPs
– Any RTP
• vSwitch will enable RTT measurements between
switches and other network entities (servers)
• OpenFlow → Innovation also in network measurements
25
Questions
Thank you
for listening
26