PISCES: A Programmable, ProtocolIndependent Software Switch
Muhammad Shahbaz
Nick Feamster
Jennifer Rexford
Sean Choi
Nick McKeown
Ben Pfaff
Changhoon Kim
Cian Ferriter
Mark Gray
PISCES: A Protocol-Independent Software Switch
P4
PISCES
vSwitch
OVS
Internal Architecture of OVS
OVS
Kernel
Fast Packet
IO
(or Forwarding)
DPDK
Internal Architecture of OVS
OVS
Parser Packet Match-Action
Processing Logic
Pipeline
Complex APIs
Kernel
DPDK
Internal Architecture of OVS
OVS
Parser
Match-Action Pipeline
Kernel
DPDK
Internal Architecture of OVS
OVS
Parser
Match-Action Pipeline
Kernel
DPDK
Internal Architecture of OVS
OVS
Parser
Match-Action Pipeline
Kernel
DPDK
Road to Protocol Independence
Domain-Specific Language
Parser
Match-Action Pipeline
Compile
OVS
Parser
Match-Action Pipeline
Kernel
DPDK
Road to Protocol Independence
P4[1]
Parser
Match-Action Pipeline
Compile
341 lines of code
Native OVS
OVS
Parser
Match-Action Pipeline
Kernel
DPDK
[1]
http://www.p4.org
14,535 lines of code
Road to Protocol Independence
P4[1]
P4 Forwarding Model
Compile
Performance Overhead!
OVS
OVS Forwarding Model
Kernel
DPDK
[1]
http://www.p4.org
P4 Forwarding Model (Post-Pipeline Editing)
Ingress
Packet
Parser
Header
Fields
Checksum
Verify
Match-Action
Tables
Checksum
Update
Packet
Deparser
Egress
OVS Forwarding Model
Match-Action
Tables
Egress
Slow-path
Flow Rule
Miss
Ingress
Packet
Parser
Match-Action
Cache
Fast-path
OVS Forwarding Model
Match-Action
Tables
Egress
Slow-path
Fast-path
Hit
Ingress
Packet
Parser
Match-Action
Cache
Egress
OVS Forwarding Model (Inline Editing)
Match-Action
Tables
Egress
Slow-path
Fast-path
Ingress
Packet
Parser
Match-Action
Cache
Egress
PISCES Forwarding Model (Modified OVS)
• Supports both editing modes:
- Inline Editing
- Post-pipeline Editing
Match-Action
Tables
Slow-path
Fast-path
Ingress
Packet
Parser
Checksum
Verify
Match-Action
Cache
Checksum
Update
Packet
Deparser
Egress
PISCES: Compiling P4 to OVS
P4
Ingress
Packet
Parser
Checksum
Verify
Match-Action
Tables
Checksum
Update
Packet
Deparser
Egress
Match-Action
Tables
modified
OVS
Ingress
Packet
Parser
Checksum
Verify
Match-Action
Cache
Checksum
Update
Packet
Depraser
Egress
PISCES Forwarding Model (Modified OVS)
Match-Action
Tables
Slow-path
Fast-path
Ingress
Packet
Parser
Checksum
Verify
Match-Action
Cache
Checksum
Update
Packet
Deparser
Egress
PISCES Forwarding Model (Modified OVS)
Match-Action
Tables
Slow-path
Fast-path
Megaflow Cache
Ingress
Packet
Parser
Checksum
Verify
Microflow Cache
Checksum
Update
Packet
Deparser
Egress
PISCES Forwarding Model (Modified OVS)
Match-Action
Tables
Slow-path
Fast-path
Megaflow Cache
Ingress
Packet
Parser
Checksum
Verify
Checksum
Update
Packet
Deparser
Egress
Naïve Compilation from P4 to OVS (L2L3-ACL)
Naïve
OVS
Performance overhead of
~ 40%
Throughput (Gbps)
50
40
30
20
10
0
64
128
192
Packet Size (Bytes)
256
Causes of Performance Overhead
Match-Action
Tables
Cache Misses
Ingress
Packet
Parser
Checksum
Verify
Megaflow Cache
CPU Cycles per Packet
Checksum
Update
Packet
Deparser
Egress
Cause: CPU Cycles per Packet
L2L3-ACL (CPU Cycles for a 64 Byte Packet)
379.5
400
15
300
200
100
10
209.5 197.5
132.5
76.5
13.5
7.6
5
43.6
0
0
Parser
Cache: Match
Naïve
OVS
Cache: Actions
Throughput (Gbps)
Factors affecting CPU Cycles per Packet
a. Extra copy of headers
b. Fully-specified Checksum
c. Parsing unused header fields
and more …
Different Optimizations for L2L3-ACL
L2L3-ACL (CPU Cycles for a 64 Byte Packet)
379.5
400
300
209.5
200
100
10
197.5
130.8 132.5
76.5
13.3 13.5
15
7.6
5
29.3 43.6
0
0
Parser
Naïve
Inline
Cache: Match
Inc. Chksm
Parsr Spcl
Act Spcl
Cache: Actions
Act Coalcng
OVS
Throughput (Gbps)
Optimized Compilation from P4 to OVS (L2L3-ACL)
Naïve
Optimized
OVS
Performance overhead of
< 2%
Throughput (Gbps)
50
40
30
20
10
0
64
128
192
Packet Size (Bytes)
256
Cause: Cache Misses
•
•
3500+ Cycles (50x Cache hit)
Throughput < 1 Mpps
Match-Action
Tables
Cache Misses
Ingress
Packet
Parser
Checksum
Verify
Match-Action
Cache
Egress
Checksum
Update
Packet
Deparser
Factors affecting Cache Misses
a. Entropy of packet header fields
b. Stateful operations in the match-action cache
(or fast path).
PISCES Forwarding Model (Modified OVS)
Match-Action
Tables
Slow-path
Fast-path
Megaflow Cache
Ingress
Packet
Parser
Checksum
Verify
Microflow Cache
Checksum
Update
Packet
Deparser
Egress
PISCES Forwarding Model (Modified OVS)
Microflow Cache
Internals of the Microflow Cache
P4
File
Packet
in
to Megaflow
Cache
Miss
Extract
Fields
Hash
Fields
Microflow Cache
Perform
Lookup
Hit
Packet
out
Performance with the Microflow Cache
Throughput (Gbps)
Phy-Phy, L3 Router Case, 64B
7.728
6.464
OVS
PISCES
Cause of Performance Degradation
Cacheline
64 Bytes
0
Metadata
1
Metadata
2
IPv4 (1st 16Bytes) IPv4 + Pad
3
TCP + Pad
Simplified “flow” Structure
Ethernet
Header
UDP + Pad
Empty
Performance with the Microflow Cache
Throughput (Gbps)
Phy-Phy, L3 Router Case, 64B
7.728
OVS
8.198
PISCES
Throughput (Gbps)
Varying the Number of Hash Fields
8.682
8.198
L2 Address (2 Fields)
Five Tuple (5 Fields)
Questions?
Disclaimers
Intel does not control or audit third-party benchmark data or the web sites referenced in this
document. You should visit the referenced web site and confirm whether referenced data are
accurate.
Intel technologies’ features and benefits depend on system configuration and may require
enabled hardware, software or service activation. Performance varies depending on system
configuration. No computer system can be absolutely secure. Check with your system
manufacturer or retailer or learn more at [intel.com].