SDN for the Cloud Albert Greenberg Distinguished Engineer Director of Networking @ Microsoft Azure [email protected] Road to SDN • WAN • Motivating scenario: network engineering at scale • Innovation: infer traffic, control routing, centralize control to meet network-wide goals TomoGravity ACM Sigmetrics 2013 Test of Time Award RCP Usenix NSDI 2015 Test of Time Award 4D ACM Sigcomm 2015 Test of Time Award • Cloud • Motivating scenario: software defined data center, requiring per customer virtual networks • Innovation: VL2 • scale-out L3 fabric • network virtualization at scale, enabling SDN and NFV VL2 ACM Sigcomm 2009 Cloud provided the killer scenario for SDN • Cloud has the right scenarios • Economic and scale pressure huge leverage • Control huge degree of control to make changes in the right places • Virtualized Data Center for each customer prior art fell short • Cloud had the right developers and the right systems • High scale fault tolerant distributed systems and data management At Azure we changed everything because we had to, from optics to host to NIC to physical fabric to WAN to Edge/CDN to ExpressRoute (last mile) Hyperscale Cloud Microsoft Cloud Network 15 billion dollar bet Microsoft Cloud Network Microsoft Cloud Network 2010 2015 Compute Instances 100K Millions Azure Storage 10’s of PB Datacenter Network 10’s of Tbps Exabytes Pbps >85% Fortune 500 using Microsoft Cloud >93,000 New Azure customers a month >60 >5 425 >18 MILLION Azure Active Directory users BILLION Azure Active Directory authentications/week 1 TRILLION Azure Event Hubs events/month Scale TRILLION Azure storage objects MILLION requests/sec 1 out of 4 Azure VMs are Linux VMs 1,400,000 SQL databases in Azure Agenda Consistent cloud design principles for SDN Physical and Virtual networks, NFV Integration of enterprise & cloud, physical & virtual Future: reconfigurable network hardware Demo Career Advice Acknowledgements Azure SmartNIC Cloud Design Principles Scale-out N-Active Data Plane Embrace and Isolate failures Centralized control plane: drive network to target state Resource managers service requests, while meeting system wide objectives Controllers drive each component relentlessly to the target state Stateless agents plumb the policies dictated by the controllers These principles are built into every component Hyperscale Physical Networks VL2 Azure Clos Fabrics with 40G NICs Regional Spine Data Center Spine Row Spine Rack T0-1 T1-1 T0-2 T1-2 … Servers T2-1-1 … T1-7 T0-20 T2-1-2 … T3-1 T3-2 T0-1 … T2-1-8 T1-1 T1-8 T0-2 T3-3 T1-2 … T2-2-1 … T1-7 T0-20 Scale-out, active-active T3-4 T2-2-2 T1-8 … T2-2-4 T1-1 T0-1 Servers T0-2 T1-2 … … T1-7 T1-8 T0-20 Servers Scale-up, active-passive Outcome of >10 years of history, with major revisions every six months L3 LB/FW LB/FW LB/FW LB/FW L2 16 Challenges of Scale • Clos network management problem • Huge number of paths, ASICS, switches to examine, with a dynamic set of gray failure modes, when chasing app latency issues at 99.995% levels • Solution • Infrastructure for graph and state tracking to provide an app platform • Monitoring to drive out gray failures • Azure Cloud Switch OS to manage the switches as we do servers Azure State Management System Architecture Traffic Engineering Fault Mitigation Infrastructure Provisioning Observed State XXx10G Border Routers Each border router connects to each Interconnect switch BGP … 16 Spine Switchs BGP XXx10G XXx10G XXx10G 64x10G eB GP P eBG eBGP 64x10G 64x10G GP eB n7k XXx10G 64x10G 64x10G … 17 pods 64x10G 64x10G 64x10G … 17 pods 64x10G 48 servers pod P BG … 16 Spine Switchs XXx10G 48 servers podset GNS/WAN Network n7k XXx10G BGP BGP BGP BGP … 16 Spine Switchs 64x10G Abstract network graph and state model: the basic programming paradigm Updater … 16 Spine Switchs XXx10G Centralized control plane: drive network to target state Target State Proposed State Monitor … Checker podset High scale infrastructure is complex: multiple vendors, designs, software versions, failures App I: Automatic Failure Mitigation Fault Mitigation Observed State Device monitor Link monitor Proposed State Checker Target State Link updater Repair bad links Device updater Repair bad devices App II: Traffic Engineering Towards High Utilization Traffic Engineering SWAN Checker 250 Observed State Topology, link load Device &Link monitor Proposed State Gbps 200 Target State 150 100 50 Routes LSP monitor Service demand Service monitor 0 Routing updater Service LB updater Add or remove LSPs, change weights Sun Mo n = Total traffic = Elastic 100 Gbps Tue We d Thu Fri Sat = Background 100 Gbps = Interactive 20-60 Gbps Azure Scale Monitoring – Pingmesh • Problem: Is it the app or the net explaining app latency issues? • Solution: Measure the network latency between any two servers • Full coverage, always-on, brute force • Running in Microsoft DCs for near 5 years, generating 200+B probes every day Normal Podset failure Podset down Spine EverFlow: Packet-level Telemetry + Cloud Analytics Cloud Network Challenge Guided prober Packet drop Loop EverFlow controller Match & mirror Load imbalance Azure Analytics Query Collector Cloud Scale Data Latency spike Collector Reshuffler Filter & aggregation Distributed storage Azure Storage Collector 22 Azure Cloud Switch – Open Way to Build Switch OS Operating System Switch Control: drive to target state Provision Deployment Automation Hw Load Balancer BGP • SAI collaboration is industry wide • SAI simplifies bringing up Azure Cloud Switch (Azure’s switch OS) on new ASICS Other SDN Apps SNMP LLDP ACL Switch State Service Sync Agent1 Sync Agent2 Sync Agent3 Switch Abstraction Interface (SAI) Switch ASIC SDK Ethernet Interfaces User space Hard ware Switch ASIC Driver Switch ASIC Front Panel Ports SAI is on github Hyperscale Virtual Networks Network Virtualization (VNet) Microsoft Azure Virtual Networks Microsoft Azure Azure is the hub of your enterprise, reach to branch offices via VPN 10.1/16 10.1/16 Efficient and scalable communication within and across VNets Internet L3 Tunnel VNet is the right abstraction, the counterpart of the VM for compute ISP/MPLS QoS Hyperscale SDN: All Policy is in the Host SDN Proprietary Appliance Management Azure Resource Azure Resource Manager Azure Resource Manager Manager Management Plane Control Data Controller Controller Controller Control Plane Switch (Host) Switch (Host) Switch (Host) Data Plane Key Challenges for Hyperscale SDN Controllers Must scale up to 500k+ Hosts in a region Needs to scale down to small deployments too Must handle millions of updates per day Must support frequent updates without downtime Microsoft Azure Service Fabric: A platform for reliable, hyperscale, microservice-based applications http://aka.ms/servicefabric App1 App2 Regional Network Manager Microservices Management REST APIs Gateway VIP Manager Service Manager (VMs, Cloud Services) VNET Manager (VNETs, Gateways) MAC Manager Fabric Task Driver Network Resource Manager (ACLs, Routes, QoS) Partitioned Federated Controller Cluster Network Managers Software Load Balancers Other Appliances Regional Network Controller Stats • 10s of millions of API calls per day • API execution time • Read : <50 milliseconds • Write : <150 milliseconds • Varying deployment footprint • Smallest : <10 Hosts • Largest : >100 Hosts Write API Transactions AddOrUpdateVnet AllocateMacs AllocateVips AssociateToAclGroup CreateOrUpdateNetworkServi ce DeleteNetworkService FreeMac Hyperscale Network Function Virtualization Azure SLB: Scaling Virtual Network Functions • Key Idea: Decompose Load Balancing into Tiers to achieve scale-out data plane and centralized control plane Router Router • Tier 1: Distribute packets (Layer 3) • Routers ECMP • Tier 2: Distribute connections (Layer 3-4) • Multiplexer or Mux • Enable high availability and scale-out Mux Mux Load-balanced Connections • Tier 3: Virtualized Network Functions (Layer 3-7) • Example: Azure VPN, Azure Application Gateway, third-party firewall Loadbalanced IP Packets Mux VNF VNF VNF Express Route: Direct Connectivity to the Cloud Connectivity Provider Infrastructure (physical WAN) Azure Edge (network virtual function) Virtual Networks Data Center-Scale Distributed Router Distributed data plane Centralized control plane Customer’s network Microsoft WAN Customer’s Virtual Network in Azure Building a Hyperscale Host SDN Virtual Filtering Platform (VFP) Acts as a virtual switch inside Hyper-V VMSwitch VM NIC vNIC VM vNIC VM Switch VFP ACLs, Metering, Security VNET SLB (NAT) Provides core SDN functionality for Azure networking services, including: Address Virtualization for VNET VIP -> DIP Translation for SLB ACLs, Metering, and Security Guards Uses programmable rule/flow tables to perform per-packet actions Supports all Azure data plane policy at 40GbE+ with offloads Coming to private cloud in Windows Server 2016 Flow Tables: the Right Abstraction for the Host VMSwitch exposes a typed MatchAction-Table API to the controller VNet Description Tenant Description • Controllers define policy • One table per policy Controller Key insight: Let controller tell switch exactly what to do with which packets VNet Routing Policy • e.g. encap/decap, rather than trying to use existing abstractions (tunnels, …) NAT Endpoints ACLs Host: 10.4.1.5 NIC VFP Flow Action Flow Action Flow Action TO: 10.2/16 Encap to GW TO: 79.3.1.2 TO: 10.1.1/24 Allow TO: 10.1.1.5 Encap to 10.5.1.7 DNAT to 10.1.1.2 Block NAT out of VNET SNAT to 79.3.1.2 10.4/16 TO: !10/8 TO: !10/8 TO: !10/8 Allow VNET LB NAT ACLS VM1 10.1.1.2 Table Typing/Flow Caching are Critical to Performance • COGS in the cloud is driven by VM density: 40GbE is here • First-packet actions can be complex • Established-flow matches must be typed, predictable, and simple hash lookups First Packet NIC Flow Action Flow Action Flow Action TO: 10.2/16 Encap to GW TO: 79.3.1.2 TO: 10.1.1/24 Allow TO: 10.1.1.5 Encap to 10.5.1.7 DNAT to 10.1.1.2 TO: !10/8 10.4/16 Block TO: !10/8 NAT out of VNET SNAT to 79.3.1.2 TO: !10/8 Allow VNET Subsequent Packets ACLS LB NAT Connection Action 10.1.1.2,10.2.3.5,80,9876 DNAT + Encap to GW 10.1.1.2,10.2.1.5,80,9876 Encap to 10.5.1.7 10.1.1.2,64.3.2.5,6754,80 SNAT to 79.3.1.2 VFP Blue VM1 10.1.1.2 RDMA/RoCEv2 at Scale in Azure Memory NIC Buffer A Write local buffer at Address A to remote buffer at Address B Application NIC Buffer B is filled Memory Buffer B Application • RDMA addresses high CPU cost and long latency tail of TCP • Zero CPU Utilization at 40Gbps • μs level E2E latency • Running RDMA at scale • RoCEv2 for RDMA over commodity IP/Ethernet switches • Cluster-level RDMA • DCQCN* for end-to-end congestion control *DCQCN is running on Azure NICs RDMA Latency Reduction at 99.9th %ile in Bing TCP 99th percentile RDMA 99.9th percentile RDMA 99th percentile Host SDN Scale Challenges • Host network is Scaling Up: 1G 10G 40G 50G 100G • The driver is VM density (more VMs per host), reducing COGs • Need the performance of hardware to implement policy without CPU • Need to support new scenarios: BYO IP, BYO Topology, BYO Appliance • We are always pushing richer semantics to virtual networks • Need the programmability of software to be agile and future-proof How do we get the performance of hardware with programmability of software? Azure SmartNIC Host • Use an FPGA for reconfigurable functions CPU • FPGAs are already used in Bing (Catapult) • Roll out Hardware as we do software • Programmed using Generic Flow Tables (GFT) • Language for programming SDN to hardware • Uses connections and structured actions as primitives SmartNIC NIC ASIC FPGA • SmartNIC can also do Crypto, QoS, storage acceleration, and more… ToR SmartNIC Northbound API Controller Controller Controller Southbound API SLB Decap Rule * SLB NAT Action Rule Decap * VNET Action Rule DNAT ACL Action Rewrite * Rule * Action Allow Metering Rule * Action Meter Encap Rewrite VM Transposition Engine VFP Flow Action 1.2.3.1->1.3.4.1, 62362->80 Decap, DNAT, Rewrite, Meter First Packet GFT Offload Engine GFT Offload API (NDIS) VMSwitch SmartNIC GFT 50G Flow Action 1.2.3.1->1.3.4.1, 62362->80 Decap, DNAT, Rewrite, Meter Crypto RDMA GFT Table QoS 43 Azure SmartNIC Demo: SmartNIC Encryption Closing Thoughts Cloud scale, financial pressure unblocked SDN Control and systems developed earlier for compute, storage, power helped Moore’s Law helped: order 7B transistors per ASIC We did not wait for a moment for standards, for vendor persuasion SDN realized through consistent application of principles of Cloud design Embrace and isolate failure Centralize (partition, federate) control and relentlessly drive to target state Microsoft Azure re-imagined networking, created SDN and it paid off Career Advice Cloud Software leverage and agility Even for hardware people Quantity time With team, with project Hard infrastructure problems take >3 years, but it’s worth it Usage and its measurement oxygen for ideas Quick wins (3 years is a long time) Foundation and proof that the innovation matters Shout-Out to Colleagues & Mentors AT&T & Bell Labs • Han Nguyen, Brian Freeman, Jennifer Yates, … and entire AT&T Labs team • Alumni: Dave Belanger, Rob Calderbank, Debasis Mitra, Andrew Odlyzko, Eric Sumner Jr. Microsoft Azure • Reza Baghai, Victor Bahl, Deepak Bansal, Yiqun Cai, Luis Irun-Briz, Yiqun Cai, Alireza Dabagh, Nasser Elaawar, Gopal Kakivaya, Yousef Khalidi, Chuck Lenzmeier, Dave Maltz, Aaron Ogus, Parveen Patel, Mark Russinovich, Murari Sridharan, Marne Staples, Junhua Wang, Jason Zander, and entire Azure team • Alumni: Arne Josefsberg, James Hamilton, Randy Kern, Joe Chau, Changhoon Kim, Parantap Lahiri, Clyde Rodriguez, Amitabh Srivastava, Sumeet Singh, Haiyong Wang Academia • Nick Mckeown, Jennifer Rexford, Hui Zhang DCQCN MSFT @ SIGCOMM’15 Congestion control for large RDMA deployments Everflow PingMesh Packet-level telemetry for large DC networks 106x reduction in trace overhead, pinpoint accuracy Corral 2000x reduction in Pause Frames, 16x better performance DC network latency measurement and analysis 200 billion probes per day! Silo Joint data & compute placement for big data jobs Virtual networks with guaranteed bw and latency 56% reduction in completion time over Yarn No changes to host stack! R2C2 Network stack for rack-scale computing Hopper Speculation-aware scheduling of big-data jobs 66% speedup of production queries Iridium Eden Enable network functions at end host Rack is the new building block! Low-latency geo-distributed analytics 19x query speedup, 64% reduction in WAN traffic Dynamic, stateful policies, F#-based API http://research.microsoft.com/en-us/um/cambridge/events/sigcomm2015/papers.html
© Copyright 2024 Paperzz