Is Neutron challenging to you?
Learn how VMware NSX is the solution for regular
OpenStack Network & Security services and Kubernetes
Dimitri Desmidt – Technical Product Manager NSX
Yves Fauser - Technical Product Manager NSX
May 2017
© 2017 VMware Inc. All rights reserved.
Agenda
•
Why Neutron is most challenging OpenStack project?
•
Why NSX plugin for your Neutron?
•
Demo of OpenStack + NSX
•
What about Kubernetes in NSX?
•
Key Takeaways
Agenda
•
Why Neutron is most challenging OpenStack project?
•
Why NSX plugin for your Neutron?
•
Demo of OpenStack + NSX
•
What about Kubernetes in NSX?
•
Key Takeaways
Neutron is #1 most challenging OpenStack project !!!
From OpenStack survey 2016
• https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf
Users called out Ceilometer and
Neutron most frequently as projects
with significant concerns.
For Neutron, they wanted the
networking service to be less
complicated to use, with more
substantial documentation and better
integration with compute functions
and PaaS layer integration
4
Neutron is #1 most challenging OpenStack project !!!
From OpenStack last survey April 2017
• https://www.openstack.org/assets/survey/April2017SurveyReport.pdf
Neutron needs to be reworked and
simpler – we don't need to include every
use case under the sun.
5
Neutron is #1 most challenging OpenStack project !!!
What are the biggest challenges
• Highly Complex implementation
– Implementation of L2 Switching (http://docs.openstack.org/developer/neutron/devref/openvswitch_agent.html)
Of course this complex solution is possible to
be masterized by OpenStack Neutron gurus 
Web-01
10.1.1.0/24
.11
.12
.13
However it doesn't removes the challenges on:
• Performance
Series of linux briges + OVS
• Troubleshooting
Doing a simple packet-walk requires highexpertise
And this shows only L2 !!!
6
Neutron is #1 most challenging OpenStack project !!!
What are the biggest challenges
• Highly Complex implementation – cont
– Layer 3 Networking in Neutron - via Layer 3 agent (http://docs.openstack.org/developer/neutron/devref/layer3.html)
And this shows only L3.
.1
Not:
• Distributed L3
.1
App-01
10.1.2.0/24
Web-01
10.1.1.0/24
.11
.12
.13
.11
.12
• Security Group
• Load Balancing
• etc
7
So why a vendor Neutron plugin?
• Simpler implementation of Neutron services
• Vendor support of that critical Cloud service
• Higher performance
• Management / Operation / Troubleshooting tools
• Stability at scale + High-Availability
8
Agenda
•
Why Neutron is most challenging OpenStack project?
•
Why NSX plugin for your Neutron?
•
Demo of OpenStack + NSX
•
What about Kubernetes in NSX?
•
Key Takeaways
Why NSX Neutron plugin?
Stability and High-Availability
• NSX offers Network & Security services Enterprise-Grade quality
– with scale tests and longevity tests run on each release
– so with vendor support at scale
• High-Availability is offered built-in in NSX
10
Why NSX Neutron plugin?
Multi Hyper-Visor support
• NSX offers Network & Security services for multiple hypervisors
• Important Note:
. All NSX Network & Security services are available on both hypervisors.
. Network & Security services are installed in kernel for best performance.
No Neutron services running in VM (which would bring challenges on HA and performance)
11
Why NSX Neutron plugin?
Advanced Network & Security services vendor supported
• NSX offers advanced Network & Security services
Animated slide
NSX
API & UI
VM1 VM2
LB-Pool
VM3
VM1 VM2
NAT
VM1 VM2
VM1
VM3
VM4
Subnet A
Subnet B
VM2
VM1 VM2
12
Why NSX Neutron plugin?
Simple and dynamic integration with physical
• NSX offers simple and unique integration with physical world
NSX router with pre-created BGP
adjacencies with physical routers.
At day0, nothing is advertised
from NSX Router to physical routers
Animated slide
NSX
API & UI
BGP
OpenStack Logical Networks
13
Why NSX Neutron plugin?
Simple and dynamic integration with physical
• NSX offers simple and unique integration with physical world
Routes learned:
. Subnet A + B
. Public-NAT 1 + 2 + 3 + 4
Animated slide
NSX
API & UI
Public-NAT4
Physical routers are NOT touched !!!
BGP
VM1 VM2
Public-NAT3
LB-Pool
VM3
VM1 VM2
Public-NAT1
Public-NAT2
VM1 VM2
VM1
VM3
VM4
Subnet A
Subnet B
VM2
VM1 VM2
OpenStack Logical Networks
14
Why NSX Neutron plugin?
Performance – Switching (1/3)
• NSX offers stunning performance
– Streamlining networking services in the hypervisors
And FYI
View with Security Gateway
VM1
VM2
ovs-vswitchd
ovs-fwd
(conntrack)
eth1
VLAN
or VXLAN
15
Why NSX Neutron plugin?
Performance – East/West Routing (2/3)
• NSX offers stunning performance
.1
.1
App-01
10.1.2.0/24
Web-01
10.1.1.0/24
.11
• Traffic Flow with Distributed Routing
.12
.13
Logical View
.11
.12
Physical View
16
Why NSX Neutron plugin?
Performance – North/South Routing (3/3)
• NSX offers stunning performance
– DPDK support
– Active/Active North/South support
External
.1
.1
DPDK
App-01
10.1.2.0/24
Web-01
10.1.1.0/24
(80Gbps / Node)
Active/Active
.11
.12
.13
Logical View
.11
(Active Cluster
up to 8 nodes)
.12
Physical View
17
Why NSX Neutron plugin?
Day2 operations (operations / troubleshooting)
• NSX makes Neutron manageable
– Typical Troubleshooting without NSX
root@compute01:~# ovs-dpctl dump-flows
recirc_id(0),in_port(6),eth(src=b6:cf:d2:e5:ac:8c,dst=00:50:56:6d:fa:8e),eth_type(0x0800),ipv4(frag=no), packets:92456,
bytes:12844792, used:0.305s, flags:P., actions:drop
recirc_id(0),in_port(6),eth(src=00:50:56:93:39:7d,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=192.168.31.32,tip=192.168.31.31,op=
1/0xff), packets:69189, bytes:4151340, used:0.105s, actions:5,4
recirc_id(0),in_port(6),eth(src=00:50:56:6d:fa:8e,dst=00:50:56:93:39:7d),eth_type(0x0800),ipv4(frag=no), packets:88150,
bytes:12167094, used:0.689s, flags:P., actions:drop
recirc_id(0),in_port(6),eth(src=b6:cf:d2:e5:ac:8c,dst=d2:7e:07:86:a1:cc),eth_type(0x0806), packets:1, bytes:60, used:0.793s,
actions:4
recirc_id(0),in_port(3),eth(src=42:7f:11:88:a4:49,dst=ff:ff:ff:ff:ff:ff),eth_type(0x8100),vlan(vid=2101),encap(eth_type(0x0806),arp(o
p=2/0xff)), packets:30018, bytes:1921152, used:1.621s, actions:2
recirc_id(0),tunnel(tun_id=0x10801000,src=192.168.31.32,dst=192.168.31.231,tos=0xc0,ttl=64,flags(df+csum+key)),in_port(8),skb_mark(0),eth_type(0x0800),ipv4(proto=17,frag=no),udp(dst=3784), packets:92255, bytes:6088830,
used:0.097s, actions:userspace(pid=4294963061,slow_path(bfd))
recirc_id(0),in_port(3),eth(src=00:50:56:93:a9:55,dst=ff:ff:ff:ff:ff:ff),eth_type(0x8100),vlan(vid=2101),encap(eth_type(0x0806),arp(s
ip=20.20.20.122,tip=20.20.20.121,op=1/0xff)), packets:10005, bytes:640320, used:6.517s, actions:2
recirc_id(0),in_port(4),eth(src=d2:7e:07:86:a1:cc,dst=00:50:56:6d:fa:8e),eth_type(0x0800),ipv4(frag=no), packets:92323,
bytes:12738740, used:0.073s, flags:P., actions:6
recirc_id(0),in_port(3),eth(src=ce:4e:c3:f2:d6:59,dst=ff:ff:ff:ff:ff:ff),eth_type(0x8100),vlan(vid=2101),encap(eth_type(0x0806),arp(s
ip=20.20.20.101,tip=20.20.20.101,op=1/0xff)), packets:3920, bytes:250880, used:1.705s, actions:2
recirc_id(0),tunnel(tun_id=0x1e001000,src=192.168.31.21,dst=192.168.31.231,ttl=64,flags(df+csum+key)),in_port(8),skb_mark(0),eth_type(0x0800),ipv4(proto=17,frag=no),udp(dst=3784), packets:88175, bytes:5819550,
used:0.289s, actions:userspace(pid=4294963061,slow_path(bfd))
recirc_id(0),in_port(6),eth(src=00:50:56:6d:fa:8e,dst=b6:cf:d2:e5:ac:8c),eth_type(0x0800),ipv4(frag=no), packets:88452,
bytes:12277876, used:0.789s, flags:P., actions:drop
<snip>
Troubleshoot OVS from KVM
18
Why NSX Neutron plugin?
Day2 operations (operations / troubleshooting)
• NSX makes Neutron manageable
– Troubleshooting with NSX
19
Troubleshoot OVS from NSX
Agenda
•
Why Neutron is most challenging OpenStack project?
•
Why NSX plugin for your Neutron?
•
Demo of OpenStack + NSX
•
What about Kubernetes in NSX?
•
Key Takeaways
OpenStack Barcelona 2016 Summit:
Done with Mirantis: https://goo.gl/zbpio0
Demo of OpenStack + NSX
OpenStack Boston 2017 Summit:
Doing with RedHat OS 10 this time
• Demo
– User1 creation of 2 Tier-App with NAT
– User2 creation of 2 Tier-App without NAT
– User2 complains MySQL access not working
Physical Router
Routes learned:
. SNAT Router-User1 + FloatingIP-User1Web1
. Subnet User2-Web + User2-DB
BGP
NSX Tier0 Router
External
30.30.30.0/24
SNAT-Router-User1
Floating-IP for User1Web1
.1
No-NAT
.1
.1
NAT_db
10.21.2.0/24
NAT_web
10.21.1.0/24
.1
NAT_db
10.22.2.0/24
NAT_web
10.22.1.0/24
MySQL
NOT working!!!
.3
.3
User1
.3
.3
User2
21
Agenda
•
Why Neutron is most challenging OpenStack project?
•
Why NSX plugin for your Neutron?
•
Demo of OpenStack + NSX
•
What about Kubernetes in NSX?
•
Key Takeaways
The Container Networking Challenges
Encap
Header
• No fine grained visibility into container networks
• Missing Network Policy definition and enforcement
• Missing Network Automation
Data
Kubernetes NSX Topology
Kubernetes NSX Topology
• Namespaces: We are dynamically building a separate
•
•
NSX/ K8s topology
•
Namespace: bar
Namespace: foo
•
10.24.0.0/24
10.24.1.0/24
10.24.3.0/24
network topology per K8s namespace, every K8s
namespace gets one or mode logical switches and one
Tier-1 router
Nodes: Are not doing IP routing, every Pod has its own
logical port on a NSX logical switch. Every Node can have
Pods from different Namespaces and with this from
different IP Subnets / Topologies
Firewall: Every Pod has DFW rules applied on its
Interface
Routing: High performant East/West and North/South
routing using NSX’s routing infrastructure, including
dynamic routing to physical network
Visibility and troubleshooting: Every Pod has a logical
port on the logical switch with:
• Counters
• SPAN / Remote mirroring
• IPFIX export
• Traceflow / Port-Connection tool
• Spoofguard
• IPAM: NSX is used to provide IP Address Management
by supplying Subnets from IP Block to Namespaces, and
Individual IPs and MAC to Pods
K8s / NSX Components
Network Container Plugin (NCP)
Network Container Plugin (NCP)
NSX Container Plugin
K8s / OS
Adapter
K8s master
CloudFoundry
Adapter
APIServer
Libnetwork
Adapter
Scheduler
More…
etcd
NSX
Manager
NSX/ K8s topology
NS: foo
NCM
Infra
NSX
Manager
API Client
• NSX Container Plugin: NCP is a software
component provided by VMware in form of a
container image, e.g. to be run as a K8s Pod
• Adapter layer: NCP is build in a modular way, so
that individual adapters can be added for different
CaaS and PaaS systems
• NSX Infra layer: Implements the logic that
creates topologies, attaches logical ports, etc.
based on triggers from the Adapter layer
• NSX API Client: Implements a standardized
NS: bar
interface to the NSX API
CNI – Container Network Interface
CNI interface spec
• Network Driver Interface:
Used by Kubernetes, OpenShift, Cloud Foundry and Mesos
• Implementation:
Uses a simple local executable call and environment variable
injection as the API between the container runtime /
orchestration and the network plugin
• Relation to Docker Networking:
The CNI spec is different to Dockers Libnetwork Container
Network Model (CNM), that uses a HTTP REST API for the
plugins and is more targeted towards the ‘Docker use-case’ [0]
[0] http://blog.kubernetes.io/2016/01/why-Kubernetes-doesnt-use-libnetwork.html
NSX-T Container Interface (CIF)
Container Interfaces in NSX-T
cif
Hypervisor
(ESXi &
KVM)
cif
mgmt
network
cif
mgmt
network
DFW
vlan 11
OVS
eth2
vlan 10
vlan 11
Minion Mgmt.
IP Stack
Vlan 10
eth2
Node
VM
OVS
NSX
CNI
Plugin
to deploy K8s Nodes as VMs today
cif
DFW
eth0
• K8s Node VMs: Most customers are looking
eth0
Node
VM
Minion Mgmt.
IP Stack
• Nested Network-Virtualization: Instead of
terminating the overlay tunnels in the Node
VM, we are extending the Hypervisor vSwitch
into the Node VM using VLAN tagging. The
Node VM vSwitch (OVS) is ‘standalone’, and
only gets programed by the NSX CNI Plugin
• Benefits:
• No double encapsulation
Pods
Pods
NSX
CNI
Plugin
• Enhanced security through strong
isolation of the Node VM from the NSX
Control-Plane
• Less transport-nodes in NSX which
equates to higher scale
K8s / NSX Demo
DEMO
K8s / NSX - FAQ
1. Q: Why is the K8s / NSX solution presented here not using the Neutron API to create network
objects?
A: We are developing a solution for NSX that works for any compute environment, vSphere,
Photon-Platform, KVM Virtualization without OpenStack, as well as public cloud. Therefore we
need to be independent from Neutron. We fully support OpenStack Kuryr for Container
Networking with Neutron and NSX if a tighter integration with Neutron is desired
2. Q: Are you supporting K8s Network-Policy?
A: Yes, our integration fully supports K8s Network Policy
3. Q: Are your supporting the K8s cloud provider for OpenStack
A: Yes, one can use the OpenStack Load-Balancing cloud provider, however we are more
focused on testing solutions that use K8s Ingress instead of the OpenStack Load-Balancing
cloud provider
4.
Q: <INSERT YOUR QUESTION HERE>
Agenda
•
Why Neutron is most challenging OpenStack project?
•
Why NSX plugin for your Neutron?
•
Demo of OpenStack + NSX
•
What about Kubernetes in NSX?
•
Key Takeaways
Key Takeaways – NSX for Neutron
Neutron offers Networks & Security services.
Neutron is made by geeks and for geeks.
If you're looking for a production grade Neutron solution, Vendors plugin are
required
Neutron + NSX plugin makes Neutron production grade thanks to:
• Multi-Hypervisor support (it's not only ESXi!!!)
• Stability + High performance
• Support and support at scale
• Management / Operation / Troubleshooting tools
Neutron + NSX can be installed within any OpenStack distro
Neutron + NSX supports Kubernetes and offers now security to Containers
31
Questions
Thank you
© 2017 VMware Inc. All rights reserved.