Does
Hypervisor
Matter in
OpenStack
Greg Elkinbard
Senior Technical Director
© MIRANTIS 2012
PAGE 1
Outline
• Brief History of Time: Hypervisor Requests
• 2011, 2012, 2013
• Trends in Different Segments
• Opportunities and Challenges
© MIRANTIS 2012
PAGE 2
2011
• Hypervisors
• XEN
• Default choice given Rackspace and Amazon use
• KVM
• Bleeding edge users
© MIRANTIS 2012
PAGE 3
2012
• Hypervisors
• KVM
• Emerges as the lead
• XEN
• Loses momentum
© MIRANTIS 2012
PAGE 4
2013
• Hypervisors
• KVM
• Maintains lead (around 90%+ for Mirantis)
• Vmware
• Emerges as a surprising second choice
• Containers (LXC, Parallels, Docker)
• Web Hosting and SAS focused
• Xen and HyperV
• Infrequent requests
© MIRANTIS 2012
PAGE 5
Trends
• Telco/ISP public clouds
• Single hypervisor
• KVM
• Internet focused companies
• Single hypervisor
• KVM
• Web Hosting and SAS sub-segments
• Single or multi hypervisors
• Containers (LXC, Parallels, Docker)
• KVM
• Enterprise
• Multi hypervisors
• KVM and vCenter/ESXi
© MIRANTIS 2012
PAGE 6
Opportunities and Challenges
• Multi-hypervisor use cases
• Advantages and Issues
• KVM
• vCenter/ESXi
• Containers (LXC, Docker, Parallels)
© MIRANTIS 2012
PAGE 7
Multi Hypervisor Use Cases
• Existing virtualization infrastructure
• Extend instead of replace
• Provide a common API
• Hedge bets against bugs and vendor pricing
• Utilize additional features
• Virtual appliances not supported by other platforms
• Advanced features not directly supported by OpenStack
© MIRANTIS 2012
PAGE 8
KVM
• Currently about 90%+ of requests, 95%+ of
deployments
• Type 2 hypervisor
• Relies on a distro
• Relatively easy to add new devices
• Easy to tune to get good performance
• Flexibility creates speedup opportunities
• Mellanox e-switch
• Intel DPDK vswitch
• Few issues
• Mostly distro related
© MIRANTIS 2012
PAGE 9
KVM Tuning
• We use KVM for HPC and NFV
• Extensive set of tunings
• Short Summary
•
•
•
•
•
•
•
BIOS to max performance
Enable huge pages
On RHEL tune for virtualization
Libvirt configure host pass-through for CPU flags
Increase TCP buffers and processor input queue
Congestion control to htcp
Enable jumbo frames
© MIRANTIS 2012
PAGE 10
KVM performance results
• KVM with OVS
• With tuning 7-8 Gbits/s VM throughput (jumbo frames)
• KVM with Mellanox
• Throughput host to host: 19.4 Gbit/s
• VM to VM on same host: 13.9 Gbit/s
• VM to VM on different hosts: 23.1 Gbit/s
• KVM with Intel OVS DPDK
• Alternative to dedicated hardware
• Good performance even for small packets
• 7Gbit/s for 64 byte packets
• 9.6Gbit/s for 512 byte packets
© MIRANTIS 2012
PAGE 11
KVM Features and Issues
• Widest set of OpenStack Features
• https://wiki.openstack.org/wiki/HypervisorSupportMatrix
• Issues
• Difficult to transfer images from other hypervisors
• Older versions of QEMU in RHEL/CentOS require virtio
drivers
• SCSI emulation in Ubuntu
• Updated CentOS with latest QEMU
© MIRANTIS 2012
PAGE 12
VMware ESXi
• Currently about 8-10% of requests, 5%+ of
deployments
• Primarily vCenter
• Type 1 hypervisor
• Does not need a linux distro
• VMware controls the code, 3rd party packages must be
signed.
• VMware is supporting both ESXi and vCenter
APIs
© MIRANTIS 2012
PAGE 13
VMware OpenStack
Compatibility
• Good compatibility with OpenStack Features
• https://wiki.openstack.org/wiki/HypervisorSupportMatrix
• Pause, Un-Pause and Resize missing
• Caveats (Grizzly)
• Network integration
• Nova-net no security groups
• Neutron requires Nicira commercial plugin
•
•
•
•
Glance integration inefficient
Only a single datastore is supported
With Cinder only iSCSI type volumes are supported
Only linked clones are supported
© MIRANTIS 2012
PAGE 14
Vmware OpenStack Havana
• Linked and Full clones are supported
• Multiple VC clusters can be managed by a
single driver
• Config drive support
• Cinder support for VMDK based Volumes
• vShield Edge Driver support for NVP plugin
(FWaaS, LBaaS)
© MIRANTIS 2012
PAGE 15
VMware Networking
• Nicira NVP/NSX
• Cisco 1000V (future)
• Accelerated Options
• NVP using STT
• VN-Tag, SR-IOV & Cisco Switches (VM-FEX)
© MIRANTIS 2012
PAGE 16
Containers
• Currently about 5%+ of requests, 0 deployments
• Low overhead alternative
• Hundreds of guests
• All Apps belong to a single tenant
• Fragmented space
• LXC
• Parallels
• Docker
• Limited OpenStack support but interest is growing
© MIRANTIS 2012
PAGE 17
LXC OpenStack Support
• VM – only launch, reboot and terminate
• Networking – basic VLANs
• Possible to get Neutron/OVS to work
• Volumes – officially no Cinder support
• Possible to hack something
© MIRANTIS 2012
PAGE 18