Operating Cloud:
NREN’s CASE
TNC 2014
Dr. Huseyin COTUK
ULAKBIM, Turkish NREN
21st May 2014, Dublin
Content
•
•
•
•
•
•
Turkish NREN ULAKBIM and TUBITAK
NREN Responsibilities
Cloud Computing
Governmental Services on Cloud
OpenStack
ULAKBIM Cloud
– FATIH Project
– PARDUS Project
– Other Projects
• Conclusion
TUBITAK
• roof organization of ULAKBIM
• is the leading agency for
management, funding and
conduct of research in Turkey
• was established in 1963 with a
mission of advancing science
and technology, conducting
research and supporting
Turkish researchers
• is responsible for promoting, developing,
organizing, conducting and coordinating research
and development in line with national targets and
priorities
Turkish NREN - ULAKBIM
• is the one of the 34 members
of the NREN consortium
• was founded as research and
development support unit of TUBITAK, in
1996
• is responsible for operating the National
Academic Network (ULAKNET) and National
Science e-Infrastructure (TRUBA)
• Departments
– Network Technologies Unit
– Educational Technologies Unit
– Cahit Arf Information Center
ULAKNET
• provides network services to more than 200
units including
–
–
–
–
•
•
•
•
•
•
universities (100K researchers and 2.5M students),
public institutions,
military and police academies
national R&D centers
IPv6 enabled backbone
TR eduGAIN
TR eduROAM
Ulak-CSIRT
20 Gbps GÈANT connection (5 Gbps backup)
25 Gbps commercial internet connection via
local ISP
TRUBA
• High performance computing
• Grid services
• Data intensive computing
– Scientific data warehouses
• Cloud computing services
• Map/reduce services via Hadoop clusters
ULAKBIM EU PROJECT PARTICIPATION
• Network Projects
– GEANT3Plus, GEANT3, BSI (coordinator) and
GEANT2
• Transition to IPv6 Project
– GEN6
• Pan-European e-Infrastructure Projects
– EGI-InSpire, PRACE2IP, EGEEIII, EGEEII
• Regional e-Infrastructure Projects
– HP-SEE, SEE-GRID-SCI, SEE-GRID2, SEE-GRID,
EUMedGrid-Support and EUMED-Grid.
• Programme and Policy Projects
– e-IRGSP3 and SEERA-EI
NRENs Responsibilities
• supporting research and education
communities like universities and research
institutes
• have important roles on shaping national
ICT strategies
• important players of innovation platform
that encourages the development of new
technologies
• also support academic communities via
providing Grid, High Performance
Computing (HPC) resources
What is Cloud?
• Cloud computing started with a dream
• Offering IT services as public utility with a
subscription like water and electricity
• “Computing may
someday be
organized as a
public utility” John McCarthy,
MIT Centennial in
1961.
Cloud Statistics
• $131 billion Estimated global cloud market at 2013
• 50 million Number of physical servers in the world
• 84% CIOs who reduced operational costs by moving to
cloud
• 80% CIOs who are delivering at least one of their
infrastructure through private cloud
• 60% CIOs who think that their first priority is cloud
• 60% Server loads that are virtualized in 2013
• 27% Governmental institutions that implemented cloud
infrastructure in 2012
• 1/3 Ratio of IT budgets spent for cloud in 2013
• 48% IT companies that spent for cloud advertisement in
2012
• 21% Amount of annual saving with the help of apps
moved to cloud
Why Cloud ?
•
•
•
•
•
•
•
Scalability
Cost effective
Pay per use
Flexibility
Self service usage
Easy metering
High availability
•
•
•
•
Distributed
Service oriented
Advanced security
Green IT (energy
saving)
• Quality of service
• Continuity
Cloud - Risks
• Data Security and
Privacy
• Service Provider
Dependency
• Management Interface
and Remote Access
• Bandwidth and Data
Transfer
• Software Licences
Cloud Models
• Public Cloud (39% – 32% in 5 years)
– No management, maintenance
– Many highly-available service providers
(Amazon, Google, Rackspace, Microsoft)
– Privacy problems
• Private Cloud (43% – 25% in 5 years)
– Security, privacy
– Control, cost, and reliability
– Management, maintenance, continuity
• Hybrid Cloud (17% – 43% in 5 years)
– Trend towards this model
Cloud Service Models
Infrastructure as a Service (IaaS)
Hipervisor, virtual machine, storage, network, load balancer, IP pools, OS
images,…
Platform as a Service (PaaS)
Database, web server, development environments, …
Software as a Service (SaaS)
Email, CRM, ERP, mobile applications, …
Source: http://cloudblueprint.wordpress.com/cloud-taxonomy/
Government Services on Cloud
• Governmental institutions
are also looking forward to
deploying their services on
cloud as well
• Governmental data is
taught as valuable enough
to conceal from
stakeholders
• most of the governmental organizations
prefer to implement these kind of private
services on their own data centers
Government Services on Cloud (Cont.)
• they are lacking of required knowledge
about innovative technologies
• they need professional and trustable
support for either implementing or
managing of these kind of services
• Rather than using commercial or
community driven platforms, getting these
services from reliable governmental
organizations is a good alternative
• NRENs could become valuable partners for
operating reputable cloud services
What is OpenStack?
• Infrastructure that provides cloud
computing technologies all together
• Open source (Apache License)
• Widespread hipervisor support (KVM, Xen,
ESXi, Hyper-V, LXC, Docker)
• Written in Python and Django
• All services provides RESTful web
services
• JSON and XML data format support
• Public, private, hybrid working models
• Widespread plugin and driver support
Why OpenStack ?
•
•
•
•
•
132 country
More than 150 vendor support
More than 13.000 developers
Completely free
Different user profiles (service providers,
governmental and educational institutions,
research institutes, and private companies)
• Flexible usage, and easy management
• Stable (9th version)
A
B
C
D
E
F
G H
I
Austin
Bexar
Cactus Diablo
Essex
Folsom Grizzly Havana Icehouse
OpenStack Developers
•
•
•
•
•
•
•
•
•
Rackspace
NASA
Canonical (Ubuntu)
Wells Fargo Bank
Dell, HP, IBM,
Fujitsu, Hitachi
Redhat, Suse
eBay
Cisco, Juniper
Intel
•
•
•
•
•
•
•
•
•
•
NSA
Mirantis
NetApp, EMC
Yahoo
Enovance
Huawei
Comcast
Sony PSP4
AT&T
Walt Disney Comp.
OpenStack in the World
• Rackspace Cloud ( All services and customers )
• NASA ( Nebula )
• CERN ( 3500 Hosts, 3 Clouds, 60K Cores, 300K
Estimated cores in 2015 )
• NSA ( Secure custom version )
• Cisco ( Webex )
• Intel ( Windows applications )
• HP Cloud ( All services and customers )
• DreamHost ( IaaS )
• RedHat ( Fully Supported, OpenShift)
• DELL ( OpenStack based projects )
• PayPal ( Most of 117 million customers are running on
OpenStack )
• AT&T ( Large scale projects )
OpenStack Components
•
•
•
•
•
•
Dashboard: Horizon
Identity: Keystone
Compute: Nova
Networking: Neutron
Block Storage: Cinder
Image: Glance
•
•
•
•
•
Object Storage: Swift
Orchestration: Heat
Telemetry: Ceilometer
DBaaS: Trove
Common Libraries: Oslo
Version History
Version
Publishment Date
Components
Austin
21st October 2010
Nova, Swift
Bexar
3rd February 2011
Nova, Glance, Swift
Cactus
15th April 2011
Nova, Glance, Swift
Diablo
22nd September 2011
Nova, Glance, Swift
Essex
5th April 2012
Nova, Glance, Swift, Horizon, Keystone
Folsom
27th September 2012
Nova, Glance, Swift, Horizon, Keystone,
Quantum, Cinder
Grizzly
4th April 2013
Nova, Glance, Swift, Horizon, Keystone,
Quantum, Cinder
Havana
17th October 2013
Nova, Glance, Swift, Horizon, Keystone,
Neutron, Cinder, Heat, Ceilometer
Icehouse
17th April 2014
Nova, Glance, Swift, Horizon, Keystone,
Neutron, Cinder, Heat, Ceilometer, Trove
Horizon
• customized web based graphical interface
for users and administrators
• virtual machines, OS images, storage,
security, network management
• suitable for adding 3rd party applications
• developed with Django
• service interaction via API calls
• usage statistics
Keystone
• defines endpoints, users, roles, groups,
and authorizations on OpenStack.
• provides role-based access control
(RBAC) between users and services.
• User-password, token, and AWS style
authentication support
• replies requests for services coming from
API.
• Key based access to virtual servers
• MySQL backend (LDAP alternative)
Nova
•
•
•
•
•
•
•
•
•
management of computing resources
virtualization layer (hypervisor)
horizontal scaling
transfers incoming new server requests to
hypervisor
resource templates (flavors)
supports live migration
supports snapshot
power management
console access
KVM (hipervisor)
Virt-manager (GUI for KVM)
Neutron
•
•
•
•
•
•
•
•
•
•
nova-network
quantum
neutron
DHCP service, Public IP allocation
L3 service (intervlan routing, subnet)
Openvswitch, Linux bridge, direct attach network
management
Bandwidth management
Security policies and access rules
Widespread plugin support for different vendors
Load Balancer as a service (LBaaS)
Firewall as a service (FWaaS)
VPN as a service (VPNaaS)
Cinder
• manages block storage units
• provides native API for several known
unified storage brand and model
• allows creating RAW disk drives for services
and attaching these disks hot-plug(PnP)
• Under VM, disk can be formatted by any file
system
• Supports snapshots and recovery
Glance
• manages image services
• Supports several image formats (qcow,
qcow2, vdi, vmdk, raw, vhd, etc.)
• Object Storage (Swift) service can be used
at the backend for storing operating
system images
• images can be modified in accordance
with the demands
Swift (Object Storage)
• provides scalable redundant storage
• does not have traditional file-system
structure
• Objects and files are placed in several
servers in a distributed manner
• Replication and data integrity are provided
by OpenStack
• can be scaled horizontally by adding new
servers
• ideal for images, static content, multimedia
and archive files
Heat
• provides template-based orchestration for
various cloud services
• works as template-driven engine for
developers and system administrators
• ensures auto resize and scale
• Uses OpenStack-native ReST API or
CloudFormation-compatible Query API.
Ceilometer
• works as telemetry service
• contains severals counters related to
usage
• possibility of parametric invoice services
• allows project-based monitoring with the
help of some monitoring tools
Trove
• Target: Scalable and trustable cloud
database service
• Relational and non-relational database
support
• CRUD, access control
• DB diagnostics
• Production level usage
– Rackspace, HP, eBay
• Single instance MySQL and Percona
support
• HA and cluster in under development
Oslo (Library)
• contains all the shared libraries and codes
that are used by whole OpenStack
projects
• provides consistency and stability between
projects
Sample Architecture
Third Party Applications
• Savanna (Hadoop as a service)
(now Sahara)
• Murano (Application catalog)
• Orchestration
• Automation (puppet, chef, juju, salt)
• Bare metal provisioning (maas, cobbler,
kick start)
Amazon WS Compatibility
• All APIs are fully compatible with
– Amazon EC2
– Amazon S3
• it is possible to migrate the applications
written for Amazon web services with
minimal effort
ULAKBIM Cloud
• Test studies were started at the end of
2011
• Old generation HPC nodes are used
• Virtual clusters for grid services on Open
Nebula (migration from physical to virtual)
• IaaS has been offered for 1.5 years
• OpenStack infrastructure will be extended
as part of EGI Federated Cloud
(certification process completed)
ULAKBIM Data Center
ULAKBIM Computing Resources - Servers
Year
Pcs
CPU / GPU
Memory
2003
128
1 core CPU
1 GB
2006
256
4 core CPU
4 GB
2007
30
8 core CPU
16 GB
2008
16
8 core CPU
16 GB
2009
154
8 core CPU
24 GB
2010
120
48 core CPU
128 GB
2011
192
24 core CPU
128 GB
2012
32
512 core GPU
24 GB
2013
128
16 core CPU
256 GB
ULAKBIM Computing Resources - Storage
Year
Capacity
2006
48 TB
2008
560 TB
2009
72 TB (Lustre)
2010
768 TB (Lustre)
2011
480 TB (Lustre)
2012
1152 TB (ZFS)
ULAKBIM Computing Resources - Interconnect
Infiniband
Year
Connection
Ports
2007
4x DDR
144
2008
4x QDR
648
Management- Network
Pcs
Conn.Speed
Ports
1
1 Gbps
576
1
1 Gbps
432
20
1 Gbps
48
1
10 Gbps
1152
OR
1
40 Gbps
288
FATIH Project
• FATIH (Movement of Enhancing
Opportunities and Improving Technology)
• Ministry of National Education
• one of the biggest education investment
projects of Turkey (8 billion TL budget)
• aims to create opportunities in education
and improve technology usage in school
– 42.000 schools
– 570.000 classes
– 10.6 million tablets
– 470.000 smart boards
– 705.000 teachers
– Distance education
centers
– Online contents
PARDUS Project
•
•
•
•
•
•
National operating system of Turkey
based on Debian architecture
Developed by ULAKBIM
KDE and Gnome desktop alternatives
Call center support
is used in several governmental
institutions including Ministry of National
Defense
• Embedded in smart boards used in FATIH
Project
• Embedded Libre Office
Other Projects
•
•
•
•
•
internal projects
other TUBITAK institutes
universities
several research institutions
various governmental institutions including
several ministries
ULAKBIM Cloud Statistics
Total Resources
Amount
Total number of cores
~ 15.000+
Total amount of memory
100 TB
Total amount of storage
2.5 PB
OpenStack Master
Amount
Total number of VMs
150
Total number of cores
1100
Total amount of memory
3 TB
Total amount of storage
60 TB
OpenStack Fatih
Amount
Total number of VMs
170
Total number of cores
765
Total amount of memory
2 TB
Total amount of storage
80 TB
Challenges
•
•
•
•
•
•
•
Version upgrades (release notes)
Maintenance, management, sustainability
Security (multi-tenancy)
L3 agent high availability issues
Bugs and patches
Optimization (kernel, user space, network)
Problem detection and elimination
Troubleshooting
•
•
•
•
•
•
https://ask.openstack.org/
https://launchpad.net/openstack
https://wiki.openstack.org/
https://www.openstack.org/blog/
docs.openstack.org
https://wiki.openstack.org/wiki/Mailing_Li
sts
• Logs under /var/log directory
Results
• ULAKBIM is working on pioneer studies on
cloud computing like as in several services
and technologies
• Server and storage services belong to
several projects are offered via this cloud
infrastructure
• Besides, FATIH and PARDUS projects, there
are many other projects running on cloud.
• After implementing production level cloud
environment, ULAKBIM has been following
the newest technologies and innovations
closely.
Any Questions?
• Dr. Huseyin COTUK
ULAKBIM, Turkish NREN
Chief Researcher
[email protected]
00-90-312-298-93-26