EGI Webinar
Porting your application to
the EGI Federated Cloud
17 Feb 2014
EGI-InSPIRE RI-261323
1
www.egi.eu
Outline
• Part 1
– General Concept and Integration Strategies
• Webinar target & Objectives
• Terminology
• Integration strategies
• Part 2
– Porting your application to the EGI Federated Cloud
•
•
•
•
•
Pre-requisites
Command-line environment
Manual server setup / Basic OS image contextualization
Custom OS images
Infrastructure & Application brokers
17 Feb 2014
EGI-InSPIRE RI-261323
2
www.egi.eu
Target & Objectives
Objective:
• Provide a general guide on how to port your application
to a Cloud Infrastructure-as-a-Service
Target:
• Applications / web services developers / administrators /
service providers, with good system administration skills
and minimal or null knowledge of Cloud technologies.
17 Feb 2014
EGI-InSPIRE RI-261323
3
www.egi.eu
The “Cloud”
Cloud:
• An IT provisioning concept of using remote
virtual services that can be rapidly provisioned
and released on-demand with minimal
management effort.
Give me a server
124.61.25.23
17 Feb 2014
EGI-InSPIRE RI-261323
4
www.egi.eu
The “Cloud”
Cloud:
• An IT provisioning concept of using remote
virtual services that can be rapidly provisioned
and released on-demand with minimal
management effort.
Store this file
http://thecloud.com/storage/myfile.data
17 Feb 2014
EGI-InSPIRE RI-261323
5
www.egi.eu
Cloud Technologies
Computing:
– Infrastructure-as-a-Service (IaaS):
• A Cloud service that provisions Virtual Machines on-demand.
– Platform-as-a-Service (PaaS):
• A Cloud service which provides a platform to integrate algorithms and
computational tasks, integrated with storage, database and other services
– Software-as-a-Service (SaaS):
• A Cloud service which provides direct access to a particular software
appliance on-demand, usually via a web interface.
– Database-as-a-Service, DNS-as-a-Service, etc…:
• A set of basic services to simplify application building, porting and scaling
Storage:
– STorage-as-a-Service (STaaS): Object Storage
– Personal Storage (Personal-STaaS): Dropbox-like storage
17 Feb 2014
EGI-InSPIRE RI-261323
6
www.egi.eu
IaaS vs PaaS vs SaaS
Software-as-a-Service
Platform-as-a-Service
Software
Algorithm
Infrastructure-as-a-Service
17 Feb 2014
EGI-InSPIRE RI-261323
7
www.egi.eu
Cloud IaaS Features
IaaS:
– Basic OS images:
• Set of pre-built generic Operative System images, with support for
contextualization, security, etc... These images are usually kept updated by
the cloud provider
– Custom OS images:
• Possibility to import virtual disks and use them as OS images for newly
created servers
– Contextualization:
• Initial setup of the virtual machine performed at startup. Standard setup
includes network and access credentials, while more advanced setup can be
specified by the user via a custom contextualization script (custom user data)
– API:
• Application Programming Interface to access the IaaS services, which can be
integrated to dynamically manage the cloud virtual servers
17 Feb 2014
EGI-InSPIRE RI-261323
8
www.egi.eu
IaaS Integration
Porting of your application can be done at different levels, according to which
and how many features of the Cloud IaaS service you want to exploit
Integration Strategies
Manual Application SetupManual server setup
Basic OS image with
Automatic Application Setup contextualization
Import Application
Custom OS image
Infrastructure broker
Integrate Application with the API
Application broker
17 Feb 2014
EGI-InSPIRE RI-261323
9
www.egi.eu
Manual Server Setup
Final users starts to use the application
Application
Connect to the virtual server via
Ask for
a virtualline
server,
with a specific
OS and
command
or graphical
interface
and hardware
resources
install manually
the application
Infrastructure-as-a-Service
The server is created and the access
details (ex. IP) are returned to the user
17 Feb 2014
EGI-InSPIRE RI-261323
10
www.egi.eu
Manual Server Setup
Advantages:
• Easy
Disadvantages:
• All the server management and configuration is done by hand
Recommended for:
• Applications testing and development
• Simple self-packaged applications with minimal effort for installation
and configuration
• “Disposable" applications (applications/web services which are
started, used for a limited time, then destroyed and never started
again)
17 Feb 2014
EGI-InSPIRE RI-261323
11
www.egi.eu
Basic OS image with contextualization
Final users starts to use the application
Deployment Script
Application
Application is encapsulated into a
deployment script
Ask for a virtual server, with a specific
OS, hardware resources, specifyiing the
deployment script
Infrastructure-as-a-Service
The server is created and the deployment script
will install and configure the application
17 Feb 2014
EGI-InSPIRE RI-261323
12
www.egi.eu
Basic OS Image with contextualization
Advantages:
• Fast server configuration
• High portability
• Possibility to customize
Disadvantages:
• Effort needed to build the contextualization script
• Need to maintain the contextualization script on OS updates
Recommended for:
• Web service applications
• Applications which usually need to stay on 24/7, with relatively infrequent
application updates and downtimes (ex. monthly).
17 Feb 2014
EGI-InSPIRE RI-261323
13
www.egi.eu
Custom OS image
Final users starts to use the application
Application
Application
Application is encapsulated into a custom
virtual disk OS image
Import
Ask
for the
a virtual
custom
server,
OS image
with ainto
the the
custom
IaaS
OS image
service
image
and library
specific hardware resources
Infrastructure-as-a-Service
The server is created and will contain
already the application
17 Feb 2014
EGI-InSPIRE RI-261323
14
www.egi.eu
Custom OS image
Advantages:
• Very fast deployment (application is already installed)
• Possibility to support legacy applications
Disadvantages:
• Possible compatibility issues
• Need to maintain a full OS image
• High effort to build the custom OS image
Recommended for:
• Servers who are started and shut down very frequently, with low frequency
of application and system update
• Legacy applications, when is not possible to port the application to a newer
environment
17 Feb 2014
EGI-InSPIRE RI-261323
15
www.egi.eu
Infrastructure Broker
Final users starts to use the application
Application
Application
Infrastructure Broker
Infrastructure-as-a-Service
On application
requestbroker
of the
user,have
the broker
perform a
The
is integrated
into
anwill
Infrastructure
Infrastructure
may
advanced
functionality,
complex
deployment
on the
Cloud
IaaS
usually
viadynamic
custom
deployment
scripts
likebroker,
high avaliability,
load
balancing,
etc…
17 Feb 2014
EGI-InSPIRE RI-261323
16
www.egi.eu
Infrastructure broker
Advantages:
• Possibility to manage complex deployments
• Possibility to access advanced features, like high avaliability, load
balancing, dynamically scaling
Disadvantages:
• Need to integrate your application into the broker
Recommended for:
• Complex applications deployments which are started manually on a
relatively frequent basis (eg. twice per week)
17 Feb 2014
EGI-InSPIRE RI-261323
17
www.egi.eu
Application Broker
Final users request the running of the
application directly to the application broker
Application
Application
Application Broker
Infrastructure-as-a-Service
The
The application
broker will perform
is integrated
the deployment
into an Application
of the
broker
application according to the requests of the user
17 Feb 2014
EGI-InSPIRE RI-261323
18
www.egi.eu
Application broker
Advantages:
• Exploit parallel processing to speed-up single processing
• Load balancing and high availability is includes
• Possibility to setup complex resources usage policies
Disadvantages:
• Need to integrate your application into the application broker
Recommended for:
• Asynchronous processing, where workload is not constant but comes in
burst and there is the need to dynamically adapt the infrastructure
utilization to the application needs.
• Applications with high resources needs, with the need to minimize the cost
of the resources usage
17 Feb 2014
EGI-InSPIRE RI-261323
19
www.egi.eu
Summary
Integration Strategy
Description
Recommended for
Manual server setup
•Basic OS image
•Manual server startup
•Manual configuration
Test, self-cointaned
applications, "disposable"
applications
Basic OS image with
contextualization
•Basic OS image
•Automatic configuration
•Startup may be slow
Web service applications, on
24/7, with monthly
application updates
Custom OS image
•Custom OS image
Specials OS flavors, complex
•Application is pre-installed
installation procedures,
•Need to build and maintain the image dynamic provisioning
Infrastructure broker
•Automate multi-server deployment
on multiple clouds sites
Application broker
•Adapting dynamically to the workload High computing resources
•Automate split for parallel
requirements or burst
applications
processing
17 Feb 2014
EGI-InSPIRE RI-261323
Complex multi-server
applications
20
www.egi.eu
Q&A
???
17 Feb 2014
EGI-InSPIRE RI-261323
21
www.egi.eu
Outline
• Part 1
– General Concept and Integration Strategies
• Webinar target & Objectives
• Terminology
• Integration strategies
• Part 2
– Porting your application to the EGI Federated Cloud
•
•
•
•
•
Pre-requisites
Command-line environment
Manual server setup / Basic OS image contextualization
Custom OS images
Infrastructure & Application brokers
17 Feb 2014
EGI-InSPIRE RI-261323
22
www.egi.eu
Target & Objectives
Objective:
• Provide a basic step-to-step guide on how to implement
the basic IaaS application porting strategy in the EGI
Federated Cloud
Target:
• Applications / web services developers / administrators /
service providers, with good system administration skills
and minimal or null knowledge of Cloud technologies.
17 Feb 2014
EGI-InSPIRE RI-261323
23
www.egi.eu
Pre-requisites
Have access to the EGI Federated Cloud:
• Get a user certificate from EUGridPMA
– http://www.egi.eu/how-to/get_a_certificate.html
• Register to a Virtual Organization (who offers cloud resources)
– For test and demo purposes, we will use the Federated Cloud Task Force VO:
•
https://perun.metacentrum.cz/perun-registrar-cert/?vo=fedcloud.egi.eu&page=apps
• Find a site supporting your resources (send an email to [email protected])
– For test and demo purposes, we will use the Federated Cloud Testbed:
•
https://wiki.egi.eu/wiki/Fedcloud-tf:Testbed
Get a test/demo environment:
• Install an hypervisor on your PC (we will use VirtualBox,
www.virtualbox.org)
• Install a basic Linux OS (we will use Scientific Linux 6.5)
17 Feb 2014
EGI-InSPIRE RI-261323
24
www.egi.eu
Step 0. Command-line environment
As a preliminary step, we will setup a command-line Linux environment with a
set of tools.
Thus not required by all the integration strategies, the command-line
environment is useful to quickly check, test and manage the IaaS virtual
machines, using the Federated Cloud API clients
Let’s login to a Linux OS (we will use Scientific Linux 6.5) and follow the
guide on https://wiki.egi.eu/wiki/Fedcloud-tf:CLI_Environment
17 Feb 2014
EGI-InSPIRE RI-261323
25
www.egi.eu
Manual Server Setup
Final users starts to use the application
Application
Connect to the virtual server via
Ask for
a virtualline
server,
with a specific
OS and
command
or graphical
interface
and hardware
resources
install manually
the application
Infrastructure-as-a-Service
The server is created and the access
details (ex. IP) are returned to the user
17 Feb 2014
EGI-InSPIRE RI-261323
26
www.egi.eu
Manual Server Setup
Steps:
•
•
•
•
•
Browse AppDB and find a basic image
Generate a set of SSH keys
Create a contextualization script to inject the key
Start the virtual server
Login and setup the application
Let’s login to a Linux OS (we will use Scientific Linux 6.5) and follow the guide on
https://wiki.egi.eu/wiki/Fedcloud-tf:Users:ApplicationPortingHowTo#1._Manual_server_setup_2
17 Feb 2014
EGI-InSPIRE RI-261323
27
www.egi.eu
Basic OS image with contextualization
Final users starts to use the application
Deployment Script
Application
Application is encapsulated into a
deployment script
Ask for a virtual server, with a specific
OS, hardware resources, specifying the
deployment script
Infrastructure-as-a-Service
The server is created and the deployment script
will install and configure the application
17 Feb 2014
EGI-InSPIRE RI-261323
28
www.egi.eu
Basic OS image with contextualization
Steps:
•
•
•
•
Browse AppDB and find a basic image
Build a deployment script
Build a contextualization script
Start the server
Let’s login to a Linux OS (we will use Scientific Linux 6.5) and follow the guide on
https://wiki.egi.eu/wiki/Fedcloudtf:Users:ApplicationPortingHowTo#2._Basic_OS_image_with_contextualization_2
17 Feb 2014
EGI-InSPIRE RI-261323
29
www.egi.eu
Custom OS image
Final users starts to use the application
Application
Application
Application is encapsulated into a custom
virtual disk OS image
Import
Ask
for the
a virtual
custom
server,
OS image
with ainto
the the
custom
IaaS
OS image
service
image
and library
specific hardware resources
Infrastructure-as-a-Service
The server is created and will contain
already the application
17 Feb 2014
EGI-InSPIRE RI-261323
30
www.egi.eu
Custom OS image
Steps:
•
•
•
•
•
•
•
Create a virtual machine on your local PC
Configure the network and contextualization on the VM
Install your software in the machine
Package the VM
Upload the image to the FedCloud repository
Register the virtual appliance and its associated image(s) in AppDB
Start the server
Let’s follow the guide on
https://wiki.egi.eu/wiki/Fedcloud-tf:Users:ApplicationPortingHowTo#3._Custom_OS_image_2
17 Feb 2014
EGI-InSPIRE RI-261323
31
www.egi.eu
Infrastructure & Application Broker
Application
Application
Broker
Infrastructure-as-a-Service
The application
broker will perform
the deployment
of the
The
is integrated
into an broker
application
17 Feb 2014
EGI-InSPIRE RI-261323
32
www.egi.eu
Infrastructure & Application Broker
Integration of applications into an Infrastructure or an Application broker is
heavily dependent on the application itself and the broker solutions.
As reference, we can follow the guides on:
https://wiki.egi.eu/wiki/Fedcloud-tf:Users:ApplicationPortingHowTo#4._Infrastructure_broker_2
https://wiki.egi.eu/wiki/Fedcloud-tf:Users:ApplicationPortingHowTo#5._Application_broker_2
17 Feb 2014
EGI-InSPIRE RI-261323
33
www.egi.eu
EGI Webinar
References for this tutorial and additional information
• https://wiki.egi.eu/wiki/Fedcloud-tf:Users
• https://wiki.egi.eu/wiki/Fedcloud-tf:Users:FAQ
• https://wiki.egi.eu/wiki/Fedcloud-tf:CLI_Environment
• https://wiki.egi.eu/wiki/Fedcloud-tf:Users:ApplicationPortingHowTo
EGI Federated Cloud resources
• Wiki site: http://go.egi.eu/fedcloud
• GitHub: https://github.com/EGI-FCTF
• Mailing List: [email protected]
• Indico site: https://www.egi.eu/indico/categoryDisplay.py?categId=56
• User support: [email protected]
EGI-InSPIRE RI-261323
www.egi.eu