DEPLOYMENT
Fredhopper Cloud Infrastructure
SDL Proprietary and Confidential
Outline
1. Problem
2. Approach
3. “Demo”
2
The Problem
• Multiple products, in different versions
• Custom on-premise deployments
• Many manual steps for routine operations work
• Diverse environments
– Hardware differences
– Operating Systems and versions
– Java VM versions
3
Approach
Deployment
Agent
Managing state
• One agent per machine, running as service/daemon
• Abstraction from operating system and hardware
• Provides access to resources on machine via REST interface
• Clients for command line and Fredhopper Data Manager
Controller
Managing change
• One controller per separate environment
• Abstraction from network topology
• Provides access to resources in network via REST interface
• Dashboard as client
• Configuration describes “should”
4
Deployment Agent - Concepts
• Installation: The ”bits”, factory for instances
• Instances: Keeps configuration and index, provides processes
• Processes: Daemons/Services/Tools in the context of an instance
• Verbs: Primitives for invoking processes
5
Controller - Concepts
• Service: a particular deployment model
– Specific to the product
• Service Instance: a particular instance of the service, owned by a
customer
• Triggers: requests from a user for a change
– Available triggers depend on service instance
6
Controller Configuration (1)
<service-plans>
<load-balancer
name="rightscale"
factory="com.fredhopper.controller.rightscale.RightScaleLoadBalancerClientFactory">
...
</load-balancer>
<installation-directory
name="default"
directory="/data/distributions"
allow-upload="true"/>
<service-plan customer="internal">
<service-instance
name="dm:dm"
installation=“fredhopper-data-manager-1.0.4“
load-balancer="none"
factory="com.fredhopper.controller.kitchen.distributed.DataManagerServiceInstanceFactory"/>
</service-plan>
..
</service-plans>
7
Controller Configuration (2)
<service-plan customer="demo">
<service-instance
name="fas:live1"
installation="fas-assembly-7.2.0"
load-balancer="rightscale“
factory="com.fredhopper.controller.fas.MasterSlaveServiceInstanceFactory">
<property name=“control.dataManagerServiceInstance">dm:dm</property>
<property name="ms.masterHasLive">true</property>
</service-instance>
<service-instance
name="suggest:live1"
installation="suggest-distribution-2.0"
load-balancer="rightscale"
factory="com.fredhopper.controller.suggest.SuggestServiceInstanceFactory">
<property name=“suggest.fasServiceInstance">fas:live1</property>
</service-instance>
</service-plan>
8
Demo
9
Copyright © 2008-2012 SDL plc. All rights reserved.. All company names, brand names, trademarks,
service marks, images and logos are the property of their respective owners.
This presentation and its content are SDL confidential unless otherwise specified, and may not be
copied, used or distributed except as authorised by SDL.
Deployment Agent - Results & Learnings
• Identification of core concepts is key
– Removing concepts is hard
– Avoid complexity
• Versioning and Fail-Fast Behaviors
– Detect problems, and report them immediately and clearly
• System Identification and “Support for Support”
– If something in the higher layers breaks the agent is the last resort for gathering valuable information
– Auditing/Logging
• Infrastructure components are hard to upgrade
– Cannot get enough testing in synthetic and production scenarios
– Stable API to ensure compatibility with future versions
– Plugins and alternative implementations of core components
• UI is important
– People want to see that everything is good, ‘no news is good news’ does not work
• Complexity in high-level language, not in scripts
– Well-known language and environment
– Easily testable
– Adaptable for new products: installations deliver the glue code, coded against available API
11
Controller - Results
• Also infrastructure glue, Deployment Agent results apply
– Concepts, Integration, Extensibility, Testing
– Testing is hard: cannot duplicate production environment easily
• Software updates are common
– New services
– Improved deployment models
• Must provide real benefit for adoption
– Upgrades of service software
– Scaling models to reduce costs
• Centralized model enables access to the big picture
– Support/Operations can solve routine tasks quickly
– I/O scaling is needed, CPU is irrelevant: partitioning seems inevitable
12
Controller - Structure
Customer A
Service Instance fas:live1
EC2 Instance 1
DA Instance
indexer
13
DA Instance
live
EC2 Instance
2
DA Instance
live
Service Instance suggest:live1
...
EC2 Instance
n
EC2 Instance
n+1
DA Instance
live
DA Instance
suggest
...
EC2 Instance
m
DA Instance
suggest
Customer
B
Customer internal
Service
Instance
service:other
Service instance dm:dm
EC2 instance(s)
DA instance
customerAfas:live1reindex
...
Trigger Execution Example: FAS Full Data Load
Trigger
•Upload Data
•Create trigger
14
Input
Validation
•Required
parameters
•MD5 checks
ETL
•create DM
instance in pool
•run kitchen
•wait for status
callback and
poll for process
exit
Output
Validation
•status
•files produced
FAS Reindex
Situation – On-Premise
Integrators
Shoppers
Frontend
Load-Balancing
Business
Users
FAS
Integrators
Custom Scripts
SysAdmins
Hardware
Monitoring
15
Integrators
Load-Balancing
Fredhopper
Deployment Agent
Hardware
Monitoring
Controller
Servi
ce
API
Monit
oring
16
Frontend
Integrators
FH
Operations
Shoppers
FH
Operations
Business Users
Integrators
Situation – Cloud Environment