IDS High Availability

IBM Data Management
IDS 11.5
MACH Your Business Continuity
Boycho Velkov
NDB Ltd.
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS SCENARIOS BY INDUSTRY
 RETAIL
 TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
2
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS SCENARIOS BY INDUSTRY
 RETAIL
 TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
3
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
TERMINOLOGY
 High Availability – The proportion of time that an application can be
used for productive work, measured against the time that it must be
functional
 Continuous Availability – Online capabilities that make the system
available all the time (near zero downtime)
– Continuous Availability maybe achieved by using High Availability
capabilities
 Global Availability – Online capabilities that allow your businesses to
be available across geographic boundaries
 Failover – Transfer of processing or services to an identical system for
operational continuity
– HA is generally achieved using Failover technology
 Cluster – A group of tightly coupled systems working together in a way
that appears as one system to users
4
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
THINGS TO CONSIDER
 Define and understand requirements
– Uptime needs (planned and unplanned outages)
– Tolerance for data and/or transaction loss
– Automatic vs manual failover
– Geographic distribution and related network availability
– Database Design – single vs multi-partition database
– Nature and size of transactions
5
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
THINGS TO CONSIDER (Cont’d)
 Understand system capabilities
 Understand “what and “where” of system
bottlenecks
– Network
– Hardware
– Application
– Database
 Understand cost of achieving High Availability
for your business
6
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Availability
Availability - Defined
 The time that the application must be functional or available to users is
called "mission time," which may be quite different than 7×24 or 24×365
operation
 The proportion of time that an application can be used for productive
work, measured against the time that it must be functional
Levels of Availability
Availability
Level Class
Uptime
Downtime
Limit
Per Year
2
99%
4 Days
3
99.9%
9 Hours
4
99.99%
1 Hour
5
99.999%
5 Minutes
6
99.9999%
30 Seconds
7
99.99999%
3 Seconds
Factors Determining Availability
 The reliability of the components
that comprise the application:
namely, how often they fail
 How long it takes for the
application to be restored once a
failure has occurred
Source: “High Availability: A Perspective,” Jane Wright, Ann Katan (Gartner Group), 11/24/2004
7
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
Availability
IBM Data Management
Availability - Business Impact
Loss of data access can stop business processing and operations
• Loss of Customer Confidence
• Loss of Employee Productivity
• Loss of Company or Share Value
Availability
Downtime
Minute per Year
99.999%
5 minutes
99.99%
50 minutes
99.9%
8 hours, 20 minutes
99%
3 days, 11 hours, 18 minutes
95%
18 days, 6 hours
90%
34 days, 17 hours, 17
minutes
85%
54 days, 18 hours
• Loss of Market Share and Revenue
• Penalties, Fines and Regulatory Fee
Application
Segment
Average Cost of
Downtime/Hour
Shipping - Distribution
$28,000 per hour
Tele-Ticket Sales
$69,000 per hour
Airline Reservations
$89,000 per hour
Home Shopping
$113,000 per hour
Pay Per View Television
$150,000 per hour
Credit Card Sales
$2,650,000 per hour
Financial Market
$6,450,000 per hour
Source: Giga Group 2004
At 99% Uptime, a financial market would
lose around $540 million per year
8
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS SCENARIOS BY INDUSTRY
 RETAIL
 TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
9
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
HIGH AVAILABILITY DATA REPLICATION (HDR)
 Two node configuration for failover – one primary and
one secondary
 Fully active-active configuration with updatable
secondaries
 Supports asynchronous and synchronous replication
 Supports manual or automatic failover
 Supports automatic client re-connect
 Simple to setup and administer
 Can be migrated to RSS (discussed shortly)
10
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS HDR
LOG TRANSFER
PRIMARY
11
HDR SECONDARY
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
SHARED DISK SECONDARY (SDS)
 Multi-node configuration – one primary and any number of
additional secondary servers
 Nodes share disk subsystem, with Primary as master of disk
 Ideal in a cluster environment
 Very fast failover, any secondary can be promoted to
primary when needed
 Additional nodes can be used for load balancing queries and
transaction updates
 Can be configured with other IDS availability components
12
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Shared Disk Secondary
Primary
SDS 1
SDS 2
Primary with Multiple SDS nodes on
a blade server
Primary
SDS #1
Shared
Disk
SDS #2
SDS #3
Blade Server
13
Business Continuity for the Modern Business | IDS Availability Suite
Shared
Disk
Mirror
© 2008 IBM Corporation
IBM Data Management
Remote Standalone Secondary (RSS)
 Similar to HDR:
 Maintains a full disk copy of the database.
 Created by performing a backup/restore of the instance
 Can be used for: Additional Backup, Load Balancing.
 Distinct from HDR:
 Uses full duplex communication (SMX)– better throughput over slower lines.
 Does not support SYNC mode, not even for checkpoints.
 Can not currently be ‘promoted’ to primary – but can be promoted to HDR
secondary (Focus is on Disaster Recovery, not HA).
 There can be any number of RSS instances
 Requires Index Page Logging be turned on.
 RSS can be used in combination with HDR secondary:
 RSS can be converted into HDR secondary
 HDR secondary can be converted into RSS.
14
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Enterprise Data Replication (ER)
 Used for Workload partitioning, capacity relief and
data distribution
 Can be used to replicate subset of data (rows,
columns)
 Supports update anywhere with very low latency
 Integrated to be compatible with all other IDS
availability solutions with the option of secure data
communications
 Applies Parallelism in update of target tables
reducing response latency back to the source
 Built-in conflict resolution
 Works in a variety of topologies for varying
business needs
Cont’d …
15
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS Enterprise Replication
Routing topologies
– Fully Connected
– Update anywhere (workload sharing)
– Hierarchical Tree
– Primary-target (consolidate or
distribute data
– Forest of Trees
16
Replication models
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Continuous Log Restore (CLR)
 Provides value as part of Disaster
Recovery solution
 Provides a secondary instance with
‘log file granularity’
Primary
 Asynchronous update of remote site
through log files replay
 Does not impact the primary server
 Can co-exist with “the cluster”
(HDR/RSS/SDS) as well as ER

Useful in unreliable, intermittent or
non-existent network environments
Cont’d …
17
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
Availability
IBM Data Management
IDS Availability Options – Solutions that fits your need
 High Availability (HA)
HDR
HDR+RSS
Continuous
 Scale Out
Availability
HDR+SDS
HA with Scale Out
HDR+SDS+RSS (The IDS Cluster)
 Enterprise Data Replication (ER)
IDS ER
 Not one size fits all
18
Business Continuity for the Modern Business | IDS Availability Suite
HA
Scale Out
ER
© 2008 IBM Corporation
Availability
IBM Data Management
IDS Availability Options – Solution that fit your need
 High Availability (HA)
HDR
HDR+RSS
 Scale Out
Continuous
Availability
IDS Global
Availability
HDR+SDS
HA with Scale Out
HDR+SDS+RSS (The IDS Cluster)
 Enterprise Data Replication (ER)
IDS ER
 Not one size fits all
HA
HDR
or
HDR
+
RSS
19
Business Continuity for the Modern Business | IDS Availability Suite
Scale Out
ER
HDR
+
SDS
ER
© 2008 IBM Corporation
Availability
IBM Data Management
IDS Availability Options – Solution that fits your need
 High Availability (HA)
HDR
HDR+RSS
 Scale Out
Continuous
Availability
IDS Global Availability
HDR+SDS
 HA with Scale Out
HDR+SDS+RSS (The IDS Cluster)
 Enterprise Data Replication
(ER)
IDS ER
 Not one size fits all
Any combination of HDR along with SDS, RSS
and ER or just ER alone can be used to meet
your requirements
20
HA
HDR
or
HDR
+
RSS
Business Continuity for the Modern Business | IDS Availability Suite
Scale Out
IDS Cluster
HDR
+
SDS
HDR
+
SDS
+
RSS
Data
Replication
ER
© 2008 IBM Corporation
IBM Data Management
IDS Availability Options – Solution that fits your need
Do you need to
Protect yourself from
Node failure?
Yes
No
Use SDS
Use HDR
Yes
Do you need to
Multilevel site failure
protection?
Do you need to
protect yourself from
Site failure?
Yes
Use RSS
No
END
Yes
Use ER
Do you need
geographically disperse
processing?
No
21
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS SCENARIOS BY INDUSTRY
 RETAIL
 TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
22
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Integrated Online Reservation System (IORS)
 Consider an online reservation system that
will provide integrated airline, hotel and car
reservations with the following model:
– Will maintain 5-7 years of customer data online for
highly customized service
 Expected data size around tens of terabytes
– Will have several lookup applications to provide an
integrated view to customer:






23
Customer preferences
Airline, car and hotels
Travel advisories
Corporate discounts
Employee verification
Previous travel/reservation listings
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
System Infrastructure and Deployment
 Operations will be managed out of
service/support and data centers:
– Continued use of agent center for customer
service and support
– IDS and web servers at data center for website
operations
– To handle peak workloads, IORS will have:
 Load balanced web servers
 Multiple IDS instances running on blade servers
configured in SDS mode
 Shared disk subsystem used by IDS instances
24
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Application Architecture
 Partner Application Interface – airlines, hotel, car
rental
 Reservation module
 Verification module
 Customer Preferences module
 Corporate Discounts module
25
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Database Architecture
 One Primary IDS instance and 3
SDS instances
 All updates will be directed to the
primary
 SDS nodes will be used to loadbalance queries
 Data resides on shared disk:
– Reservations Database
– Customer Database
– History Database
26
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Configuring SDS
 Set SDS_TIMEOUT on Primary
 Make Primary active in SDS environment
– onmode –d set SDS primary server_name
 On secondary set SDS_ENABLE, SDS_PAGING,
SDS_TEMPDBS
 Start secondary using oninit
 Monitor SDS stats using onstat –g sds
27
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS: Application Logic and System Design
Achieving Availability and Scalability with IDS SDS
Airline Systems
Partner App Interface
Hotel Systems
Car Rental Systems
Reservation App
Updates to
Updates to
Primary
New Primary
Verification Process App
Primary
Customer Preferences App
Corporate Discounts Process
Customer History App
SDS_1
Reads configured
On 3 SDS nodes
SDS_2
SDS_3
Blade Server hosting IDS
SDS instances
28
Business Continuity for the Modern Business | IDS Availability Suite
Shared Disk
Subsystem
© 2008 IBM Corporation
IBM Data Management
Application Availability with SDS
 If the primary fails:
– promote an SDS node to primary
 onmode –d make primary sds_servername
 restart old primary later as new SDS node
 Primary can also be brought down for maintenance
– issue onmode –d make primary to promote SDS to primary
– this will also shutdown the primary
29
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Availability: Incremental improvements
 As application continues to service more users,
stakeholders look to improved availability:
– The shared disk itself is not protected from failure
– The blade server itself can be a single point of failure
 Two approaches:
– Add a disk mirror and additional blade server with
SDS nodes
– Add HDR secondary
 Useful if secondary needs to be hosted at a different
facility within same vicinity to account for site failover (with
n/w ping rates under 50 ms)
30
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Redundant SDS for Additional Failover
Redundant SDS nodes
SDS 4
Primary
SDS 5
SDS 1
SDS 6
SDS 2
SDS 3
SDS 7
Disk Mirror
31
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Site Failover Protection with HDR
HDR Secondary
Primary
SDS 1
SDS 2
SDS 3
DATA CENTER
BLDG 1
32
SERVICE CENTER
BLDG 2
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS team decides on
Additional site failover with HDR – with
existing SDS nodes capable of handling
growth requirements, site failure protections
is critical
Secondary can also be used for scale as
needed
The HDR secondary will be housed at the
support center building
33
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS : Improved Availability with IDS HDR
Improving Business Continuity with IDS SDS and HDR
Airline Systems
Partner App Interface
Hotel Systems
Car Rental Systems
Reservation App
Updates to
Primary
HDR
Verification
Process App
Primary
Customer
Preferences App
Corporate
Discount Process
SDS_1
Primary
Reads configured
On 3 SDS nodes
SDS_2
SDS_3
Secondary
Shared Disk
Blade Server hosting IDS Subsystem
SDS instances
34
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS : Improved Availability with IDS HDR
Improving Business Continuity with IDS SDS and HDR
Airline Systems
Partner App Interface
Hotel Systems
Car Rental Systems
Reservation App
Updates re-routed to new Primary
Verification
Process App
Primary
Customer
Preferences App
SDS_1
Primary
Queries
re-routed to new Primary
SDS_2
Corporate
Discount Process
SDS_3
Secondary
Shared Disk
Blade Server hosting IDS Subsystem
SDS instances
35
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
HANDLING SUCCESS
MANAGING IORS GROWTH
36
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS SYSTEM - REVIEW
 Application interface to third party systems
 Application manages load balancing with writes on
primary and reads on SDS nodes
 Additional HDR node for site failover
With this existing configuration, the IORS team were
successfully able to manage scale and availability
per business requirements
37
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
NEW BUSINESS AND NEW REQUIREMENTS
 IORS stakeholders have signed up three
new airline and three new hotel partners
essentially doubling current accounts
 Two additional service centers will be
opened one additional in the United States
and another in India
 As part of improving customer service and
overall system, data will be collected on
customer visits to the site
38
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
REVISITING SYSTEM REQUIREMENTS
The IORS team has made the following
recommendation of the infrastructure to handle
new requirements:
 Additional IDS SDS cluster and a disk mirror
 Deployment of IDS Connection Manager to
dynamically load balance and arbitrate failover
 Additionally configured IDS ER node at each
customer service site
 Deployment of the Open Admin Tool to
remotely monitor the cluster
The IORS team was also able to present the additional SDS
cluster as incremental improvement to availability as it provides
node failover for current blade server
39
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS CONNECTION MANAGER
The benefits of IDS Connection Manager:
 Dynamically routes connection requests to
most appropriate server in HA cluster

Connection Manager Arbitrator enables
automatic failover logic for cluster
 Works
by using a well-defined Service Level
Agreement (SLA) that allows different
services levels such as OLTP (update
intensive) or Reports (read only)
40
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS Connection Manager
Clients
db_server
Reports
Connection
Redirection
db_sds_rw
OLTP
OTHER
db_sds_ro
41
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
CHANGE IS GOOD!
Since IORS modules have implemented
some custom load-balancing, using the
Connection Manager to its best potential
requires some change to IORS modules.
However, this change is for the better:
 Higher level of separation of business
logic
– modules no longer have to know or manage
the server connectivity of individual servers
– lower application maintenance
 Dynamic management of load-balancing
including read/write scaling and failover
42
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS Connection Manager (CM)
 Connection Manager is installed with IDS CSDK
 CM can run on any machine independent of the IDS
server instances
 Defined as named “SLAs” in a configuration file or
through command line
 Clients connect to SLAs in CM, so CM SLA itself
must have entry in sqlhosts
43
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS CM – configuring for load balancing
Using the default configuration file
$INFORMIXDIR/etc/cmsm.cfg :
Name
iors_db
SLA
reports=sds_2_ro+sds_4_ro+sds_3_rw
SLA
oltp=(ids_primary+sds_5_rw)
LOGFILE
/products/ids11.50/logs/oncmsm.log
Tells CM to try sds_2_ro first, then sds_4_ro next and
finally sds_3_rw if 2 and 4 are unavailable
Parenthesis tells CM to treat ids_primary and sds_5_rw
with equal priority but use one with the least load
44
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS CM – SQLHOSTS for CM
The CM entries must be entered into sqlhosts:
#primary server
ids_primary
onsoctcp
Iors.dc
9088
oltp
onsoctcp
Iors.dc
19088
reports
onsoctcp
Iors.dc
19089
#connection mgr
CM requires one server definition to go about it’s business
45
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS CM – SQLHOSTS for IDS Server
#primary server
46
ids_primary
onsoctcp
Iors.dc
9088
#SDS servers
sds_2_ro
onsoctcp
Iors.dc
9098
sds_3_rw
onsoctcp
Iors.dc
9108
sds_4_ro
onsoctcp
Iors.dc
9208
Sds_5_rw
onsoctcp
lors.dc
9308
sds_6_ro
onsoctcp
Iors.dc
9408
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS CM – SQLHOSTS for Client
Client only needs entries for Connection Manager SLA definitions:
#connection mgr
oltp
onsoctcp
Iors.dc
19088
reports
onsoctcp
Iors.dc
19089
Applications now connect to their databases via SLA:
‘reserve_db@oltp’ or ‘service_db@oltp’
47
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS – The New Look
Airline Systems
Partner App Interface
Hotel Systems
Car Rental Systems
Connection
Manager
Reservation App
HDR
Mirror
Verification
Process App
Customer
Preferences App
Corporate
Discount Process
= Reads
= writes
48
Primary
SDS_4
SDS_1
SDS_5
SDS_2
SDS_6
SDS_3
SDS_7
Business Continuity for the Modern Business | IDS Availability Suite
Secondary
Shared Disk
© 2008 IBM Corporation
IBM Data Management
Monitoring the IDS Connection Manager
With the default configuration file, CM can be instantiated
from the command line :
$prompt$ oncmsm
Monitoring the CM can be done using:
$prompt$ onstat –g cmsm
IBM Informix Dynamic Server Version 11.50.UC1 -- OnLine -- Up 1 days 00:29:38 -- 166296
Kbytes
CM name host
sla
define
foc
flag
connections
ids_db
iors.dc
reports
sds_2_ro+sds_4_ro+sds_3_rw
3
30
ids_db
iors.dc
oltp
ids_primary+sds_5_rw
3
130
Ok, but what is foc ?
49
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IDS CM Arbitrator – configuring for failover (contd.)
 FOC – Fail Over Configuration parameter
 Another SLA in CM SQLHOSTS for failover
model
 Defaults to SDS+HDR+RSS, 0
Timeout in seconds
 Can be disabled using keyword DISABLE
Name
iors_db
SLA
reports=sds_2_ro+sds_4_ro+sds_3_rw
SLA
oltp=(ids_primary+sds_5_rw)
LOGFILE
/products/ids11.50/logs/oncmsm.log
FOC
50
SDS+HDR,2
DISABLE
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Monitoring IDS Clusters using OpenAdmin Tool
51
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Monitoring Connection Manager using OpenAdmin Tool
52
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Service Center
Data Distribution
IDS Enterprise
Replication
53
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
IORS Customer Service - Distributed Call Centers
Call Center 1, USA
Let’s take a look at IRS customer
service model:
Company has 3 call centers
distributed geographically
Distributed call centers help to
manage costs and service
customers effectively
 Each call center has several call
center representatives (CSR)
 Each call is routed to an available
CSR
 Any subsequent calls maybe
routed to a different CSR and/or
different call center
54
Call Center 2, USA
Business Continuity for the Modern Business | IDS Availability Suite
Call Center 3,
India
© 2008 IBM Corporation
IBM Data Management
Benefits of Data Replication
Why distribute data across the
service centers ?
– Data distribution enables faster
local access to information
– Provides additional load
balancing by distributing
workload across centers
– Helps provide higher data
availability
– Can help facilitate Decision
Support reporting activities
55
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Call Center 1, USA
service table and customer history table
are marked for replication
Customer connected first to a CSR in Call
Center 1 and Problem report created
CSR
Customer calls end of the day , connected
to Center 3; service record has already
been replicated, new updates recorded
Call Center 2, USA
Call Center 3, India
CSR
56
Business Continuity for the Modern Business | IDS Availability Suite
CSR
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS SCENARIOS
BY INDUSTRYCOMPONENTS
•CONFIGURING
IDS AVAILABILITY
 RETAIL
•BUSINESS
SCENARIOS BY INDUSTRY
 TELCO - TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
57
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
AUTOMATED TELECOM SYSTEM
Financial Services Trading
 Consider an automated telecom
Telecom Service Connect
system:
Authenticate user
Check availability
Validate end service
Provision the service
Validate billing
Authorize service
OSS/BSS Applications
– performs user authentication and
service provisioning
– manages call connection and call state
– manages generation of Call Detail
Records (CDRs)
– integrates data from CDRs to billing
systems
– makes data available to warehouse for
trend analysis
Database
OSS : Operations Support Systems
BSS : Business Support systems
58
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
System Requirements
 The system has the following requirements on
performance, scale and availability:
– Perform authentication using very quick lookups on
subscriber profile in provisioning database
– Perform very quick reads and updates for dynamic state
management
– capacity to manage call volumes/data to the order of tens of
millions a day
– generate timely data to be sent to carriers during periodic
intervals in a day
59
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Database Architecture
 Provisioning database
– used to authenticate caller, verifies account
– Very quick lookups required
 Call state management database
– records call details after call is connected
– very frequent updates of call information
 Account database
– staging DB for CDR data
– data is used by warehouse applications for trend analysis
– data is also used to build billing info
 Billing database
– holds data pruned from Account DB for billing
– ability to service hundreds of millions of subscribers’ bill view
60
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
The technology team has decided on the following system configuration
Operations
Cell phone
users
Provisioning
Server
Billing
DATA
WAREHOUS
E FOR
TREND
ANALYSIS
Prov DB
RADIUS
Server
Account DB
RMS
Server
State DB
Billing DB
Secondary
RMS : Resource Management Server
RADIUS : Remote Authentication Dial In User Service
61
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
HANDLING SUCCESS
MANAGING GROWTH
62
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
As business grows, our Telecom Service Provider
realizes the need for unifying and optimizing systems
as well as improving overall system availability:
 Unified billing systems from disparate ones from
acquisitions
 Handle twice the capacity on unified systems
 Handle expected growth of 15% every year for the next
5 years
63
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Recommended Configuration
 Provisioning DB
– An additional ER node to handle workload balance
 State DB
– One primary IDS instance, 2 SDS instances with query load
balancing on SDS instances and one remote HDR secondary
– Connection Manager for dynamic capacity management and
failover
 Account DB
– One primary IDS instance and one HDR secondary
 Billing DB
– One primary IDS instance, 3 SDS instances, one HDR secondary
for remote failover
– Connection Manager for dynamic capacity management and
failover
64
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Call Operations
Cell phone
users
Provisioning
Server
Connection manager
Prov DB Primary
BILLING
RADIUS
Server
Prov_SDS2
Connectio
n Manager
RMS
Server
State DB Primary
HDR Secondary
State_SDS1
State_SDS2
Cont’d …
RMS : Resource Management Server
RADIUS : Remote Authentication Dial In User Service
65
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Billing
HDR Secondary
From call operations
DATA
WAREHOUS
E FOR
TREND
ANALYSIS
Account DB
IDS
RSS
Billing_Pri
Billing_SDS1
Connectio
n Manager
Billing_SDS2
W
e
b
A
p
p
HDR Secondary
66
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS SCENARIOS
BY INDUSTRYCOMPONENTS
•CONFIGURING
IDS AVAILABILITY
 RETAIL
•BUSINESS
SCENARIOS BY INDUSTRY
 TELCO - INTERACTIVE ENTERTAINMENT
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
67
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
The World of 3D Online Games
 Massively Multi-player Online Games
(MMOGs)
– Networked Game play
– Highly Immersive Experience
– Persistent worlds that exist after players sign off
– Hundreds of thousands of simultaneous players
– Very high demands on performance and scale
– Divided Worlds to manage scale
– Complex design, infrastructure
– Data management critical part of game play and
account management
68
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Optimizing Data Management for MMOGs
Consider a company building a new MMO with a “One World”
experience breaking the divided worlds barrier. The advantages:
 A seamless game experience - all players able to play against
any other
 Easier game management, simpler upgrades and rollout
 Easier Account/player management
 Ultimately, will provide a competitive advantage
69
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
MMOG Architecture
 The company has made an initial analysis of its requirements:
– Handle 50 thousand concurrent users on average
– Handle growth of about 20% concurrent users every year
– Handle expected peak load of about 100,000 concurrent users
 The infrastructure will consist of :
– Login servers to manage the accounts
– Game servers that will host the Game World
– Database servers for game play and account management
– Game client to provide necessary graphical rendering
70
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Optimizing MMO Data Management with IDS
71
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Information Architecture
The MMO information architecture consists of:
– Accounts database: login application will use this for
authentication and new subscriber registration
– Game world database: hosts world specific data such as layouts,
game strings, Non-Playing Characters (NPC); read only
– Player database: persist changes to player character, skill
levels, assets, health levels and milestones reached
– Logging database: used to log player movements , behavior
trouble shooting and data for service improvements
– Reports and Statistics database: relevant data offloaded from
logging database for business analysis, marketing, etc.
72
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
The First Experience – Game Login
 An important first experience:
– Long waits or lockouts can frustrate players
– Can subsequently lead to bad PR
 Given critical nature of the process, and based on data activity
model, the MMO team has decided on the following:
– IDS instances configured with SDS nodes
– Query Load will be balanced across secondary nodes
– Primary will be used for updates
73
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Login
Server 1
Login
Server 2
Connection
Manager
Primary
SDS_1
Accounts
SDS_2
DB
SDS_3
74
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Let the Games Begin! – MMO Game Play
 On successful login, players connect to
game servers
 Game world data is read only, but is a
critical foundation
 MMO team has decided to use a primary
with an offsite HDR instance to handle
failover
75
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
success
Login Servers
Game Servers
Primary
World DB
76
Business Continuity for the Modern Business | IDS Availability Suite
HDR
Secondary
Failover
© 2008 IBM Corporation
IBM Data Management
MMO Player Data Management – Player Database
 The most critical data management of the game:
– Lots of small transactions that occur frequently
– Response to changes need to be very fast
– Transaction integrity crucial
– Failover and recovery need to be smooth and reliable
 MMO team has decided on:
– IDS configured with SDS nodes; updates on primary, scale
to secondaries as needed
– A redundant SDS cluster to protect against node failover
– An HDR instance to protect against site failover
77
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Client
traffic
Connection
Manager
Game
Servers
Failover
Client
traffic
SDS_4
Primary
SDS_1
SDS_5
SDS_2
SDS_6
SDS_3
SDS_7
Offsite HDR
Secondary
Player
DB
Shared Disk
78
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Collecting Data – MMO Game Logging
 Collects data on player movements, behavior,
troubleshooting, marketing
 Size and volume are custom to the game
– Could be collected very frequently or less frequently
– Could collect data on a variety of objects or detail or fewer
objects and lesser detail
 MMO team has decided to capture detailed data
 To optimize performance, workload partitioning will
be realized using ER
79
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Choosing a Replication Model and Topology
 The MMO team is deploying 20 game servers in a
cluster
 Each game server will have an IDS ER participant to
manage the updates to log tables
 The team wants the data from each IDS instance
replicated to other instances using an Update
Anywhere replication model
 But what about the routing topology - fully
connected or hierarchical?
80
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Fully connected Topology
Ids_inst_1
Ids_inst_3
Game Server3
Game Server1
More efficient with no
routing required between
participants
Useful when h/w resources
are constrained : memory,
CPU, threads
Game Server2
81
Business Continuity for the Modern Business | IDS Availability Suite
Ids_inst_2
© 2008 IBM Corporation
IBM Data Management
Hierarchical Tree Topology
Ids_er2
Ids_er1
IDS root
instance
IDS root instance
synchronizes routing of
data
82
Ids_er3
Useful topology when
convenience/flexibility
important and there are a large
number of participants
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Choosing a Replication Model and Topology (Contd.)
 The team decided on the hierarchical tree
topology as:
– it offered convenience and flexibility of adding more
participants
– hierarchical trees can be interconnected to form a
“forest of trees” topology
– Additional data centers can now be interconnected
using this topology
83
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
ids_us_1
ids_us2
ids_us3
Forest of Trees
Topology
ids_eu1
ids_root1
Update Anywhere
model with a
hierarchical tree
topology in the USA
ids_eu3
All participants synchronized via
interconnected root nodes
Update Anywhere
model with a
hierarchical tree
topology in Europe
84
ids_eu2
ids_root2
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
But what happens if the synchronization
server goes down for some reason?
Wouldn’t all the participants be out of
sync ?
– The MMO team realizes that adding an
HDR secondary to each root node is
straightforward and alleviates their
concerns
– Not only will the synchronization server
have failover protection, all the participants
will automatically recognize the new
primary as synchronization instance
85
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
ids_us_1
ids_us2
ids_us3
Forest of Trees
Topology
ids_eu1
Secondary
Failover
ids_root1
ids_eu2
ids_eu3
Failover
ids_root2
Secondary
86
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS
SCENARIOS BY INDUSTRY
SUMMARY
 RETAIL
 TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
87
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
SUMMARY
 Downtime = $$
 Understand your RCC
– Requirements for availability
– Your existing Capabilities
– Understand the Cost of achieving high availability
 IDS offers a range of options for continuous availability
– Clustering and availability via Shared Disk Secondary
instances
– Site failover protection and additional scale with HDR
– Disaster Recovery with multiple Remote Standalone
Secondary instances and CLR
– Combine these options to build a continuous availability
model that fits your business needs
88
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
Content Summary
 AVAILABILITY CONCEPTS
 IDS AVAILABILITY OFFERINGS OVERVIEW
 CONFIGURING IDS AVAILABILITY COMPONENTS
– BUSINESS
SCENARIOS BY INDUSTRY
RESOURCES
 RETAIL
 TELCO
 INTERACTIVE ENTERTAINMENT
 SUMMARY
 RESOURCES
89
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation
IBM Data Management
RESOURCES
 Product Family Homepage:
http://www.ibm.com/software/data/informix/
 Platform Roadmap:
http://www.ibm.com/software/data/informix/pubs/roadmaps.html
 Product Life Cycle (PLC) Roadmap:
http://www.ibm.com/software/data/informix/support/plc/
 Developer’s Homepage:
http://www.ibm.com/developerworks/db2/zones/informix/
 International User Group (IIUG): http://www.iiug.org
 Interoperability:
http://www.ibm.com/software/data/informix/ids/interop/
 Documentation/Manuals:
http://www.ibm.com/software/data/informix/pubs/library/
 IDS Redbooks: http://www.redbooks.ibm.com/cgibin/searchsite.cgi?Query=Informix&SearchMax=250&SearchOrder=4
90
Business Continuity for the Modern Business | IDS Availability Suite
© 2008 IBM Corporation