Hitachi Unified Compute Platform 6000 for SAP
S/4HANA
Reference Architecture Guide
By Milind Pathak, Prasad Patkar, and Mark Walker
December 2015
Feedback
Hitachi Data Systems welcomes your feedback. Please share your thoughts by sending an email message to
[email protected]. To assist the routing of this message, use the paper number in the subject and the title of this white
paper in the text.
Contents
Solution Overview................................................................................................................ 2
Key Solution Elements......................................................................................................... 4
Hardware Elements......................................................................................................................... 4
Software Elements .......................................................................................................................... 8
Solution Design................................................................................................................... 10
SAP Application and Database Architecture and Sizing...............................................................
High Availability for Production Environment................................................................................
Fibre Channel Architecture ...........................................................................................................
Management Server......................................................................................................................
Network Architecture ....................................................................................................................
Storage Architecture .....................................................................................................................
Parity Group, RAID, and LUN Configuration.................................................................................
SAP HANA Configuration..............................................................................................................
S/4HANA Configuration ................................................................................................................
10
14
23
26
27
32
41
47
49
Engineering Validation ....................................................................................................... 51
Test Methodology ......................................................................................................................... 51
Test Results .................................................................................................................................. 51
Appendix .............................................................................................................................. 52
SUSE Documentation ................................................................................................................... 52
SAP Documentation and SAP Notes ............................................................................................ 52
1
Hitachi Unified Compute Platform 6000 for SAP S/4HANA
Reference Architecture Guide
This reference architecture guide describes an example architecture of how to deploy a three system landscape of S/
4HANA on Hitachi Unified Compute Platform using the following:

Hitachi Compute Blade 2500 (CB 2500) with 520X B2 server blades

Hitachi Compute Blade 2500 with 520H B3 server blades

Hitachi Virtual Storage Platform G400

SAP HANA 1.0 SPS09 or later

S/4HANA (SAP ECC6 EHP7 with SimpleFinancials)

Suse Linux Enterprise Server 11 SP3 with High Availability Extension
This technical paper assumes that you have familiarity with the following:

Storage area network (SAN)-based storage systems

General storage concepts

Common IT storage practices

SAP HANA

SAP ECC

VMware vSphere 5.5 update 2
1
2
Solution Overview
This reference solution uses Hitachi Unified Compute Platform for the SAP HANA Platform with the following components
for the scale-up configuration and S/4HANA application servers:








Hitachi Compute Blade 2500 (CB 2500) — An enterprise-class blade server platform. This solution uses one CB
2500 chassis with four 520X B2 for the SAP HANA Platform and eight 520H B3 server blades for the SAP S/4HANA
application, as shown in Figure 1.
Hitachi Virtual Storage Platform G400 (VSP G400) — Storage virtualization system designed to manage storage
assets more efficiently. The persistent storage of the SAP HANA server and SAP application resides on this storage
device. This solution uses one VSP G400.
Quanta Cloud Technology QuantaPlex T41S-2U chassis — An ultra-dense design equipped with four independent
nodes with the flexibility to set up different workloads independently in one 2U shared infrastructure. This solution
uses one QuantaPlex T41S-2U chassis with a single node that acts as a central device for managing the SAP HANA
Platform.
Brocade VDX 6740 switch — A 48-port switch that provides 10 GbE connectivity to the appliance. This solution
uses two Brocade VDX 6740 switches.
Brocade ICX 6430-24 switch — A 24-port 1 GbE switch that provides the network for management of all hardware
components using the Quanta Cloud Technology QuantaPlex T41S-2U chassis with one node (used as a
management server). This solution uses one Brocade ICX 6430-24 switch.
SAP HANA Platform — A flexible, data source-agnostic, in-memory data platform that allows customers to analyze
large volumes of data in real time.
SAP S/4HANA - SAP S/4HANA is the short form for SAP Business Suite 4 SAP HANA. It is a new product fully built
for in-memory platform – SAP HANA. SAP S/4HANA creates unique opportunities to simplify the landscape and help
reduce total cost of ownership.
VMware vSphere -- A cloud computing virtualization operating system platform.
This reference architecture describes an example configuration to deploy a three system landscape of S/4HANA,
Development, QA, and Production systems in a highly available configuration.
2
3
Figure
1 shows the topology of this reference architecture.
Figure 1
3
4
Key Solution Elements
The following are the key hardware and software components used in this reference architecture.
Hardware Elements
Table 1 lists the hardware used to deploy the specific scale-up configuration of Unified Compute Platform for the SAP
HANA Platform for the different sized solutions.
Table 1. Hardware Elements
Hardware
Quantity
Hitachi Compute Blade 1
2500 (CB 2500) chassis
Configuration (per unit)
Role

8 full width blade slots
Server blade chassis

2 management modules

10 cooling fan modules

5 Power Supply Modules (PSM)

28 I/O board modules


Hitachi Compute Blade 4
520X B2 server blade
Hitachi Compute Blade 8
520H B3 server blade
SMP connector module 2





12 × 10 Gb 2-port LAN PCIe
adapters
12 × 16 Gb 2-port Hitachi FIVE-FX
FC PCIe adapters
2 × 18-core processors
SAP HANA production
and QA servers
2 × pass through mezzanine card.
One on mezzanine slot 2 and one on
mezzanine slot 4 of primary blade
2 × 18-core processors
SAP application servers
and SAP HANA DEV
1 × pass through mezzanine card on
mezzanine slot 2
SMP connector board: 2-blade
connector board for 4-Socket

SMP expansion module

SMP connector cover
SMP connector for SAP
HANA QA and PRD server
4
5
Table 1. Hardware Elements (Continued)
Hardware
Quantity
Configuration (per unit)
Role
Hitachi Virtual Storage
Platform G400
Single
frame

Block storage for SAP
HANA nodes and SAP
application servers
Quanta Cloud
1
Technology QuantaPlex
T41S-2U Chassis with
one node





Single frame
Intel Xeon E5-2620 v3 processor, 2.4
GHz CPU, 32 GB RAM

2 × 500 GB 7200 RPM SATA drives
1 dual port 10GigE Intel 82599ES
SFP+OCP Mezzanine card
1 dual port 1 GigE Base-T Intel i350
Mezzanine Card
Emulex Dual Port 8 Gb/sec Fibre
Channel HBA
Brocade VDX 6740
switch
2

Brocade ICX6430-24
port switch
1

1 GbE

24 ports




One node of the
server serves as the
management server
running the following:
NTP
Hitachi Command
Suite
Hi-Track® Remote
Monitoring system
SAP HANA Studio
Two switches with dedicated VLANs 10 GbE network
for production, non-production, NFS connection for
production, nonand replication network
production, NFS and
replication network
1 GbE management
network
Hitachi Compute Blade 2500
Hitachi Compute Blade 2500 delivers enterprise computing power and performance with unprecedented scalability and
configuration flexibility. Lower your costs and protect your investment.
Flexible I/O architecture and logical partitioning allow configurations to match application needs exactly with Hitachi
Compute Blade 2500. Multiple applications easily and securely co-exist in the same chassis.
Add server management and system monitoring at no cost with Hitachi Compute Systems Manager. Seamlessly integrate
with Hitachi Command Suite in Hitachi storage environments.
5
6 configuration uses a maximum of four 520X B2 server blades and maximum eight 520H B3 server blades in the
The
Hitachi Compute Blade 2500 chassis for the different sized solutions. Supported scale-up SAP HANA platforms and the
required number of server blades are listed in Table 6 on page 13 and Table 7 on page 13.
Table 2 lists the configurations for the 520X B2 server blades used for Unified Compute Platform for SAP HANA scale-up
configuration for production and QA databases.
Table 2. 520X B2 Server Blade Configuration
Feature
Configuration
Small (2-Socket)
Medium (4-Socket)
Processors
2 Intel Xeon E7-8880
processors
4 Intel Xeon E7-8880
processors
Processor SKU
Intel Xeon E7-8880 v3
Processor frequency
2.3 GHz
Processor cores
18 cores
Number of blades
1
2
Number of 128 GB
DIMMs per
blade
256 GB
8 × 16 GB DIMMs
8 × 8 GB DIMMs
16 × 16 GB DIMMs
16 × 8 GB DIMMs
384 GB
24 × 16 GB DIMMs
24 × 8 GB DIMMs
512 GB
32 × 16 GB DIMMs
16 × 16 GB DIMMs
768 GB
48 × 16 GB DIMMs
24 × 16 GB DIMMs
1024 GB
32 × 32 GB DIMMs
32 × 16 GB DIMMs
1536 GB
48 × 32 GB DIMMs
48 × 16 GB DIMMs
2048 GB
N/A
32 × 32 GB DIMMs
3072 GB
N/A
48 × 32 GB DIMMs
Network Ports
2 × 2-port 10GBASE-SR LAN PCIe adapters on two
I/O board modules.
Fibre Channel Ports
2 × 2-port 16 Gb/sec Hitachi FIVE-FX Fibre Channel
PCIe adapters on two I/O board modules.
6
7
Table
3 lists the configurations for the 520H B3 server blades used for the scale-up SAP HANA platform used for the
development environment and S/4HANA application servers for production and non-production environments.
Table 3. 520H B3 Server Blade Configuration
Feature
Configuration
Processors
2 Intel Xeon E5-2699v3 processors
Processor SKU
Intel Xeon E5-2699v3
Processor frequency
2.3 GHz
Processor cores
18 cores
Number of DIMMs per blade
24 x 32 GB DIMMs (768 GB)
Network Ports
1 × 2-port 10GBASE-SR LAN PCIe adapter on one I/O board
module.
Fibre Channel Ports
1 × 2-port 16 Gb/sec Hitachi FIVE-FX Fibre Channel PCIe
adapters on one I/O board module.
Hitachi Symmetric Multiprocessing Connector
For multiple server blades, the solution uses Hitachi symmetric multiprocessing (SMP) technology to combine multiple
server blade resources into a single server. The 520X B2 server blades use SMP to combine two server blades with a 2blade SMP connector.
Hitachi Virtual Storage Platform G400
Hitachi Virtual Storage Platform G400 (VSP G400) belongs to the Hitachi Virtual Storage Platform family systems, which
are based on industry-leading enterprise storage technology. With flash-optimized performance, these systems provide
advanced capabilities previously available only in high-end storage arrays. With the Virtual Storage Platform family, you
can build a high-performance, software-defined infrastructure to transform data into valuable information.
Hitachi Storage Virtualization Operating System provides storage virtualization, high availability, superior performance,
and advanced data protection for all models in the Virtual Storage Platform family. This proven, mature software provides
common features to consolidate assets, reclaim space, extend product life and reduce migration effort. New management
software improves ease of use to save time and reduce complexity. The infrastructure of Storage Virtualization Operating
System creates a management framework for improved IT response to business demands.
This solution uses Hitachi Virtual Storage Platform G400. The following LUNs reside on this storage array:

Operating system LUNs for SAP HANA servers and SAP application servers

SAP HANA data, log and shared binaries, and LUNs for production, QA, and development environments.

SAP application LUNs: sapmnt, transport directory, and instance directory.
7
8
Table
4 shows the Hitachi Virtual Storage Platform G400 components used in this solution.
Table 4. Hitachi Virtual Storage Platform G400 Components
VSP G400 Component
Quantity
Cache
128 GB
MPU
2 pair
DKB (BED)
1 pair
CHB (FED)
3 pairs (8 Gb/sec)
Software Elements
Table 5 describes the software products used to deploy this solution. If using higher versions, please check the
corresponding product vendor's compatibility and support matrix.
Table 5. Software Elements
Software
Version
SUSE Linux Enterprise Server for SAP Applications
SLES11 SP3
SAP HANA Platform
1.0 SPS09, Rev. 97
Hitachi Storage Navigator
Microcode dependent
Hitachi Command Suite
8.1.2
S/4HANA
ECC 6 EHP 7, NW 7.40 with SimpleFinancials
VMware vSphere
5.5 update 2
SAP HANA
The SAP HANA platform is flexible, multipurpose in-memory software. It combines SAP software components optimized
to specific hardware. These components come from leading hardware partners of SAP, including Hitachi Data Systems.
The SAP HANA platform allows customers to analyze large volumes of data in real time. It is also a development platform,
providing an infrastructure and tools for building high-performance applications based on SAP HANA Extended
Application Services (SAP HANA XS). It is the foundation of various SAP HANA editions, like the SAP HANA Platform
Edition, providing core database technology, and the SAP HANA Enterprise Edition, bundling additional components for
data provisioning. The SAP HANA Platform Edition integrates a number of SAP components, including the SAP HANA
database, SAP HANA studio and SAP HANA clients. As a SAP customer, you can get more information on the SAP HANA
platform at the SAP Service Marketplace and SAP Help Portal.
SUSE Linux Enterprise Server
Hitachi Unified Compute Platform for the SAP HANA Platform in a scale-up configuration runs on the following SUSE
Linux operating system:

64-bit SUSE Linux Enterprise Server (SLES) for SAP Applications 11 SP3.
Note - The initially delivered configuration of the operating system should persist. Do not make any modifications to
the operating system, except as noted or approved by SAP.
For more details of the operating system changes, see the section, “Updating and Patching the Operating System,” in the
SAP HANA Technical Operations Manual.
8
9
S/4HANA
SAP S/4HANA is a short form for business suite for HANA. It is a new product fully built on an in-memory platform – SAP
HANA – and the modern design principles with the SAP Fiori user experience (UX). SAP S/4HANA delivers massive
simplifications (customer adoption, data model, user experience, decision making, business processes, and models) and
innovations (Internet of Things, Big Data, business networks, and mobile-first) to help businesses run simply in the digital
economy. You can get more information on the SAP S/4HANA Frequently Asked Questions page.
VMware vSphere 5.5
VMware vSphere 5.5 is a virtualization platform that provides a datacenter infrastructure. It features vSphere Distributed
Resource Scheduler (DRS), high availability, and fault tolerance.
VMware vSphere 5.5 has the following components:

ESXi
A hypervisor that loads directly on a physical server. It partitions one physical machine into many virtual machines that
share hardware resources.

vCenter Server
Management of the vSphere environment through a single user interface. With vCenter, there are features available
such as vMotion, Storage vMotion, Storage Distributed Resource Scheduler, High Availability, and Fault Tolerance.

VMware vSphere 5.5
Enterprise virtualization solution to create a dynamic and flexible data center with integrated management and
reporting capability for a high level of server and service uptime
Unified Compute Platform for SAP HANA on the VMware vSphere solution combines the benefits of the Hitachi Unified
Compute Platform for the SAP HANA Platform with the flexibility and manageability of the VMware vSphere solution.
VMware vSphere customers can download more information about the ESXi Platform on the VMware vSphere
Documentation page. See the vSphere installation and setup guide in the ESXi and vCenter Server Product
Documentation section for ESXi documentation. In addition, you can download the Performance Best Practices for
VMware vSphere 5.5 PDF.
9
10
Solution Design
The detailed design for this solution includes the following.
SAP Application and Database Architecture and Sizing
Compute Hardware Layout
This solution shows an example configuration of how a single CB 2500 chassis with four 520X B2 blades, eight 520H B3
blades, and one VSP G400 storage array can be used for deploying a three system landscape of S/4HANA. All of the
hardware components can be installed in a single rack. The following is the server distribution in a CB 2500 chassis used
in this reference architecture:


Two slots are used for two full width 520X B2 blades in the two blade SMP configuration for medium size SAP HANA
QA database instance.
Two slots are used for two full width 520X B2 blades in the two blade SMP configuration for medium size SAP HANA
production database instance.

One slot is used for half width 520H B3 blades for the small size Development HANA Database instance.

Four slots are used for four half width 520H B3 blades for production application servers.

Three slots are used for three half width 520H B3 blades for non-production application servers.
Note – The number of blades used in this reference architecture for application and database are just an example.
Your actual requirement depends on the number of users and type of workloads. Perform hardware sizing to find out
your hardware requirements. Assistance is available from your Hitachi Data Systems pre-sales representative or by
contacting Hitachi Data Systems via email: [email protected]
Figure 2 shows the location of server blades based on their usage.
Figure 2
SLES11 SP3 is used as the operating system for SAP HANA database and application. VMware is used to virtualize
application tier.
10
11
Software
Layout
SAP NetWeaver is the underlying technology platform to power SAP Business Suite. SAP NetWeaver is a three-layer
application stack: Presentation, Application, and Database. This three-layer application stack has two deployment
scenarios: “2-Tier” and “3-Tier.”
In a 2-Tier scenario, Database and Application server runs on the same operating system, e.g. the server, while the
presentation layer runs on a separate operating system, e.g. the client.
In a 3-Tier scenario, Database and Application servers run on separate operating systems, e.g. two different servers, while
the presentation layer runs on a separate operating system, e.g. the client.
For the validation of this reference architecture, 3-Tier installation of S/4HANA and SAP HANA database was performed in
the Hitachi Data Systems lab for Production and QA environments. Figure 3 shows an example layout of the three system
landscape adapted for validation for this reference architecture. The production system was installed as a Highly Available
configuration.
Figure 3
11
12
Sizing,
Scalibility, and Intergration with External IT Components
Sizing
The unit used to perform SAP system sizing is SAP Application Performance Standard (SAPS). SAPS is a hardwareindependent unit of measurement that describes the performance of a system configuration in the SAP environment. It is
derived from the Sales and Distribution (SD) benchmark, where 100 SAPS is defined as 2,000 fully business processed
order line items per hour. SAP documentation regarding sizing can be found at SAP Service Marketplace (quick link
sizing). SAP application sizing can be performed using the SAP Quicksizer. The SAP documentation and the SAP notes
regarding sizing referred to in this reference architecture are a good starting point, but per SAP's guidelines, SAP system
sizing must always be performed by an experienced SAP consultant.
SAP Application Sizing
The SAP Quicksizer results provide SAPS required for the application tier and database tier separately. This can be used
as a starting point for sizing the application layer for the SAP system. In this reference architecture, 520H B3 server blades
are used for the application tier. These blades are certified by SAP on behalf of the SAP Benchmark Council for 82,180
SAPS, certification number: 2015008. This can be used as a reference to perform SAP application layer sizing. This
reference architecture also uses virtualization using VMware ESX Hypervisor for production additional application servers
and the entire application tier of non-production servers. Each 520H B3 server blade has 2 CPUs and 36 cores. Assuming
Hyperthreading is activated (2 threads per physical core), each blade will have 72 threads or 72 vCPUs (virtual CPU). This
translates to 2282 SAPS per physical core and 1141 SAPS per vCPU. The benchmark score is achieved by generating
workload using a benchmarking tool and consuming close to 100% CPU. Perform the sizing by taking into consideration
that the CPU utilization should not go beyond 70%. Also, about 10% overhead for virtualization must be considered.
SAP HANA Database Sizing
Refer to following SAP Community Network(SCN) page, SAP note and SAP QuickSizer to perform SAP HANA Platform
sizing for S/4HANA database:
Sizing for SAP S/4HANA
1514966 - SAP HANA 1.0: Sizing SAP In-Memory Database
Note - These documents refer to several web links from the SAP website. S-user ID with proper authorization may be
required to access some of those links (for example to access SAP Notes).
This solution supports:


Small and medium size scale-up SAP HANA platforms for production and QA environments. All supported
configurations are listed in Table 6
Small size scale-up SAP HANA platform for development environments. All supported configurations are listed in
Table 7
12
13
Table 6. Supported Scale-Up Configuration Sizes for Production and QA Environments
Description
2-Socket
4-Socket
Number of CPUs
2
4
Number of Server Blades
1
2
Memory

128 GB

128 GB

256 GB

256 GB

384 GB

384 GB

512 GB

512 GB

768 GB

768 GB

1024 GB (SoH only)

1024 GB

1536 GB (SoH only)

1536 GB

2048 GB (SoH only)

3072 GB (SoH only)
Table 7. Supported HANA Scale-Up Configuration Sizes for Development Environment
Description
2-Socket
Number of CPUs
2
Number of Server Blades
1
Memory

64 GB

128 GB

256 GB

384 GB

512 GB

768 GB
Scalibility
This reference architecture only describes the configuration that supports a maximum of two 520X B2 server blades and
the 4-socket scale-up SAP HANA Platform for production HANA databases, and four 520H B3 server blades for the
production application tier. It is possible to install the large size (8-socket) scale-up SAP HANA Platform with Hitachi
Compute Blade 2500 (CB 2500) chassis and 520X B2 server blades for SAP HANA and use more blades to add additional
application servers. Refer to the reference architecture paper Tailored Datacenter Integration Implementation of Multiple
SAP HANA Scale-up HSW Appliances on Hitachi Virtual Storage Platform G400 (VSP G400) and Hitachi Virtual Storage
Platform G600 (VSP G600) for all supported scale-up SAP HANA Platform sizes and design. Such a configuration can be
achieved by using more than one CB 2500 chassis and using the same VSP G400 storage, Brocade switches, and a
management server. It is also possible to install production and non-production applications in separate CB 2500 chassis.
Please contact your Hitachi Data Systems representative to design a solution based on your requirements.
13
14
Integration
with SAP and Non-SAP Components External to this Solution
It is very common that SAP Business Suite systems need to communicate with other SAP and non-SAP applications and
databases. There are uplink ports available on Brocade VDX 6740 switches that can be used to communicate with other
systems that are not part of this solution.
One such example is SAP Fiori. SAP S/4HANA is natively designed with SAP Fiori UX, offering an integrated user
experience with modern usability and instant insight on any device. Even if a customer is using SAP S/4HANA, the onpremise edition can still use the traditional user interfaces, and therefore many customers might want to use SAP Fiori
User Experience (UX). This solution will serve as the ABAP backend server which provides the business logic and the
back-end data. Network connection can be established as required to the external SAP Web dispatcher or external
gateway used as central hub. For details refer to the Setup of SAP Fiori System Landscape with SAP HANA Database
document.
High Availability for Production Environment
SAP runs mission critical applications and it is important that these systems remain available to users at all times. This
requires that these systems are able to make faster recovery after a system component failure (High Availability) or after a
disaster (Disaster Recovery). Needless to say, this should happen without any data loss (zero RPO) and in very short
recovery time (low RTO). This section explains how to achieve high availability for production systems using the HA
features of Hitachi Data Systems servers and storage for hardware failures and the HA features of SAP Netweaver and
SAP HANA software (along with SUSE HAE) for software failures as follows:


Hardware Failures: Hitachi Data Systems SAP HANA solutions using Hitachi Compute Blade 2500 with 520X B2 and
520H B3 server blades offer redundant hardware components to provide fault tolerance such as redundant power
supplies and fans, two hot-swappable management modules, and multiple Ethernet and Fibre Channel HBA
interfaces. Similarly, Hitachi Virtual Storage Platform family storage arrays offer redundant hardware components
such as dual controllers, redundant front-end and back-end I/O modules, and power supply units. Hitachi Data
Systems storage design for SAP Application and SAP HANA use striping and double parity to provide redundancy for
automatic recovery from disk failures. Redundant network components are used (two Brocade VDX 6740 switches)
with at least two network connections from each physical server (four for SAP HANA production and QA nodes), one
to each switch. At least two FC connections (four for SAP HANA production and QA nodes) per server are used.
Software Failures: This reference architecture achieves High Availability configuration by addressing the possible
single point of failures (SPOFs) of SAP software:

Database Instance (SAP HANA)

ABAP SAP Central Services (Message Server and Enqueue Server)

Central File System
For more details about High Availability of SAP systems, refer to High Availability - Frequently Asked Questions.
14
15
Database
Instance (SAP HANA)
This reference architecture uses SAP HANA System Replication to provide High Availability for a production SAP HANA
Platform. The secondary production node is installed on the server blades of the QA SAP HANA Platform in the cost
optimized SAP HANA System Replication scenario. In case of a primary failure, the QA HANA database is shut down and
the secondary node takes over production operations. Per SAP note 1681092 - Multiple SAP HANA DBMSs (SIDs) on one
SAP HANA system, this is a supported configuration by SAP. This process is automated using SUSE HAE and SUSE
SAPHanaSR resource agents.
Figure 4
Refer to SAP document How to Perform System Replication for SAP HANA to setup HANA System Replication and SAP
HANA System Replication on SLES for SAP Applications to setup automated failover using SUSE HAE.
15
16
Table
8 shows the information used for HANA System Replication Setup in this reference architecture.
Table 8. Information used for HANA System Replication Setup
HANA Node 1
HANA Node 2
Hostname
prdhana
sechana
SAP HANA SID
HIT
HIT
SAP HANA Instance Number
10
10
IP Address for client network
192.168.100.245
192.168.100.246
IP Address for replication network
192.168.160.245
192.168.160.246
Logical Name for system replication
HITSRP
HITSRS
Virtual IP Address
192.168.100.100
16
17
Figure
4 shows the failover process used in SUSE HAE HAWK (HA Web Konsole). During normal operation, the production
database is running on host prdhana and the QA database and failover production databases are running on host
sechana. The failover production database running on sechana is not available for users. It only receives data from the
primary database. If the primary database server fails, the QA database is stopped automatically by the cluster, and the
secondary database is promoted as the primary database. Application connectivity to the database is through a virtual IP
address and is also managed by the cluster. The virtual IP address is tied to the primary node, and is therefore moved to
the host on which the primary node is running. Refer to SAP HANA System Replication on SLES for SAP Applications and
HOW TO SET UP SAPHanaSR IN THE COST OPTIMIZED SAP HANA SR SCENARIO for details. Set the memory usage
parameters for SAP HANA on quality and failover production according to the SAP HANA administration guide. Provide
separate storage for the QA SAP HANA System, and use the latest SAPHanaSR version available, and also use SAP
HANA Platform 1.0 SPS09, Rev. 97 or later.
17
18
ABAP
SAP Central Services (Message Server and Enqueue Server)
SAP separates the message server and the enqueue work process of the Central Instance for High Availability
configuration and groups them in ASCS (ABAP SAP Central Services). The enqueue server contains a lock table in the
memory of the host (primary node) on which the ASCS instance is running. The enqueue Replication server is installed to
replicate the lock table to a different host (secondary node). In case of failure of ASCS on the primary host, services are
failed over to the secondary node. In this case, the lock table replicated to this secondary host is used to build the lock
table for the enqueue server. There is more than one way to achieve High Availability using SUSE HAE. Refer to Best
practices for mission-critical SAP applications to find the right solution for your requirements. The solution used in this
reference architecture is described in the SUSE guide SAP NetWeaver on SUSE Linux Enterprise Server with High
Availability as it matches the architecture of this reference architecture. Figure 5 shows the two servers sapnode1 and
sapnode2 as cluster nodes. During normal operations, the ASCS Instance and one Additional Application Server run on
sapnode1, The enqueue replication server and primary application server run on sapnode2. SUSE Linux Enterprise High
Availability 11 is certified by SAP (NW-HA-CLU_730), NW 7.40, and is also supported by SUSE. Table 9 shows the
parameters used by cluster components for SAP application.
Table 9. Parameters used by Cluster Components for SAP Application
Cluster Node
(Original
Location)
Virtual
Hostname
Virtual IP
Instance
Number
Role
sapnode1
saps4has
192.168.100.101
00
ASCS Instance
sapnode2
saps4her
N/A
01
Enqueue Replication Server
sapnode1
saps4hci
192.168.100.102
02
Primary Application Server
sapnode2
saps4hd03
192.168.100.103
03
Additional Application Server
18
19
Figure 5
19
20
Figure
6 shows one failover scenario. If sapnode1 fails, ASCS and PAS running on sapnode1 are moved to sapnode2. The
lock table is re-created using the replication table. The enqueue replication server is stopped. All of the virtual IP
addresses are also moved to sapnode2. When the primary node comes back online, the enqueue replication server is
started on this node.
Figure 6
20
21
Figure
7 shows the failover process in SUSE HAE HAWK (HA Web Konsole).
Figure 7
Application Connectivity to Database after Failover
Install all of the application servers using the virtual hostname of the database that is managed by the hdbuserstore
Commands. Also configure work process reconnection parameters based on the time required by databases in failover.
21
22
Central
File System
File system /sapmnt hosts the kernel and profiles of an SAP system and is therefore very critical. This file system must be
mounted on all the instances of a SAP system for the system to be up and running. For the validation of this reference
architecture, Highly Available NFS Storage with distributed replicated block device (DRBD) and SUSE HAE was used. The
DRBD allows you to create a mirror of two block devices across an IP network. When used with SUSE HAE, DRBD
supports distributed high-availability Linux clusters. As shown in Figure 3 on page 11, two virtual machines are used in
this reference architecture as two SUSE HAE cluster nodes to host the Highly Available NFS server that provides the
central file system. When SUSE HAE is installed, all of the packages required by DRBD are also installed by default. Refer
to Highly Available NFS Storage with DRBD and Pacemaker for implementation instructions. It is possible to use Brocade
VDX 6740 switches to connect an external Highly Available NFS Filer (such as Hitachi NAS Platform (HNAS) to provide a
central file system for this solution. VLAN 150 is created on these switches for this purpose. Hitachi NAS Platform is an
advanced and integrated network attached storage (NAS) solution. It provides a powerful tool for file sharing, file server
consolidation, data protection, and business-critical NAS workloads.
SUSE HAE Configuration
Refer to the SUSE Linux Enterprise High Availability Extension documentation for installation and configuration of SUSE
HAE cluster for SAP application and SAP HANA database. The following are the best practices followed in this reference
architecture.
STONITH Device
STONITH (shoot the other node in the head) is the way to implement fencing in SUSE HAE. If a cluster member is not
behaving normally, it must be removed from the cluster. This is referred to as fencing. A cluster without the STONITH
mechanism is not supported by SUSE. There are multiple ways to implement STONITH, but STONITH Block Devices
(SBD) is used in this reference architecture.
Create a small LUN (1 MB) on the VSP G400 array that is shared between the cluster members. Map this LUN to both
primary and secondary SAP HANA servers (First SUSE HAE Cluster) and SAP Application server (Second SUSE HAE
Cluster) through storage ports. Make note of the SCSI identifier of the block device (/dev/disk/by-id/scsi*) of this LUN
(The SCSI identifier should be the same on both primary and secondary HANA servers). It is recommended to add more
than one SBD device in a cluster for redundancy.
Cluster Communication Network
A separate communication network is used by creating VLAN 120 on the cluster servers. Access to VLAN 120 is required
only for SAP application servers and SAP HANA servers installed on SUSE HAE clusters. For details on VLANs created
and the related operating system configuration required in this reference architecture, refer to the section Compute
Network. An NFS network is used as a backup communication network, but if required, a separate VLAN can be created
for this as well. Access to the VLAN must be allowed at the switch level and a VLAN must be created at the operating
system.
22
23
Fibre
Channel Architecture
As shown in Table 10, the solution uses two 16 Gb/sec 2-port Hitachi FIVE-FX Fibre Channel adapters installed on the
PCIe slot of the I/O board modules of the SAP HANA database and application servers. This solution uses the following:


Four 8 Gb/sec Fibre Channel ports on the VSP G400 storage directly attached to the Hitachi Compute Blade 2500
server chassis via the Fibre Channel PCI-Ex adapters for production and QA HANA server blades.
Two 8 Gb/sec Fibre Channel ports on the VSP G400 storage directly attached to the Hitachi Compute Blade 2500
server chassis via the Fibre Channel PCI-Ex adapters for development HANA and all application server blades.
Table 10. Fibre Channel Port Mapping
SAP
Application /
Database
Blade
Number
PCIe Slot
Number
Port
Numb
er
VSP G400 Port
SAP production
application
server
Blade 1
IOBD 01A
0
1A
1
2A
SAP production
application
server
Blade 2
0
3A
1
4A
SAP production
application
server
Blade 3
0
5A
1
6A
SAP production
application
server
Blade 4
0
7A
1
8A
SAP HANA
Development
server
Blade 5
0
1B
1
2B
SAP Nonproduction
application
server
Blade 6
0
3B
1
4B
SAP Nonproduction
application
server
Blade 7
0
5B
1
6B
SAP Nonproduction
application
server
Blade 8
0
7B
1
8B
IOBD 02A
IOBD 03A
IOBD 04A
IOBD 05A
IOBD 06A
IOBD 07A
IOBD 08A
23
24
Table
10. Fibre Channel Port Mapping (Continued)
SAP
Application /
Database
Blade
Number
SAP HANA
Blade 9
production server
PCIe Slot
Number
Port
Numb
er
VSP G400 Port
IOBD 09A
0
1C
1
2C
0
3C
1
4C
0
1D
1
2D
0
3D
1
4D
IOBD 10A
SAP HANA QA
server
Blade 13
IOBD 13A
IOBD 14A
This configuration supports high availability by providing multiple paths from the host within Hitachi Compute Blade 2500
to multiple ports on VSP G400 storage.
24
25
Figure
8 shows the direct connect Fibre Channel connections for SAP HANA production and QA nodes. The HANA
development node and all application servers installed on 520H B3 server blades have one 2-port 16 Gb Hitachi Fibre
Channel adapter and two Fibre Channel connections to storage per serve blade as listed in Table 10 on page 23.
Figure 8
25
26 the port properties for the point-to-point connection between Hitachi Compute Blade 2500 and VSP G400 storage, as
Set
shown in Table 11.
Table 11. Port Properties
Property
Value
Port Security
Disabled
Port Speed
Auto (8 Gb/sec)
Fabric
ON
Connection Type
P-to-P
Management Server
This solution uses one node of a four-node Quanta Cloud Technology QuantaPlex T41S-2U server for the management
server. The management server acts as a central device for managing the SAP HANA platform.
Manage the following from the management server:

Hitachi Compute Blade 2500 chassis

520X B2 server blades

520H B3 server blades

Brocade ICX 6430 — 24 port switch

Brocade VDX 6740 switches

SAP HANA nodes

SAP application servers

Hitachi Virtual Storage Platform G400

NTP configuration and maintenance

Hi-Track Remote Monitoring system from Hitachi Data Systems

Hitachi Command Suite and management of the server blades

SAP HANA Studio
Figure 9 shows the management server network ports using one dual port 1 GbE Base-T Intel i350 mezzanine card.


Slot 01 Port 2 — Connect this port to the customer network. It provides a 1 GbE network to the management server.
Slot 01 Port 1 — Connect this port to the Brocade ICX 6430-24 port switch that provides the
other switches, chassis, and Hitachi NAS Platform nodes.
1 GbE network to all
The management server has the following additional components:

One dual port 10 GbE Intel 82599ES SFP+ OCP mezzanine card

One Emulex 2-port 8 Gb/sec Fibre Channel HBA on the PCIe slot
26
27
Connect
the 10 GbE network ports to the two Brocade switches, VDX 6740-A, Port #20 and VDX 6740-B, Port #20 to
provide management access to the SAP HANA and SAP application nodes from the Quanta 2U4N server using the NFS
network.
Figure 9
Install the following software on the management server:

Hitachi Command Suite

Hitachi Compute Systems Manager software

NTP server service

PuTTY

JRE version jre-7u51-windows-i586 (no 64 bit)

Adobe Flash Player

Remote client software

SAP HANA Studio
Network Architecture
The network architecture of this solution can be divided into the following two parts:

Management Network

Compute Network
27
28
Management
Network
Management Network — Used for management of the chassis, server blades, VSP G400 and Brocade VDX 6740
switches. This provides a 1 Gb/sec network for management of these hardware components through the Quanta Cloud
Technology QuantaPlex T41S-2U management server.

This resides on a 1 GbE Brocade ICX 6430-24 port switch.

The management network does not need to have a VLAN assigned to it.

The Brocade ICX 6430-24 port switch uses the default switch configuration.

The Hitachi Compute Blade 2500 chassis has two management modules to secure fault tolerance. Both
management modules are connected to the Brocade ICX 6430-24 port switch.

VSP G400 management server (MiniMapp server) is connected to the Brocade ICX 6430-24 port switch.

The two Brocade VDX 6740 switches are connected to the Brocade ICX 6430-24 port switch.

The Quanta Cloud Technology QuantaPlex T41S-2U server is connected to the Brocade ICX 6430-24 port switch.
Compute Network
There are 10GBASE-SR 2-port LAN adapters installed on the PCIe slots of the I/O board module of blades of the Hitachi
Compute Blade 2500 chassis as shown in Table 12 on page 29. The solution uses the following:


Two 10 Gb ports on the 10GBASE-SR 2-port LAN adapters installed on each 520H B3 server blade for connectivity
with the two Brocade VDX 6740 switches.
Four 10 Gb ports from two 10GBASE-SR 2-port LAN adapters installed on the primary 520X B2 blades of the
production and QA SAP HANA servers for connectivity with the two Brocade VDX 6740 switches.
This solution requires the following five separate networks.





SAP Production Client Network — Communication between the SAP HANA production system and SAP production
application servers. It is set with an MTU size of 9100 in accordance with Brocade best practices, and isolated using
a VLAN of 100.
SAP Non-Production Client Network — Dedicated to traffic between the SAP HANA DEV and QA database. It is set
with an MTU size of 9100 in accordance with Brocade best practices, and isolated using a VLAN of 110.
SAP NFS Network — Access SAP shared binaries, transport directory, and profiles using the NFS protocol. It is set
with an MTU size of 9100 in accordance with Brocade best practices, and isolated using a VLAN of 150.
SAP HANA System Replication Network — Used for the HANA system replication between the SAP HANA
production primary and secondary node if high availability is configured. It is set with an MTU size of 9100 in
accordance with Brocade best practices, and isolated using a VLAN of 160.
Cluster Communication Network - Used for communication between two SUSE HAE cluster nodes. It is set with an
MTU size of 9100 in accordance with Brocade best practices, and isolated using a VLAN of 120. Access to VLAN 120
is required only for SAP Application servers and SAP HANA servers installed on SUSE HAE clusters
28
29
Make
the following network connections for the client network setup of the SAP HANA node as an uplink network setup:


520X B2 Server Blades: Bond the eth9901 and eth9902 ports as bond0 and eth9911, and eth9922 as bond1 at the
operating system level using active-active network bond mode with options "mode= 802.3ad miimon=100
xmit_hash_policy=layer3+4 updelay=5000 lacp_rate=fast", which acts as the client and replication network for the
SAP HANA node. The replication network is shared with the NFS network. Refer to Table 12 on page 29 for port
numbers.
520H B3 Server Blades:




SAP HANA Development server blade: Bond the eth9901 and eth9902 ports as bond0 at the operating system
level using active-active network bond mode with options "mode= 802.3ad miimon=100
xmit_hash_policy=layer3+4 updelay=5000 lacp_rate=fast", which acts as the client network and NFS network for
SAP HANA database. Refer to Table 12 for port numbers.
SAP Application server blades: Bond the eth9901 and eth9902 ports as bond0 at the operating system level using
active-backup network bond mode with options "mode= active-backup miimon=100 updelay=5000
primary=eth9901", which acts as the production client network, non-production client network and NFS network
(based on the role of the server blade: production application or non-production application). Refer to Table 12 for
port numbers.
Create separate VLANs at the operating system level for the bonds that need access to multiple networks as listed in
Table 12 Assign IP addresses to the VLANs.
Configure the external switch ports with a short LACP timeout value to speed up detecting corrupted connections.
Note - It is strongly recommended network bandwidth is reserved for critical VLANs that are used for production
environments.
The switch placement is as follows:

The switch at rack unit 42 is the Brocade VDX 6740-B switch.

The switch at rack unit 41 is the Brocade VDX 6740-A switch.
The compute network setup uses the ports on the 10GBASE-SR 2-port LAN adapters connected to the Brocade VDX
6740 switches, as listed in Table 12.
Table 12. Network Setup Using 10GBASE-SR 2-Port LAN Adapter
SAP
Application/
Database
SAP
production
application
server
SAP
production
application
server
Blade
Number
PCI-e slot
number, Port
Bond
IOBD 01B, Port 0
Blade 1
IOBD 01B, Port 1
Bond0
IOBD 02B, Port 0
Blade 2
IOBD 02B, Port 1
Bond0
VDX 6740
Switch and
Port
VDX 6740-A,
Port #3
VDX 6740-B,
Port #3
VDX 6740-A,
Port #4
VDX 6740-B,
Port #4
Accessible
VLAN IDs
100,110,120,150
100,110,120,150
29
30
Table
12. Network Setup Using 10GBASE-SR 2-Port LAN Adapter (Continued)
SAP
Application/
Database
SAP
production
application
server
SAP
production
application
server
SAP HANA
development
server
SAP nonproduction
application
server
SAP nonproduction
application
server
SAP nonproduction
application
server
SAP HANA
production
server
Blade
Number
PCI-e slot
number, Port
IOBD 03B, Port 0
Blade 3
IOBD 03B, Port 1
Bond0
IOBD 04B, Port 0
Blade 4
IOBD 04B, Port 1
Bond0
IOBD 05B, Port 0
Blade 5
IOBD 05B, Port 1
Bond0
IOBD 06B, Port 0
Blade 6
IOBD 06B, Port 1
Bond0
IOBD 07B, Port 0
Blade 7
IOBD 07B, Port 1
Bond0
IOBD 08B, Port 0
Blade 8
Blade 9
IOBD 08B, Port 1
Blade 13
Bond0
VDX 6740
Switch and
Port
VDX 6740-A,
Port #5
VDX 6740-B,
Port #5
VDX 6740-A,
Port #6
VDX 6740-B,
Port #6
VDX 6740-A,
Port #7
VDX 6740-B,
Port #7
VDX 6740-A,
Port #8
VDX 6740-B,
Port #8
VDX 6740-A,
Port #9
VDX 6740-B,
Port #9
VDX 6740-A,
Port #10
VDX 6740-B,
Port #10
IOBD 09B, Port 0 Bond0
VDX 6740-A,
Port #1
IOBD 10B, Port 0
VDX 6740-B,
Port #1
IOBD 9B, Port 1
SAP HANA
QA server
Bond
Bond1
VDX 6740-A,
Port #21
IOBD 10B, Port 1
VDX 6740-B,
Port #21
IOBD 13B, Port 0 Bond0
VDX 6740-A,
Port #2
IOBD 14B, Port 0
VDX 6740-B,
Port #2
IOBD 13B, Port 1 Bond1
VDX 6740-A,
Port #22
IOBD 14B, Port 1
VDX 6740-B,
Port #22
Accessible
VLAN IDs
100,110,150
100,110,150
110,150
110,150
110,150
110,150
100
120,160,150
100,110
120,160,150
30
31
Network
Architecture for Virtual Machines
The 520H B3 server blades for production application servers and all of the non-production application server blades are
installed in virtual environment and VMware vSphere is used as virtualization platform in this reference architecture.
Following best practices are used:



Each ESXi host is configured with a Standard vSwitch (VSS) with two physical adapters (vmnic0 and vmnic1) for
redundancy in active-backup configuration. It is possible to create a virtual distributed switch (VDS) and create a
vLAG to achieve an active-active configuration if required. (All ports connected with physical servers are in activeactive configuration).
Use the VMXNET3 network adapter - this is a paravirtualized device that works only if VMware Tools is installed on the
guest operating system. The VMXNET3 adapter is optimized for virtual environments and designed to provide high
performance. It is recommended to install VMware tools on the guest operating system SLES.
Refer to the SAP Solutions on VMware Best Practices Guide for information regarding implementing SAP solutions on
VMware vSphere.
Table 13 shows the network configuration for production and non-production ESXi hosts.
Table 13. Network Configuration for Production and Non-production ESXi Hosts
ESXi Host
Virtual Network
VLAN ID
Client Production 1
100
Physical Adapters
Client Production 2
Production
520H B3
server blades
Client Non-Production 110
Network 1
Client Non-Production
Network 2
NFS Network 1
vmnic0 and vmnic1
(active-backup)
150
NFS Network 2
NonProduction
520H B3
server blades
Client Non-Production 110
Network 1
Client Non-Production
Network 2
NFS Network 1
150
vmnic0 and vmnic1
(active-backup)
NFS Network 2
31
32
Storage
Architecture
Hitachi Virtual Storage Platform G400 is used for this SAP S/4HANA solution. Sizing and configuring of storage including
storage drive box trays (DBS), spare drives, the operating system boot volume (OS), SAP HANA shared volume (/hana/
shared), SAP HANA log volume (/hana/log), SAP HANA data volume (/hana/data), SAP Application Server, the operating
system boot volume (OS), and other SAP Application Server volumes varies for different size solutions by taking into
account requirements including I/O, capacity, etc. The reference architecture distinguishes between two groups of
storage based on purpose:

SAP HANA Database Storage

SAP System Application Server Storage
SAP HANA Database Storage
The following four figures show the disk configuration of the storage subsystem for different memory sizes of production
and QA scale-up SAP HANA Platform solutions.

Figure 10 — Production and QA environment HANA database memory less than or equal to 1TB.

Figure 11 — Production and QA environment HANA database memory equal to 1.5 TB and 2 TB.

Figure 12 — Production and QA environment HANA database memory equal to 3 TB.

Figure 13 —Development environment HANA database
32
33
Figure 10
33
34
Figure 11
34
35
Figure 12
35
36
Figure 13
36
37
Table
14 lists the storage configuration for different memory sizes for production and QA environment SAP HANA
platforms.
Table 14. Storage Configuration for Production and QA Environments
SAP HANA
Database
Memory =<1 TB
OS

1 RAID6 (6D+2P)
/hana/shared

8 × 600 GB HDD

OS: 100 GB
/hana/data


Memory = 3 TB
/hana/shared: 3 TB
1 RAID6
(6D+2P)

8 × 600
GB
HDD

/hana/log
Memory =1.5 or 2 TB

2 RAID6 (6D+2P)

3 RAID6 (6D+2P)

16 × 600 GB HDD

24 × 600 GB HDD

8 × 750 GB LUNs

12 × 750 GB LUNs
4 × 750
GB
LUNs

1 RAID6 (6D+2P)

8 × 600 GB HDD

4 × 150 GB LUNs
Table 15 lists the storage configuration for different memory sizes for development environment SAP HANA platforms.
Table 15. Storage Configuration for Development Environments
SAP HANA
Database
RAID Group and File System
SAP Development
HANA

1 RAID6 (6D+2P)

8 × 600 GB HDD

OS: 100 GB

/hana/shared: 750 GB

/hana/data: 1 TB

/hana/log: 500 GB
37
38 System Application Server Storage
SAP
Figure 14 and Figure 15 show the disk configuration of the storage subsystem for different memory size solutions. A 250
GB disk space is provisioned per operating system with 256 GB RAM for swap space. Refer to SAP Note 1597355 Swap-space recommendation for Linux for more details.

Production Application Storage Volumes

Non-Production Application Storage Volumes
Figure 14
38
39
Figure 15
Table 16 lists the storage configuration for different production SAP applications.
Table 16. Storage Configuration for Production Applications
SAP Application
RAID Group and File System
SAP Production
Application

1 RAID6 (6D+2P)

8 × 600 GB HDD

ESXi: 4 × 50 GB LUNs

OS and Swap: 6 × 300 GB LUNs

Transport Directory: 200 GB LUN

SAPMNT_PRD Directory: 100 GB LUN

/usr/sap Directory LUN: 6 × 100 GB LUNs
39
40
Table
17 lists the storage configuration for different non-production SAP applications.
Table 17. Storage Configuration for Non-Production Applications
SAP Application
RAID Group and File System
SAP NonProduction
Application

1 RAID6 (6D+2P)

8 × 600 GB HDD

ESXi: 3 × 50GB LUNs

OS and Swap: 6 × 300 GB LUNs

SAPMNT_QAS Directory: 100 GB LUN

SAPMNT_DEV Directory: 100 GB LUN

/usr/sap Directory LUN: 6 × 100 GB LUNs
40
41
Storage
Architecture for Virtual Machines
The 520H B3 server blades used for production (only additional application servers) and non-production application
(primary application servers and additional application servers) are virtualized in this reference architecture. VMware
vSphere is used as the virtualization platform. Figure 16 shows the storage architecture of one virtual machine. All virtual
machines use the same storage architecture.
Figure 16
Parity Group, RAID, and LUN Configuration
This reference architecture uses multiple LUNs created in various RAID 6 (6D+2P) groups on Hitachi Virtual Storage
Platform G400 for SAP applications and different sized SAP HANA systems. There are four microprocessor units (MPUs)
used to distribute the workload for all LUNs needed for the solution.
41
42
Table
18 shows the parity group, RAID, and LUN configurations used in this reference architecture.
Table 18. Parity Group, RAID, and LUN Configurations
Parity Group
RAID Level
and Disks
LDEV ID
LDEV
Size
MPU ID
Description
Size Applied
1-1
RAID-6
00:00:00
(6D+2P) on 600
GB 10k RPM
SAS drives
100 GB
MPU10
Operating
system boot
volume for
Production
HANA
All sizes
3.0 TB
MPU11
SAP HANA
shared volume
for Production
HANA
All sizes
RAID-6
00:00:02
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU20
All sizes
RAID-6
00:00:03
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU10
SAP HANA log
volume for
Production
HANA for
Production
HANA
RAID-6
00:00:04
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU11
RAID-6
00:00:05
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU21
RAID-6
00:00:06
(6D+2P) on 600 00:00:07
GB 10k RPM
00:00:08
SAS drives
00:00:09
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
RAID-6
00:00:10
(6D+2P) on 600 00:00:11
GB 10k RPM
00:00:12
SAS drives
00:00:13
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
RAID-6
00:00:14
(6D+2P) on 600 00:00:15
GB 10k RPM
00:00:16
SAS drives
00:00:17
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
00:00:01
1-2
1-3
1-4
1-5
SAP HANA data All sizes
volume for
Production
HANA
SAP HANA data >=1.5 TB
volume for
Production
HANA
SAP HANA data =3 TB
volume for
Production
HANA
42
43
Table 18. Parity Group, RAID, and LUN Configurations (Continued)
Parity Group
RAID Level
and Disks
LDEV ID
LDEV
Size
MPU ID
Description
Size Applied
2-1
RAID-6
00:01:00
(6D+2P) on 600
GB 10k RPM
SAS drives
100 GB
MPU20
Operating
system boot
volume for QA
HANA
All sizes
3.0 TB
MPU21
SAP HANA
shared volume
for QA HANA
All sizes
RAID-6
00:01:02
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU20
SAP HANA log
volume for QA
HANA
All sizes
RAID-6
00:01:03
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU10
RAID-6
00:01:04
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU11
RAID-6
00:01:05
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU21
RAID-6
00:01:06
(6D+2P) on 600 00:01:07
GB 10k RPM
00:01:08
SAS drives
00:01:09
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
RAID-6
00:01:10
(6D+2P) on 600 00:01:11
GB 10k RPM
00:01:12
SAS drives
00:01:13
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
RAID-6
00:01:14
(6D+2P) on 600 00:01:15
GB 10k RPM
00:01:16
SAS drives
00:01:17
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
00:01:01
2-2
2-3
2-4
2-5
SAP HANA data All sizes
volume for QA
HANA
SAP HANA data >=1.5 TB
volume for QA
HANA
SAP HANA data =3 TB
volume for QA
HANA
43
44
Table 18. Parity Group, RAID, and LUN Configurations (Continued)
Parity Group
RAID Level
and Disks
LDEV ID
LDEV
Size
MPU ID
Description
Size Applied
3-1
RAID-6
00:02:00
(6D+2P) on 600
GB 10k RPM
SAS drives
100 GB
MPU10
Operating
system boot
volume for
Production HA
HANA
All sizes
3.0 TB
MPU11
SAP HANA
shared volume
for Production
HA HANA
All sizes
RAID-6
00:02:02
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU20
SAP HANA log
volume for
Production HA
HANA
All sizes
RAID-6
00:02:03
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU10
RAID-6
00:02:04
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU11
RAID-6
00:02:05
(6D+2P) on 600
GB 10k RPM
SAS drives
150 GB
MPU21
RAID-6
00:02:06
(6D+2P) on 600 00:02:07
GB 10k RPM
00:02:08
SAS drives
00:02:09
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
RAID-6
00:02:10
(6D+2P) on 600 00:02:11
GB 10k RPM
00:02:12
SAS drives
00:02:13
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
RAID-6
00:02:14
(6D+2P) on 600 00:02:15
GB 10k RPM
00:02:16
SAS drives
00:02:17
750 GB
MPU21
750 GB
MPU11
750 GB
MPU20
750 GB
MPU10
00:02:01
3-2
3-3
3-4
3-5
SAP HANA data All sizes
volume for
Production HA
HANA
SAP HANA data >=1.5 TB
volume for
Production HA
HANA
SAP HANA data =3 TB
volume for
Production HA
HANA
44
45
Table
18. Parity Group, RAID, and LUN Configurations (Continued)
Parity Group
RAID Level
and Disks
LDEV ID
LDEV
Size
MPU ID
Description
Size Applied
4-1
RAID-6
00:03:00
(6D+2P) on 600
GB 10k RPM
00:03:01
SAS drives
50 GB
MPU21
ESXi LUN for
PRD host
All Sizes
50 GB
MPU11
ESXi LUN for
PRD host
00:03:02
50 GB
MPU20
ESXi LUN for
PRD host
00:03:03
50 GB
MPU10
ESXi LUN for
PRD host
00:03:04
300 GB
MPU21
SAP PRD
Application Boot
and Swap
Space LUN
00:03:05
300 GB
MPU11
SAP PRD
Application Boot
and Swap
Space LUN
00:03:06
300 GB
MPU20
SAP PRD
Application Boot
and Swap
Space LUN
00:03:07
300 GB
MPU10
SAP PRD
Application Boot
and Swap
Space LUN
00:03:08
300 GB
MPU21
SAP PRD
Application Boot
and Swap
Space LUN
00:03:09
300 GB
MPU11
SAP PRD
Application Boot
and Swap
Space LUN
00:03:10
200 GB
MPU20
Transport
Directory LUN
00:03:11
100 GB
MPU10
SAPMNT_PRD
LUN
00:03:12
100 GB
MPU21
/usr/sap PRD
LUN
00:03:13
100 GB
MPU11
/usr/sap PRD
LUN
00:03:14
100 GB
MPU20
/usr/sap PRD
LUN
00:03:15
100 GB
MPU10
/usr/sap PRD
LUN
45
46
Table
18. Parity Group, RAID, and LUN Configurations (Continued)
Parity Group
RAID Level
and Disks
LDEV ID
LDEV
Size
MPU ID
Description
Size Applied
4-2
RAID-6
00:04:00
(6D+2P) on 600
GB 10k RPM
00:04:01
SAS drives
50 GB
MPU21
ESXi LUN for
Non-PRD host
All Sizes
50 GB
MPU11
ESXi LUN for
Non-PRD host
00:04:02
50 GB
MPU20
ESXi LUN for
Non-PRD host
00:04:03
300 GB
MPU10
SAP Non-PRD
Application Boot
and Swap
Space LUN
00:04:04
300 GB
MPU21
SAP Non-PRD
Application Boot
and Swap
Space LUN
00:04:05
300 GB
MPU11
SAP Non-PRD
Application Boot
and Swap
Space LUN
00:04:06
300 GB
MPU20
SAP Non-PRD
Application Boot
and Swap
Space LUN
00:04:07
300 GB
MPU10
SAP Non-PRD
Application Boot
and Swap
Space LUN
00:04:08
300 GB
MPU21
SAP Non-PRD
Application Boot
and Swap
Space LUN
00:04:09
100 GB
MPU11
SAPMNT_QA
LUN
00:04:10
100 GB
MPU20
SAPMNT_DEV
LUN
00:04:11
75 GB
MPU10
/usr/sap NonPRD LUN
00:04:12
75 GB
MPU21
/usr/sap NonPRD LUN
00:04:13
75 GB
MPU11
/usr/sap NonPRD LUN
00:04:14
75 GB
MPU20
/usr/sap NonPRD LUN
00:04:15
75 GB
MPU10
/usr/sap NonPRD LUN
46
47
Table
18. Parity Group, RAID, and LUN Configurations (Continued)
Parity Group
RAID Level
and Disks
5-1
LDEV ID
LDEV
Size
MPU ID
Description
Size Applied
RAID-6
00:05:00
(6D+2P) on 600
GB 10k RPM
SAS drives
00:05:01
100 GB
MPU21
SAP HANA OS
volume for
Development
All Sizes
750 GB
MPU11
SAP HANA
shared volume
for Development
00:05:02
500 GB
MPU20
SAP HANA Log
volume for
Development
00:05:03
1 TB
MPU10
SAP HANA Data
volume for
Development
SAP HANA Configuration
This section explains the SAP HANA configuration.
Operating System Configuration
The SAP HANA scale-up configuration requires an OS volume. It uses one 100 GB LUN on Hitachi Virtual Storage
Platform G400 for the operating system boot volume for the SAP HANA Platform.
The four ports of the 16 Gb Hitachi FIVE-FX Fibre Channel 2-port adapters have the 100 GB operating system OS LUN
configured as the primary boot device. The operating system LUN holds partitions for SUSE Linux for SAP Applications
version 11 SP3, /usr/sap/ and the Linux swap space.
Activate Device-Mapper Multipath
This reference architecture uses Device-Mapper Multipath, a component of the native Linux operating system.
Using Device-Mapper Multipath allows the configuration of multiple I/O paths between the server blades, and Hitachi
Virtual Storage Platform G400. Multipathing aggregates all physical I/O paths into a single logical path. The LUNs are
always available, unless all paths fail.
Device-Mapper Multipath is used for the following I/O paths:

SAP HANA server OS LUN.

SAP HANA data volume LUN.

SAP HANA log volume LUN.

SAP HANA shared volume LUN.
47
48 HANA Data Volume Configuration
SAP
The Logical Volume Manager configures the SAP HANA persistent storage volumes. Table 14 on page 37 and Table 15 on
page 37 show the size of the SAP HANA data volume LUNs for all three environments defined by the solution.
Logical Volume Manager creates a single striped volume on which the XFS file system is created to store the SAP HANA
data volume. The striped volume acts as the persistent layer for the SAP HANA server.
SAP HANA Log Volumes Configuration
This reference architecture uses Logical Volume Manager to configure the SAP HANA log volumes.
Table 14 on page 37 and Table 15 on page 37 list the size of the SAP HANA data volume LUNs for all three environments
defined by the solution. With the four 150 GB SAP HANA log volume LUNs for the production, QA and HA SAP HANA
nodes, Logical Volume Manager creates a single four-way striped volume on which the XFS file system is created to store
the SAP HANA log volumes.
SAP HANA Shared Volume
This reference architecture uses one or two LUNs for the SAP HANA shared volume for different sized solutions. Logical
Volume Manager creates a single striped volume on which the XFS file system is created to store the SAP HANA binaries,
configuration, and trace files.
Table 14 on page 37 and Table 15 on page 37 list the number and size of SAP HANA shared volume LUNs.
SAP HANA Software Installation
After configuring the file system for the SAP HANA data volume and log volume, the latest SAP HANA SPS stack is
installed on the SAP HANA server.
The following SAP HANA software components are installed on the HANA server node:

SAP HANA database.

SAP HANA client.

SAP Host agent.
48
49
S/4HANA
Configuration
Perform the following steps to install S4/HANA:


Perform the system sizing. Refer to the Sizing, Scalibility, and Intergration with External IT Components section for
more information.
Design the landscape based on sizing requirements. Refer to the Compute Hardware Layout and Software Layout
sections for more information.

Download SLES for SAP Applications 11 SP3 media from the Novell website.

Download the SAP installation documentation from https://service.sap.com/instguides.

Download the installation media from https://support.sap.com/swdc.

Install ESXi and create virtual machines on the 520H B3 server blade selected for virtualization.

Install SLES11 SP3 on the 520H B3 server blades or virtual machines.




Make sure that settings recommended in SAP note 1275776 - Linux: Preparing SLES for SAP environments are
implemented.
Create file systems as described in the SAP Installation Guide. File systems used in this reference architecture are
shown in Table 19.
Create operating system users and groups as described in the SAP Installation Guide. In a distributed installation, the
same user IDs and group IDs must be used.
Perform the installation and post-installation steps as described in the SAP installation guide.
SAP Application File system
Table 19 shows the file systems used in this reference architecture for SAP applications.
Table 19. SAP Application File System
File System
Description
Type
Location
/usr/sap/trans
Transport directory
NFS
Mounted on every
application OS. The same
file system for all three
environments.
/sapmnt/<SID>
sapmnt
NFS
Mounted on application and
database OS. A separate file
system for each environment
(production, QA, and
development).
/usr/sap/<SID>/ASCS<Instance Number>
Instance directory
for ASCS (only for
HA)
Local
Mounted on both production
cluster nodes. Two different
file systems for two nodes
with same name.
/usr/sap/<SID>/ERS<Instance Number>
Instance directory
for ERS (only for
HA)
Local
Mounted on both production
cluster nodes. Two different
file systems for two nodes
with the same name.
49
50
Table 19. SAP Application File System (Continued)
File System
Description
Type
Location
/usr/sap/<SID>/DVEBMGS02
Instance directory NFS
for production
primary application
server
Mounted on both production
cluster nodes.
/usr/sap/<SID>/D03
Instance directory
for production
additional
application server
NFS
Mounted on both production
cluster nodes.
/usr/sap/
Instance directory
for production or
non-production
primary or
additional
application server
Local
Mounted on production or
non-production primary or
additional application server.
50
51
Engineering Validation
Validation of this reference architecture paper has been performed by installing the production environment in a highly
available configuration as shown in Figure 3 on page 11. The validation testing includes KPI performance test cases using
the SAP benchmarking tool kit executed by Hitachi Data Systems.
Test Methodology
Because of intellectual property limitations, this paper does not include the test methodology.
Test Results
Because of intellectual property limitations, this paper does not include the test results or an analysis of those results.
51
52
Appendix
SUSE Documentation
SAP HANA Technical Operations Manual
SAP HANA System Replication on SLES for SAP Applications
Highly Available NFS Storage with DRBD and Pacemaker
Best practices for mission-critical SAP applications
SAP NetWeaver on SUSE Linux Enterprise Server with High Availability
SAP Documentation and SAP Notes
SAP Note 1597355 - Swap-space recommendation for Linux
SAP note 1681092 - Multiple SAP HANA DBMSs (SIDs) on one SAP HANA system
SAP note 1275776 - Linux: Preparing SLES for SAP environments are implemented
SAP note 1310037 SUSE LINUX Enterprise Server 11: Installation notes
SAP S/4HANA Frequently Asked Questions
High Availability - Frequently Asked Questions
HOW TO SET UP SAPHanaSR IN THE COST OPTIMIZED SAP HANA SR SCENARIO
SAP HANA administration guide
SAP Solutions on VMware Best Practices Guide
52
1
Corporate Headquarters
2845 Lafayette Street
Santa Clara, CA 96050-2639 USA
www.HDS.com
community.HDS.com
Regional Contact Information
Americas: +1 408 970 1000 or [email protected]
Europe, Middle East and Africa: +44 (0) 1753 618000 or [email protected]
Asia Pacific: +852 3189 7900 or [email protected]
HITACHI is a trademark or registered trademark of Hitachi, Ltd., Other notices if required. All other trademarks, service marks and company names are properties of their respective
owners.
Notice: This document is for informational purposes only, and does not set forth any warranty, expressed or implied, concerning any equipment or service offered or to be offered by
Hitachi Data Systems Corporation.
AS-442-00 December 2015.