Technical Report
Oracle Performance Using NetApp Private
Storage for SoftLayer
Saad Jafri, Mark Beaupre, John Fullbright, NetApp
January 2015 | TR-4373
A Performance Comparison
This technical report compares the performance of the single-instance Oracle Database 11g
R2 using random I/O workloads running on premises and on NetApp Private Storage using
SoftLayer Bare Metal Compute Services. The report demonstrates comparable performance.
TABLE OF CONTENTS
1
2
Introduction and Executive Summary ................................................................................................ 3
1.1
Introduction .....................................................................................................................................................3
1.2
Target Audience..............................................................................................................................................3
Test Configurations .............................................................................................................................. 3
2.1
Oracle Direct NFS ...........................................................................................................................................7
2.2
Storage and Database Layout ........................................................................................................................7
3
Performance Test Results ................................................................................................................... 8
4
Conclusion .......................................................................................................................................... 11
5
References .......................................................................................................................................... 11
LIST OF TABLES
Table 1) Storage layout for all test configurations. .........................................................................................................8
LIST OF FIGURES
Figure 1) Simulated on premises test configuration with local server connected to NetApp storage. .............................5
Figure 2) NPS connected to the SoftLayer cloud. ..........................................................................................................6
Figure 3) Read performance comparison between NPS/SoftLayer and on-premises configurations. ..........................10
Figure 4) Read/write performance comparison between NPS/SoftLayer and on-premises configurations. .................11
2
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
1 Introduction and Executive Summary
1.1
Introduction
This technical report contains performance information for running single-instance Oracle Database 11g
Release 2 (Oracle 11gR2) on NetApp Private Storage (NPS) using SoftLayer Bare Metal Compute
Services and compares the performance of running the same workload on premises. Customers can use
this information to make informed decisions about deploying Oracle databases on NPS using SoftLayer
Bare Metal Compute Services.
The test configuration environments described in this report includes the following elements:

Oracle 11g R2 on Red Hat Enterprise Linux 6.5 64-bit

NPS systems running clustered Data ONTAP 8.2.1 at an Equinix colocation data center

SoftLayer Bare Metal Compute Server to drive the database workload to NetApp Private Storage at
Equinix colocation data center

Local compute servers connected to the same NPS controllers at Equinix colocation to drive the
database workload to simulate an on-premises configuration.
®
®
The protocol used in this architecture is NFS, using a 10Gb Ethernet connection. Specificially the NFS
client is the Oracle Direct NFS client. This is provided by the Oracle 11gR2 RDBMS software. It runs as
part of the Oracle database and is optimized specially for database workloads. DNFS can be used only to
access the Oracle database files.
This report demonstrates that NPS systems using SoftLayer Bare Metal Compute provide performance
comparable to that attained running the same workload on local servers connected to NetApp storage on
premises. For customers who are considering using the SoftLayer cloud to take advantage of elasticity
and the economics of SoftLayer compute, NPS offers excellent performance and enterprise-class storage
features.
1.2
Target Audience
This report is for NetApp customers and partners who are investigating whether to deploy their Oracle
databases using NPS and SoftLayer compute services.
2 Test Configurations
These tests compare performance when running Oracle RDBMS workloads on local compute servers
connected via an Ethernet network to NetApp storage to the performance of an architecture running the
same Oracle RDBMS workload on similarly capable SoftLayer Bare Metal Server instances connected to
NPS using a dedicated Ethernet network.
®
For the baseline tests, in order to simulate an environment typically found on premise, two Cisco UCS
servers were connected using 10GbE to two FAS8060 controllers in a cluster. This setup was part of NPS
solution architecture in an Equinix colocation data center. Each of the two servers ran a single-instance
Oracle database, the binaries, data files, and log files of which were hosted on each of the two FAS8060
storage controllers, respectively.
All test configurations used the Oracle DNFS protocol over 10GbE network connections between
database servers and NetApp storage systems. For each database used in our configuration, we set up a
10GbE path to be shared by all data and log volumes on each of the databases to be accessed using
DNFS. In addition, we stored Oracle binaries for each database on NetApp storage volumes that were
accessed using the same 10GbE path by the OS NFSv3 client so that same database could be used in
all test configurations for comparison without the need to rebuild the database.
3
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
After the baseline tests were completed, the databases on the two Cisco UCS servers were shut down,
connected to two SoftLayer Bare Metal Server instances with capabilities similar to the NPS FAS8060
controllers, and then brought back online. This allowed the use of the same database to run the
workloads for as close as possible a comparison.
The objectives of this testing were to drive the workloads at progressively higher intensities to determine
the IOPS and average database read latencies that each system could deliver, and to compare the
performance of the Oracle RDBMS in a simulated on-premises configuration with the SoftLayer Bare
Metal Compute/NPS configuration. Depending on the configuration tested, the I/O patterns used during
our tests were as follows:

100% reads, 100% random I/O using 8K block size

90% reads, 10% writes, 100% random I/O using an 8K block size
For all tests, we used the Silly Little Oracle Benchmark (SLOB2) workload generator to simulate the I/O
patterns that are likely to be encountered in an actual Oracle production environment. SLOB2 uses the
actual database engine to generate real database I/Os for the workload. SLOB2 drives different levels of
simulated users, each generating the specific I/O patterns described previously.
Multiple test iterations using a progressively higher workload were executed. This allowed us to observe
the impact on performance at low, medium, and high loads. On the results graphs shown on the X axis,
the starting load point is referred to as X workload representing a simulated number of users. The
remaining load points are linearly represented relative to X as we incrementally increased the load on the
database. At each load point, we recorded the IOPS and average read latency reported by the Oracle
database using automatic workload repository (AWR) reports.
Performance tests evaluated the following configurations:

Baseline (simulated on-premises architecture). The baseline architecture consists of two singleinstance Oracle 11gR2 databases running on two physical servers (one instance per server)
connected using 10GbE to a two-node NetApp FAS8060 cluster. The size of each database instance
was 6TB. The two FAS8060 storage controllers used in this scenario were part of a NPS system
residing in an Equinix colocation data center to simulate an on-premises configuration. The baseline
environment consists of the following hardware and software:


Two Cisco UCS C220 servers, each equipped with:
®
®
o
Intel Xeon CPU E5-2650 at 2.6 GHz, 2 sockets, 8 cores per socket
o
64GB physical memory
o
Single Intel 10GbE port
o
Red Hat Enterprise Linux Server release 6.5 64-bit
o
Oracle Database 11g Enterprise Edition Release 11.2.0.4 64-bit with DNFS enabled
Two-node FAS8060 cluster equipped with:
o
NetApp clustered Data ONTAP 8.2.1
o
Two FAS8060 storage controllers equipped with:

Eight DS2246 disk shelves

24 900GB SAS 10K rpm disks per disk shelf

Single 10GbE port
Figure 1 provides a network diagram of the FAS8060 test configuration built to simulate an on-premises
environment.
4
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
Figure 1) Simulated on premises test configuration with local server connected to NetApp storage.

NPS connected to SoftLayer cloud. The cloud environment consists of single-instance Oracle
11gR2 databases running on two SoftLayer Bare Metal Servers connected using 10GbE network to
NPS systems using FAS8060 storage controllers running clustered Data ONTAP 8.2.1. The FAS8060
controllers used in these tests were also used in the baseline tests and were part of the NPS system
residing in the Equinix colocation data center. The two SoftLayer Bare Metal Compute instances were
the same in the number of CPUs, CPU cores, CPU speed, memory, and 10GbE network bandwidth
as the two UCS C220 servers used in the baseline configuration. The difference was that we used
two SoftLayer Bare Metal Servers for compute connected to NPS instead of local server connected to
the same storage. The NPS for SoftLayer environment includes the following hardware and software:


Two SoftLayer Bare Metal Compute instances, each equipped with:
o
Intel Xeon CPU E5-2650 V2 at 2.6 GHz (Ivy Bridge), 2 sockets, 8 cores per socket
o
64GB physical memory
o
Single Intel 10GbE port
o
Dual 450GB SAS 15K rpm local disk
o
Red Hat Enterprise Linux Server release 6.5 64-bit
o
Oracle Database 11g Enterprise Edition Release 11.2.0.4 64-bit with DNFS enabled
Two-node FAS8060 cluster equipped with:
o
NetApp clustered Data ONTAP 8.2.1
o
Two FAS8060 storage controllers equipped with:

Eight DS2246 disk shelves

24 900GB SAS 10K rpm disks per disk shelf

Single 10GbE port
Figure 2 provides a network diagram of NetApp Private Storage connected to the SoftLayer Cloud. In the
network diagram, the top black box (delineated by black dotted lines) shows the SoftLayer cloud where
the Bare Metal Compute instances are hosted. The bottom black box (delineated by black dotted lines)
shows the Equinix colocation data center where the NPS controllers reside. The same ones where used
in the baseline configuration. The SoftLayer cloud and the NPS were connected using a dedicated highspeed Ethernet network through SoftLayer DirectLink. The effective network bandwidth between
5
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
SoftLayer Bare Metal Compute instances and NPS controllers is 10GbE due to 10GbE network interfaces
on the SoftLayer Bare Metal Compute instances and NPS storage controllers.
Figure 2) NPS connected to the SoftLayer cloud.
6
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
2.1
Oracle Direct NFS
With Oracle Direct NFS client, NFS functionality is actually provided by the Oracle RDBMS stack. Unlike
native OS NFS clients, DNFS is fully optimized for Oracle data access I/O patterns and provides
significantly higher performance than traditional NFS clients.
Key features and benefits of using a DNFS client include the following:

Standard NFS client implementation for all hardware and operating system platforms

Concurrent direct I/O

Bypasses OS-level caches, thereby reducing host memory utilization

Reduces CPU overhead

Offers asynchronous I/O, which allows overlap of processing and I/O operations, resulting in
improved performance.

Support for parallel network paths

Automatic load balancing across network paths, which helps avoid network-related performance
bottlenecks

Automatic failover between network paths, providing network fault tolerance and high availability
In summary, Oracle DNFS provides better performance than native OS NFS clients and provides I/O path
redundancy.
NetApp FAS systems fully support the features of DNFS. In addition, they provide I/O redundancy and
high-performance I/O as follows:

NFS shared volumes are created in storage aggregates, which are made up of disk RAID groups.
®
The RAID groups provide disk redundancy, protecting data against disk failures. With RAID DP , data
is actually protected against double-disk failures.

The NFS shared volumes, along with the data files they contain, are spread across all the disk drives
in the aggregate.

I/O performance benefits from the total bandwidth of disks in the aggregate.

I/O is load balanced across all disks in the aggregate.
Implementation of Oracle DNFS with NetApp FAS systems provided optimized I/O performance, network,
and disk redundancy, and load balancing across network paths and disk drives. Best practices for use of
NetApp storage with Oracle DNFS are the same as with native OS NFS clients and can be found in TR3633: Best Practices for Oracle Databases on NetApp Storage.
2.2
Storage and Database Layout
We started by creating a single aggregate on each FAS8060 controller (in addition to the existing root
volumes). Each aggregate was created from 190 SAS disks with a RAID group size of 19, using NetApp
RAID DP. The required volumes for each database instance were created in those two aggregates. The
following is a summary of the storage configuration:


7
Database 1: An aggregate on controller 1 made up of 190 900GB 10K rpm SAS disks

Binaries volume, 100GB

Data volume, 10TB

Log volume, 500GB

Temp tablespace volume, 500GB
Database 2: An aggregate on controller 2 made up of 190 900GB 10K rpm SAS disks

Binaries volume, 100GB

Data volume, 10TB
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373

Log volume, 500GB

Temp tablespace volume, 500GB
In the tested configuration, the data volume contained a single tablespace for tables and data. The
tablespace was a bigfile tablespace using a single 7TB data file for physical storage. The temp
tablespace volume provided storage for the database’s temporary tablespace. The log volumes contained
three redo log group members, with each member having a size of 40GB.
Table 1 summarizes the storage configuration.
Table 1) Storage layout for all test configurations.
Database Name
Aggregate
Volume
Volume Capacity
db1
aggr1_controller1
binaries
100GB
datavol
10TB
logvol
500GB
tempvol
500GB
binaries
100GB
datavol
10TB
logvol
500GB
tempvol
500GB
db2
aggr1_controller2
3 Performance Test Results
This section provides the performance tests results of the baseline and NPS/SoftLayer Bare Metal
Compute configurations and compares the two. When evaluating performance of storage arrays, it is
critical to establish the definition of terms such as I/O per second (IOPS). For example, the values for the
databases are often taken from the physical reads statistics, located in the Load Profiles statistics of an
Oracle AWR report. This can be highly misleading because at the storage layer, 128 physical reads could
be performed as 128 individual random I/O operations, or they could be read as a 1MB unit as part of a
longer sequential read.
In our tests, the read IOPS were calculated based on the physical reads statistics, while making sure that
essentially all I/O was performed as random db file sequential read I/O or db file parallel read I/O. This
makes sure that the IOPS level, in terms of Oracle block operations, is an accurate reflection of the IOPS
level that occurred at the storage layer.
Write performance was not a focus of these tests. Although low-latency redo logging is critical to a
production database, virtually all arrays on the market, from any vendor, already commit inbound writes to
some type of solid-state cache. The use of high-performance drives for persistent storage has little impact
on this activity because the write is complete from the database viewpoint as soon as it is stored in the
array cache itself. Likewise, database block write operations are generally back group operations and are
not latency sensitive.
For each set of tests, the workloads were incrementally increased in intensity and executed across both
Oracle databases to observe the impact on performance at low, medium, and high loads. We then
measured the total IOPS delivered along with the associated average read latency at each workload
level. The specific workloads used were as follows:

100% reads, 100% random I/O using 8K block size
8
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373

90% reads, 10% writes, 100% random I/O using an 8K block size
We used the SLOB2 workload generator to simulate the previous I/O patterns. SLOB2 uses the actual
database engine to generate real database I/Os for the workload. SLOB2 drives different levels of
simulated users, each generating the specific I/O patterns described previously.
Figure 3 and Figure 4 show the graphs of total IOPS and average read latency observed by the database
for the simulated on premises configuration compared to the configuration where NetApp storage is
connected to the SoftLayer Bare Metal Server instances. On the results graphs, on the X axis, a starting
load point is referred to as X workload, representing a simulated number of users. The remaining load
points are linearly represented relative to X as we incrementally increased the load on the database. At
each load point, we recorded the IOPS and average read latency reported by the Oracle database using
AWR reports.
For tested workloads, we observed the following items of interest:

As we incrementally increased the workload, IOPS generated in NPS/SoftLayer are lower compared
to IOPS generated in the simulated on premises configuration; however, they track reasonably
closely in both configurations and are comparable.

The average read latencies for NPS/SoftLayer at each load point are slightly higher compared to the
simulated on premises configuration; however, they are within 1ms to 2ms of the simulated on
premises configuration. This aligns with expectations, because read latencies are slightly higher in
NPS/SoftLayer configurations compared to on premises configurations because storage resides
closer to the SoftLayer cloud compared to on premises configurations, in which both storage and
compute reside locally.

When we added writes to the I/O mix, the overall read IOPS drop at each load point compared to the
100% random reads case. This is expected because there are now write I/Os in the mix consuming
resources. However, introducing writes in the I/O mix did not drastically change the observations
mentioned in the previous two bullet items, which still apply in this case as well.
9
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
Figure 3) Read performance comparison between NPS/SoftLayer and on-premises configurations.
10
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
Figure 4) Read/write performance comparison between NPS/SoftLayer and on-premises configurations.
4 Conclusion
NetApp has a long history of providing high-performance, feature-rich storage systems. NetApp Private
Storage extends this legacy to the SoftLayer cloud. With the advent of NetApp Private Storage, NetApp
continues to develop leading-edge storage solutions that provide the agility and mobility NetApp current
and future customer’s desire. Our testing shows that NetApp Private Storage, when combined with
SoftLayer, delivers performance comparable performance to that of on-premises environments when
running comparable workloads. NetApp Private Storage allows customers to benefit from the elasticity
and economics of the cloud combined with the control, availability, and performance of NetApp storage.
5 References

TR-4326: NetApp Private Storage for SoftLayer Solution Architecture and Deployment Guide
http://www.netapp.com/us/media/tr-4326.pdf

TR-3633: Best Practices for Oracle Databases on NetApp Storage
http://www.netapp.com/us/media/tr-3633.pdf
11
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373
Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact
product and feature versions described in this document are supported for your specific environment.
The NetApp IMT defines the product components and versions that can be used to construct
configurations that are supported by NetApp. Specific results depend on each customer's installation in
accordance with published specifications.
Copyright Information
Copyright © 1994–2015 NetApp, Inc. All rights reserved. Printed in the U.S. No part of this document
covered by copyright may be reproduced in any form or by any means—graphic, electronic, or
mechanical, including photocopying, recording, taping, or storage in an electronic retrieval system—
without prior written permission of the copyright owner.
Software derived from copyrighted NetApp material is subject to the following license and disclaimer:
THIS SOFTWARE IS PROVIDED BY NETAPP "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, WHICH ARE HEREBY
DISCLAIMED. IN NO EVENT SHALL NETAPP BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.
NetApp reserves the right to change any products described herein at any time, and without notice.
NetApp assumes no responsibility or liability arising from the use of products described herein, except as
expressly agreed to in writing by NetApp. The use or purchase of this product does not convey a license
under any patent rights, trademark rights, or any other intellectual property rights of NetApp.
The product described in this manual may be protected by one or more U.S. patents, foreign patents, or
pending applications.
RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the government is subject to
restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software
clause at DFARS 252.277-7103 (October 1988) and FAR 52-227-19 (June 1987).
Trademark Information
NetApp, the NetApp logo, Go Further, Faster, ASUP, AutoSupport, Campaign Express, Cloud ONTAP,
Customer Fitness, Data ONTAP, DataMotion, Fitness, Flash Accel, Flash Cache, Flash Pool, FlashRay,
FlexArray, FlexCache, FlexClone, FlexPod, FlexScale, FlexShare, FlexVol, FPolicy, GetSuccessful,
LockVault, Manage ONTAP, Mars, MetroCluster, MultiStore, NetApp Insight, OnCommand, ONTAP,
ONTAPI, RAID DP, SANtricity, SecureShare, Simplicity, Simulate ONTAP, Snap Creator, SnapCopy,
SnapDrive, SnapIntegrator, SnapLock, SnapManager, SnapMirror, SnapMover, SnapProtect,
SnapRestore, Snapshot, SnapValidator, SnapVault, StorageGRID, Tech OnTap, Unbound Cloud, and
WAFL are trademarks or registered trademarks of NetApp, Inc., in the United States and/or other
countries. A current list of NetApp trademarks is available on the Web at
http://www.netapp.com/us/legal/netapptmlist.aspx.
Cisco and the Cisco logo are trademarks of Cisco in the U.S. and other countries. All other brands or
products are trademarks or registered trademarks of their respective holders and should be treated as
such. TR-4373-0115
12
Oracle Performance Using NetApp Private Storage for SoftLayer
TR-4373