Business white paper
Peak traffic can happen
anytime. Be prepared.
Tips and tricks for using mobile app and web cloud performance
load testing to improve the user experience
Business white paper
Table of contents
3
Are you ready for peak traffic—whenever it hits?
4
4
4
5
6
7
7
7
Seven steps to peak performance
1. Design for scale
2. Test early, test often, test always
3. Build a mobile testing strategy
4. Focus on the end-user experience
5. Plan for diverse network conditions
6. Consider performance testing from the cloud
7. Test in production
8 Have a happy holiday
Business white paper
Page 3
Section 1: The challenges
Are you ready for peak traffic—whenever it hits?
For retailers, these days it’s all about online shopping. Consider these findings:
•U.S. online orders totaled more than $3 billion USD on Cyber Monday 2015, up
16 percent from the previous year.1
•U.S. mobile sales also set a sales volume record on Cyber Monday 2015, with nearly
$800 million USD in online spending coming from smartphones or tablets.2
•In China, meanwhile, sales on the nation’s biggest online shopping day, Single’s Day,
topped a staggering $14 billion USD in 2015.3
’Tis the season for performance
problems
Seasonal spikes in demand for online
services aren’t just a concern for retailers.
They happen to the best of us. University
enrollment windows open and traffic
booms as students vie for seats in
classes. The health insurance industry
gets hit hard during open enrollment
periods. Auto sales volumes spike in the
spring and fall. Every industry has its
seasons—and its associated threats to
web and app performance.
Clearly, holiday shopping is big business, with many companies pulling in as much as half of
their annual revenue in the November-through-January shopping surge. But this doesn’t mean
that the rest of the year they get to sit back and dream about record profits—big holiday sales
numbers are the result of meticulous planning, preparation, and engineering for success.
In different terms, this same story plays out across virtually all industries. From insurance to
entertainment, from government to higher education, organizations have to prepare their
websites and apps for massive volumes of concurrent users. Sometimes they can see the
deluge coming in the form of predictable seasonal spikes in demand, but sometimes they
can’t—for example, when an online promotion goes viral.
In this new era, the shoppers and end users will come. The question is, will your web and mobile
apps be ready? The stakes are high, and delays and crashes can be catastrophic. We all know
that consumers have many choices about where to shop, and that your competitors are always
just a click or two away.
So how do you positon your organization for success in today’s competitive online world?
Follow these seven mobile app and web performance testing steps to achieve peak
performance for your web and mobile apps.
1, 2
3
“ Cyber Monday hits $3 billion USD sales
record,” CNN Money, December 1, 2015.
“ China’s ‘Cyber Monday’ smashes records—again,”
CNN Monday, November 11, 2015.
Business white paper
Page 4
Section 2: Tips and tricks
Seven steps to peak performance
1. Design for scale
The first step forward is to design for scale, a process enabled by performance engineering.
Simple performance testing is not enough—you need to do performance engineering. Whether
you are working in an agile testing or waterfall development environment, building performance
into the fabric of your app is essential.
A key here is to analyze your data to optimize performance. Use production analytics to
understand data and trends from past peak periods, such as holidays, and the past 12 months.
Things change, but insight into consumer behaviors, common browsing platforms, devices,
locations, and network condition will help you get an idea about what to test for best results.
Here are a few best practices for the design stage:
•Define performance budget in every page or service design.
•Create clear non-functional requirements. Learn from your usage patterns in production over time.
•Include performance engineers in technical planning and architectural discussions.
•Pick the right technology and architecture from the start to avoid performance bottlenecks.
•Plan for peak traffic like campaigns, sales, and promotions—anything that may increase traffic
and/or change consumer behavior.
•Include performance testing targets in your “definition of done” for the software release.
2. Test early, test often, test always
To ensure top performance of web and mobile apps, there are three essential steps in
the performance testing process: Test early, test often, and test always. The goal is to
test throughout the software development cycle so you can identify issues while they are
still inexpensive to fix.
One of the keys here is to use service virtualization software to emulate third-party
components that are not yet available. Service virtualization allows your development and load
testing teams to simulate a service’s behavior in a production environment, regardless of access
to production systems.
Other best practices are to test your app in production and to plan testing around business
cycles. For example, many companies either shut down or don’t allow developers to introduce
app changes over the holidays.
Business white paper
Page 5
3. Build a mobile testing strategy
Mobile commerce is growing rapidly—and that’s both an opportunity and a problem for many
organizations. While it is great to have customers who can contact you from virtually anywhere,
your mobile apps have to deliver the performance they expect.
Consider these findings from a mobile app user study conducted by Dimensional Research.4
Mobile app users are impatient:
49% expect apps to respond in
2 seconds or less.
61% expect apps to start in
4 seconds or less.
Users are intolerant of issues and are quick to abandon mobile apps:
53% uninstall or remove a
mobile app with severe issues,
like crashes, freezes, or errors.
80% will attempt to use a problematic
app just three times or fewer.
4
F ailing to Meet Mobile App User
Expectations: A Mobile App User Survey.
Dimensional Research, February 2015.
Business white paper
Page 6
When they encounter problems, users blame the mobile app and the company who made it:
37% say that mobile app
crashes or errors make
them think less of a
company’s brand.
55% hold the app responsible
for performance issues.
Findings like these underscore the need for comprehensive mobile testing strategies that
validate the performance and stability of apps—in both the lab and the real world. But don’t
test just the front-end app. Be sure to also evaluate the impact of mobile traffic on back-end
servers and enterprise systems.
In addition, keep the broader shopping experience in mind when you’re building your
mobile performance testing strategy. Search can originate on a mobile device—a phone or a
tablet—yet the purchase might be completed online via a different device, such as a laptop
or desktop system. For the optimum experience, customers should not have to restart their
shopping journey. The shopping experience should be seamless across one shopping channel.
The same holds true for performance—it should be seamless and stellar across all devices.
Searching on the go
More Google™ searches now take place
on mobile devices than on computers in
10 countries, including the United States
and Japan.5
4. Focus on the end-user experience
With all your preparations, don’t lose track of what’s most important: the user experience.
Web and mobile app users have little tolerance for slow load times and sluggish app
performance.
In your performance testing, focus on load times and intuitive flows. And keep the user’s
perspective in mind. There are many ways to improve perceived load times compared to actual
load times, such as optimizing image sizes and rendering certain content to appear first, such as
the top of a page. These sorts of optimizations can make it appear that a site is loading quickly.
Include performance testing at both the user interface and the application programming
interface (API) layers of your web and mobile apps. And then check off items on your
performance testing list to satisfy your customers’ online shopping lists.
5
oogle blog. Building for the next
G
moment, May 05, 2015.
Business white paper
Are you ready for this?
In the United States, e-commerce is
projected to grow at a compound annual
growth rate (CAGR) of 10 percent, hitting
$480 billion USD in online sales in 2019.7
Page 7
5. Plan for diverse network conditions
App performance and the user experience are affected by network latency. Excessive network
latency or packet loss doesn’t just make your app respond slower, it can cause the app to
behave erratically or even fail outright. Moreover, many apps are sensitive to the impact of the
network—their behavior and scalability change as users encounter different network conditions.
All of this means that realistic performance engineering must include the testing of network
impacts on apps during development and prior to infrastructure changes. This is an essential
piece of a performance testing strategy.
This focus on diverse network conditions applies to even the world’s largest and most
respected technology companies. For example, in 2015 Facebook launched an initiative called
“2G Tuesdays” to give its employees the chance to spend an hour each week experiencing
the slow network speeds found in emerging markets. A company spokeswoman explained
that on the lower end of the 2G networks used in many emerging markets, it can take about
two minutes to download a webpage. She noted, “We need to understand how people use
Facebook in different Internet connections in all parts of the world so we can build the best
experience for them.”6
One of the keys to addressing this challenge is to use network virtualization to simulate realistic
network conditions in your load tests. Network virtualization helps you accurately test the
impact of the network on apps long before they are put into production. Once you understand
the impact of the network, take the next step: Optimize and fine tune your app to help ensure
that it will perform well in the expected network conditions.
6. Consider performance testing from the cloud
Your customers are coming from all over, so to gain a realistic view of performance you need to
test from all over—or all representative geographies. This step is now easily accomplished via
cloud-based load testing resources that are always just a click away.
Just design and create your cloud load tests and then run them in the cloud to see how your
apps perform under different virtual user loads. There’s no need for your test and development
teams to spend time and energy managing cloud testing infrastructure—everyone can stay
focused on what they do best.
Cloud-based load testing is also essential for testing performance against spikes in demand.
Using cloud load testing generators is the only way to scale quickly and add users from
different worldwide e-locations.
7. Test in production
With today’s complex and high-volume applications, your production environment will never
be similar to your test environment. To test your content delivery networks (CDNs) and
load balancers, you have to test in production. You can accomplish in-production testing in
a non-disruptive manner by introducing new features gradually to real users and by doing
load testing when actual production usage is low.
6
he Wall Street Journal. “Facebook
T
Slows the Internet for Staffers on
‘2G Tuesdays.’” October 27, 2015.
7
F orrester Research e-commerce
Forecast, 2014 to 2019 (U.S.).
Business white paper
Have a happy holiday
When it comes to your web and mobile apps, there’s a lot on the line—from your product
revenues to your business reputation. This new reality for businesses amplifies the need for
load testing and performance engineering strategies that help ensure your apps will stand
strong under the weight of seasonal spikes in demand.
With advanced planning and rehearsing, you won’t have to dread the holiday season or any
similar seasons for your business. Just follow the seven steps outlined here to enable peak
performance for your web and mobile apps. These steps will help you avoid common IT disasters
and enjoy the thought of maintaining peak web and mobile app performance all year round.
Learn more at
hpe.com/software/holidayreadiness
Sign up for updates
© Copyright 2016 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change
without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty
statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty.
Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
Google is a registered trademark of Google Inc. All other third-party trademark(s) is/are the property of their respective owner(s).
4AA6-7432ENW, September 2016