Vendor Selection Matrix –
Next Generation Application Performance
Management: Key Highlights
Scope: Global 2015
Dr. Thomas Mendel Ph.D.
Managing Director
September 2015
© 2015, Research In Action GmbH Reproduction Prohibited
1
The Research In Action GmbH
Vendor Selection Matrix Methodology
Data Summary:
 Unique, primarily survey-based methodology for comparative vendor evaluation.
 Roughly 60% of evaluation results are based on enterprise buyers’ survey results.
 Analyst’s opinion accounts for roughly 40% of evaluation results (not 100% as in most other vendor
evaluations).
 More than 40000 data points were collected.
 Data was collected in Q1 of 2015, covering 900 IT buyers in a telephone survey, and in Q2 of 2015,
covering 700 IT buyers in an online survey of Next Generation Application Performance
Management solutions.
 20 vendors of Next Generation Application Performance Management solutions (selected by the
buyers in the survey) were evaluated. This document provide the highlights of the complete report.
 The evaluations results and forecasts are based on customer and vendor feedback, publicly available
information, triangulation, as well as the analyst’s opinion.
© 2015, Research In Action GmbH Reproduction Prohibited
2
What Tools Do You Use To Create The Vendor Longlist?
MQ/VSM
Google
Peers
Social Media
Press
Decision Makers use
a mix of traditional
and online tools
N=931 IT and Business Managers in Enterprises (Companies with more than 10,000 employees only)
© 2015, Research In Action GmbH Reproduction Prohibited
3
Market Overview: Requirements For Next Generation
Application Performance Management
What are the key requirements for a Next Generation Application Performance Management solution?
N=1600 IT Managers in Enterprises
© 2015, Research In Action GmbH Reproduction Prohibited
4
Market Overview: A Market In Transition
 The importance of Application Performance Management is at an all time high. This is a very dynamic market,
highly complex and with many different facets. Application Performance Management solutions today have many
faces and definitions, few vendors can be compared on an apples to apples basis. There are more than 400 active
software and SaaS vendors generating around $ 4,6 B in annual revenue. However, the overall importance of
Application Performance Management for the IT Service, Application and Operation Management space as a
whole, is still growing every year. This makes it worthwhile to look ahead to see what the future will look like for the
Next Generation of Application Management solutions.
 SaaS will be the platform of choice for RoI conscious buyers. Next Generation Application Performance
Management will be predominantly a SaaS play. The vast majority of new vendors are already SaaS-only. A much
better RoI as well as ease of deploy and upgrade are the major drivers for this.
 Next Generation Application Performance Management will be more streamlined. The technical requirements
for Next Generation Application Performance Management will ultimately lead to a vendor landscape that is much
more comparable than the one in today’s highly fragmented market. For the IT decision makers in our survey, the
key requirements for the future are: (1) Hybrid Cloud management, (2) Analytics and Big Data, (3) End-to-end and
real-time SLAs, (4) Application Quality Life Cycle and DevOps integration, (5) Virtual CMDB integration and (6)
Application discovery and application component deep-dive capabilities.
© 2015, Research In Action GmbH Reproduction Prohibited
5
Market Overview: Market Trends 2015
What is your number one investment area in the Application Performance Management space for 2015?
N=900 IT Managers in Enterprises
© 2015, Research In Action GmbH Reproduction Prohibited
6
Market Overview: The End-User Pain Points Today
 The Cloud challenge in the extended enterprise. The Cloud phenomenon is driving investment
from IT buyers to address issues in Application Performance Management around new
applications in the Cloud, as well as extended real-time and Big Data requirements. More
traditional challenges around N-tier Applications and instrumentation, are further exemplified by
the Cloud.
 More business orientation. DevOps is only the beginning, but already high on the agenda for
2015. As the business becomes more entrenched in Application Management as a whole,
business SLAs are a key focal point for 2015 as well.
 More integration issues than one can handle. IT decision makers are still struggling with the
many areas of integration in today’s Application Performance Management implementations. Very
few companies have worked this out completely. The most important integration challenges
come from the areas of: (1) Cost, (2) Capacity, (3) BPM/BTM, (4) Business process and (5) The
CMDB.
© 2015, Research In Action GmbH Reproduction Prohibited
7
Market Overview: Key Points To Remember
 A changing market with new and clearer requirements. Application Performance
Management has a long history in many enterprises. Many implementations are already more
than 25 years old. Vendors and IT buyers alike a struggling with updating their solution sets to
accommodate the quickly changing requirements. The Next Generation Application
Performance Management solutions, however, will be based a clearer set of requirements,
making it easier to implement and maintain, as well as to compare vendors.
 SaaS will be the future delivery platform. SaaS is now widely accepted. While new players
are already predominantly SaaS-only, established players are either revamping their solutions,
or are at least offering SaaS options.
 One clear market leader and the comeback of an established player. Dynatrace, the rebranded former Compuware solution set is the clear winner in our evaluation and well
positioned for future market dominance. BMC Software, with a new strategic direction for its
Application Performance Management solution, is making serious headway in the market and
will be very likely the long-term number two.
© 2015, Research In Action GmbH Reproduction Prohibited
8
Vendor Selection Matrix – Next Generation Application
Performance Management: Evaluation Criteria
Strategy
Vision & Go-To-Market
30%
Innovation & Partner Ecosystem
20%
Company Viability & Execution Capabilities
15%
Differentiation & USP
Execution
Breadth & Depth Of Solution Offering
Market Share & Growth
Customer Satisfaction & Mindshare
Price Versus Value
35%
Does the company have a coherent vision in line with the most probable future market scenarios?
Does the go-to-market and sales strategy fit the target markets and customers?
How innovative is the company?
How is the partner ecosystem organized and how effective is the partner management?
How likely in the long-term survival of the company?
Does the company have the necessary resources to execute the strategy?
Does the solution have a Unique Selling Proposition (USP) and clear differentiators?
30%
15%
25%
30%
Does the solution cover all necessary capabilities expected by the customers?
How big is the market share and is it growing above market rate?
How satisfied are customers with the solution and the vendor?
How do customers rate the relationship between the price and perceived value of the solution?
© 2015, Research In Action GmbH Reproduction Prohibited
9
Vendor Selection Matrix – Next Generation Application
Performance Management: Ranking Of Selected Vendors
1 Dynatrace
2 CA
3 AppDynamics
4 New Relic
5 HP
6 Microsoft
7 IBM
8 Oracle
9 AppNeta
10 Riverbed
© 2015, Research In Action GmbH Reproduction Prohibited
10
Strategy Execution
4,28
4,43
3,90
4,08
4,00
3,85
3,93
3,73
3,88
3,70
3,30
3,68
3,48
3,35
3,63
3,13
2,90
3,55
2,80
3,48
Total
8,70
7,98
7,85
7,65
7,58
6,98
6,83
6,75
6,45
6,28
Vendor Selection Matrix – Next Generation Application
Performance Management: Ranking Of Selected Vendors
AppDynamics
AppNeta
CA
Dynatrace
HP
IBM
Microsoft
New Relic
Oracle
Riverbed
Strategy Execution Total Revenue ($m)* Market Share*
1,85%
85
7,85
3,85
4,00
0,98%
45
6,45
3,55
2,90
5,43%
250
7,98
4,08
3,90
8,70%
400
8,70
4,43
4,28
2,87%
132
7,58
3,70
3,88
5,43%
250
6,83
3,35
3,48
6,09%
280
6,98
3,68
3,30
2,50%
115
7,65
3,73
3,93
1,85%
85
6,75
3,13
3,63
5,22%
240
6,28
3,48
2,80
* Research In Action forecast CY2015, in Application Performance Management
© 2015, Research In Action GmbH Reproduction Prohibited
11
Vendor Selection Matrix – Next Generation Application
Performance Management: The Number One
Dynatrace: The champion of Application Performance Management
 General: The re-branding from Compuware to Dynatrace, was a risky endeavor indeed, but it is
paying off. Clients value he renewed focus of the company, in particular with the Keynote
integration which completes this impressive portfolio.
 Strategy: Dynatrace is highly differentiated, clients give Dynatrace the highest scores when it
comes to innovation. Strategically, Dynatrace is second to none.
 Execution: Dynatrace is the current market share leader by a wide margin and will continue to be
so. The company already covers all major growth areas for the future. Clients are highly satisfied
and see the value in Dynatrace’s solutions.
 Customer Quote: ”Dynatrace continues to impress us. This going private thing was a concern for
sure, but now I cannot even imagine going with another vendor.” CIO North-American retail
company.
 Bottom Line: Being focused is paying off. Dynatrace will be the reference company in Next
Generation Application Performance Management for the foreseeable future.
© 2015, Research In Action GmbH Reproduction Prohibited
12
Vendor Selection Matrix – Next Generation Application
Performance Management
CA Technologies: Integration will be a key competitive advantage for future
growth
 General: CA’s mission is to cover all aspects of the IT Service, Application and Operation
Management space and the company does this well. CA is addressing all key client requirements of
Next Generation Application Performance Management.
 Strategy: CA is highly differentiated, the vision remains strong. However, Clients would like to see a
clearer articulated strategy for Next Generation Application Performance Management.
 Execution: CA is one of the market share leaders in Application Performance Management, still
capitalizing on the Wily acquisition. Clients are very satisfied with CA today, but there were of course
some complaints regarding CA’s overall pricing.
 Customer Quote: ”When it comes to platform integration, CA is second to none. This works well for
us and we will continue to work with CA in the future.” CIO North-American financial services
company.
 Bottom Line: The overall IT Service, Application and Operation Management champion is back on
track in Application Performance Management.
© 2015, Research In Action GmbH Reproduction Prohibited
13
Vendor Selection Matrix – Next Generation Application
Performance Management
AppDynamics: The fall of a rising star
 General: Founded in 2008 by former Wily leaders, and well-funded indeed, the rise of AppDynamics
has recently come to a more or less abrupt halt.
 Strategy: AppDynamics is highly differentiated, clients still like the strategic approach which is very
much in-line with where the market is going, but would like to see a more mature approach to
company growth and profitability forecasts.
 Execution: AppDynamics is liked by their customers for their capability of delivering value for
money. Clients also like the SaaS and non-SaaS options, as the all SaaS approach is still questioned
by many IT decision makers today, despite the fact that almost everyone agrees that this is the
strategic future direction.
 Customer Quote: ”We are happy with the AppDynamics solution. However, we are concerned about
the accounting and reporting principles of the company.” CIO North-American financial services
company.
 Bottom Line: AppDynamics needs to grow-up to become a long-term competitor in the Next
Generation Application Performance Management space.
© 2015, Research In Action GmbH Reproduction Prohibited
14
Vendor Selection Matrix – Next Generation Application
Performance Management
New Relic: A top Application Performance Management player in-themaking
 General: New Relic, founded in 2008, also from former Wily Technology leaders, and like
AppDynamics well-funded, is a pure-play SaaS Application Performance Management vendor.
 Strategy: Clients see New Relic as highly differentiated, they value the company’s innovation
capabilities and future-proof architecture, in particular as it comes to Big Data integration.
 Execution: New Relic today has above average top 20 scores for client satisfaction as well as on
a price versus value level. However, as clients become more familiar with the SaaS-only
approach of the company, Research In Action believes that the scores will improve even more.
 Customer Quote: ”We decided to buy-into SaaS wholeheartedly. New Relic is the therefore the
number one choice for us.” CIO North-American services company.
 Bottom Line: New Relic is moving up the ranks quickly. Though still a larger niche player, the
future looks bright for New Relic.
© 2015, Research In Action GmbH Reproduction Prohibited
15
Vendor Selection Matrix – Next Generation Application
Performance Management
HP: The ITIL champion is becoming more important in Application
Performance Management
 General: HP has been a key player in the IT Service Management market almost from its inception.
HP’s legacy in Network Performance Management is helping the company to expand its reach into
Application Performance Management.
 Strategy: HP’s differentiation is only at a medium level. However, clients like the partnership
approach and the strategic direction. Clients see also huge value in HP’s Big Data and Analytics’
solutions.
 Execution: HP is still extremely strong in Infrastructure Performance management, Network Node
Manager still being the market reference. Clients are therefore excited about the future prospects in
the Application Performance Management market.
 Customer Quote: ”We like HP’s flexibility and customer orientation. This helps us a lot when talking
to our business leaders.” CIO European manufacturing company.
 Bottom Line: HP is now focusing more on this growth market. The future looks bright.
© 2015, Research In Action GmbH Reproduction Prohibited
16
Vendor Selection Matrix – Next Generation Application
Performance Management: Detailed Results
Weighting
Dynatrace
Score
Strategy
Vision & Go-To-Market
Innovation & Partner Ecosystem
Company Viability & Execution Capabilities
Differentiation & USP
Execution
Breadth & Depth of Solution Offerring
Market Share & Growth
Customer Satisfaction & Mindshare
Price versus Value
Weighted
Score
CA
Score
AppDynamics
Weighted
Score
Score
Weighted
Score
New Relic
Score
Weighted
Score
HP
Score
Weighted
Score
30%
20%
15%
35%
100%
4
4,5
4
4,5
1,20
0,90
0,60
1,58
4,28
3,5
3
4,5
4,5
1,05
0,60
0,68
1,58
3,90
4
3
3
5
1,20
0,60
0,45
1,75
4,00
4
4
3,5
4
1,20
0,80
0,53
1,40
3,93
4,5
4
4,5
3
1,35
0,80
0,68
1,05
3,88
30%
15%
25%
30%
100%
4,5
5
4,5
4
1,35
0,75
1,13
1,20
4,43
4,5
4,5
4
3,5
1,35
0,68
1,00
1,05
4,08
4
4
4
3,5
1,20
0,60
1,00
1,05
3,85
4
4
3,5
3,5
1,20
0,60
0,88
1,05
3,73
3,5
4
4
3,5
1,05
0,60
1,00
1,05
3,70
Scale Explanation: 1 (Low) to 5 (High)
© 2015, Research In Action GmbH Reproduction Prohibited
17
The Research In Action GmbH
Vendor Selection Matrix Methodology
Vendor Selection Matrix Disclaimer:
Research In Action GmbH does not endorse any vendor, product or service depicted in our
research publications, and does not advise technology users to select only those vendors
with the highest ratings. The information contained in this research has been obtained from
both enterprise as well as vendor sources believed to be reliable. Research In Action GmbH’s
research publications consist of the analysts’ opinions and should not be considered as
statements of fact. The opinions expressed are subject to change without further notice.
Research In Action GmbH disclaims all warranties, expressed or implied, with respect to this
research, including any warranties of merchantability or fitness for a particular purpose.
© 2015, Research In Action GmbH Reproduction Prohibited
18
Contact
Dr. Thomas Mendel Ph.D.
[email protected]
+49 160 99492223
RESEARCH IN ACTION
Research In Action GmbH
Hauptstraße 9
56244 Hartenfels
Germany
Office: +49 2626 9249845
Fax: +49 2626 9249845
Email: [email protected]
© 2015, Research In Action GmbH Reproduction Prohibited
19