EMA Radar™ for Advanced Performance
Analytics (APA) Use Cases: Q4 2012
By Dennis Drogseth
ENTERPRISE MANAGEMENT ASSOCIATES® (EMA™) Radar Report
December 2012
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Table of Contents
Executive Introduction...................................................................................................................... 1
Investing in APA’s Power – More a Tidal Wave than a Market..................................................... 1
Methodology..................................................................................................................................... 4
Core Criteria ..................................................................................................................................... 4
Deployment and Administration................................................................................................. 5
Cost Advantage............................................................................................................................ 5
Architecture and Integration........................................................................................................ 5
Functionality ............................................................................................................................... 5
Vendor Strength........................................................................................................................... 6
Reading the Bubble Charts.......................................................................................................... 6
Additional Criteria-related Insights.................................................................................................... 7
Cost, Overhead and Time to Value.............................................................................................. 7
Overall Cost......................................................................................................................... 7
Overall Function: Architecture, Integration and Features............................................................. 8
Other Tradeoffs in Evaluation.................................................................................................... 10
Technical Performance Analytics..................................................................................................... 11
Change Impact and Capacity Planning and Optimization ............................................................. 15
Business Impact Management......................................................................................................... 17
The 22 APA Vendors in Summary................................................................................................... 20
Awards ............................................................................................................................................ 28
AccelOps – Best Integrated Security .......................................................................................... 28
Compuware – Best DevOps Support......................................................................................... 28
IBM – Vendor to Watch in 2013............................................................................................... 28
Netuitive – APA Market Definer................................................................................................ 28
SevOne – Best Infrastructure-optimized Analytics..................................................................... 28
Zyrion – Most Balanced APA Performer.................................................................................... 28
Conclusion...................................................................................................................................... 29
Appendix A: Additional APA Vendors ............................................................................................ 31
2
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Table of Contents (continued)
3
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Executive Introduction
Advanced Performance Analytics (APA), as Enterprise Management Associates (EMA) defines it, brings
real-time or near real-time “big data” to IT operations, architects, service managers and even applications
development, as well as IT executives and non-IT business stakeholders. APA has evolved out of classic
service performance management, but with a twist – rather than siloed approaches to monitoring and
analysis, APA demands more eclectic data collection, huge levels of analytical scalability and strong
insights into cross-domain and/or business outcomes.
This EMA Radar™ report targets APA in three critical use cases:
• Technical performance analytics – focused on optimizing the resiliency of critical application and
business services – including VoIP and rich media – in cloud (public/private) as well as non-cloud
environments with a strong focus on triage, diagnostics, roles supported, self-learning capabilities
and associated automation.
• Business impact management – including user experience, customer experience and customer
management, business process impacts, business activity management, and data such as revenue
per transaction, abandonment rates, competitive impact, and IT operational efficiency.
• Change impact and capacity optimization/planning – which are admittedly two use cases combined
into one, but which share requirements in terms of understanding interdependencies across the
application/service infrastructure as volumes increase, changes are made, configuration issues arise
and actions, including relevant automated actions, are required.
It should also be mentioned that while security is not singled out as a separate use case, there is a
modest but growing overlap between advanced performance analytics and Security Information and
Event Management (SIEM), and this EMA Radar included some insights there as well. Similarly, while
DevOps requirements are not a separate use case in this EMA Radar, it was called out as a value-add
across all three use cases – although primarily in technical performance and change impact.
A record twenty-two vendors are included in this industry assessment. They are: AccelOps, AppFirst,
Appnomic Systems, BMC, Compuware, eG Innovations, eMite, FireScope, Fluke Networks, HP, IBM,
Interlink Software, ManageEngine, Neebula, Netuitive, OPNET, OpTier, Prelert, Quest Software, SevOne,
Splunk, and Zyrion. While this literally A-to-Z list doesn’t cover every conceivable entrant, it’s certainly
more than representative of many of the real APA innovations available in the market today.
Investing in APA’s Power – More a Tidal Wave than a Market
APA challenges traditional notions of “fixed markets” with linear boundaries and rigid, technical
criteria – as it is more like a tidal wave shattering traditional industry assumptions than a boxed-in,
technically defined market. But it does have a center of gravity, much like a star can be the center of an
entire planetary system. APA differs from classic warehousing in that discovery, modeling, techniques
for data collection and prioritization, and the use of “trusted sources” for relevant KPIs provide a level
of efficiency for IT service management that most classic “big data” approaches cannot.
1
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
And not surprisingly, many of the more advanced APA capabilities leverage a wide range of advanced
heuristics (See Figure 1) – most classically algorithms for anomaly detection, optimization, and
predictive trending, along with advanced forms of correlation – but EMA has also seen everything from
Chaos Theory to Fuzzy Logic, to neural networking, to DNA resequencing among the vendors here.
Some of the vendors also utilize warehousing, OLAP, and data mining for advanced historical analysis.
Anomaly detection
100%
Comparators
95%
Correlators
91%
Application transaction analysis
91%
Predictive Algorithms
86%
Object-based modeling
77%
Data mining and OLAP
77%
Optimization algorithms
50%
Case-based reasoning
45%
Fuzzy logic
36%
Chaos theory
27%
Neural networks
18%
0%
20%
40%
60%
80%
100%
120%
2 or 1
Figure 1: APA Analytics spread across many multiple heuristics, as reflected in the chart above.
If your IT or service provider organization is seeking greater efficiencies, insights, and relevance, then
APA solutions are clearly worth your attention for a number of reasons. For instance, as a group, APA
solutions are:
• Self-learning and adaptive: While generally not the cheapest in software purchases, APA solutions
often bring very fast ROI and typically require minimal administrative overhead, especially
once integrated and deployed. In fact if a solution can’t be effective without huge administrative
investments, it’s categorically not true APA. And this goes for changing conditions across the
enterprise or service provider environment. For instance, across all twenty-two vendors the average
full-time employee count for ongoing administration after deployment is well under a single
full-time administrator. (See Figure 2)
2
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
What is the average required Full Time Employee (FTE) allotment on a
percentage basis for ongoing administration?
45%
Less than 25% of an FTE
27%
25% to 50%
14%
50% up to 100%
100% up to 150%
9%
150% up to 200%
5%
200% up to 250%
250% up to 300%
300% up to 400%
400% up to 500%
500% up to 1000%
1000% or more
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Figure 2: Average post-deployment administration for APA across all 22 vendors was well under .5 of a fulltime employee thanks to APA’s advanced, self-learning and self-adaptive heuristics.
• Assimilative – optimize existing investments: Some APA solutions are what might be called “overlays,”
designed exclusively to leverage existing investments. Others can and often do collect at least some
of their data directly. However, most true APA solutions are versatile in how they integrate with
other toolsets and all are versatile in how and where they collect data. Categorically, APA solutions
can support effective toolset consolidation by providing broader, more cohesive insights than other
more siloed or niche-defined tools where simply collecting data is more important than actually
doing something with it. Hence the economies they provide in both opex and capex investments
can be dramatic.
• A cohesive set if insights for many multiple stakeholders: In a very positive sense, APA might be seen
as the “CMDB of performance management” in which many different perspectives are unified,
analyzed and presented meaningfully. APA may not provide a “single source of truth,” but it does
offer a single, multi-dimensional resource to support many different constituencies with cohesive
and actionable information. This not only brings economies of scale and opex and capex efficiencies,
it provides a foundation for breaking through siloed politics to promote far more useful dialogs and
more directed and consistent actions.
3
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• A way to unify IT and business objectives in a manner that’s substantively new: Like user experience
and customer experience management, some APA solutions can provide a bi-directional mirror
looking backwards at applications and their supporting infrastructures, and forwards at user and
consumer behavior and business outcomes. Since so many business activities involve interacting
with applications and application transactions, both in terms of back office efficiencies and frontoffice business growth, APA’s dual mirror is uniquely well positioned to support what one of the
forty-one APA deployments demanded – “a way to tune my business on an hourly basis versus
monthly, weekly, or even 24-hour trending.”
Methodology
EMA interviewed twenty-two vendors for this EMA Radar. They are: AccelOps, AppFirst, Appnomic
Systems, BMC, Compuware, eG Innovations, eMite, FireScope, Fluke Networks, HP, IBM, Interlink
Software, ManageEngine, Neebula, Netuitive, OPNET, OpTier, Prelert, Quest Software, SevOne, Splunk,
and Zyrion. EMA spoke to 41 distinct APA deployments, an average of two per vendor. To complete
this research, each participating vendor completed a comprehensive questionnaire consisting of more
than 100 questions and over 500 data points. All of the twenty-two vendors were interviewed and
sometimes re-interviewed for current status as the report includes some functionality just available by
the end of Q4 2012.
Core Criteria
In all Radar Reports, EMA evaluates solutions based on five key areas: Architecture and Integration,
Functionality, Deployment and Administration, Cost Advantage, and Vendor Strength.
Figure 3: A chart like this will be generated for each vendor based on their overall score.
A “perfect” score of 500 will generate a pentagon, as shown above in red.
4
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Deployment and Administration
This area addresses overall efficiencies for deployment and administration. It keys on a number of factors
such as FTEs for ongoing administration, time to value, automation and administrative advantages
in deployment and adaptability to changing conditions, as well as maintenance, customer support
and range of services available to support core requirements along with unique use cases. Customer/
deployment interviews are especially critical in validating vendor perspectives here.
Cost Advantage
Costs include core software or SaaS investments, associated costs for any hardware and software
dependencies, basic services costs associated with deployment and basic administration, and maintenance
costs. Some questions, such as pricing models, are more descriptive than evaluative, to help provide a
more rounded insight into solution adoption.
Architecture and Integration
Targeted areas of interest here are:
• Scalability – addresses supported devices or managed entities, metrics and events, users and other
scalability-related factors.
• Domain coverage – includes classic IT domains (network/system, etc.) as well as use-case specific
areas touching on business impact, change impact and capacity optimization.
• Cloud capabilities – captures both cloud as a resource for SaaS, etc., as well as reach into managing
cloud-dependent environments. Questions about support for virtual applications, VDI, virtual
infrastructure and other cloud-related components were spread throughout the questionnaire.
• Data collection and data harvesting – includes types of events, metrics and other data access, and this
includes a balance of evaluative as well as descriptive insights.
• Integrations – looks at core use-case related integrations in support of performance, business impact,
change impact and capacity optimization from a bi-directional (import and/or export) perspective.
• Modeling and interdependencies – keys on contextual insights into application or other services as
they map across infrastructure, business, change and configuration and capacity-related issues.
• Standards – provides a look at standards supported.
Functionality
Targeted areas of interest here are:
• Application/service reach – looks at application and other services supported as well as technical
environments ranging from cloud, to VPN, to Java, etc.
• Technical/performance analytics – is naturally focused on the “performance” management use case,
keying on strengths in triage, diagnostics and core performance analytics.
• Business impact analytics – addresses everything from Service Level Management (SLM) and User
Experience Management (UEM), to business activity monitoring, business process impact, user
productivity and financial impacts ranging from revenue generation and competitiveness to overall
business or organizational efficiencies.
5
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• Change/capacity analytics – target unique insights into change, configuration and shifting capacity
requirements, as well as supported linkages to other resources such as a CMDB/CMS.
• Predictive capability – looks at both advanced trending as well as near-term predictive insights.
• Automation – provides a view of how natively an APA solution can trigger automated actions in a
variety of contexts, from diagnostics, to change management, to incident and problem workflows.
• Visualization/reporting – addresses the richness of an APA solution in exposing and externalizing
its insights based on type of visualization technology, and roles supported natively to the solution.
Vendor Strength
This section provides a quick visual way of assessing marketplace presence in terms of breadth of
customer base, partners, revenue if offered, and overall vision and strategy in the APA arena.
Reading the Bubble Charts
This EMA Radar is specifically intended to provide a useful set of insights into the design points and
unique strengths of each of the twenty-two APA solutions. And while there will be some sorting based
on Value Leader, Strong Value and Specific Value for each use case, IT and service provider buyers
will be well advised to first define their objectives and then seek out the solution that fits them best
– regardless of apparent “rank” or “award.” Similarly, the report summary makes every effort to offer
high-level insights into design and function for each vendor so that it too can be used as a starting point
for planning an APA investment.
The survey questions covered the five key functions common to all EMA Radar Reports, which include
Architecture, Functionality, Deployment & Administration, Vendor Strength, and Cost Advantage.
EMA has produced a report targeted at presenting and explaining Radar Reports in general, entitled
How to Use the EMA Radar Report, which is available free of charge on the EMA website. We encourage
readers of this APA Radar to begin by reading that document.
In this APA Radar, there are three Bubble Charts to emphasize use-case diversity. However, there are
some core things to keep in mind as you look at each of these, or look across them in combination:
• The vertical axis (Product Strength) will be primary for seeking strengths in functionality and
overall architectural breadth and power.
• The horizontal axis (Cost Efficiency) favors cost advantages, as well as advantages in administrative
overhead and overall administrative efficiency.
• The size of the bubble is reflective of (Vendor Strength) – as described above represents overall
market strength as relevant to the Radar.
• Because of the nature of APA solutions, proximate vendor positions may still reflect radically
different advantages. For instance one vendor’s core, analytic power may place it next to a vendor
at a similar cost/administrative advantage but with weaker core analytics and broader assimilative
and modeling strengths. So they may, in fact, represent significantly different types of investment
choices. Therefore, no investment decisions should be made without thoroughly considering each
vendor’s specific profile.
6
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Additional Criteria-related Insights
The APA market and its instantiations reflect an especially broad range of design points, served
constituencies, and domain targets. And while the three use cases go a long way to clarifying these
differences, they remain primarily an informative departure point for further investigation. So using
this EMA Radar well requires viewing it less as a scorecard – as if a sporting event had been played
out, terminated, and won – and more as a vehicle for focusing a deeper dive through examination of
individual vendor profiles and other discussions.
Cost, Overhead and Time to Value
Overall Cost
• Core software and hardware costs: Few APA solutions are cheap. On the other hand many bring
added value not only in their richness of functionality, but also in their capabilities to learn their
environments and adapt dynamically to change. Costs were provided confidentially by each vendor
assuming a 1,000-server, or 10,000-person environment across all three use cases. And while this
began to create a level playing field, in some cases, this favored enterprise vendors natively targeting
large-scale deployments over vendors optimized for smaller business and mid-tier businesses. In
some cases, the reverse was true. Other native inequalities included costs where monitoring was
fully factored in, versus those solutions optimized to draw from third-party and other monitoring
tools as more of an analytic overlay. And needless to say, relative cost-efficiency is likely to change
for much larger or smaller proposed solutions based on individualized pricing models and designs.
• SaaS and packaging flexibility: Vendors with SaaS options along with on-premise options were
modestly favored over those with only one choice. Moreover, if SaaS deployment interviews validated
strong savings in terms of administrative and deployment benefits, those were also factored in.
• Quick time to value (self-learning): APA solutions should categorically provide quick time to
value compared to more traditional monitoring tools. In this equation, self-learning was viewed
as a strong plus – and in fact nineteen of the twenty-two vendors answered “yes” to the question:
Once installed can your solution “learn” its environment dynamically without added administrative
intervention?
• Easy administrative overhead: Similarly, no APA solution should require huge resources for
on-going administration, or a lot of human intervention to make it “smart.” However, many
solutions added options for policy creation, modeling and other associations that could drive
core APA intelligence to yet higher strategic value. Sixteen of the twenty-two vendors indicated
that for on-going administration, they required only .5 full time employees, or less, for ongoing
administration of their APA intelligence. In suite situations, EMA worked with vendors to separate
costs for analytics administration versus basic monitoring tool setup in order to maintain a more
equitable balance between monitoring suites and analytic overlays.
• Services and support: While APA solutions should ideally not require much in the way of basic
deployment services, having good reputations for listening to customers and stepping up to their
unique requirements remains important. Feedback along these lines was highly valued across our
forty deployment interviews. Moreover, since most APA initiatives have strategic value, services
designed to support strategic process and organizational planning across use cases were also ranked
as important.
7
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Overall Function: Architecture, Integration and Features
• Scalability and “Big Data:” APA solutions are by definition “big data” tools and eighteen out of
the twenty-two could assimilate a million or more KPIs in five minutes or less. The lowest number
was 100,000-500,000 and nine vendors support 50 million or more KPIs in five minutes or less.
(See Figure 4) Here KPIs might be events, time series data, log file queries, Web 2.0-unique data,
configuration data, and data from business planning and other data warehousing investments, just
to provide a partial list. So qualitative and quantitative breadth and power in assimilating KPIs
were given high marks in this APA Radar.
How many metrics, events, or other data sources can you assimilate for
your advanced analytics within a period of no more than five minutes?
1-100
101-500
501-1,000
1,001-5,000
5,001- 10,000
10,001-50,000
50,001-100,000
100,001-500,000
9%
500,001-1,000,000
9%
1,000,001-10,000,000
27%
10,000,001-50,000,000
14%
More than 50,000,000
41%
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Figure 4: Big Data in APA analytics – as viewed across our twenty-two vendor participants.
• Frequency of sampling: While both historical and “real-time” capabilities were evaluated here,
one requirement for APA is fundamental support for real-time insights into performance, change,
capacity and business impact. When frequency of data collection was evaluated, sixteen out of
twenty-two could assimilate data in 30 seconds or less, and six could do it in real-time. Frequency
of data assimilation was also given high marks.
• Cross-domain support: EMA looked for balanced and broad cross-domain support for APA
vendors. While this made sense overall, it clearly favored some vendors targeting breadth over
others that were narrower and more focused in scope, and deliberately selective in the types of data
they assimilated.
8
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• Integrations and monitoring: For APA, both integrations and basic monitoring were viewed as a
means to an end. In other words, EMA was “monitoring neutral,” as APA is fundamentally about
what’s done with the data after it’s collected. As a result, EMA did not favor vendors that did their
own monitoring versus those that depended on supported integrations. Both have value, and these
differences were made clear in individual vendor profiles. However, it should be pointed out that
EMA is seeing a growing trend among suite vendors to develop analytic overlays to assimilate many
multiple sources, and this capability was given high marks when it appeared.
• Cloud: EMA evaluated vendors vis-à-vis cloud from two perspectives. One of them was their
ability to leverage cloud as a resource for scalability, SaaS ease of administration, and pure analytic
compute power. The other perspective looked at their ability to analyze outcomes from inside
public and private cloud environments.
• DevOps: While DevOps was itself not a separate use case here, it was examined in terms of four
areas: application development directly, Q/A Test, production/operations hand-offs, and feedback to
development after deployment. Twenty out of the twenty-two vendors evaluated provided some level
of DevOps support along at least one of these lines.
• Applications over VoIP, rich media and data streaming: Since most IT business services are
applications related, application and particularly transaction insights were favored over VoIP,
rich media and other network-centric business services, although all were evaluated. Out of the
twenty-two vendors, twelve supported VoIP or telephony at some level, while ten supported rich
media, media streaming or video on demand.
• Applications over infrastructure: Similarly, the EMA Radar for APA favored top-down over
bottoms-up approaches to APA, which tends to be native to both APA design and benefits, but
which put some infrastructure-centric APA solutions at a disadvantage.
• Modeling, application dependency and CMDB support: EMA believes that service modeling
and APA advanced analytics should go hand in hand, and this APA Radar ranked service modeling
capabilities accordingly. While all three use cases can in some way benefit from modeling, application
dependency and CMDB support, EMA weighted these capabilities as especially important for change
impact management, and capacity planning and optimization. All twenty-two vendors had some
capabilities for capturing application and/or infrastructure interdependencies. Seventeen claimed
to have a performance-optimized model of applications or other services, and twelve could integrate
their application dependency models with CMDBs (their own and/or third-party CMDBs).
All twenty-two vendors had some capability for linking physical and logical interdependencies,
which is especially important for business impact management and for service providers seeking to
manage multiple customers.
• Realized analytic value: Needless to say, this EMA Radar heavily weighted high-powered analytics,
and especially self-learning, predictive analytics, across multiple heuristic strengths ranging from
correlation to anomaly detection, to optimization algorithms, to fuzzy logic, OLAP, data mining and
chaos theory, just to mention some examples. However, the APA Radar weighted realized analytic
value strongly over just raw potential. “Realized” is a key word here, as some of these solutions have
the raw analytic power to do far more than they were given credit for in this report – if that power
wasn’t yet harnessed through visualization, reports and articulated stakeholder support.
9
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• Visualization to support diverse stakeholder constituencies: This APA Radar also favored APA
solutions that could reach effectively across a wide variety of constituencies, whether domainspecific (network, systems, apps, storage, etc.) or cross-domain (architects, engineers, infrastructure
managers, DevOps, IT execs, etc.) and business constituencies (business execs, line of business managers,
online operations, market development, etc.).
Other Tradeoffs in Evaluation
• Cost versus function: Needless to say, more often than not greater function comes at greater cost,
albeit this is far from universally true. For the purposes of this evaluation, function came first. For
instance, if a vendor had a strong application dependency mapping solution its price was added
into the core package for evaluation and this generally positioned the vendor more strongly. On
the other hand, if a vendor with strong functionality and overall lower cost could match more
expensive solutions on per-function/feature basis that was seriously taken into account in price
weightings.
• Use-case shifts in the Cost Efficiency axis: While some vendor positions remained relatively
constant across all three use-case scenarios, most did not, which raises the question: why if vendor
“x” is a low-cost and easy to deploy, and vendor “y” pricey, doesn’t that remain constant in all situations?
However, there can be substantial differences, both in terms of the product contents of the solutions
– e.g. unique products targeted at capacity planning or business impact – as well as administration
and integrations required for optimal results. Fitting a square peg in a round hole is bound to jack
up administrative costs as it attempts to compensate for functional deficiencies.
10
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Technical Performance Analytics
Technical Performance Analytics is a mainstay for all twenty-two vendors, which are, by dint of their
APA leadership, natively all leaders in this space. A few highlights in the evaluation included:
• Technical triage: Vendors were evaluated on both cross-domain (network/systems/storage,
database) and intra-domain triage, as well as isolating problems across application tiers, middleware
issues, problems in provisioning applications over internal cloud, isolating cloud service provider
issues, triage across virtualized systems, visibility into branch office issues including QoS, security
(SIEM) and end-device issues (OS, browser, etc.).
11
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
What types of technical triage and diagnostics can your advanced
analytics solution provide?
Isolate infrastructure issues internal to (systems)
91%
Isolate infrastructure issues within the DB
91%
Isolate whether the problem is within the
application, server, network, or DB
86%
Triage across application tiers
86%
Triage across virtualized systems
86%
Isolate infrastructure issues in storage
86%
Isolate infrastructure issues in the network
82%
Isolate middleware issues
77%
Visibility from the branch into issues such as QoS,
bandwidth, application latency
73%
Isolate the relevant cloud service provider
55%
Isolate problems in provisioning applications over
internal cloud
50%
Isolate end device issues/ hardware and OS
50%
Isolate end device issues/ browser
45%
Isolate mobile-specific end-device issues
41%
Isolate security-related issues (SIEM)
Other (Please specify)
23%
14%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Figure 5: APA solutions are typically strong in their triage reach.
12
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• Integrations: Some of the integrations targeted were WAN optimization and load balancing
support, application development tools, workload automation, service desk integration, SIEM and
an application or operational dashboard.
What is the range of domains that you can analyze directly through your own
advanced analytics capabilities?
Systems
91%
Virtual systems
91%
Web /Web2.0 Applications
91%
91%
Client/server applications
Network
86%
Virtualized applications
86%
86%
Database
77%
Storage
73%
Virtual cross domain
59%
Mainframe
59%
Desktop
55%
VoIP and Unified Communications
50%
Mobile
41%
Video
41%
Security/ vulnerability
32%
Asset Inventory
23%
Other (Please specify)
9%
We depend on integration with other monitoring tools
0%
10%
20%
30%
40%
50%
60%
70%
80%
90% 100%
What is the range of domains that you can analyze through fully supported thirdparty or outside sources?
Systems
95%
Network
89%
Virtual systems
89%
Database
84%
Web /Web2.0 Applications
79%
Client/server applications
79%
Virtualized applications
79%
Storage
79%
Virtual cross domain
74%
Mainframe
74%
VoIP and Unified Communications
63%
Desktop
58%
Mobile
58%
Video
58%
Asset Inventory
42%
Security/ vulnerability
42%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Figure 6-7: Cross domain support is endemic to APA for performance management and other use cases as indicated
here, whether it’s directly through the APA solution, or through vendor and third-party integrations.
13
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
What types of data/alerts can your advanced analytics
solution collect directly?
Events – performance related
100%
Time series (performance)
100%
Transaction
86%
Log file queries
82%
Events- business related
77%
Configuration
73%
Flow
64%
Web 2.0 (Ajax etc.)
59%
Events—security related
55%
End user keystroke
18%
Other (Please specify)
14%
0%
20%
40%
60%
80%
Figure 8: Eclectic data gathering is also typical of APA solutions. These options were
often significantly expanded through integrations with other tools.
14
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
100%
120%
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Change Impact and Capacity Planning and Optimization
Similarly all twenty-two vendors had some APA offering to support either change impact or
capacity planning and optimization. However, the solutions varied significantly in where and how
they approached these problems, as well as the depth and quality of APA capabilities they delivered.
Highlights relevant to this use case include:
• Change impact scope: All twenty-two vendors had some support for change impact at some level,
with the majority favoring system configuration (nineteen) and virtualized infrastructure (eighteen)
service impact. Sixteen supported application release changes and network configuration changes.
Least served were end devices or desktops where only three supported change impact management.
15
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• Change impact analytics: These included if/then change impact, correlation, anomalous behaviors,
predictive trending OLAP, data mining, and self-learning heuristics. While correlation and anomaly
detection were most prevalent at twenty vendors, thirteen supported predictive trending and five
indicated OLAP support. Most solutions offered both real-time and historical trending.
• Capacity analytics: Twenty of the twenty-two vendors offered capacity analytics with a mix
of real-time and historical trending support. Across the twenty vendors, all domains had some
capacity analytics support, with the most prevalent being virtual systems (nineteen), and the least
prevalent being asset-related and security-related capacity analytics (four each). However, this APA
Radar gave significantly heavier weight to advanced optimization and capacity analytics versus just
basic trending – which was still the most prevalent approach among the 22 vendors.
• Integrations: EMA gave high marks to integrations with capacity planning resources, CMDB
and other configuration sources and application dependency mapping, chargeback or usage-based
analysis, as well as an IT data warehouse for advanced trending.
How do you support change impact analytics?
Primarily through our own analytics
55%
Primarily through integration with a CMDB
14%
Either through our own analytics or via CMDB
integrations - -both are good
32%
0%
10%
20%
30%
40%
50%
Figure 9: Change impact analytics significantly involved CMDB integrations with nearly half the APA vendors – in
some cases making the APA solution a “trusted source” for critical CI attributes in a federated CMS.
16
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
60%
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Business Impact Management
Business impact is in some ways the most revelatory arena for understanding APA potential, as it not
only crosses traditional market boundaries, but also pre-set cultural and political boundaries as well.
One could argue persuasively that business impact-oriented APA, more than any other technology,
holds the key to the kingdom of true IT-to-business alignment and optimization, as a clear and current
reference point for joint dialogs and planning. Some highlights here include:
• Logical-to-physical connections and metrics: Vendors were asked to examine linkages to business
outcomes with transaction or other performance insights including revenue, consumer identification
for internal and external service consumers, Business Activity Management (BAM), business process
impacts, project-management related impacts, as well as costs and committed SLAs. They were also
asked if they could highlight such things as response time to complete a business service, end-user
17
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
application preferences, application usage from a cost perspective, end-user effectiveness in interacting
with applications, brand impact metrics, conversions from competitive websites and supply-chain-related
outcomes. Twenty-one out of the twenty-two vendors showed support for business impact along
one or multiple of these metrics with an average of four business metrics supported per vendor.
What business impact metrics does your advanced
analytics solution support?
Internal OLA/SLA commitments
95%
External SLAs (with service providers including
cloud)
Response time for multiple transactions to support
a complete business service
Metrics to show end user effectiveness in
interacting with applications
90%
86%
76%
Revenue-specific metrics
71%
Business Activity Metrics (BAM) including revenue
71%
Metrics to show end user application preferences
71%
Metrics to show application usage from a cost
perspective
67%
Business process impact metrics
57%
Industry compliance–related metrics
38%
Operations opex cost savings
38%
Service Desk opex cost savings
33%
Brand impact-related metrics
29%
Supply-chain-related outcomes
29%
Conversions from competitive Web sites
19%
Other (Please specify)
19%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Figure 10: Business Impact-oriented APA can already capture an impressive array of metrics and clearly holds the potential to do more.
• Integrations: EMA looked for integrations with business and financial planning systems, executive
dashboard, security audit or compliance systems, business process management systems, social
media, chargeback and user demand profiling.
• The User Experience Management (UEM) foundation: Understanding user behavior in context
with transactional efficiency was highly weighted as EMA views this as a core foundation for APA
business impact.
18
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
• True business impact means serving both IT and non-IT constituencies: All Value Leaders
in business impact had defined and realized support for business as well as IT stakeholders in
optimizing actual business versus just IT service outcomes. In this EMA Radar, APA business
impact means providing a common ground for IT and business stakeholders to align and optimize
beyond mere service performance.
Which of the following non-IT-related roles do you support natively or as
fully integrated extensions to your advanced analytics?
On-line Operations
95%
Line of Business
91%
Executive non-IT
82%
User Experience Management
77%
Business Development/ Planning
50%
Partner management
27%
Supply chain management
27%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Figure 11: Not surprisingly, online operations and line of business led APA-supported non-IT
stakeholders, but breadth of non-IT support is an already impressive area of growth.
19
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
90%
100%
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
The 22 APA Vendors in Summary
AccelOps
EMA has watched AccelOps evolve since its founding in 2007 with heightened interest as it combines
two key technologies in a unique way – analytics and service modeling – which are key to this report.
AccelOps is also the clear leader in this research in integrated security (SIEM) with cross-domain, serviceaware monitoring and analytics. Moreover, AccelOps is one of the very most cost-effective options
among the 22 vendors surveyed here and is strongly competitive in mid-tier and smaller businesses,
as well as some enterprises and service providers. In multiple interviews with AccelOps deployments,
it has lived up to its promise of “changing the narrative around infrastructure monitoring.” AccelOps
is one of only six vendors to be a Value Leader in more than one use case. Thanks to its combination
of functionality and cost-effectiveness, AccelOps gained the Value Leader category in change impact
and capacity optimization, as well as technical performance analytics. AccelOps is a Strong Value in
business impact management.
AppFirst
In the vendor’s own words, “AppFirst was founded on the principle that software should just work.”
Focused very much on the application ecosystem with more than just an eye to business outcomes,
AppFirst bores deeply into systems-centric awareness of application processes and interactions with
unique benchmark data that help to set it apart and complement its own self-learning heuristics.
AppFirst also profits from having its data (application, infrastructure and business) in a single location
so that its core analytic strengths can be harvested more efficiently, and more dimensionally, than data
lost in isolated monitoring tools or suites. Many of AppFirst’s customers are typically smaller and
mid-tier environments seeking in-depth and efficient ways of optimizing and managing critical business
applications – although the solution can scale easily to much larger enterprise environments. Thanks
to AppFirst’s combination of application performance and business impact information, AppFirst is a
Value Leader in business impact management. The vendor is a Strong Value in technical performance
and change impact and capacity optimization.
Appnomic Systems
Appnomic Systems has focused its sales efforts on mid-tier enterprises, representing about eighty
percent of its customer installed base. AppsOne, the Appnomic product in the APA Radar, analyzes
metrics across three dimensions of application performance, end-user transaction response time,
selected application and underlying infrastructure components, and application usage patterns.
This multi-dimensional approach to monitoring application performance analyzes diverse amounts
of application and infrastructure operations data which is then used to optimize performance and
end user productivity. The usage patterns correlate the other two dimensions to provide a variety of
application stack operations such as: Early Warning Alerts™ of impending application issues, preventive
infrastructure configuration changes, capacity planning and reducing transaction response times.
Because of Appnomic’s innovative approach, fast time to value and strong analytics capabilities EMA
has rated the vendor as one of the Value Leaders in business impact management and a Strong Value in
the two other APA use cases.
20
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
BMC
BMC gained one of the APA crown jewels as far back as 2007 when the vendor acquired ProactiveNet.
This was significantly enhanced by two additional acquisitions – Neptuny for cross-domain capacity
planning analytics in 2010, and Coradiant in April of 2011 for one of the industry’s most versatile
capability in end-user-experience-driven analytics. So not surprisingly, for this APA Radar, BMC
has leveraged its ProactiveNet Performance Management Suite, which includes BMC ProactiveNet
Performance Management (BPPM), BMC Capacity Planning, and BMC End User Experience
Management (EUEM), as well as its BMC Cloud Operations Management capabilities for assimilating
and optimizing unique cloud-related requirements into its overall service management capabilities.
BMC is consistently very strong in functionality and architectural strength in technical performance
analytics and business impact where it is a Strong Value, and a clear Value Leader in change impact and
capacity planning where it received the highest overall functional rating across all twenty-two vendors.
As BMC continues to evolve and further integrate its analytic capabilities, it should further enhance its
position as one of the very strongest APA solution sets in the industry.
Compuware
Compuware Corporation’s dynaTrace and Gomez solutions have made a not only a true leader in
Application Performance Management (APM), but also in Advanced Performance Analytics (APA).
Its suite of solutions is one of the most complete and innovative in APA in the market today. This
includes dynaTrace Enterprise with dynaTrace Deep Transaction Management, Data Center RUM,
enterprise Synthetic Monitoring and APM for Mainframe, which brings distributed and mainframe
transaction-aware analytics together for uniquely cohesive triage and diagnostics. Its Gomez SaaS
offerings leverage one of the industry’s largest global testing environments for mobile computing,
user experience, cloud and cloud ecosystem support and industry benchmarking. Compuware is
one of only six vendors to achieve a Value Leader ranking in more than one use case – specifically
technical performance analytics and business impact management. A Strong Value in change impact
and capacity optimization, Compuware also achieved the award for Best DevOps Support against
some very competitive APA contenders.
eG Innovations
eG Innovations’ diverse customer base is spread across all sized segments of enterprises as well as
service providers, which is a testament to the solution’s flexibility and efficiency. eG Enterprise is the
company’s solution for advanced performance. The product offers performance management solutions
that simplify and accelerate the discovery, diagnosis, and resolution of service performance issues in
physical, virtual and cloud service infrastructures. eG Innovations offers 360-degree service visibility
with automated, virtualization-aware performance correlation across all layers of an IT infrastructure,
from desktops to applications and from network to storage. This unique approach delivers actionable
insights into the root causes of cross-domain service performance issues, enabling administrators to
preemptively detect, diagnose and fix root-cause issues. eG Enterprise has strengths across all three of
the use cases in this EMA Radar, where it is ranked as a Strong Value – although the primary use case
for which customers have deployed the product is performance analytics.
21
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
eMite
eMite’s Service Intelligence Platform (SIP) is powerful and use-case adaptable. Based in Australia, the
company is beginning to expand more aggressively into the United States and establishing some key
industry partnerships, such as with ServiceNow. The eMite Service Intelligence Platform includes
modules for analytics, correlation, capacity, performance, availability and SLA, and has strong modeling
extensions to support business impact outcomes. While it is primarily used in large enterprises, eMite
is also downwardly scalable and efficient enough in cost and ease of administration to support mid-tier
and smaller environments as well. eMite’s predictive capabilities include both advanced historical
analytics through OLAP and real-time predictive analytics. This in part enabled through its dynamically
generated eMite’s OLAP cube and the eMite Metric Warehouse.
eMite’s next-generation architecture and low cost of entry combined to make it one of only two vendors
that were APA Value Leaders in all three categories.
FireScope
FireScope’s solution set: FireScope Analytics, Stratis, Unify and Orchestrate combine to position the
company as one of the stars of this APA Radar. One proof of its FireScope’s analytic strengths is that
its solutions can serve smaller businesses as well as very large enterprises (where it is currently growing
most aggressively) – an endemic value of truly advanced analytic designs. FireScope has also taken the
bull by the horns when it comes to capitalizing on cloud scalability for empowering analytics with
a designed-in flexibility that also support both multi-layered redundancy and secure multi-tenancy.
FireScope featured its cloud-based Stratis and Analytics for this EMA Radar, although use cases could
also draw on its on-premise Unify and its CMDB capabilities in Orchestate. FireScope is only one of
six vendors to receive a Value Leader ranking in more than one use case – specifically business impact
management and technical performance analytics. It is a Strong Value in change impact and capacity
optimization.
Fluke Networks: Visual Network Systems
Visual Network Systems (VNS) is a brand of Fluke Networks, which is part of Danaher Corporation,
a large manufacturer of professional measurement instrumentation products. Its VPM Xpress is an
integrated single appliance solution that measures performance, helps isolate problems, and delivers
with root cause analysis and remediation. VPM Xpress’ acceptance is well distributed across different
customer sizes, among the broadest acceptance in this APA Radar. The product provides in-depth,
cause-and-effect visibility, along with correlated application and network response time analytics that
demonstrate how application performance degradation impacts user experience. VPM Xpress utilizes
a patented workflow analysis engine, IntelliTrace™, to provide problem area identification – network,
server or application. These solid capabilities and very broad technical metric support led EMA to rate
Visual Networks as a Value Leader in technical performance analytics. VPM Xpress is rated as a Strong
Value in change impact and capacity optimization and business impact management.
22
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Hewlett-Packard (HP)
HP is unique in this EMA Radar in its ability to combine two threads – a single analytic overlay
as embodied in its recently introduced Service Health Analyzer (SHA) product, and a broader suite
solution optimized for HP to participate in all three use cases here with maximum functional impact.
This is not entirely a black-and-white situation, as SHA does leverage and currently largely depends on
integration with HP Business Service Management 9.1, which is itself a suite. But SHA can assimilate
other third-party sources, and checks out brilliantly in early-phase deployments in terms of time-tovalue and analytic power. Along with HP’s recently introduced Operational Analytics (OpsAnalytics),
which combines and integrates the correlation power of Operations Manager i (OMi) with the
predictive analytics of HP Service Health Analyzer (SHA) and HP ArcSight Logger – Service Health
Analyzer is the lead reason for HP’s strong Value Leader showing in technical performance analytics.
HP is rated a Strong Value functional leader in business impact management, with its unique crown
jewel – Business Process Insight, and change impact and capacity optimization, which leverages HP’s
Service Health Optimizer (SHO) targeted at optimizing virtualized environments. HP SHA, and SHO
are both a part of HP’s Service Intelligence Suite, which also includes Service Health Reporter, and
Service Level Management.
IBM
IBM’s solution suite for this APA Radar was carefully thought out and delivers one of the most
functionally exciting options across the entire Radar. IBM achieved a Value Leader position for its
diverse and powerful analytics in change impact and capacity optimization. IBM is a Strong Value in
business impact management where it received the highest single functional score thanks in part to its
Cognos and Web Analytics investments. IBM is also ranked as a functionally potent Strong Value in
technical performance analytics. Finally, given its extensive recent investments in APA, IBM received
the award for Vendor to Watch in 2013. Since the Bubble Charts were frozen in November, 2012, IBM
has already moved forward on more aggressive pricing, for instance, across all three use cases.
IBM’s APA solution includes: IBM Netcool OMNIbus, for cross-domain event-driven analytics for
service performance, change impact and compliance; Tivoli Business Service Manager, for optimizing
the health and impact of critical IT-delivered business services; Tivoli Application Dependency and
Discovery Manager, for discovering applications, configurations and interdependencies across the
full application infrastructure; IBM SmartCloud Monitoring, for capacity and optimization analytics
targeted at virtualized infrastructures; IBM Tivoli Monitoring-Tivoli Performance Analyzer SPSS
Module, for selecting the best forecasting module for performance and capacity optimization; Workload
and log file analytics, designed to quickly identify and resolve problems leveraging log files in support
of service performance and business-impact-related outcomes; Netcool Network Analytics, targeted
primarily at telecommunications-related service providers to trend customer experience across time,
geographies and other business and technical contexts; Service performance analytics, for identifying
emerging performance problems based on learning the behavior of the environment and recognizing
when deviations from normal are important.
23
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Interlink Software
Interlink Software’s Automated Service Intelligence (ASI) is only one of two solutions to achieve
Value Leader status in all three use cases. Interlink not only has a well developed set of top-down
and bottoms-up analytics, with strong operational, real-time predictive and historical trending, but
it has a true CMDB with Service Configuration Manager, optimized to service impact, and strongly
proven integrations with most leading third-party vendors. Interlink’s support for nearly 40 third-party
integrations, now being expanded in the field by Web Services, makes it a standout, as well, in its ability
to leverage third-party investments. ASI recently succeeded Interlink’s prior solution, Business Enterprise
Server (BES), and ASI’s strengths in business activity monitoring have been seriously amplified with
integrated, BI-based functions. With ASI, Interlink can now correlate service performance and change
impact issues with business outcomes across a singularly powerful and cohesive analytic matrix, which
Interlink calls its “Impact Calculation Engine” or ICE.
ManageEngine
ManageEngine has a broad portfolio of IT Operations products ranging from performance
management for applications, network and servers to help desk to log file management. ManageEngine
has a customer installed base spread across all sized segments of enterprises as well as service providers.
The two products evaluated for this APA Radar are: ManageEngine Applications Manager, an
application performance monitoring product with out-of-the-box support for 50+ application servers,
servers, databases and transactions spanning physical, virtual and cloud infrastructures and end user
monitoring capabilities; and Site24x7, a Web infrastructure monitoring service that helps to monitor
the uptime and performance of websites, online applications and servers. The monitoring is done from
40+ locations worldwide to provide a global perspective of end-user experience. Based on Site 24x7
and Applications Manager’s exceptional cost effectiveness and well-targeted analytic capabilities, EMA
has made ManageEngine a Value Leader in business impact management, where it competed against
far more expensive and enterprise-centric solutions. ManageEngine is a cost-effective Strong Value in
technical performance analytics and change impact and capacity optimization.
Neebula
Neebula’s ServiceWatch is a true innovator in combining performance-optimized service modeling
with cross-domain analytics in support of all three use cases discussed here. Neebula sells effectively to a
balance of mid-tier and enterprise environments in testament to the versatility and scalability (upward
and downward) of its design. ServiceWatch has proven integrations leveraging its own modeling studio
and Web Services with HP, CA, BMC and ServiceNow CMDBs. The relevance ServiceWatch brings to
service interdependencies across applications and infrastructure, as well as its proven CMDB integrations,
make it particularly strong in change impact optimization. And its capabilities for modeling logical
business outcomes in context with physical service models also makes it strong in business impact
management. Finally, the breadth and facility with which ServiceWatch can receive events from many
multiple monitoring investments give it a solid footprint in technical performance analytics as well.
Neebula is ranked as a well-balanced Value Leader in change impact and capacity optimization, with
Strong Value strengths in technical performance analytics and business impact management.
24
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Netuitive
Netuitive 6.0 heralds a landmark moment not only in Netuitive’s own evolution, but in the evolution
of APA as a whole. This is in large part because Netuitive, arguably more than any other single vendor,
helped to create the APA marketplace more than ten years ago with its self-learning, predictive analytics
that not only capture anomalies, but actually clustered anomalous behaviors as they addressed a defined
business or technical outcome, from transactional latencies to completed reservations. Moreover, the
eclecticism of Netuitive’s design – it is data or source neutral – allows it to freely address issues across
application and infrastructure interdependencies. Over the course of its history Netuitive has been able
to focus variously on WAN optimization, server performance and efficiency, virtualized infrastructure
performance optimization, application service performance, and ultimately business service performance
and outcomes. Not surprisingly, then, Netuitive is a clear Value Leader in technical performance analytics,
and a very Strong Value performer in business impact management and change impact and capacity
optimization. Netuitive also received an award for being the lead APA Market Definer.
OPNET
OPNET is a mid-sized vendor in network and application performance with special strengths in
government, defense, telecommunications, financial services, healthcare and a wide range of other
verticals. In Q4 2012, Riverbed Technology announced its intentions to acquire OPNET in order to
broaden its reach into the application and service management arena.
OPNET is a suite vendor with products that are cost-effective and versatile in scope, and which can play
variously to use-cases and constituencies primarily across IT. Given this breadth of support, OPNET
consistently scored within the top two or three vendors in terms of breadth of coverage, scalability
and overall analytic power – for both technical service performance and change impact and capacity
optimization. The core solution for this report was OPNET’s APM Xpert Suite, which includes which
includes AppResponse Xpert for capturing application flows and user experience across the distributed
infrastructure, and AppMapper Xpert for capturing performance-optimized application-infrastructure
interdependencies, along with several other very strong analytic solutions. OPNET is one of only six
vendors to receive a Value Leader ranking in more than one use case – specifically technical performance
analytics and change impact and capacity optimization. OPNET is rated a Strong Value in business
impact management.
OpTier
OpTier is one of the true trend-setting innovators in the APA arena, with a unique focus on business
impact and performance. OpTier led with its Big Data Analytics Solution and its Advanced Analytics
Module, but it also has strong capabilities for transaction centric application service performance,
optimization and user experience management. No vendor in this APA Radar appreciates the
dual-sided mirror of APA, in which service performance and business impact come together through
transactions, more than OpTier. And no other vendor has equaled OpTier in tuning its solution to
mining this promising, and critical, combination of insights. OpTier APM 5.0 is popular in both
mid-tier and enterprise markets, and it is optimized to support environments where IT services and
business performance are closely linked. OpTier APM is also designed to support complex transactional
interdependencies, where in some of its accounts, a single business transaction may actually depend on
the performance of as many as fifty different applications.
25
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
OpTier is a clear Value Leader in business impact management in this APA Radar, with one of the two
very highest functional scores – and with a clear lead across this Radar in business impact effectiveness
(functionality plus efficiency), as well as being a solid Strong Value in both technical performance
analytics and change impact and capacity optimization.
Prelert
In terms of raw potential, Prelert may well be the ultimate powerhouse in a pack of twenty-two already
powerful APA vendors. The vendor describes its solution as “Third Generation Machine Intelligence,”
to distinguish it from less automated and self-adaptive machine-learning capabilities. Prelert analytics
can assimilate millions of metrics across sources as diverse as logs, events/traps, SNMP polls, usage data,
performance (time series) data, transactions, or any other type of time stamped data. It offers continual
evaluation of its assimilated data in order to catch unobvious or complex patterns that may not surface
otherwise. Focused on large enterprises initially, Prelert is also one of the very most cost-effective vendors
in this APA Radar, with low cost of entry and fast time to value. Prelert is a balanced Value Leader
in technical performance analytics and a Strong Value in change impact and capacity optimization,
where the vendor expects do more in the future, as well as business impact management, which is not
currently a top-tier priority for Prelert.
Quest Software
Quest Software is a large software company acquired by Dell Corporation in September 2012.
Traditionally the company sold primarily to small and medium businesses but in recent years has
made a concerted effort to sell to larger companies. Foglight is the company’s solution for application
monitoring across multiple technologies (e.g. Java, .NET, virtual or physical servers, databases, networks,
etc.) and measuring end user experience interacting with those applications. Foglight also aids users in
finding the root cause of business-impacting incidents, to guide users towards problem identification
and resolution quickly. Powered in part by its leading-edge capabilities to capture user behaviors via
its User Experience Monitoring and Replay, Foglight now supports business-related metrics as well as
integrations with business and financial planning systems, so it can perform analysis on these metrics
as well as traditional IT measurements and KPIs. EMA has rated Quest a Value Leader in the business
impact management category, with balanced and functionally strong positioning as Strong Value in
technical performance analytics as well as change impact and capacity optimization.
SevOne
SevOne is one of only two of the twenty-two companies in this EMA Radar that use an appliance-based
approach. The vendor has created a proprietary, next-generation distributed technology, called the
SevOne Cluster™, that utilizes both advanced technologies of peer-to-peer sharing and big data clusters
to offer excellent scalability for huge numbers of network elements, across all monitoring technologies.
These elements can be monitored and displayed in a single view to the operator.
Historically, the company has established itself as a product provider in very large customer deployments
as a result of the solution’s significant scalability. Large enterprises, defined as customers with greater
than 10,000 employees, now represent a major part of SevOne’s installed base. These large organizations
include verticals such as service providers that rely on SevOne technology to be able to scale to millions
of elements, and billions of baseline analytics, and financial and banking companies that need immediate
26
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
and instant visibility across their large networks. SeveOne is a Strong Value with very high functionality
in Technical Performance Analytics and Change Impact and Capacity Optimization. As it has not yet
explored much in the way of business impact management as evaluated here, SevOne was at the lower
end of Strong Value there. SevOne also received the award for Best Infrastructure-Optimized APA
where its power and scalability clearly positioned it in a position of dominance.
Splunk
Splunk has garnered a great deal of attention as one of the primary companies involved in Big Data,
in part due to its recent Initial Public Stock Offering. Splunk is an engine for analyzing machine data
in files such as logs, performance metrics and events from application components. Splunk customers
find relationships and information for troubleshooting, trending and analytics. Splunk uses a powerful
search language to help customers extract information from both real-time and historical machine data,
correlate it across various data types (both structured and unstructured data), study transactions across
multiple application components and trend critical operational parameters. Sophisticated statistical
and reporting commands let users detect anomalies, eliminate noise and update transaction counts and
durations. These strong functions and the ability of Splunk to scale up or down have led to a fairly even
distribution of customer sizes among the Splunk installed base.
Splunk is data-source agnostic in the sense that its capabilities can be used to analyze almost any type
of data from IT performance and event data to complex business event processing and other business
categories such as revenue. As such Slunk supports a wide range of business metrics including revenue,
business activity, business process outcomes and business service responsiveness. EMA has rated it
among the Value Leaders in the business impact analytics category with solid Strong Value capabilities
in technical performance analytics and change impact and capacity optimization.
Zyrion
Zyrion’s Traverse has established itself for its combination of innovation, scalability, breadth of domain
support, cohesiveness and its pragmatic approach to service-centric analytics. Its versatility is reflected
in its diverse customer base, scaling upwards from smaller and mid-tier businesses with a growing
emphasis on enterprise-scale, MSP and service provider deployments, with strong support for multitenancy. Traverse combines strong capabilities for cost-effectiveness and time to value with a rich set of
change impact and capacity-related options, where it ranks as a Value Leader. Traverse also combines
a value-based investment with a broad range of triage capabilities across: applications, systems and
virtualized systems, networks, databases and application tiers including isolating middleware issues
– a Strong Value garnering a near Value Leader ranking in technical performance analytics. As a
Strong Value in business impact management, Traverse leverages its service containers to enhance
its capabilities to measure business service response times, as well as SLAs, OLAs, business process
impact, business activity, and application utilization. Because of its well-tuned strengths, Zyrion was
awarded Most Balanced APA Performer.
27
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Awards
AccelOps – Best Integrated Security
AccelOps has a centralized monitoring and analytic foundation that draws in balanced fashion to support
SOC and NOC requirements. This has placed it squarely ahead of all the other 21 vendors in this APA
Radar for integrated security analytics. Moreover, AccelOps’ powerful cross-domain capabilities for
discovering and modeling infrastructure and application-to-infrastructure interdependencies resonates
well across both use cases, so that the impacts of changes are readily captured and applied to both SIEM
and service performance issues.
Compuware – Best DevOps Support
The combination of dynaTrace, with is strengths in code-level and data-center analytics, Gomez with its
rich support for mobile, Web-2.0, and geographically relevant environmental testing, and Compuware’s
leadership in User Experience Management (UEM), established Compuware as the uncontested leader
in lifecycle application and DevOps support.
IBM – Vendor to Watch in 2013
Four out of the eight products included in IBM’s APA suite are new, one in limited availability and one
still in field-test-level deployments at the time of writing this report. This may well represent the single
largest collective vendor investment in APA in the industry bar none. EMA expects that a year from
now, as these solutions have a chance to mature, IBM’s APA strengths will be even more dramatically
enhanced – making it a “vendor to watch” in 2013.
Netuitive – APA Market Definer
More than any other vendor in this EMA Radar, Netuitive has helped to define the APA market with
its versatile capabilities for analytics, long before the term “analytics” or even “big data” got much
play. Netuitive’s leadership continues to show in its solid enterprise and service provider adoption, the
maturity and power of its visualization to harvest the raw power of its analytic strengths, and its focus
on broadening its integration base.
SevOne – Best Infrastructure-optimized Analytics
SevOne’s peer-to-peer architecture and innovative approach placed them in the lead in the infrastructureoptimized analytics. EMA was very impressed with SevOne customer enthusiasm directed at the
power, performance, and ease of deployment of SevOne Cluster. SevOne is one of the true analytic
powerhouses in this EMA research.
Zyrion – Most Balanced APA Performer
Zyrion’s Traverse garnered the Most Balanced Performer award with near Value Leader placements in
performance and business impact to match its Value Leader position in change impact and capacity
optimization. “Balance” is also an apt word to describe Traverse’s overall design, which has evolved
from more than a decade of industry innovation and real-world deployments to support a wide variety
of customer-driven use-cases.
28
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Conclusion
Given the diversity across the many vendors included in this report, it may be useful to recall how strongly
these twenty-two vendors do share some critical common ground. For instance, nineteen of the twenty-two
vendor solutions can “learn” their environments dynamically, once installed, on average three to four days
before bringing dependable values. In addition, twenty of the twenty-two vendors support predictive
capabilities. These tend to skew from technical performance management to capacity optimization, to
business management to change impact – in that order. Moreover, as shown in Figure 12 – most vendors
could deliver some balance between trended and real-time operational predictive values.
How are your predictive capabilities manifested?
Advanced trending analytics through our own
solution (e.g. OLAP/data mining)
76%
Advanced trending analytics through supported
integrations (e.g. OLAP/data mining)
24%
Real-time operational predictive analytics short
term (up to two hours out)
71%
Real-time operational predictive analytics longer
term (beyond two hours out)
43%
0%
10%
20%
30%
40%
50%
60%
70%
80%
% Valid Cases (Mentions / Valid Cases)
Figure 12: APA predictive capabilities show a balance of trending and operational real-time values.
Automation is also critical to APA and a cornerstone of many of the vendor offerings cited above.
Logically, analytics should enable faster and more automated decision making and so enable more
automated actions ranging from basic trouble ticketing, to more complex diagnostics, to active
configuration and change-related remediation – among the top three (Figure 13).
29
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
What types of automation capabilities do you support?
Service desk trouble ticketing
90%
Automated diagnostics
67%
Change and configuration management tools
52%
Service desk workflows
48%
ITPA or Run-book automation tools
38%
ITIL Process automation
33%
Other Process Automation
24%
Load balancing
19%
Other (Please specify)
19%
Workload automation tools
14%
Automated security compliance actions
5%
WAN optimization
5%
0%
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
% Valid Cases (Mentions / Valid Cases)
Figure 13: Integrated automation is a key growth area for APA. Not surprisingly, service desk trouble ticketing and
diagnostics lead automation, but change and configuration is supported by more than half the APA vendors as well.
One could argue that little in IT service management technologies – or little anywhere else – could
challenge APA’s preeminence in predictive, especially real-time operational predictive, power, and
therefore its primary role in enabling more pervasive and “intelligent” forms of automation. And yet,
in doing this Radar, EMA was clearly taking a chance. No other analyst firm or industry watcher has
defined APA as a category in and of itself – in part because it is multi-faceted and tends to cross more
established, conventionally defined markets.
EMA believes, however, that to ignore APA just because it is more polymorphic than traditional ways
of looking at IT-related technologies is a grave mistake. APA holds the potential not only to reshape
conventional service management boundaries, it should also help to pave the way for changes in how
IT works, including IT processes, governance, and IT-to-business alignment. As it matures APA may
indeed help to reshape the role of IT towards a more value-driven versus cost-driven organization, and
may actually help businesses themselves redefine how to function and optimize in a world in which
business services and IT services are increasingly one and the same.
30
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
EMA Radar™ for Advanced Performance Analytics
(APA) Use Cases: Q4 2012
Appendix A: Additional APA Vendors
While this EMA Radar for Advanced Performance Analytics is surprisingly complete, there really
aren’t that many more APA vendors out there, at least in Q4 2012. For various reasons a few vendors
were not included that are nevertheless still worthy of APA-related consideration. This is, itself, not
meant to be definitive, but will add a few insights into other types of innovations and innovators. In
alphabetical order, these are:
1. Accretive Technologies, Inc. – Accretive Technologies bridges traditional approaches to
BI and Big Data with a clearly defined vision of the more dynamic linkages between IT
services and business outcomes-- with an eye to business transformation. Accretive modeling,
simulation and predictive technologies allow for the discovery of unknowns and help to
make “what if ” analysis affordable. The technology maps data center architecture and IT
service performance interdependencies to critical business systems, such as, for instance, a
critical in-house payment system or customer-facing credit-approval system. These in turn are
analyzed in context with broader operational and business efficiencies.
2. ExtraHop Networks – ExtraHop is an operational intelligence platform for business-critical
transaction environments. In the vendor’s own words, ExtraHop “offers a simple, non-invasive
way to provide holistic cross-tier correlation of network, applications, infrastructure, and
transactions.” In addition to auto-discovery and auto-classification, ExtraHop analyzes
the payload for all application transactions in real time, and integrates with external Syslog
solutions like Splunk for additional analytics and long-term trending.” In this way ExtraHop,
can, for instance, isolate high-priority anomalies and performance outliers according to criteria
such as order ID, merchant ID, and transaction ID without the need of agents, probes, or
modifying applications.
3. Nastel Technologies – Nastel’s AutoPilot delivers big data performance and compliance
analytics across application/middleware ecosystems with real-time and predictive value.
AutoPilot can also link IT and business KPIs with dynamic awareness of business impact
through the detailed analysis of Web service, message and transaction payloads. Nastel
recently announced a partnership with CA for reselling Nastel’s middleware monitoring and
analysis solutions.
4. NEC Invariant Analyzer – The NEC Invariant Analyzer is a new and pure-play analytic
solution designed for predictive and self-learning capabilities in discovering and capturing
anomalies across KPI interdependencies – what the vendor calls “Invariant Relationship
Monitoring.” It currently provides closed-loop integration with CA Service Assurance for
application performance and business impact.
5. TeamQuest – TeamQuest software delivers advanced analytics in support of what the
vendor calls “single-pane capacity management” targeted at cross-domain support for
applications. TeamQuest offers predictive analytic insights and assimilates data from a
wide-range of third-party monitoring tools. TeamQuest also has extended support for
storage and virtualized infrastructure.
31
©2012 Enterprise Management Associates, Inc. All Rights Reserved.
About Enterprise Management Associates, Inc.
Founded in 1996, Enterprise Management Associates (EMA) is a leading industry analyst firm that provides deep insight across the full
spectrum of IT and data management technologies. EMA analysts leverage a unique combination of practical experience, insight into
industry best practices, and in-depth knowledge of current and planned vendor solutions to help its clients achieve their goals. Learn
more about EMA research, analysis, and consulting services for enterprise line of business users, IT professionals and IT vendors at
www.enterprisemanagement.com or blogs.enterprisemanagement.com. You can also follow EMA on Twitter or Facebook.
This report in whole or in part may not be duplicated, reproduced, stored in a retrieval system or retransmitted without prior written permission of
Enterprise Management Associates, Inc. All opinions and estimates herein constitute our judgement as of this date and are subject to change without notice.
Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. “EMA” and “Enterprise Management
Associates” are trademarks of Enterprise Management Associates, Inc. in the United States and other countries.
©2012 Enterprise Management Associates, Inc. All Rights Reserved. EMA™, ENTERPRISE MANAGEMENT ASSOCIATES®, and the mobius symbol
are registered trademarks or common-law trademarks of Enterprise Management Associates, Inc.
Corporate Headquarters:
1995 North 57th Court, Suite 120
Boulder, CO 80301
Phone: +1 303.543.9500
Fax: +1 303.543.7687
www.enterprisemanagement.com
2584.121012