Embedding Analytics for Competitive Advantage Three Market‐Driven Approaches for Software Development Companies A White Paper by Visual Numerics, Inc. July 2008 Visual Numerics, Inc. 2500 Wilcrest Drive, Suite 200 Houston, TX 77042 USA www.vni.com
Embedding Analytics for Competitive Advantage Three Market‐Driven Approaches for Software Development Companies by Visual Numerics, Inc. © 2008 by Visual Numerics, Inc. All Rights Reserved Printed in the United States of America Publishing History: July 2008 Trademark Information
Visual Numerics, IMSL and PV-WAVE are registered trademarks. JMSL, TS-WAVE, and JWAVE are trademarks of
Visual Numerics, Inc., in the U.S. and other countries. All other product and company names are trademarks or
registered trademarks of their respective owners.
The information contained in this document is subject to change without notice. Visual Numerics, Inc. makes no
warranty of any kind with regard to this material, included, but not limited to, the implied warranties of merchantability
and fitness for a particular purpose. Visual Numerics, Inc, shall not be liable for errors contained herein or for
incidental, consequential, or other indirect damages in connection with the furnishing, performance, or use of this
material.
TABLE OF CONTENTS Embedding Analytics for Competitive Advantage ...................................... 4
Analytical Building Blocks ........................................................................ 4
Alternatives for Embedding Analytics......................................................... 5
In‐House Analytics ................................................................................... 5
Open Source Analytics ............................................................................. 6
Commercial Options ................................................................................ 6
IMSL® Numerical Library Introduction ........................................................ 7
Case Studies ................................................................................................ 8
Solving Specific Numerical Challenges..................................................... 9
Putting Analytics in the Hands of Users................................................... 9
Creating a Custom Solution ................................................................... 10
Other Approaches..................................................................................... 10
Summary ................................................................................................... 10
Embedding Analytics for Competitive Advantage As customer demand for advanced analytics grows, enterprise software vendors are finding ways to add new business intelligence (BI) capabilities to their current offerings to stay competitive. With embedded analytics, vendors can empower their users to do more with their applications, enhancing the “stickiness” of their software. For example, nCode International1, a developer of data analysis software, data acquisition systems and durability solutions, recently added Visual Numerics statistical tools to their ICE‐flow Glyphworks software. These additional tools enable customers to analyze and understand durability testing and product usage data from within nCode’s solutions, as opposed to moving the data to a separate application for analysis. For businesses, having more analytics embedded in the software solutions that their employees use on a daily basis delivers significant value. Employees get the decision‐making tools they need to make good business decisions, when they need them. Organizations that have employees making better decisions faster gain a significant competitive advantage over organizations without similar abilities. This white paper is intended for software vendors that want to make analytics an embedded part of their products. It will discuss analytical building blocks, alternatives for building solutions, and through case study examples, it will outline three approaches taken by vendors to embed analytics into their software applications. Analytical Building Blocks In a cover story discussing analytics in business, “Math will Rock Your World2” (Business Week, January 2006), the author writes about “imposing the efficiencies of math” on finance, science, engineering and business problems. This article and others that followed outlined the many ways organizations and industries could leverage numerical analysis to solve problems and make decisions analytically. As discussed in the Business Week article, the building blocks for analytical applications are mathematical and statistical techniques. Math functions such as interpolation and optimization, and statistics functions such as correlation, regression or random number generation help organizations understand data, simulate situations by asking “what‐if” questions, and make fact‐based decisions. A key decision for software vendors that want to make analytics part of their application is how to embed the mathematics and statistics that enables analytics. Three alternatives are discussed in the next section. 1
2
http://www.ncode.com/productDetails.aspx?display=details&product_id=67 http://www.businessweek.com/magazine/content/06_04/b3968001.htm Page 4 Alternatives for Embedding Analytics Software vendors have many alternatives for adding analytic functionality to their applications. Three commonly considered alternatives are: • Build math and/or statistical functions for their application using in‐house development resources • Use “free” open source options • Leverage a commercial option The following sections will explore some advantages and disadvantages of each. In­House Analytics Many software vendors first consider developing analytics functionality in‐house. Because of the nature of their business, software vendors employ intelligent programmers who can write the code for math and statistical algorithms. Publications such as the Numerical Recipes3 reference books contain complete commented source code in several languages for hundreds of algorithms. Leveraging Numerical Recipes or a similar source of algorithms can be as easy as copying the source into an application and compiling the code. Once complete, the in‐house developed algorithm is owned by the software vendor and is an embedded part of the application. For novel or cutting‐edge algorithms, the development process is considerably more in‐depth than simply consulting existing references of source code. Working from a journal article outlining the mathematics involved in an analytical solution can be a considerable challenge for even seasoned software developers if the goal is to create commercially viable and usable general source code. In addition, such development is typically serial in nature meaning that adding more resources will not shorten the development cycle. If time‐to‐market is a consideration, then the time required to develop in‐house solutions may be a costly drawback. While in‐house development may seem easy, there are both direct and ongoing costs to consider. There is the initial, direct cost of the development time to write, test and document the algorithm. For simple algorithms, this might be a quick process. For complex algorithms or custom solutions, development can take much longer – months or even years in some cases. In addition, there are ongoing costs of maintaining algorithms, including upgrading code for new hardware or software platforms, potentially migrating code to new environments, and making updates as necessary. Maintenance costs can easily run twice the cost and time of initial development. Aside from development costs, an important consideration when weighing whether to develop analytical algorithms in‐house is how to best deploy resources. Asking the question “should our internal development staff be used to write math or statistical functions or should they work on 3
http://www.nr.com/ Page 5 our core application” can help a software development company decide if it is better to create in‐house analytics or leverage open source or a commercially available option. Open Source Analytics To avoid the cost of writing and supporting algorithms developed in‐house, software vendors might look to one of the thousands of open source options, such as the GNU Scientific Library4. The commonly cited attraction of open source is, of course, that it is free and supported by the general development community. For software vendors looking to add analytics to their applications, open source has potential drawbacks. Since many open source options come with third‐party obligations or viral type license agreements, some customers will simply not accept software that contains open source code to avoid dealing with multiple companies and organizations for support. Additionally, utilizing open source codes in a commercial application may require the entire application’s source to be made available, which is not an option for many software vendors. For developers, not all open source projects are easy‐to‐use. Many are sparsely documented, occasionally in foreign languages, and contain few, if any, examples. Other projects are available only in source code form, which means the onus is on the user to compile necessary binaries. When an open source project designed for Linux is required on Windows, this compiling step can be a daunting task. Projects rarely have resources available to address technical support questions. Users are left to email busy developers or visit unresponsive forums. Finally, open source projects often stagnate after initial activity. Once the original developers get it working for their specific problem, there is no impetus for continuing development. While there are many examples of successful open source projects, software vendors should consider the risks before pursuing open source options for their applications. Commercial Options In computer science, a library is a collection of subprograms used to develop software. Libraries contain "helper" code and data that provide services to independent programs. Visual Numerics is one of several vendors that offer a commercial library option for embedding analytics. Visual Numerics IMSL Numerical Libraries5 provide numerical computation code in the form of math and statistical algorithms. The intent of a commercial option like the IMSL Libraries is to empower software developers to add analytical capabilities to their application, but avoid re‐inventing the wheel through writing complex math or statistical routines, or introducing risk by using untested open source software in their application. 4
5
http://www.gnu.org/software/gsl/ http://www.vni.com/products/imsl/ Page 6 A commercial option also offers extensive numerical algorithm development expertise. Visual Numerics focuses on the development, sales and maintenance of analytical algorithms, supporting a wide range of languages and platforms, and has years of experience working with thousands of customers. A commercial option like the IMSL Libraries therefore provides a wide breadth of available algorithms, significant depth of capabilities, and extensive testing by the vendor and a large and diverse user community. Table 1 summarizes the key advantages and disadvantages of these alternatives for embedding analytics into a software application. Table 1. Embedded Analytic Alternatives Summary In‐house analytics ƒ
ƒ
Open source analytics ƒ
Commercial options ƒ
Key Advantages Source code examples are often available Analytic functionality developed is owned by the software vendor Many algorithms are freely available ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
Depth and breadth of ƒ
available algorithms Reduce development time through the use of pre‐built algorithms Reduce risk through commercially tested and widely used algorithms Key Disadvantages Time to code, test, document and maintain algorithms Opportunity cost of developers writing algorithms instead of core software application Third‐party obligations and viral‐type license agreements are common Documentation, testing and support are often minimal Cost to license IMSL® Numerical Library Introduction The IMSL Libraries are a collection of mathematical and statistical algorithms that developers can integrate into their applications. IMSL Libraries cover a wide range of functions ranging from linear algebra and curve fitting to numerical optimization in the math domain and time series analysis and data mining functions in the statistical domain. IMSL Numerical Libraries are implemented in the widely used computer programming languages of C, Java, C#/.NET, and Fortran. Software developers embed algorithms from these Page 7 libraries into their software applications using their preferred programming language. The first IMSL Library for the Fortran language was released in 1971, followed by a C language version originally called C/Base in 1991, a Java language version in 2002 and most recently a C# language version in 2004. Over time, the libraries have grown in supported languages, supported platforms, and in depth and breadth of available math and statistics algorithms. Table 2 outlines general math and statistics areas found in the IMSL Numerical Libraries. These areas illustrate a broad and full‐featured spectrum of critical analytic capabilities that can be used to build advanced numerical software applications in many fields. Table 2. IMSL Numerical Library Math & Statistics Areas ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
Mathematics Matrix Operations Linear Algebra Eigensystems Interpolation & Approximation Quadrature Differential Equations Nonlinear Equations Optimization Special Functions Finance & Bond Calculations ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
Statistics Basic Statistics Time Series & Forecasting Nonparametric Tests Correlation & Covariance Data Mining Regression Analysis of Variance Transforms Goodness of Fit Distribution Functions Random Number Generation Neural Networks The items listed in the table represent entire areas of functionality with numerous algorithms within each area. Within the IMSL Libraries, the actual count of available math and statistical algorithms runs into the thousands, giving developers many options to mix and match algorithms as needed to create unique and competitively engineered analytical applications. How to use numerical algorithms in a software application depends on the type of application, the needs of the users of the application, and the desires of the application vendor. Some software vendors might add a few algorithms that provide specific functionality; others might want to offer a broad selection of numerical analysis functions; while others might need highly customized functionality. The next section outlines three recent approaches by software vendors to embed analytics into their software applications. Case Studies The following sections highlight examples of customers that have embedded IMSL Library functions into their software applications. Page 8 Solving Specific Numerical Challenges Many companies need analytics to enable their software application to solve a particular type of problem. An example is Moore Nanotechnology Systems6, a leading manufacturer of ultra‐
precision machining systems. Customers in defense, aerospace, consumer electronics, and ophthalmic industries use Moore Nanotech’s ultra‐precision machining equipment to create 3D freeform surfaces such as optical lenses, mirrors and inserts. With their machining equipment, Moore provides NanoCAM, a full 3D CAM software that enables customers to define the precise surface to be machined using their ultra‐precision tools. IMSL Library functions are embedded into the NanoCAM application. Math algorithms, including spline definition and interpolation functions, enable the accurate definition of 3D freeform surfaces. Accurate definition is critical for ensuring quality output from the manufacturing process – typically in nanometer range. The IMSL Library functions embedded into NanoCAM also provide the fast computation capability required by the application. Manufacturers who need to be completely precise with the surfaces commonly interpolate point data sets ranging from 300,000 to 1,000,000 points. The ability of IMSL Library functions to process these large datasets in a fraction of a second speeds up the design and analysis process. For Moore Nanotech developers, in‐depth analysis functions from the IMSL Libraries were easily embedded in the application, saving months of development time versus trying to write the functions from scratch. Putting Analytics in the Hands of Users For many software applications, vendors can deliver value by providing a complete set of analysis tools for customers. Customers can then use these tools in unique ways on their own data and make business decisions to create competitive advantages. An example is Teradata7, a provider of enterprise data warehousing and business intelligence solutions. To provide users with a wide range of embedded statistical functions for in‐database analysis, Teradata embedded multiple IMSL Library probability distribution functions and inverses and random number generators into a new product, the Teradata Statistical Library Add‐in, Powered by IMSL, part of Teradata Warehouse Miner. These statistical functions are provided as embedded user defined functions (UDFs) accessible through Teradata Warehouse Miner or standard SQL Queries. Teradata Warehouse Miner’s UDF capabilities work like an index from which users can select prepackaged functions which are linked to the detailed data. From a desktop environment, business users can click, drag and drop the needed analytic function from a list of UDFs and run it against the appropriate detailed data in the warehouse without wasting time by manually coding the function or moving data between systems. Combined with other analytic functions in Teradata Warehouse 6
7
http://www.nanotechsys.com/ http://www.teradata.com/ Page 9 Miner, the Statistical Library functions, Powered by IMSL can be used in a variety of applications, including classification, forecasting and others. Creating a Custom Solution Some software applications can benefit from customized analytics. An example is Teksouth8, a privately held firm headquartered in Birmingham, Alabama. The company has served both the commercial markets and Federal government for more than a generation. Teksouth products and services include data warehouse development, implementation, and support; web‐based decision support tools; secure high‐speed data transport tools, as well as system engineering, integration, and program management. To enable private sector and government organizations to more accurately project and budget spending changes, and help companies address accountability and regulatory compliance issues, Teksouth, working with Visual Numerics Consulting Services, developed an innovative, customized neural network forecasting application for its customers based on neural network forecasting functionality in the IMSL Library. The application extracts information from historical cost data, applying the information to forecast future costs. The analytic engine has the flexibility to prioritize historical data and exclude anomalous data to improve its predictive capabilities. One of the most powerful features of this solution is its ability to mimic the brain's own problem‐solving process by applying knowledge gained from historical data to new problems, fine‐tuning its forecasting accuracy even more over time. Many organizations evolve their predictive analytic capabilities over time. Visual Numerics Consulting Services can help organizations like Teksouth get to the next stage of analytical maturity by identifying and then helping to implement methods to bring more sophisticated analysis into the organization. Other Approaches The previous section highlighted three examples of how software vendors can embed advanced analytics into their applications. With more than 3000 functions across four native development languages on dozens of hardware platforms and operating systems, the ways that advanced analytics with the IMSL Numerical Libraries can be embedded into software are virtually limitless. Summary Software vendors can help their customers solve problems and make decisions analytically by embedding analytic functionality into their applications. While there are alternatives to embedding commercially‐developed and supported analytical functions: building in‐house or 8
http://www.teksouth.com Page 10 leveraging open source; commercial libraries deliver the optimum approach for many vendors because of their: •
•
•
•
Breadth and depth of numerical algorithms Broad platform and language support Proven reliability Technical and consulting support Analytics can be embedded in many ways. Depending on the software vendor, application and customer needs, the best approach will vary. Regardless of approach, the end goal remains the same – to give customers the ability to create competitive advantage by making decisions analytically. Visual Numerics is an established provider of commercial libraries. Math and statistical functions in the IMSL Libraries provide analytic capabilities to many software vendors today and offer unique value, including: ƒ
Fast time to market – Software companies like nCode International and Moore Nanotechnology brought products to market significantly faster using pre‐built IMSL Libraries as opposed to developing functionality in‐house. ƒ
Execution & Efficiency – Analyzing data at the source is often more efficient than moving it outside the database. Teradata provides IMSL Library statistical functions to users as embedded user defined functions (UDFs) accessible through Teradata Warehouse Miner or standard SQL Queries. IMSL Libraries, available in C, Fortran, .NET and Java make it easy to embed analytics in the same native language in which an application is written. ƒ
Customization – Working with Visual Numerics Consulting Services, Teksouth was able to customize advanced neural network technology to develop a unique budget forecasting application. The ability to customize algorithms or simply mix and match algorithms from the thousands available, IMSL Libraries enable developers to create unique and competitively engineered analytical applications. Finally, IMSL Libraries ensure numerical consistency across different languages and platforms. Developers do not have to hard code libraries to fit each application environment. This advantage is particularly helpful when software companies need to upgrade or migrate their applications to new hardware or software platforms. To learn more: •
•
Visual Numerics IMSL Numerical Libraries9 product information Visual Numerics OEM/ISV partner10 information 9
http://www.vni.com/products/imsl/index.php http://www.vni.com/company/partners/oemIsvPartners.php 10
Page 11