W-277
ONLINE FILE W14.6
ADVANCED METHODS
FOR
EVALUATING IT INVESTMENTS
The value analysis method evaluates intangible benefits on a low-cost, trial basis
before deciding whether to commit to a larger investment in a complete system.
Keen (1981) developed the value analysis method to assist organizations considering investments in decision support systems (DSSs). The major problem with justifying a DSS is that most of the benefits are intangible and not readily convertible
into monetary values. Some—such as better decisions, better understanding of business situations, and improved communication—are difficult to measure even in nonmonetary terms. These problems in evaluating DSSs are similar to the problems in
evaluating intangible benefits for other types of systems. Therefore, value analysis
could be applicable to other types of IT investments in which a large proportion of
the added value derives from intangible benefits.
The value analysis approach includes eight steps, grouped into two phases. As
illustrated in Figure W14.6.1, the first phase (first four steps) works with a low-cost
prototype. Depending on the initial results, this prototype is followed by a full-scale
system in the second phase.
In the first phase the decision maker identifies the desired capabilities and the
(generally intangible) potential benefits. The developers estimate the cost of providing the capabilities; if the decision maker feels the benefits are worth this cost, a
small-scale prototype of the DSS (or other IT application) is constructed. The prototype then is evaluated.
The results of the first phase provide information that helps with the decision
about the second phase. After using the prototype, the user has a better understanding of the value of the benefits, and of the additional features the full-scale system
needs to include. In addition, the developers can make a better estimate of the cost
of the final product. The question at this point is: What benefits are necessary to justify this cost? If the decision maker feels that the system can provide these benefits,
development proceeds on the full-scale system.
Though it was designed for DSSs, the value analysis approach is applicable to
any information technology that can be tested on a low-cost basis before deciding
whether to make a full investment. The current trend of buying rather than developing software, along with the increasingly common practice of offering software on
a free-trial basis for 30 to 90 days, provide ample opportunities for the use of this
approach. Organizations may also have opportunities to pilot the use of new systems in specific operating units, and then to implement them on a full-scale basis if
the initial results are favorable. For further discussion see Fine et al. (2002).
VALUE ANALYSIS
Phase 1
1
Identify value
(intangible benefits)
2
3
Establish maximum
cost willing to pay
4
Build prototype if
cost is acceptable
Evaluate
prototype
Phase 2
8
Enhance functionality
of full-scale system
7
Build full-scale
system if benefits
justify it
6
Identify benefits
required to
justify cost
Figure W14.6.1 Steps in the value analysis approach.
5
Establish cost of
full-scale system
W-278
INFORMATION
ECONOMICS
The information economics approach is similar to the concept of critical success factors in that it focuses on key organizational objectives, including intangible financial
benefits, impacts on the business domain, and impact on IT itself. Each area has several components (see Figure W14.6.2). In addition, more areas can be added (see
McKay and Marshall, 2004). Information economics incorporates the familiar technique of scoring methodologies, which are used in many evaluation situations.
A scoring methodology evaluates alternatives by assigning weights and scores
to various aspects and then calculating the weighted totals. The analyst first identifies all the key performance issues and assigns a weight to each one. Each alternative in the evaluation receives a score on each factor, usually between zero and
100 points, or between zero and 10. These scores are multiplied by the weighting
factors and then totaled. The alternative with the highest score is judged the best (or
projects can be ranked, as in the R.O. Iowa case at the beginning of the chapter).
Then one can perform sensitivity analysis, to see the impact of changing weights.
A Closer Look W14.6.1 shows an example of using a scoring methodology to evaluate two different alternatives.
The information economics approach uses organizational objectives to determine which factors to include, and what weights to assign, in the scoring methodology. The approach is flexible enough to include factors in the analysis such as
impacts on customers and suppliers (the value chain). Executives in an organization
determine the relevant objectives and weights at a given point in time, subject to
revision if there are changes in the environment. These factors and weights are then
used to evaluate IT alternatives; the highest scores go to the items that have the
greatest potential to improve organizational performance.
Note that this approach can incorporate both tangible and intangible benefits.
If there is a strong connection between a benefit of IT investment (such as quicker
decision making) and an organizational objective (such as faster product development), the benefit will influence the final score even if it does not have a monetary
value. Thus the information economics model helps solve the problem of assessing
intangible benefits by linking the evaluation of these benefits to the factors that are
most important to organizational performance.
Approaches like this are very flexible. The analyst can vary the weights over
time; for example, tangible benefits might receive heavier weights at times when
earnings are weak. The approach can also take risk into account, by using negative
weights for factors that reduce the probability of obtaining the benefits. Information
Traditional cost-benefits
Value-linking
Value-acceleration
Value-restructuring
Figure W14.6.2
Information economics.
(Source: Willcocks, 1994,
p. 376.)
V
A
L =
U
E
of
IT
Innovation
Enhanced
ROI
+
Business domain
assessment
+
Technology domain
assessment
Strategic match
Strategic IS architecture
Competitive advantage
Definitional uncertainty
Management information
Technical uncertainty
Competitive response
IS infrastructure risk
Organization risk
W-279
A Closer Look
W14.6.1
A Scoring Worksheet for Evaluation of Alternatives A versus B
Decision
Participant
Most
Interested
in Criteria
CEO
CEO
CFO
CFO
VP, Human
Resources
(HR)
CIO
CIO
Criteria
VP-HR
VP-HR
Dir.
Employee
Relations (ER)
VP-HR
Dir.-ER
2
3
4
3
2
8
12
16
12
4
2
5
2
3
1
8
20
8
12
2
Manage risk of organizational resistance to change.
Manage risk of project failure.
2
2
–1
–1
–2
–2
–3
–2
–6
–4
22
48
2
2
3
3
4
4
6
6
40
Increase earnings per share.
Improve cash flow.
Close books faster.
Expand profitability by better product line reporting.
2
2
2
2
Total Finance
8
Improve employee productivity.
Attract, retain high-quality employees.
Strengthen labor relations.
2
2
2
3
3
3
6
6
6
3
2
2
6
4
4
Enhance “employee service” image of HR.
Manage risk of insufficient communications with employees.
2
2
3
–2
6
–4
2
–3
4
–6
2
3
2
3
20
10
4
6
4
6
20
20
12
Rapid implementation.
Openness and portability.
2
2
4
4
8
8
2
3
4
6
Easier software customization.
Less software modification over time.
Global processing and support.
2
2
2
4
4
2
8
8
4
3
4
4
6
8
8
Total Information Systems
CEO
CEO
Alternative B
Grade Score
4
4
4
4
2
Total Human Resources
CIO
Director,
Systems
Dir.-Sys
Dir.-Sys
CIO
Alternative A
Grade Score
Intangibles (Benefits and Risks)
Improve revenues, profits, and market share.
Integrate global operations.
Have flexibility for business changes and growth.
Have more end-user self-sufficiency.
Improve employee morale.
Total Senior Management
CFO
CFO
Dir. Acctg.
Director, Fincl.
Reporting
Weight
10
36
32
Total Intangibles
50
124
104
Tangible Benefits
Return on investment.
Payback period
20
20
Total Tangibles
Grand Total
The results favor option A (total of 244 vs. 204).
3
3
60
60
3
2
60
40
40
120
100
90
244
204
Source: Compiled from “Peoplesoft Strategic Investment Model,” Peoplesoft.com (accessed August 1997).
W-280
TABLE W14.6.1 Analyzing a Build versus Buy Decision
Tangible ROI with Enhanced Analysis ($ in thousands)
Build Only (A)
Buy and Build (B)
2004
2005
2006
2007
2004
$
$
$ 75
375
$450
50
$400
$200
575
$775
50
$725
$
0
0
$ 0
250
($250)
Cumulative savings
($150)
($250)
$150
Gross four-year tangible savings
$1,225
Four-year cost of each choice
350
Net 4-year saving from each choice
$875
Net present value (NPV) from each choice
(at 6.5%)
$709
Internal rate of return (IRR)
150%
Choice based on NPV analysis: Buy and build (choice B)
Choice based on IRR analysis: Build only (choice A)
$875
($250)
Standard tangible savings
(Reduced head count)
Overlooked tangible savings
Total tangible savings
Cost of new system
Net annual savings
0
0
$ 0
150
($150)
0
0
$ 0
100
($100)
2005
2006
2007
0
275
$275
175
$100
$175
600
$775
100
$875
$ 275
625
$ 900
100
$ 800
($150)
$525
$1,950
625
$1,325
$1,325
$
$1,100
124%
Source: Adapted from Datamation, January 7, 1994, p. 46.
economics studies appear in various shapes depending on the circumstances. An
example in banking is provided by Peffers and Sarrinen (2002). Note that in this
study, as in many others, special attention is paid to the issue of risk assessment. (See
also Gaulke, 2002.) Table W14.6.1 shows an analysis of a decision of whether to
develop a system in-house or buy it. Information economics can be implemented by
software packages such as Expert Choice (expertchoice.com).
REAL-OPTION
VALUATION OF
IT INVESTMENT
A promising new approach for evaluating IT investments is to recognize that they
can increase an organization’s performance in the future. This is especially important for emerging technologies that need time to mature. The concept of real options
comes from the field of finance, where financial managers have applied it to capital
budgeting decisions. Instead of using only traditional measures like NPV to make
capital decisions, financial managers are looking for opportunities that may be
embedded in capital projects. These opportunities, if taken, will enable the organization to alter future cash flows in a way that will increase profitability. These opportunities are called real options (to distinguish them from financial options that give
investors the right to buy or sell a financial asset at a stated price on or before a set
date). Common types of real options include the option to expand a project (so as
to capture additional cash flows from such growth), the option to terminate a project that is doing poorly (in order to minimize loss on the project), and the option to
accelerate or delay a project (e.g., the delay of airport expansion cited earlier). Current IT investments, especially for infrastructure, can be viewed as another type of
real option. Such capital budgeting investments make it possible to respond quickly
to unexpected and unforeseeable challenges and opportunities in later years. If the
organization waits in its investment decisions until the benefits have been established, it may be very difficult to catch up with competitors that have already
invested in the infrastructure and have become familiar with the technology.
W-281
Step 1
Steps
Objectives
Step 2
Compute Base Case
Present Value (PV)
without Flexibility, Using
DCF Valuation Model
Compute base case
present value
without flexibility
Comments
Model the
Uncertainly,
Using Event
Trees
Outputs
Project's PV without
flexibility
Identify and
Incorporate Managerial
Flexibilities by Creating
a Decision Tree
Step 4
Calculate RealOption Present
Value (ROA)
Identify major uncertanities in each stage
Understand how those
uncertainties affect
the PV
Analyze the event tree
to identify and incorporate
manageial flexibility to
respond to new
information
Value the total
project using a
simple algebraic
methodology
Still no flexibility; this
value should equal the
value from Step 1
Explicitly estimate
uncertainly
Incorporating flexibility
transforms event trees,
which transforms them
into decision trees
The flexibility
continuously alters the
risk characteristics of
the project, and hence
the cost of capital
ROA includes the base
case present value
without flexibility plus
the option (flexibility)
value
Under high uncertainty
and managerial
flexibility, option value
will be substantial
Detailed event tree
capturing the possible
present values of the
project
A detailed decision tree
combining possible
events and management
responses
ROA of the project and
optimal action plan for
the available real
options
Figure W14.6.3 Realoptions analysis. (Source:
Based on Rayport and
Jaworski, 2001, p. 304.)
Step 3
Applying just the NPV concept (or other purely financial) measure to an investment in IT infrastructure, an organization may decide that the costs of a proposed
investment exceed the tangible benefits. However, if the project creates opportunities for additional projects in the future—that is, if it creates opportunities for real
options—the investment also has an options value that should be added to its other
benefits (see Trigeorgis, 2005, and Devaraj and Kohli, 2002).
The mathematics of real-option valuation are well established but unfortunately
are too complex for many managers. (See Dixit and Pindyck, 1995, for details.) For a
discussion on using real-option pricing analysis to evaluate a real-world IT project
investment in four different settings, see Benaroch (2002). Rayport and Jaworski
(2001) applied the method for evaluating EC initiatives (see Figure W14.6.3).
REAL-OPTIONS
ANALYSIS
An illustration of real-options analysis is demonstrated in Figure W14.6.3. It is
shown as a four-step process, each with its own output.
References for Online File W14.6
Benaroch, M., “Management Information Technology Investment Risk:
A Real Options Perspective,” Journal of Management Information
Systems, Fall 2002.
Datamation, January 7, 1994, p. 46.
Devaraj, S., and R. Kohli, The IT Payoff. New York: Financial
Times/Prentice Hall, 2002.
Fine, C. H., et al., “Rapid-Response Capability in Value-Chain Design,”
MIT Sloan Management Review, Winter 2002.
Gaulke, M., “Risk Management in TI Projects,” Information Systems
Control Journal, November–December 2002.
Keen, P. G. W., “Value Analysis: Justifying DSS,” Management Information Systems Quarterly, March 1981.
McKay, J., and P. Marshall, Strategic Management of e-Business. Milton,
Australia: Wiley, 2004.
Peffers, K., and T. Saarinen, “Measuring the Business Value of IT Investments: Inferences from a Study of a Senior Bank Executive,” Journal
of Organizational Computing and Electronic Commerce, January–
March 2002.
“Peoplesoft Strategic Investment Model,” Peoplesoft.com (no longer
available online).
Rayport, J., and B. J. Jaworski, E-Commerce. New York: McGraw-Hill,
2001.
Trigeorgis, L., “Making Use of Real Options Simple,” The Engineering
Economist, 50 (1), 2005.
Willcocks, J., “Managing Information Technology Evaluation—
Techniques and Processes,” in R. G. Galliers and B. S. H. Baker (eds.),
Strategic Information Management: Challenges and Strategies in Managing Information Systems. Oxford: Butterworth Heinemann, 1994.