Performance Metrics
CPS 181s
April 10, 2003
Should Performance Be
Measured?
Metrics
 Assessing
the progress and health of on-line businesses
 Financial
progress: sales, margins, profit, market share
 Strategic metrics require entire examination of the company:
perception of value proposition, marketplace offering,
effectiveness of implementation
 Branding and implementation metrics focus on
> supply-chain performance
> organizational dynamics
> marketing communication effectiveness – including branding
 Senior
and stock market analysts tend to have a bias
toward financial metrics
 provide
history of the company in the marketplace
 do not reflect the strategy of the company
 Managers
need early warning system – metrics
Should Senior Managers be
Concerned About Metrics?
 Organization
metrics represent performance targets of
the company
 Percent
increase in revenue
 Percent increase in unique visitors
 Length of time visitors remain on site
 Cost of customer acquisition
 Metrics
valuable as employee reward or taking strategic
action
 Change
processes
 Change strategy
 Change product strategies
Metrics Drive Behavior in a
Number of Ways
 Help
define the business model
 Concrete
goals with precise measurements
 Focusing on measurement can increase the precision of the value
proposition
 Help
communicate strategy
 Document
performance targets
 Communicates strategy to workforce
 Creates buy-in to the metric-setting processes
 Help
track performance
 Instantaneous
feedback concerning site performance real-time
 Usage, visitors, length of time on site, average sales, page views
Metrics Drive Behavior in a
Number of Ways...
 Help
increase accountability
 Metrics
must be linked to the performance appraisal system
 Individual performance can be appraisals tied to companywide,
team-specific, and individual metrics
 Site usability can be tied to the interactive design function
 Customer service metrics can be tied to the customer service
department
 Help
Align Objectives
 Align
precise objectives, departmental functional goals,
companywide strategic activities as a whole
Current Challenges to Specifying
Metrics for Online Businesses
Strategies Change Rapidly – Internet time
requires rapid change
 Measurement is resource intensive
 Companies
 Setup
and maintenance of capital intensive/human resource
intensive systems and procedures
 Data capture, data mining, and information use require time and
commitment of senior executives
measurement systems are vulnerable – easy to
manipulate final results
 Online
Current Challenges to Specifying
Metrics for Online Businesses...
 Soft
metrics are not valued by the investment community
 Investment
community looks at revenues, margins, number of
visitors, length of time on site, customer acquisition costs
 But, ease of customer use, best value perception, and other market
research measures are frequently early warning indicators of site
performance
 Meaningful metrics change on Internet time – often as quickly as
the time it takes to learn best how to interpret the retained original
data
What Should We Measure?
How Can We Assess the Health
of eCommerce Firms?...
 The
Balanced Scorecard
 Introduced
by Kaplan and Norton
 Finance balanced by other domains of the business
 Includes internal business processes and customer responses
 Four categories
>
>
>
>
Financial
Customer
Internal business systems
Learning and growth
Kaplan-Norton Balanced Scorecard
FINANCIAL
To succeed financially, how
should we appear to our
shareholders?
CUSTOMER
To achieve our vision, how
should we appear to our
customers?
Vision
and
Strategy
INTERNAL BUSINESS
PROCESS
To satisfy our shareholders and
customers, at which businesses
must we excel?
LEARNING AND GROWTH
To achieve our vision, how will
we sustain our ability to change
and improve?
Source: Kaplan, Robert S. and David P. Norton. 1996. Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review 74 (January-February): 76.
K-N Financial and Customer
Metrics
 Financial
Metrics
 Revenue,
revenue growth, gross margins, operating income,
net margin, earnings per share, cash flow
 Reflect recent planning period , to some degree all previous
planning periods
 Customer
Metrics
 Management
 Market
relationship of customers by the firm
share
 Customer acquisition
 Customer satisfaction
 Customer profitability
 Customized by the target segment
K-N Internal Business Process
Metrics
 Innovation
includes customer’s perception of
innovativeness of the company
 Operations – quality of entire supply
 Postsale service – quality being offered to
customer
K-N Learning and Growth
Metrics
 Employee
 Selection
 Training
 Retention
 Satisfaction
 Information
Systems
 Quality
of infrastructure
 Timeliness, accuracy, and utility of data
 Motivation
 Empowerment
 Alignment
of company and personal goals
Limitations to the K-N
Balanced Scorecard
No
clear definition of strategy or business
models
Unclear location of organizational
capabilities or resources in framework
Unclear where partnerships reside in
framework
The Rayport-Jaworski Performance
Dashboard and Strategy Framework
 The
Performance Dashboard - intended to reflect the
health of the business
 The Strategy Framework Drives the Necessary Metrics
 Six
>
>
>
>
>
>
critical steps
opportunity assessment
business-model metrics
customer-interface design
market communications and branding
implementations
financial
Introducing the R-J Performance Dashboard
Steps of Strategy:
Market Opportunity
Assessment
Business Model
Customer Interface
Implementation
and Branding
Business Model
Metrics
Implementation
& Branding
Metrics
Customer
Interface &
Outcome Metrics
Evaluation
Performance
Dashboard:
Opportunity
Metrics
Financial
Metrics
Areas
Addressed:



Size of market
opportunity

Uniqueness of value
proposition
Attractiveness of
target segments

Competitive
landscape

Firm capabilities
relative to
competition

Sustainability of
competitive position
Attractiveness of
offering

Branding

Customer acquisition

Revenue

Technology
infrastructure

Changes in customer
behavior

Profit

Cost

Internal organization


Balance sheet

Relationship with
suppliers and
partners
Site experience and
usability

Fulfillment capability
Components of the R-J
Performance Dashboard
 Opportunity
Metrics
 Ability
to discern unfulfilled needs in the market place is critical
 Market-opportunity metrics asses the degree to which the firm
can accurately gauge the market opportunity
 Business-model
Metrics
 Capture
subcomponents of the business model: egg diagram,
resource systems, financial metrics
 Value Proposition or Cluster Benefits Metrics
> target segments, benefits offered, capabiliti4s that drive benefits
> focus on customer’s perceptions
> performance relative to the competitor’s benefits
R-J Branding and
Implementation Metrics
Focus
on the supply-chain performance,
organizational dynamics, marketing
communication effectiveness (including
branding)
Fulfillment of the brand process metrics:
customer awareness of the brand
Innovation metrics: ability to rapidly innovate,
even if in a potentially discontinuous fashion
R-J Customer-Interface
Metrics
 Customer-interface
and customer-outcome metrics
 Customer’s perception of the firm’s performance on
each level of the 7 C’s of the customer interface
 Example:
customer’s rating of customization compared to
competitors’
 Other
critical interface metrics
 Including
value proposition
R-J Customer-Outcome
Metrics
 Capture
process measures that the firm will produce
favorable customer responses such as satisfaction
and loyalty
 Focus on subjective customer-outcome metrics
 satisfaction
 length
of stay at site (stickiness)
 overall evaluation of the experience
 Focus
on objective customer-outcome metrics
 customer
acquisition costs
 average order size
 customer profitability
 number of visits per month
Financial Metrics
Revenues
Costs
Profits
Balance–sheet
metrics
Lise Buyer
Life Cycle of a Company
Strategy
Examples*
STARTUP/
BETA
CUSTOMER
ACQUISITION
MONETIZATION
6 Months- 1 Year
1 Year - 2 Years
2 Years - 5 years
Develop a
platform for rapid
growth by
building a strong
team and
creating a flexible
site.
Build market
share as quickly
as possible by
aggressively
spending on
partnerships and
promotion.
Increase
revenues and
customer lock-in
by developing
new revenue
streams.
Business.com
TNBT.com
mValue.com
Paypal.com
Evite.com
* At time of writing (3/2000)
Yahoo.com
Amazon.com
MATURITY
> 5 Years
Control costs and
optimize
marketing
expenditures to
achieve profitable
growth.
Schwab
AOL
What are the Steps to Implement
the R-J Performance Dashboard?
 Step
One: articulate business strategy
 market
opportunity assessment
 business model
 customer-interface design
 branding
 implementation
 evaluation
Steps Two and Three
 Step
Two: translate strategy into desired outcomes
 not
 Step
settling target levels but specifying outcomes
Three: devise metrics
 identifies
specific metrics that reflect the desired
outcomes
 look-to-book ratios
Bill Gurley on the Power of
Conversion Rates
Conversion Rate
Advertising Spend
2%
$
Visitors
Transactions
10,000
4%
$
10,000
8%
$
10,000
5000
5000
5000
100
200
400
Cost/Transaction
$
100
$
50
$
25
Revenue
$
10,000
$
20,000
$
40,000
Marketing/Revenue (%)
Average transaction size = $100
100%
50%
Source: J William Gurley
Source: Gurley, J. William. 2000. The Most Powerful Metric of All. CNET News.com, 21 February.
URL: http://www.news.com/Perspectives/Column/0,176,403,00/html?tag=st.ne.
25%
Steps Four and Five
 Step
Four: link metrics to leading and lagging
indicators
 map
entire set of indicators
 including leading & lagging indicators
 Step
Five: calculate current and target performance
 current
conversion rates
 advertising expenditures
 degree of positive word of mouth
Blueprint to the R-J Performance Dashboard
Develop resource system required to deliver the strategy
Define goals and value proposition
Step One:
Articulate
Business
Strategy
Step Two:
Translate
Strategy Into
Desired
Outcomes
Step Three:
Devise
Metrics
Market Opportunity
• Opportunity size?
• Competitive environment?
Market Opportunity
• Market size and growth
• Average age and income
• Competitor concentration
Business Model
• Unique value proposition?
• Capabilities vs. competition?
Business Model
• Customer perceived benefits
• Exclusive partnerships
• $ invested in technology vs
competition
Implementation and
Branding
• How to develop brand?
• How to go to market?
Implementation
• Customer brand awareness
• System uptime percentage
• Number of IT staff
• % inaccurate orders
Customer
• How to acquire customers?
• How will customers change?
• The customer experience?
Customer
• Market share
• Purchases / year
• Success rate
• Service requests / customer
Financial
• Financial consequences in
terms of revenue, profit, cost
and balance sheet?
Financial
• Revenue
• Profit
• Earnings per share
• Debt to Equity ratio
Step Four:
Link Metrics
To Leading
And Lagging
Indicators
• For each metric,
determine the metrics
that it affects and that
affect it
• Map the linked set of
metrics, indicating
leading and lagging
indicators
• Ensure that there is a
balance between
leading and lagging
indicators
Step Five:
Calculate
Current And
Target
Performance
• For each metric,
calculate current level
of performance
• Determine target
level required to meet
outcomes described
in Step 2
• Ensure that targets
are consistent with
each other
What are the Three Sources of
Metrics Information That Firms Can
Use to Chart Their Progress?
 Industry
standard metrics
 Benchmark values
 On-line Information
Online Information
research – primary customer data
 Analysts reports – blending market data
 Financial information – income statement, balance sheet,
statement of cash flow
 Use of Bizrate – use of dimensions of service
 Use of Forrester Reports – e-commerce reports –
competitors, consumers, technology evolution in a segment
of the industry
 Use of Hoover’s Online – financial and market information for
both online and offline companies
 Market
U.S. Top 25 Web & Digital Media Properties (March 2000)
Rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Digital Media/Web
AOL Network
Yahoo Sites
Microsoft Sites
Lycos
Excite @ Home
Go Network
NBC Internet
Amazon
Time Warner Online
Real.com Network
Go2Net Network
AltaVista Network
About.com Sites
Ask Jeeves
eBay
LookSmart
ZDNet Sites
CNET Networks
eUniverse Network
JUNO/JUNO.COM
EarthLink
Infospace Impressions
Viacom Online
FortuneCity Network
CitySearch-TicketMaster
Online
Unique Visitors (000)
59,858
48,336
46,581
32,899
28,571
23,006
17,169
15,217
13,636
13,482
13,041
12,557
12,329
12,269
11,155
10,557
10,226
10,023
9,198
9,177
8,526
8,305
8,139
7,809
7,689
Source: Media Metrix (www.mediametrix.com/usa/press/releases/20000424.jsp) March 31, 2000
Market Research Source - Bizrate.com
Analyst Source - Forrester
Financial Information Source - Hoovers Online
Mapping Internet Research Onto The Performance Dashboard
Market Research
Media
Metrix
Market Info
Market
Traffic
Fulfillment
Implementation
Implementation
Privacy
Usability
Content
Customer
Customer
Satisfaction
Customer
Service
Financial
Financial
Performance
AC
Nielsen
Financial
Information
Analyst
BizRate
Forrester
Gomez
Jupiter
Creative
Good
Hoovers