PERFORMANCE LEADERSHIP SERIES
NUMBER GAMES: MAKING
SURE METRICS DRIVE THE
RIGHT BEHAVIOR
FROM BEST PRACTICES TO NEXT PRACTICES:
THIRD IN A FOUR PART SERIES
by Frank Buytendijk, Beingfrank Research
September 2012
SPONSORED BY:
NUMBER GAMES: MAKING SURE METRICS DRIVE THE RIGHT BEHAVIOR
p2
ABOUT THE SERIES
In a series of papers, author and industry expert
Frank Buytendijk examines the people side of
business performance and business intelligence.
How does measurement affect human behavior? How
does organizational culture contribute to business
intelligence? What does a rational decision making
process look like? Which number games do we
recognize in business? What is the best methodology
to manage business performance? In this third article
of a series on how measurement drives behavior, Frank
discusses how recognize which games people play and
how to drive the right behaviors .
ABOUT THE AUTHOR
Frank Buytendijk (Beingfrank Research) is a
well-known industry expert specialized in strategy,
performance management and organizational behavior.
He is an exceptional speaker at conferences all over
the world, known for his out-of-the-box, entertaining
and slightly provocative style. Frank is also a TDWI
fellow, a visiting fellow at Cranfield University School
of Management, and author of various books, including
“Performance Leadership” (McGraw–Hill, September
2008), “Dealing with Dilemmas” (Wiley & Sons, August
2010) and “Socrates Reloaded: The Case of Ethics in
Business and Technology” (Beingfrank Publications,
September 2012). Follow Frank on Twitter at @
FrankBuytendijk or visit www.frankbuytendijk.com.
ABOUT TABLEAU
Tableau Software helps people see and understand
data. Ranked by Gartner and IDC in 2011 as the world’s
fastest growing business intelligence company, Tableau
helps anyone quickly and easily analyze, visualize and
share information. More than 9,000 companies get
rapid results with Tableau in the office and on-the-go.
And tens of thousands of people use Tableau Public
to share data in their blogs and websites. See how
Tableau can help you by downloading the free trial at
www.tableausoftware.com/trial.
The content of this paper is based on Buytendijk, F.A., 2010, “Performance
Leadership,” McGraw-Hill, United States
1
NUMBER GAMES: MAKING SURE METRICS DRIVE THE RIGHT BEHAVIOR
p3
M
etrics can tell you anything you want.
lead to cheating the system. For instance, a group in a
For every event, metrics can be found
back office asks one member to work late, but to punch
that present different, even opposing, conclusions.
the time clock for all. Many of the recent bookkeeping
Managers have an interest in finding and presenting
scandals have been caused by pure misrepresentation.
the metrics that make their performance look good.
For instance, consider a multinational owning off-
It is hardly debated that many companies suffer from
balance sheet entities that buy products and services
“gaming the numbers” and “cheating the system.” But
at the end of the quarter so that the main entity makes
it would be too easy to blame middle management or
the numbers it forecasted to the shareholders and
divide-and-conquer and other forms of opportunistic
financial analysts. Another example plays on a more
behavior.
operational level. A salesperson promised the customer
The problem is not bad people. The real problem is bad
measurement practices that makes people behave in
opportunistic ways. All these dysfunctional behaviors
tend to be hidden. So to understand them, and be
able to predict and perhaps prevent the unintended
consequences, we need to know where to look for
an additional discount if he would order more products
than were needed. Secretly the salesperson advised
the customer to ship back the surplus of goods the day
after receiving the order. Returns were not part of the
compensation plans of the salesperson, and returns
these behaviors.
The real problem is bad
We can distinguish two basic types of consequences:
measurement practices that make
•
people behave in opportunistic ways.
People impacting measurement. People trying to
play the numbers so that they don’t have to alter
their actions.
•
Measurement impacting people. The metrics put in
place drive dysfunctional behavior.
There are various ways that managers play the
numbers. Perhaps the most well-known example of
dysfunctional behavior is measure fixation. It happens
when running the numbers becomes more important
for managers than running a successful business. An
example of this is the railway organization that saw its
accuracy deteriorate. Accuracy here is defined as the
percentage of trains that leave the station on time and
arrive at their destination on time. Confronted with the
performance problems on this metric, the operations
manager decides to widen the margin of the definition.
Previously, “on time” was defined with a margin of two
minutes, one minute before the listed time until one
minute after. Now the metric is redefined and trains
are considered to ride on time within a margin of four
were not correlated to the revenues.
But measurements also affect the way people react
and behave. Gaming is the opposite of measure fixation
and misrepresentation. It means manipulating the
business to make the numbers look good (instead of
manipulation of the numbers to make the business look
good). Both transgressions are equally serious. Gaming
occurs when managers start to underachieve once
the target has been reached. The next period’s targets
therefore are forced not to be higher any more than
absolutely necessary. Another variant is overspending
at the end of the year to make sure all the budget is
used and thus secure an equally high or higher cost
budget for the next round.
Due to compliance and shareholder pressures, there
is an increased focus on short-term objectives instead
of the longer-term strategy. Myopia is the result. For
minutes.
instance, a professional services firm found out it had
When measure fixation grows out of control, it can
the account managers were urged to work with their
a problem with its days-sales-outstanding (DSO). All
NUMBER GAMES: MAKING SURE METRICS DRIVE THE RIGHT BEHAVIOR
p4
clients to have the invoices paid sooner. One particular
HOW DO YOU DRIVE THE RIGHT BEHAVIORS?
account manager was extremely successful—all
In every organization, one of the most important
the outstanding invoices were paid immediately.
Unfortunately, these were the last invoices the firm
could send. The account manager had pushed his
client to such an extent that the client paid the bills and
terminated the relationship. The actual performance
indicator was considered more important than the
overall customer relationship.
Another common behavior can be observed when
success factors is collaborative behavior—a synergy
that makes the contributions of individuals into more
than just the sum of its parts. However, it is remarkable
that cooperation within an organization generally
remains underexposed in management reports. One
of the reasons for this is that collaboration is generally
considered not quantifiable and therefore cannot be
measured. However, the results of collaboration are
managers focus not on the important targets and the
easy to measure. Think about the following examples:
key performance indicators, but on the targets and
•
indicators that are easy to measure. This phenomenon
installations in a telecom company that offers
is called tunnel vision. A widespread example is the
telephony, high-speed internet, and television
use of “revenue” as a target for salespeople. This often
leads to high discounting by the salesperson who needs
to reach his or her target at the end of the quarter or
through its various divisions
•
able to answer difficult questions in a photocopier
margin pressures. This sales behavior is a logical
service company
consequence of measurement on revenue because it
The basis of every set of balanced
Tracking the use of an expertise location system
that helps engineers find colleagues who may be
year. At the same time, the CFO will complain about
is the easiest metric to track. Measuring salespeople’s
First-time-right percentage of “triple play”
•
Cross-sell percentage of products in a large bank
UNDERSTAND THE IMPORTANCE OF FEEDBACK
Every person needs and likes feedback, even when it
metrics consists of three different
is negative feedback. It is important to hear how we are
elements: cost or revenue, quality,
doing and how we are perceived. When feedback is
and speed.
back is negative (but constructive), there is a good
positive, it will spur the displayed behavior. If the feedchance it will alter behavior.
performance based on contribution margin is more
difficult, but also much more worthwhile.
A common best practice states that performance
indicators should have a single owner and that
management should provide this person with all the
means to make the target on the performance indicator.
Although this sounds good as a plan, in practice it can
lead to suboptimization. Managers look at maximizing
their own targets, even at the expense of the overall
strategic objectives. Many small suboptimal results then
lead to one big negative result.
Keep performance measures close to the operational
processes and systems, instead of just creating highlevel management reports in a distant data warehouse.
Measures are most useful when used as feedback
for continuous review and continuous improvement
of activities in your business operations. Aggregated
general management reporting can be an automated
derived result from the measurement process.
BALANCING THE METRICS
The basis of every set of balanced metrics consists of
three different elements: cost or revenue, quality, and
speed. There needs to be a balance, if quality is the
NUMBER GAMES: MAKING SURE METRICS DRIVE THE RIGHT BEHAVIOR
p5
only aim, then the speed of processes will decrease,
be weighed in internal decision making. Competitive
and the costs will increase. Perfectionism comes at
intelligence is a crucial discipline in an organization’s
a price. If cost saving is the only factor, then quality
performance management. Another reason for external
will almost immediately suffer, and often speed will
focus is the added value of benchmarking on an
deteriorate too. You get what you pay for. If speed is of
organization’s behavior. At the same time, organizations
utmost importance, then quality may suffer, and cost
shouldn’t be led by the outside world. Strategies should
will most definitely be an issue. It is not possible to
not be copied from the competition, they should be
optimize all three of the elements. Speed and cost will
authentic.
require quality trade-offs, speed and quality will come
at a premium, and low cost and quality will take time.
In most organizations, performance management and
risk management are seen as separate disciplines.
There are additional balances when implementing
Yet, the two disciplines are very much related. Key
performance management. There needs to be a
performance indicators complement key risk indicators.
balance between short-term and long-term issues.
The balanced scorecard speaks of the financial,
Reaching strategic goals usually takes a while, up
customer, process, and learning/growth perspective;
to a number of years. Step by step, you manage to
risk management distinguishes financial, customer,
get closer to the goals until you reach them and it is
and operational risk. There are multiple advantages
time to stretch those goals again. In order to reach
to combining the two disciplines. First of all, risk
those goals in the long term, today’s action is needed.
management allows the organization to establish
Short-term focus and long-term focus go hand in hand.
improvement projects before the perform- ance
Focusing on just the long-term leads to a lack of sense
metrics show that a problem is looming, which leads
of urgency. Focusing on just the short- term leads to
to preventive action. Secondly, risk management
myopia.
challenges the intuitive belief of performance
A balance between financial and non-financial
performance indicators is also needed. Although the
financial bottom line is important, the value drivers of
the organization reside in the operations. A too-strong
focus on finance easily leads to misinterpretation, even
misrepresentation. A too-strong focus on operations
may lead to nonaccountable behavior. The balance
between financial and operational information leads
to a higher predictability of financial results as well as
a better understanding of financial consequences of
measurement that everything will proceed as planned.
When there are discontinuities that threaten corporate
objectives, having gone through a risk management
exercise prepares the organization better for dealing
with them.
CREATE A PROCESS OF ACCOUNTABILITY
Performance evaluation is often very control-oriented
and is restricted to evaluating the metrics themselves.
The numbers come into the management information
operational decisions.
system, are compared against target, and are color-
Organizations should also balance leading and lagging
and underperformance is shown by red numbers.
metrics. Lagging metrics are put in place in order to be
able to report and to justify. Leading metrics support
decision-making processes. Both should drive the
right behaviors. Internal and external focus should be
balanced as well. Organizations should realize that
the market and the environment at large have a huge
impact on the organization. These influences should
coded. Superior performance leads to green numbers,
The management information systems apply a filter to
manage by exception and produce a list of indicators
where people have underperformed. The results are
predictable: people try to “game” the numbers, coming
up with better scores that look good on paper. What
happens if we break that rule? Color coding should
not be automatically added the moment the new data
NUMBER GAMES: MAKING SURE METRICS DRIVE THE RIGHT BEHAVIOR
p6
comes into the management information system. By
comparing results with the target, the responsible
manager manually assigns the color coding. Perhaps a
target is met, but the manager still assigns the color red
to the metric as it could have been done even better.
Or a target has not been met, but is assigned a green
color, because external circumstances significantly
changed. With the analysis at hand, the managers go
into the management meeting where each manager
explains the color coding. The manager then is queried
about choices made and in the end receives a sign-off
(or not) by senior management.
This process may come across as peculiar. “Writing
your own report card” is something we instinctively
reject. All managers would immediately score
themselves a “green” on all metrics. But in the new
process there is still a sign-off. Senior management
approves or modifies the end evaluation. And the
opposite effect, allowing people to apply their endless
creativity to game automated systems, is worse. Dare
to break the rules!
© Copyright Tableau Software, Inc. 2012. All rights reserved. 837 North 34th Street, Suite 400, Seattle, WA 98103 U.S.A.
NUMBER GAMES: MAKING SURE METRICS DRIVE THE RIGHT BEHAVIOR
p7
READ THE WHOLE SERIES:
How Measurement Drives Behavior. In this introductory paper to a four-part series, author and industry expert Frank
Buytendijk examines the people side of business performance and business intelligence.
1. Fact-based decision-making, or is it? Take a close look at the process of decision making to ensure a rational,
appropriate process is applied.
2. What is the best methodology? Getting to double-loop learning. Understanding the right measurement
methodology is essential to achieving your requirements.
3. Number games: making sure metrics drive the right behavior. Why it’s important to use measurements that
incent actions that benefit the organization as a whole.
4. It’s the culture, stupid! Unless measurements are linked to an organization’s culture, the effect will be lost.