The Human Side of Metrics
Dennis Opacki, CISSP QDSP
Metricon 1.0
[email protected]
Vancouver, Canada
Covestic, Inc.
August 1, 2006
How do organizations use metrics?
Most purposes cited reduce to: (Hauser & Katz 1998)
•
Metrics enable firms to take stock of where they are, and help
them plan for the future.
•
Metrics provide estimates of future performance.
•
Managers use metrics to allocate assets and select strategies.
•
Metrics form the basis for bonuses and promotions.
Even measures derived from automated systems
reflect and influence human behavior.
slide 2
Common metrics pitfalls
Goal displacement (Kerr, 1975)
•
•
Means becomes the ends in themselves.
People seek rewards to the exclusion of all non-rewarded behavior.
Unintended consequences
•
•
Metrics affect decisions even if those decisions inadvertently
sacrifice long-term benefits. (Hauser & Katz, 1998)
Broken reward systems encourage behavior organizations are
trying to discourage. (Kerr, 1975)
Passive or active resistance
•
•
Staff will find ways to avoid metrics they oppose. (Neal, 2006)
Zone of indifference – “a person can and will accept a
communication as authoritative only when… at the time of his
decision, he believes it to be compatible with his personal interests
as a whole.” (Kerr, 1975)
slide 3
Factors contributing to failure
Evolutionary psychology (Economist, 2005)
•
Humans are hard-wired not for logic, but for detecting injustice.
•
People will even forego their own self-interests to punish those
behaving unfairly.
Social psychology (Neal, 2006)
•
Locus of control
•
Internal – Sense of self-reliance – “master of own destiny”
•
External –Sense of helplessness – external parties are in control
•
Fundamental attribution errors – incorrectly assuming bad behavior
is irrational or malicious
•
Diffusion of responsibility – bystander effect
•
•
Everyone believes someone else is better suited to act.
•
This can occur if responsible parties lacks authority to affect change.
Reciprocity norm – What’s in it for me?
slide 4
Factors contributing to failure (cont.)
Behavioral Economics (Kahneman, 2003)
•
Intuition vs. reason – people don’t think very hard
•
•
•
•
•
•
Prospect theory
•
•
•
Intuition uses heuristics, prototypes and predictions – fast and parallel.
Reason is slow and single-threaded.
Natural attributes enhance intuition – size, color, valence.
Good/bad can be substituted for any attribute.
Default options have a natural advantage.
People anticipate emotions associated with changes in state.
People are willing to ignore scope when presented with definite solutions to
emotional problems.
Framing
•
•
How we present situations greatly influences outcomes.
Multiple frames increase the chance of specific outcomes.
slide 5
Improving chances for success
Select a few good metrics
•
•
•
•
•
Focus on attributes and scales that people gauge intuitively good/bad, like/dislike, percentile.
Use a small set of metrics to keep time pressure from driving
cognitive processes from reason to intuition. (Kahneman, 2003)
Survey employees to understand which behaviors are actually
being rewarded. (Hauser & Katz, 1998)
Express metrics in dollars; research shows this improves peoples’
ability to assess probability. (Kahneman, 2003)
Use clear metric names and descriptions to increase accessibility
of metrics’ traits. (Kahneman, 2003)
Present them well
•
Don’t ignore entertainment value. (Pijpers & Montfort, 2006)
•
•
•
•
Drill-down is key to getting middle-management buy-in. (Gartner, 2005)
Slicing and dicing – bells and whistles
Use common visualizations to improve mental prototyping.
Give bad news first - peak/end rule (Kahneman, 2003)
slide 6
Works Cited
[1] Buytendijk, F. and Gassman, B. (2005). Management Update: Just Give Me a CPM
Dashboard. Gartner whitepaper.
[2] Hauser, J. R. and Katz, G. M. (1998) Metrics: You are what you measure!
European Management Journal, 16(5).
[3] Kerr, Steven (1975).On the Folly of Rewarding A, While Hoping for B. Academy of
Management Journal. 18(4).
[4] Kahneman, Daniel (2003). Maps of Bounded Rationality: Psychology for Behavioral
Economics. The American Economic Review. 93(5).
[5] Neal, Russ. (2006). Social Psychology Variables that Contribute to Resistance to
Security Assessment Findings. Information Systems Security, 15(1).
[6] No Author Cited, (2005). Survey: The concrete savannah. The Economist,
377(8458).
[7] Pjipers, G.G.M. and van Montfort, K. (2006). An Investigation of Factors that
Influence Senior Executives to Accept Innovations in Information Technology.
International Journal of Management. 23(1).
slide 7