Objectives:
1. Learn about how NZers’ feel about
Retirement Income Policies (RIPs)
specifically
2. Test Multi-Criteria Analysis as a method –
in general – for understanding citizens’
preferences as an input to the policymaking process
Main findings:
1. What matters to NZers concerning RIPs
2. Preference heterogeneity – i.e. people’s
preferences are idiosyncratic, and not
closely related to observable sociodemographics and backgrounds
3. Multi-Criteria Analysis is useful in general!
1
Don’t be confused by the terminology …
“Multi-Criteria Analysis” (MCA)
“Conjoint Analysis”
“Multi-Criteria Decision-Making” (MCDM)
“Choice Modelling”
“Multi-Attribute Decision Analysis”
(MCDA)
“Discrete Choice Experiments”
(DCE)
… considering trade-offs between
multiple criteria (or ‘objectives’) in
order to prioritise, rank or choose from
the alternatives being considered by
decision-makers …
… surveying ‘stakeholders’ (e.g. citizens,
taxpayers, consumers) about how much
they value and are prepared to trade-off
key attributes of particular goods or
services …
→ discover the relative importance of
the criteria
→ discover which attributes are most
important
→ rank / prioritise alternatives, usually
subject to a budget constraint
→ design the ‘ideal’ good / service
(e.g. to maximise citizen satisfaction)
2
Making ‘good’ decisions has always mattered ...
“Choices are the hinges of destiny.”
- Pythagoras (570 BC-495 BC)
“Nothing is more difficult, and therefore more precious,
than to be able to decide.” - Napoleon Bonaparte (1769–1821)
“Man is man because he is free to operate within the framework of his
destiny. He is free to deliberate, to make decisions, and to choose
between alternatives.”
- Martin Luther King (1929-68)
“Making good decisions is a crucial skill at every level.”
- Peter F Drucker (1909-2005)
“Ever notice that 'what the hell' is always the right decision?”
- Marilyn Monroe (1926-62)
3
Multi-Criteria Decision-Making (MCDM)
 Prioritising, or ranking, or choosing from amongst competing alternatives or
individuals, based on considering multiple criteria (or objectives) …
(very similar to ‘Conjoint Analysis’ / ‘Choice Modelling’)
1000s and 1000s of applications across huge range of areas …
e.g.
Investment Appraisals
– Ranking investment or project proposals
- CAPEX
Procurement Decision-Making
– Evaluating supplier proposals
– Assessing value for money
- Allocating budgets
Homeland Security, Military & Defence
– Prioritising security / terrorism threats
– Assessing intelligence
– Prioritising ‘special projects’
Strategic Planning
– Prioritising organisation’s objectives
Health Care
– Prioritising patients for treatment
– Managing waiting lists
– Health Technology Assessments
– Advanced planning for pandemics
Environmental Management
– Environmental impact assessments
– Planning notifications
Government Decision-Making, in general
– Prioritising investment projects
– Ranking community-funding grant
applications
...
4
1000s and 1000s of MCDM applications … (2)
Immigration
–Points Systems for selecting immigrants
Utilities Management
– Ranking project proposals
– Environmental planning
Research Funding
– Ranking grant applications
Education
– Awarding scholarships
– Admitting students to med, pharmacy & law
schools
Marketing Research & New Product Design
– Choice-based Conjoint Analysis
Police & Emergency Services
– Prioritising police & emergency services
responses
Human Resource Management
– Short-listing job applicants
– Appraising CEO candidates
– Staff appraisals for bonuses & promotions
Finance, Banking & Insurance
– Short-listing assets for portfolios
– Capital budgeting
– Credit scoring
– Detecting fraud
Business Opportunities
– Ranking start-ups, company acquisitions &
investment opportunities
– Allocating ‘angel’ & venture capital
– Portfolio management
Competitions & Awards
– Judging
... etc. etc. etc (ie. anything involving ranking
according to decision criteria
5
Making ‘good’ decisions is often difficult …
due to our (humans!) cognitive limitations
 Even when we have precisely the relevant facts in front of us,
we often fail to make rational decisions
 Moreover, we often draw the wrong inferences about
what the relevant facts are (i.e. compounding the problem)
VS
Common mistakes people commit when making important decisions ...
Decision by Objectives, E Forman & M Selly (2001), p. 4; http://mdm.gwu.edu/forman/DBO.pdf
7
3 Key Elements of ALL MCDM Decisions
(‘ART’ and ‘SCIENCE’)
1. ALTERNATIVES to be prioritised
e.g. project proposals for a govt. agency to invest in
2. CRITERIA (or OBJECTIVES) by which
alternatives are to be prioritised / ranked
... can be quantitative or qualitative, as appropriate
3. RELATIVE IMPORTANCE of the criteria
e.g. ‘weights’ or ‘points’
 Describe ALTERNATIVES on the CRITERIA
via the Points Systems …
 Ranking of Alternatives …
 Make decisions subject to a budget constraint
e.g. Project proposals for a govt. agency to invest in …
e.g. Living Standards Framework
Economic growth
negative
nil / neutral
weak-to-moderately positive
strongly positive
0
12
25
33
Sustainability for the future
negative
nil / neutral
weak-to-moderately positive
strongly positive
Increasing equity
negative
nil / neutral
weak-to-moderately positive
strongly positive
Social infrastructure
negative
nil / neutral
weak-to-moderately positive
strongly positive
Managing risks
negative
nil / neutral
weak-to-moderately positive
strongly positive
0
7
17
22
0
4
16
19
0
12
14
15
0
3
10
11
Plus Net Cost (NPV - include later)
8
($ m)
Proposal A
Proposal B
Proposal C
Proposal D
Proposal E
Proposal F
Proposal G
Proposal H
Proposal I
Proposal J
Proposal K
Proposal L
Proposal M
Proposal N
Proposal O
Proposal P
Proposal Q
Proposal R
Proposal S
Proposal T
Proposal U
Proposal V
Proposal W
Proposal X
Proposal Y
Proposal Z
30
100
130
14
40
40
80
50
140
30
180
70
120
160
180
125
60
15
10
110
50
130
80
70
40
90
$2144 m
9
With a budget of, say, $600 million, which ‘proposals’ would you select?
 Which represent good value for money?
 Which is wasteful (low quality spending)?
+ CONFIDENCE in
Net Cost Estimates
i.e. larger the bubble
= greater certainty
(index)
NET COST (NPV, $ million)
10
FAQ #1
How does Multi-Criteria Decision-Making (MCDM)
relate to Cost-Benefit Analysis (CBA)?
Answer:
They’re complementary ...
Use MCDM to combine ‘Benefit’ defined on multiple
dimensions – including financials – into a single
measure (‘Benefit index’)
And leave ‘Cost’ in NPV terms
 Benefit-Cost Ratios, etc
11
MCDM Software Industry
1. 2014 Software Survey: http://lionhrtpub.com/orms/surveys/das/das.html
2. http://en.wikipedia.org/wiki/Decision-making_software#DMS_comparison_table
Software
1000Minds
Ahoona
Altova MetaTeam
Analytica
Criterium
DecisionPlus
D-Sight
DecideIT
Decision Lens
Super Decisions
Expert Choice
Hiview3
Logical Decisions
M-MACBETH
PriEsT
WISED
MCDM Methods
PAPRIKA
WSM, Utility
WSM
Pairwise
Comparison
Yes
No
No
No
Sensitivity
Analysis
Yes
No
No
Yes
Group
Evaluation
Yes
Yes
Yes
No
Webbased
Yes
Yes
Yes
Yes
AHP, SMART
Yes
Yes
No
No
PROMETHEE, UTILITY
MAUT
AHP, ANP
AHP, ANP
AHP
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
No
No
No
No
Yes
AHP
MACBETH
AHP
MACBETH
12
Why not give MCDM and Choice Modelling a try?!
THANK YOU!!!
13