The Federal Resource Management and
Ecosystem Services Guidebook
NON-MONETARY VALUATION:
MULTI-CRITERIA EVALUATION
ACES Workshop 2014
Dean Urban, PhD
Professor of Landscape Ecology
Senior Associate Dean
Nicholas School of the Environment
Duke University
Lynn Maguire, PhD
Professor of the Practice of
Environmental Decision Analysis
Nicholas School
Duke University
Federal Resource Management and Ecosystem Services Guidebook
nespguidebook.com
Structured Decision-Making
 Encompasses much/most of the assessment
framework outlined for FRMES:
– Stakeholder engagement, desired outcomes
– Management and ecological outcomes
– Stakeholder preferences for outcomes
 Levels of performance on a single service
 Trade-offs among competing services
– Aggregated information for decision support
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
SDM vs. Alternatives
 Relative to benefit-relevant indicators:
– Directly engages stakeholders
 Relative to monetization:
– Does not require monetary valuation
– Can be applied to benefits that are hard to monetize
– Values options relative to the best option on the table
(not necessarily business as usual)
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Structured Decision-Making
This illustration:
 Multi-attribute utility theory (MAUT)
 Steps:
– Engage stakeholders, identify desired outcomes
– Select empirical indicators for desired outcomes
– Identify management means to achieve ends
– Elicit stakeholder preferences for levels of performance
(per service)
– Elicit preferences for services
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Objectives Hierarchy
The objectives hierarchy:
 Declares what outcomes are valuable to stakeholders
 Outlines relationships among desired outcomes:
– Categories (independent)
– Nestedness
– Final vs intermediate goods and services
 Identifies empirical indicators for the final outcomes
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Objectives Hierarchy
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Objectives Hierarchy
Topology:
 Left side: categories of services (independent)
 Middle: refinement
 Right side: “final” goods and services, with
measurement units
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Means-Ends Model
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Means and Ends
The objectives hierarchy and means-end models are
overlapping constructs …
 Objectives hierarchy: a static depiction of desired
outcomes and how they will be measured
 Means-ends models: a depiction of dynamics via
which management might effect these outcomes
 The right-hand side of the M-E models is the left-hand
side of an objectives hierarchy
 These are elaborated simultaneously
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Alternatives Matrix (1)
Measures
Alternative actions
Status quo
Downstream dam
Upstream release
Numbers of bird 1
(breeding pairs on
forest)
200
220
205
Wildlife viewing at
walkway site
(qualitative scale)
One iconic sp < 5
One iconic sp < 5,
one >5
Both >5
Flood events
(annual average)
0.2
0.15
0.2
Cost ($MM NPV)
0.1
1.0
0.8
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Stakeholder Preferences
Preferences for levels of performance …
 Given different expectations from management
alternatives (the options on the table):
– Which option provides the least satisfaction?
– Which is the best/most preferred?
– How to other (intermediate) options compare, relative
to the endpoints?
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility
Utility
1.0
0.0
worst
MULTI-CRITERIA EVALUATION
Performance
best
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Alternatives Matrix (2)
Status quo
Downstream dam
Upstream release
Numbers of bird 1
(breeding pairs on
forest)
200
(0)
220
(1)
205
(0.25)
Wildlife viewing at
walkway site
(qualitative scale)
One iconic sp < 5
Both >5
(0.14)
One iconic sp < 5,
one >5
(0.86)
Flood events
(annual average)
0.2
(0)
0.15
(0.8)
0.2
(0)
Cost ($MM NPV)
0.1
(1)
1.0
(0)
0.8
(0.6)
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
(1)
Preferences Across Services
 Which services are preferred by stakeholders?
 Which are they willing to trade off against other
services?
 For which are they willing to accept losses or
reduction of services?
 Competing services implies that preferences for some
require sacrifices on others
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Preferences Across Services
Cumulative weight
Cost
Breeding Pairs
Flooding
0.55
0%
10%
20%
MULTI-CRITERIA EVALUATION
30%
Bird viewing
0.11
40%
50%
60%
0.28
70%
80%
0.06
90%
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
100%
Alternatives Matrix (3)
Measures (Weights)
Alternatives
Status quo
Downstream dam
Upstream release
Numbers of bird 1
(breeding pairs on
forest) (w = 0.11)
200
(0)
220
(1)
205
(0.25)
Wildlife viewing at
walkway site
(qualitative scale)
(w = 0.06)
One iconic sp < 5
(0.14)
One iconic sp < 5, one Both >5
>5
(0.86)
(1)
Flood events
(annual average)
(w = 0.28)
0.2
(0)
0.15
(0.8)
0.2
(0)
Cost ($MM NPV)
(w = 0.55)
0.1
(1)
1.0
(0)
0.8
(0.6)
Overall value
0.56
0.39
0.42
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Decision Support
 At this point: stakeholder preferences for outcomes,
weighted by preferred criteria (services) and by
preferences for different levels of performance for
each criterion (utility).
 This information should inform the decision.
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
But …
 What is embedded in the utilities?
 How consistent are stakeholder preferences?
 How to reconcile heterogeneous preferences among
stakeholder factions?
→ unpack and explore the elicitation process …
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Exercise
 Utility:
– Bird population numbers (context)
– Wildlife viewing (qualitative scales)
– Flood risk (asymmetric stakeholders)
 Preferences across criteria:
– Heterogeneous stakeholder populations
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (1)
Bird populations:
Option:
Status Quo
Downstream Dam
Upstream Release
Birds (#)
200
220
205
 What is the relative value of these population levels?
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (1)
1.0
Option A:
Linear
interpolation
MULTI-CRITERIA EVALUATION
0.25
0.0
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (1)
 Option A: Interpolation … What if?
the range were 200-500?
there was a minimum
viable population size
of 500?
200
500
200
MULTI-CRITERIA EVALUATION
500
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility Estimates
How might we estimate these curves?
 Linear (or nonlinear) interpolation
 Model-based (e.g., utility = survival likelihood)
 Elicitation (of stakeholder, by expert)
 Survey (with attention to sampling frame!)
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (2)
Wildlife viewing:
Option:
Wildlife viewing
Status Quo
Downstream Dam
Upstream Release
1 spp, < 5
1 spp < 5, 1 < 5
both spp, > 5
 How to rate preferences for qualitative measures?
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (2)
Option B: the Ratio method for qualitative data …
 Step 1: list all possible observations (cases)
Category (obs)
Ratio
Points
Utility
Neither
1 spp, < 5x
1 spp, > 5x
Both spp, < 5x
1 spp < 5x, 1 > 5x
Both spp, > 5x
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (2)
Option B: Ratio method …
 Step 2: rate each case relative to the worst case
Category (obs)
Neither
Ratio
Utility
(worst)
1 spp, < 5x
2x
1 spp, > 5x
2.5x
Both spp, < 5x
5x
1 spp < 5x, 1 > 5x
7x
Both spp, > 5x
8x
MULTI-CRITERIA EVALUATION
Points
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (2)
Option B: Ratio method …
 Step 3: multiply ratios by worst-case base score (10)
Category (obs)
Ratio
Points
(worst)
10
1 spp, < 5x
2x
20
1 spp, > 5x
2.5x
25
Both spp, < 5x
5x
50
1 spp < 5x, 1 > 5x
7x
70
Both spp, > 5x
8x
80
Neither
MULTI-CRITERIA EVALUATION
Utility
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (2)
Option B: Ratio method …
 Step 4: divide through: (points-min)/(max-min)
Category (obs)
Ratio
Points
Utility
(worst)
10
0.00
1 spp, < 5x
2x
20
0.14
1 spp, > 5x
2.5x
25
0.21
Both spp, < 5x
5x
50
0.57
1 spp < 5x, 1 > 5x
7x
70
0.86
Both spp, > 5x
8x
80
1.00
Neither
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (3)
Flood risk reduction:
Option:
Flood risk (avg)
Status Quo
Downstream Dam
Upstream Release
0.20
0.15
0.20
 How might different stakeholder groups value these
options?
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility (3)
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Utility
Utility estimates are dependent on:
 Range of options on the table
 Extremes of this range (worst, best cases)
 Which stakeholders are engaged
– Who (which groups/factions)
– Where (geographic location and extent)
 Timing of the ratings, including
– Immediate temporal context
– Planning horizon
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Preferences Across Services
Cumulative weight
Cost
Breeding Pairs
Flooding
0.55
0%
10%
20%
MULTI-CRITERIA EVALUATION
30%
Bird viewing
0.11
40%
50%
60%
0.28
70%
80%
0.06
90%
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
100%
Preferences Across Services
Eliciting weights for services …
 “Slider bar” method:
– Forces the weights to add up properly to 1.0
 Ratio method:
– Rank services from least to most preferred
– Assign ratios relative to least preferred
– Compute scores and relativize to sum to 1.0
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Preferences Across Services
 Ratio method across services:
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Alternatives Matrix
Measures (Weights)
Alternatives
Status quo
Downstream dam
Upstream release
Numbers of bird 1
(breeding pairs on
forest) (w = 0.11)
200
(0)
220
(1)
205
(0.25)
Wildlife viewing at
walkway site
(qualitative scale)
(w = 0.06)
One iconic sp < 5
(0.14)
One iconic sp < 5, one Both >5
>5
(0.86)
(1)
Flood events
(annual average)
(w = 0.28)
0.2
(0)
0.15
(0.8)
0.2
(0)
Cost ($MM NPV)
(w = 0.55)
0.1
(1)
1.0
(0)
0.8
(0.6)
Overall value
0.56
0.39
0.42
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Extensions & Caveats
 Uncertainty:
– Set ranges of values to bound uncertainty
– Recompute alternatives matrix for range
– Does the favored outcome vary?
 All of this is localized to the decision context:
– Which services, range of values, stakeholders, location,
timing
– None of this is likely to be very transferable
 Heterogeneity of stakeholders?
– Alternative alternatives matrices
– Use difference to frame further discussion
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
Conclusions
 This approach can be applied in a wide variety of
decision contexts, to a wide variety of services
 The results are context-dependent
 The mechanics are simple but not easy
– Get help! (Help is increasingly available)
MULTI-CRITERIA EVALUATION
Federal Resource Management and Ecosystem Services Guidebook | nespguidebook.com
nespguidebook.com
For more information, contact Lydia Olander:
[email protected]
Federal Resource Management and Ecosystem Services Guidebook
nespguidebook.com