Transportation Decision Making
Principles of Project Evaluation and Programming
Chapter 18
Evaluation of Transportation Projects
and Programs Using Multiple Criteria
Kumares C. Sinha and Samuel Labi
„ Decision criteria can have multiple dimensions
„ Dollars
„ Number of crashes
„ Acres of land, etc.
„ All criteria are not of equal importance
„ For a given criterion, different stakeholders may have
different weights.
Typical Steps in Multi-Criteria
Decision Making
Scaling
1. Establish
Transportation
Alternatives
2. Establish
Evaluation Criteria
3. Establish
Criteria Weights
Weighting
Amalgamation
4. Establish Scale to be Used for
Measuring Levels of Each Criterion
5. Using Scale, Quantify Level
(Impact) of Each Criterion for Each
Alternative
6. Determine Combined Impact of all
Weighted Criteria for Each Alternative
11. Determine the Best Alternative
Establishing Weights
„ Weights reflect the relative importance attached by
decision makers to various criteria
„ In some cases, the decision maker refers to the
agency as well as the facility user. In those cases, the
weight used for each criterion is a weighted average of
the weights from these two parties.
Weighting Techniques
1.
2.
3.
4.
5.
6.
7.
Equal Weights
Direct Weighting
Derived Weights
Delphi Technique
Gamble Method
Pair-wise comparison: AHP
Value Swinging
Equal Weights - Example
Project Cost
33.3%
Travel Time Saving
33.3%
VOC Saving
33.3%
Direct Weighting
1. Point Allocation – A number of points are
allocated among the criteria according to their
importance.
2. Ranking – Simple ordering by decreasing
importance.
Point allocation is preferred because unlike ranking, it
yields a cardinal rather that an ordinal scale of
importance.
Point Allocation (0-100)
(Cardinal)
Ranking
(Ordinal)
Project Cost
70
1
TT Saving
50
3
VOC Saving
60
2
Regression-Based Observer-Derived Weighting
1.
Survey respondents assign scores of overall
“benefit” or “desirability” for a given combination
of criteria levels achieved by each alternative
2.
Weights are then the resulting regression
coefficients
Minimize ∑ ε i2
TVi = ∑ ( w jV ji ) + ε i2
j
i = alternative
j = Criterion
TV = score or desirability
Regression
„ 7 Respondents
„ 21 Data Points
„ TV = wcost* Cost + wtime * Time
„
„
wcost = 0.214
wtime = 0.786
R2 = 0.98
Delphi Technique
„ Individual responses aggregated
„ Effect of assessment of other respondents
„ Consensus building
„ Iterative, generally 2 rounds to achieve stable
values
Scaling Methods
GAMBLE METHOD
1.
Carry out an initial ranking of all criteria in order of decreasing
importance. set the first criterion at its most desirable level and
all other criteria at their lest desirable levels
2.
Compare between the following two outcomes:
„
„
3.
Sure thing: The outcome is that the criterion in question is at its
most desirable level while all other criteria at their least desirable
levels
Gamble: In this outcome, all criteria attained their most desirable
levels p% of the time, their least desirable levels (1-p)% of the time
At a certain level of ‘p’ the two situations (sure thing and gamble)
are equally desirable. At that level, the value of ‘p’ represents
the weight for the criterion in question
Example:
Bus Route Assessment
Headway (from 5 to 15 minutes)
Population Served (from 5,000 to 10,000)
Solution:
1. Sure Thing: Bus headway is 5 minutes and population served is 5,000
2. Gamble: Two outcomes:
a. A p% chance of an outcome that headway is 5 minutes and
population is 10,000
b. A (1-p)% chance of an outcome that headway is 15 minutes
and population is 5,000.
Suppose the respondent were found to be indifferent between sure
thing and gamble at p = 60%, then, the relative weight for bus
headway is 0.6.
Pair Wire Comparison
Analytical Hierarchy Process (AHP)
⎡1
⎢1/ a
⎢ 12
⎢...
⎢
⎢⎣1/ a1n
................... a1n ⎤
⎥
1
................... a2 n ⎥
.... .... .... .... .... ....... ⎥
⎥
1/ a2 n ..... ... ....
1 ⎥⎦
a12
aij = relative importance of two criteria I and j on the basis
of a scale of 1 to 9
= wi / w j
Table 18.1: Ratios for Pair wise
Comparison Matrix
Value Swinging Method
1.
2.
3.
4.
Consider a hypothetical situation where all criteria at
their worst values
Determine the criterion for which it is most preferred to
“swing” from its worst value to best value, all other
criteria remaining at their worst values.
Repeat steps 1 and 2 for all criteria.
Assign the most important criterion the highest weight
in a selected weighting range (100 for 1-100 scale) and
then assign weights to the remaining criteria in
proportion to their rank of importance.
Scaling of Performance Criteria
Certainty
- Value Function
Risk
- Utility Function
Uncertainty
- Scenario Analysis
Value Function
a. Direct Rating Method – direct assignment
of value to various levels of a criterion
b. Mid Value Splitting Technique – based on
“indifference” between changes in levels of
criterion.
c. Regression – based on data from direct
rating
Discrete Value Function
Discrete Dis-Utility Function for Performance Measure
of Impact on Natural, Socio-Economic, Historical &
Cultural Resources
0
Extreme
Impact
Utility
Huge
Impact
-100
Major
Impact
-80
Moderate
Impact
-60
Minor
Impact
-40
No
Impact
-20
Continuous Value Function
Dis-Utility Function for Single Performance Measure
of Emissions
Percentage Increase in Emissions
0
0
-20
-40
-60
-80
-100
Utility
20
40
60
80
100
120
Developed Value Functions
Utility Function
Direct Questioning Using the Gamble
Approach
Guaranteed prospect of an outcome vs. risky
prospect of a more favorable outcome.
Example 18.7
„ Utility Functions for agency cost, ecological damage, and vulnerable
population served.
Solution:
For Agency Cost:
Ucost ($30 Million) = 0 (Worst)
Ucost ($ 0 Million) = 1 (Best)
Sure Thing: The outcome is that agency cost is guaranteed to be $20 Million
Gamble: There is a 50% chance that cost is 0 and 50% change it is $30 Million
X50 = $20 Million is the Certainty Equivalent because the expected utility is 0.5
120
100
Cost (in $millions)
Population served (in thousands)
Utility
80
60
40
Wetland lost in acres (in tens)
20
0
0
5
10
15
20
25
Criteria Level
30
35
40
45
Combination of Performance Criteria
„ Pareto Optimality
„ Difference Approach
Net Utility = U(B) – U(C)
„ NPV = PV (B) – PV(C)
„
„ Ratio Approach
Utility Ratio = U(B)/U(C)
„ BCR = PV(B) / PV(C)
„
Cost Effectiveness
„ Costs and Benefits are not necessarily
expressed in the same metrics
„ Indifference Curves
„
Tradeoffs – marginal rates of substitution
between criteria
„
TV = 2*TTR + PCC
Indifference Curves Using Mathematical Form of
Utility/Value Function for Combined Performance
Measures
Ranking and Rating Method
Scorei = Pi ∑ w j Oij
j
For each i
Impact Index Method
Ii = ∑ R j S j X ij + e j R j S j X ij
j
R j = relative weight for criterion j =
wj
∑w
j
j
S j = scaling factor of measurement X of criterion j = 1
e j = RN drawn froma rec tan gular distribution
(−0.5 ≤ e ≤+ 0.5)
max( X1 j , X 2 j ,........, X nj )
Table E18.10.1: Performance of Alternatives
Figure E18.10: Plot of Confidence
Intervals