Asia Pacific Managemenl Review (2001) 6 (1), 39-52
A validation procedure for multicriteria analysis:
application to the selection of scholarship students
Chung-Hsing Yeh' and Robert J. Willis"
Different multicriteria analysis (MA) methods often produce different outcomes for selecting or ranking a set of decision altematives involving multiple criteria. This paper presents a new
procedure to the selection of compensatory MA methods based on multiattribute value theory via
sensitivity analysis of criteria weights. In line with the context-dependent concept of informational
importance , the procedure examines the consistency degree between the relative sensitivity of individual criteria of 叩 MA method and the 悶 lative degree of influence of criteria indicated by Shannon 's entropy concep t. The decision outcome produced by the most appropriate method c.n best
reflect 出 e decision information embedded in the problem data se t. An empirical study of a scholarship student selection problem is conducted to demonstrate how the procedure can validate thc
decision outcomes produced by different MA methods. With its simplicity in both concept 個d
computation , the procedure c.n be applied in general decision problems solvable by compensatory
MA methods. It is particularly suited to large-scale MA problems where the decision outcomes
produced by different methods differ significantly
Keywords:
Multicriteria Analysis; 5election; Criteri. Weigh阻 ; Entropy; 5ensitivity Analys的
1. Introduction
Multicriteria analysis (MA) has been widely used in ranking or selecting
one or more altematives from a finite number of altematives with respect to
multiple , usually conflicting criteria or attributes. Numerous MA methods have
been proposed for a large variety of decision problems. [n most decision situations , different methods often produce inconsistent outcomes for the same problem [20 , 26]. [n other words , the decision outcome is dependent on the method
used. The outcome inconsistency of MA methods increases as the number of
altematives to be selected or ranked increases , or when the altematives have
similar performance [1 1]. Selecting a valid method for reflecting the values of
the decision maker (DM) 的 thus important, in particular ifthere are a number of
MA methods available and the alternatives involved have similar performance
lf the ranking outcomes of different methods are significan tI y different, the validity issue becomes crucial [8]. This is exact[y the case for the scholarship student selection problem to be exemplified in this paper. To address the validity
issue in the context of the selection of scholarship students , we develop an empirical validation procedure for examining different se lection outcomes produced by different MA methods. The procedure suggests the use of the decision
• Co凹閒rπT閏p仰ond
叫din
時
嗯
n
2咆
g author.
5c仙
h ∞叫10fBu
閏SIn間
ne
旭
es盟s5
句
戶
y
血
s
t包c間 ， Mo
叩na
旭a訓
晶
sl
hU
山"'附
鴨
v
e店
JSlty
吼
扎， Cαla'
Y
晰戶
lytO
叩
n， 叫
V l叫
啪
cα
t怕
叮叩n旭
o
.
.3
兒8∞， A固倒1吭
丸,
a
恤叩叫叫
E
訓I:Cα
a
臼
h】u
叩
m
嗯
1
.. 5chool of Bus別iness 5ystems, Monash University , Clayton , Victoria 3800, Australia
39
Chung-Hsing Yeh and Roberl J. WiII is
outcomes that can best ref1ect 也e decision inforrnation embedded in the problem data se t.
In subsequent sections, we first describe a scholarship student selection
problem and forrnulate it as an MA problem, fo11owed by a discussion of suitable MA methods. We then develop a new validation procedure for choosing
the most valid selection outcome for situations where the rankings produced by
different MA methods di叮叮. Fina11y we conduct an empirical study to demonS甘ate the effectÌveness of the procedure
2. The scholarship student selection problem
An Australian university departrnent has recently offered a number of industry sponsored scholarships to the frrst year students in its Bachelor of Business Systems program on a yearly basis. The scholarships are for duration of
tbree years , su阿 E此ct
叫 tωo 伽
t he 叫o
仙
la
帥
吋
r
shi中
p 岫
h 01恤
岱
d
ersκs' 叫
sa
剖訓
tt岫
s
st削
ud
副le
仿s. During 航
t hei叮r s仇
ωJdi昀
es丸， 臼
s cholar臼ship students are requi叮red tωo work wit血
h
industry 叩
s po
∞
nsωor昀s 伽
fio
仗r a total pe
凹
no
的
od of one year under industry-based learning
programs. Scholarship students who fail to pass a perforrnance review process
at the end of an academic year wi11 have their scholarships withdrawn. This is
not desirable due to the waste of the lirnited financial resource and the negative
impact on the departrnent 's overall profile. Despite the fact that there is no “ correct" decision and the perforrnance of scholarship students is genera11y beyond
the departrnen t' s con甘01 ， the departrnent needs to justify that scholarships are
granted to the best-qualified applicants in a fa吼叫ional manner
The applicants for the scholarships are selected based on their perfo口TI­
ance on non-academi仇 恨心 tative selection criteria via an interview process
The reason for excluding the academic criteria is that a11 the applicants have
overcome a considerable academic hurdle to become eligible for study at the
departrnent. Th erefore they are expected to be capable of completing a11 the
spec凶ed acadernic requirements of the scholarship. Based on comprehensive
discussions with industry sponsors , a set of eight criteria relevant to the industry-based learning program is determined. These criteria are brie f1 y discussed
below
(1) Community services (C,). Voluntary work within the community by applicants is viewed favorably. Examples include activities involved in social welfare , coaching, peer suppo前， etc
(2) SportslH obbies (C2). Non-work related
40
Chung-Hsing Yeh and Robert J. Willis
(4)
Energy (C4 ). Future demands placed on the applicants will require energy that indicates a positive attitude and a willingness to participate in
demanding tasks
(5) Communication skills (C5). The applicants' ability to communicate is
lmporta瓜， 的 they need to interact with other individuals in their industry-based learning placements. Their manner of speaki月， writing ability
and appearance are all communication enablers or disablers
(6) Attitude to business (C6). Most applicants , after finishing their s個dies ，
will work in the business world. Their attitude to , and ambitions in the
corporate world are crucial in indicating what kinds of employees they
will make
(7) Maturity (C7). This is related to the applicants ' willingness and ability to
take on responsibility for their current situations. The applicants' perform且nce in acadernic studies and indus甘于based leaming placements is
highly dependent on the degree of responsibility they undertake
(8) Lead ership (Cg) . Potentialleadership qualities are preferred， 的 they ref1 ect the applicants ' overall perforrn且nce for their acadernic studies and
indus甘于based lea口也19 placements
In this scholarship student selection proble m, each criterion is weighted
equally, because the DM or the stakeholders (indus甘y sponsors) cannot deterrnine other acceptable weights in a fa咒 convincing manne r. This is in line with
the principle of insufficient reason [1 呵， which suggests the use of equal
weights if the DM has no reason to prefer one criterion to anothe r. In addition,
no single criterion weighting method can guarantee a more accurate result , and
the same decision makers may elicit different weights using different methods
[5 , 21 , 24]. In practical applications , this implies that there is no easy way for
deterrnining criteria weights and there are no criteria for deterrnining what the
true weight is [21]
The perforrnance of 出e applicants on the eight criteria will be assessed on
a 6-point Likert-type scale, ranging from 5 (ex仕emely high) to 0 (ex仕emely
low). With this problem setting, an intuitive method , referred to as the total sum
(TS) method, can be used. The TS method simply SU ITlS the perforrnance ratings
of each applicant on all criteria to a total score. The TS method implies tradeoffs between criteria - a high value on one criterion can compensate for low
values on other criteria [2]
In the MA context, the TS method is in line with the compensatory methods based on multiattribute value theory (MA VT) [10] , which is t
41
Chung-Hsing Yeh and Robert J. Willis
ers [26]. This is mainly due to 也e仕 simplicity in both concept and computation
In addition , MA VT-based MA is the most appropriate quantitative tool for
group decision support systems [1]
To facilitate the presentation of suitable MAVT-based MA methods for
the scholarship student selection proble m, we fo口nulate the problem as an MA
problem
3. The multicriteria analysis problem and methods
1, 2 ，、 m)
The MA problem involves a set of m altematives A; (i
These altematives are to be evaluated with respect to a set of n criteria (or attributes) ç (j 斗 ， 2 ， ..., n). Two data sets are to be given: (a) a decision rnatrix
X = {旬， i = 1, 2, ..., m; j = 1 ， 丸
， n } for representi ng 也e perforrnance ratings
of altemative Aj (i = 丸之 ..， m) with respect to criterion ç (j = 1, 2, ..., n) and
(b) a weighting vector W = (叭 ， W 2. ... , w.) for representing the relative importance of the criteria. The perforrnance ratings and the criteria weights are cardinal values that represent the DM 's absolute preferences. The decision problem
is to rank all the altematives in terms of their overall preference value , which is
obtained based on the two data sets
With the selection problem fo口nulated above , the TS method and three
MA VT-based MA methods described below can be used. These methods assume that the criteria weights and perforrnance ratings of alternatives are given
on an interval scale. Another widely used MA method is the analytic hierarchy
process (AHP) [1 呵 ， with which the values for 的 and x ij are given based on a
ratio scale of preferences. In practical applications where effective guidance is
not available , the use of simple MA VT-based MA methods may be more justifiable than AHP [19]
3. 1. Th e Simple Additive Weighting
(SA 的 Method
The SA W method, also 1cnown as the weighted sum method , is probably
the best 1cnown and most widely used MA method [9]. The basic logic of 帥
SA W method is to obtain a weighted sum of the performance ratings of each
altemative over all criteria. The SA W method norrnally requires normalising
the decision matrix (均 to allow a comparable scale for all ratings inXby
I
X;;
| 一一-一 ，
I maxX;;
，η
j;
=i
I
if j is a benefit attnbute
; i = 1, 2, ...， ，1 叫，
rrun A ij
(l )
1.→于一 ， if jisacost 甜咖 te
/HJ
where r ij (0 三 rij $ 1) is defmed as the norrnalized perforrnance rating of altemative A; on criterion Cj • This norrnalisation process transforms all 伽叫 tngs m a
42
Clwng-Hsing
Yeh Gnd Robert J.
Willis
linear (proportional) way , so that the relative order of rnagnitude of the ratings
rernains equa l. The overall preference value of each altemative (只) is obtained
by
L
V; =
} =
Wj
勻
i =
1, 2, ... , m
(2)
J
The greater the value (V;), the more preferred the altemative (A;). Re.
search results have shown that the linear form of 甘ade.offs between criteria
used by the SA W method produces extremely close approxirnations to compli.
cated nonlinear forn芯， while maintaining far easier to use and understand [9]
The TS method discussed in the previous section is the same as the SA W
method , except for the normalization process. To facilitate the comparison be.
tween the TS method and other three MA methods , the overall preference value
of each altemati閱 (V，) by the TS method is given as
X;;
只=于 17i t=1 , 2, , m
(3)
where M is a constant which equals the rnaximum score on the measure scale
(e.g. M = 5 in the problem setting)
3.2. Th e Weighted Product (WP) Method
The WP method use multiplication for connecting criteria ratin郎， each of
which is raised to the power of the corresponding criteria weight. This multipli.
cation process has the same effect as the normalisation process for handling
different measurement units. The overall preference score of each altemative (S;)
is given by
n
n
S;
i = l ，丸
X ijWj
， m
(4)
where I j=l w j = 1. wj is a po叫ve power for benefit criteria and a negat附
power foi cost criteria. In this study, for easy comparison with other methöds ,
the relative preference value of each alternative (只) is given by
、-)
vi
u-n巾 4
nH尸
",
-*,
-
、
ν
-
1J
,
J
J
43
I
弓L
-
一一
-.
f
XII-x
yt
m
(5)
Chung-Hsing Yeh αnd Robert 1. Willis
where x' j = max xυand 0 三只三 1. The greater the value (V;), the more preferred the altemative (A;)
3. 3. The Techniqlle for Order pr，φrence by Similarity /0 Ideal Sollltion (fOPSIS)
TOPSIS is based on the concept that the most preferred altemative should
not only have the shortest distance from the positive ideal solution , but also
have the longest di stance from the negative ideal solution [9 , 27] . This concept
has been widely used in various MA models for solving practical decision problems (e.g . [13 , 25]). This is due to (a) its simplicity and comprehensibility in
conce阱， (b) its computational efficiency , and (c) its ab ility to measure the relative performance ofthe decision altematives in a simple mathematical form
TOPSIS norm ally requires normalising the perfo口nance ratings of alternative A; on criterion Cj by
1
1m
rij = X lf/ 、 I 2:
i = I ， 2 ，... ， m;j = l ， 丸
x'/ ;
， n.
(6)
The pos itive ideal solution A+ and the negative ideal solution k can be
determined based on the weighted normalized ratings (Yij) by
Yij =
Wj 旬 ;
A + = (y
i = I , 2, ... , m; j = I , 2, ..., n.
t ' y; ,... , y:) ;
刀
A - = (y i一 ， y 2" , .. , y;; )
J
-1 min Yij ' if j is a cost attri bu峙
where
L
1
(8)
max Yij' if j is a benefit att巾 ute
I
".+ _
(7)
，
I
min yij , if j is a benefit attribute
f=1mix yv , lf
j
j = 1, 2,. ., n
(9)
att伽te
lsamt
The distance between altematives A; and the positive ideal solution and the
negative ideal solution can be calculated respectively by
)
=
J
The overall preference value of each alternative (V;) is given by
44
，
勻h
νJ
"
v'
H
d
1.
=
,.'
/'、、
D
叫 UB
V
寸臼
YM ) 2
n
+
一-
ZJ
y
...
Ilt-
rs.
、
n寸 ，=
一一
D+
m (
)
AU
Clmng-Hsing
Yeh and Robert J. WiII is
V,
~二一
D ;+ + D 戶
，
The greater the value (V;) , the more
i;
1. 2. .. .. m
prefeπed
the
(l l )
altemative 徊 ;)
3.4. Differences between MA Methods
The main differences between the four methods described above lie in (a)
the normalisation process for comparing all performance ratings on a common
sca le , and (b) the aggregation of the normalized decision matrix and weighting
vector for obtaining an overall preference value for each altemative. Due to
these structural differences , the ranking outcome produced by the four methods
may not always be consistent for a given decision matrix and weighting vector
In fact , the empirical study presented in this paper shows that the rankings are
so different that the relative effectiveness of the methods used needs to be examined to help the decision makers make rational decisions. To this end , we
develop a new procedure for empirical validation of the four suitable methods
via sensitivity analysis of criteria we ights
4. The approach to the validation of decision outcomes
The valid ity issue of the selection outcome, particularly for admission decisions , has been conventionally addressed along the lines of the predictive validi ty of the selection instruments (criteria). A typical study of the predictive
validity is to examine the correlation between the selected students' results on
the chosen selection criteria and an indicator of subsequent academic performance [23]. However, the selection criteria used in higher education admission
processes varies widely among programs and no consistent conclusions can be
reached on the predictive values of these criteria [22] . This may pa此 ly be due
to the fact that the predictive validity of the selection instruments is not in itself
sufficient for an assessment of the validity of a selection , although it can be a
critical factor [23]
ln this paper , prediction is not the stated pu巾的 e for the scholarship student selection problem , partly because we do not have sufficient empirical data
to conduct a predictive validity study. Thus , the selection of scholarship students is made on the grounds of the app licants ' merits (performance ratings) ,
based on a g iven set of criteria in accordance with the requirements of the industry-based leaming program for which the scho larsh ips are designed. As such ,
we focus the validity issue on the selection of suitable MA methods
MA research on the validity issue has focused on the problem of selecting
an MA method under various decision contexts along two Ii nes of development
(a) experimental comparisons of MA met
45
Chllllg-Hsillg Yeh and Robert 1. Wi l/ is
ness of use an d/or theoretical validity [2旬 ， and (b) method selection procedures
for specific characteristics of the decision problem and distinct features of
available methods in the form of décision support systems [12 , 17] or as general
selection principles [7] . While the results of experimental comparisons cannot
be used as guidelines for a DM to select a proper MA method for an application
[12] , the method selection procedures may not always make a clear unequivocal
choice 凹 ， in particular between methods of the same category. Due to their
implicit and explicit assumptions, the applicability of the methods selected remams uncerta凹， as evidenced by the fact that these selection procedures do not
normally examine 曲e validity of decision outcomes. Despite the significant
development in MA method selection research , the validity of decision outcomes remains an open issue. This is rnainly due to the fact that the “ true" cardinal ranking of altematives is not known. To address this issue for the scholarship student proble m, in what follows we present an empirical validation procedure based on the context-dependent concept of informational importance ,
given the selection criteria and studen筒 ， performance ratings
The decision m且trix for performance ratings of alternatives contains a certain amount of decision information for 由e MA proble m. For a given weighting
vector, the decision outcome is largely dependent on the degree of divergence
of altematives ' performance ratings on individual criteria. The more divergent
the perfo口nance ratings for a criterion, the more important the criterion for 也e
problem [16 , 27]. This means 也at the criterion has more in f1 uence on the decision outcome , thus transmitting more information to 出e DM. This also implies
that a criterion is less important or inf1 uential for a specific problem if all alternatives have similar performance ratings for that criterion. This concept of decisive information has been used as objective weights of criteria importance in
inter-company comparison problems which requ叮e being conducted on a commonly accepted basis [3, 4]
Shannon's en甘opy concept [1 5] is well suited for measuring the relative
contrast intensities of performance ratings to represent the average intrinsic
information transmitted to the DM [9] . This concept coincides with the contextdependent concept of informational importance [27]. For decision rnatrix X , the
侃呻
Pμ
叫
cte
吋
d 叮
i r枷 m
by the en甘 opy value (令
砂) as
m
ej
wher E K =
=-
TL
mm
ls
k
L
am
P iJ ln PiJ;
s叫
t徊a缸
叫
m
伽恤圳
ntw
咖
划
h泌叫
ic
1沁叫
cl凶t
46
j
= 1, 2, ...n
(1 2)
ChU fJ g-Hsing Yeh and Robert 1. Willis
Pij=Xij/L>qj;
I
i = I , 2 , ..., m;j=I , 2 , ..., n .
(1 3)
q~1
The degree of d附rgence (dj ) of the average 叫rinsic inforrnation provided by
the correspo吋呵 performance ratings on criterion Cj ca的e defined as
dj
= 1-
e j;
j
= 1, 2, ...n
(14)
The value of 吋 in (14) 即 resents the inherent con甘ast I忱的 ity of criterion ý. The 叫ative degree of inf1 uence of criterion C; to the decision outcome
can be deterrnined by
力=作ldq
j=l2n
(1 5)
The value ofjj in (1 5) depends on the decision matrix of the proble m. The
decision outcome of an MA method , determined based on the same decision
matrix , should ideally re f1 ect this decisive information implicitly transmitted to
the DM. However，也is requirement may only be met to some degree by the MA
method due to its structural characteristics. The degree to which the method
meets this requirement can be re f1 ected by the relative degree of inf1 uence of
individual criteria on the decision outcome , as compared to other criteria. The
degree of inf1 uence of a criterion can be measured by the degree of sensitivity
of the decision outcome to weight changes of the criterion. This can be carried
out by a typical sensitive analysis process
In this paper, we use sensitivity analysis as a means of determining how
sensitive (the degree of sensitivity) the decision outcome of an MA method is to
weight changes of a criterion. This degree of sensitivity implies the relevance
(the degree ofi nf1 uence) ofthe criterion to the decision outcome. The method is
va Iid , if the relative degree of sensitivity of individual criteria consists with the
va1ue of 吋 in (14) , indicati時 the method has the abil叮 to ref1ect 出e decision
information embedded in the problem data se t. In the selection of MA methods
for a given problem in terrns of this validity measure , the method with the highest degree of consistency should be used. As a result , the most rational decision
outcome can be identified as it best re f1 ects thti decision inforrnation embedded
in the problem defined by a given decision matrix and weighting vector
5. Empirical study
To illustrate how the validation procedure can help the DM select the
most rational decision outcom巴 ， we use the data in a recent year. There were 57
app Ii cants attended the interview. The result of 也is in紀rVlew process constl-
47
Chung-Hsing Yeh and Roberl J. WiIl is
Table 1 Comparison oftop 10 rankings between four methods
2
WP
SA W
TS
Ranking
TOPSIS
A;
V;
A;
V;
A;
V;
A;
V;
A,
1. 000
A,
1. 000
1. 000
1. 000
0.915
,
0.975
A
,
0.975
A,
A2
0.973
A,
A2
AJ
0.950
AJ
0.950
AJ
0.946
AJ
0.892
,
,
0.925
0.900
A
,
,
0.925
A4
0.868
As
0.920
0.895
A.
0.900
A
,
A.
0.900
0.900
A.
。 900
A
,
0.895
As
0.855
0.847
A
A9
0.846
。 900
A6
0.895
0.887
A9
A
,
,
。 900
0.900
AIO
0.831
。 900
A
,
。 887
AII
0.815
0.875
A ,o
0.870
A
,
0.81 1
A
A
A
A
6
7
A
8
A
9
A9
。 900
A9
10
AIO
。 8 75
AIO
,
,
tuted the decision matri x X with m = 57 and n = 8. The weighting vector W used
was (0 .1 25 , 0.125 , 0 .1 25 , 0.125 , 0.125 , 0.125 , 0 .1 25 , 0. 125) which satisfies
L.J=' wj
=J
The ranking outcomes obtained by the four methods are not consistent ,
which may cause some decision difficuJties. As an iJJustration , TabJe 1 shows
the top 10 rankings with the four methods. For easy comparison , applicants A; (i
= 丸之. .， 57) are denoted in order of their overaJJ preference vaJue Vj (i = 1,
2 ，叫 57) by the TS method. If there were onJy 10 appJicants to be seJected , A6
wouJd not be seJected using TOPSIS , and AII wouJd not be seJected using TS ,
SA W, or WP. For most decision situations where the number of appJicants to be
selected varies , there wiJJ be some appJicants being in cJ uded using some methods and being excJuded with other methods
To vaJidate the decision outcomes produced by the four methods , a sensitivity analysis procedure was carried out for each method . Th e procedure aimed
at deterrnining the degree of sensitivity of each criterion to the decision outcome of each method . With a method , the procedure carried out for a criterion
is given beJow
1. Assign all criteria a weight vaJue of 1,called basic weights (i .e 句= 1;) = 1 立，. . .， 8)
2. Change the weight for the criterion in the range between 1 and 2 , with an
increment ofO.I , whiJe other criteria are kept at their basic weights
3. Norm但 m叫i叫但naw咖 by
川 = bi
月1bJ
叫叫小 l
4. Apply the method with the weights obtained at step 3
5. Calcu1ate the percentage of ranking changes ， 的 compared to the ranking
outcome with equal weights
48
Chllllg -Hs;llg Yeh and Robert J. W; /lis
The range sening of criteria weight changes used in the procedure is
based on the assumption that no single criterion is more than twice important as
any other criter間; a senillg confirmed by the DM. The result of 也is anal ys is is
summarized in Figure 1, which shows the average degree (in percentage) of
influence of individual criteria to the decision outcome by fo山 different me也­
ods
The result in Fig叮e I indicates the criteria weights have significant influence on .the ranking outcome. Although the sensitivity of criteria weights is
largely dependent on the data in the decision matrix, its relative degree is also
influenced by the method used. For all methods used , C5 is 由e least sensitive
criterion, while the most sensitive criterion is C1 or C3 depending on the method
used. The average degrees of influence of all criteria by TS , SA W , WP and
TOPSIS are 43 .4%, 43 .1%, 35 .2% and 53.0% respectively. This indicates 也at
TOPSIS is the most sensitive method , while WP is the least sensitive method
for the problem data se t.
The degree to which each method reflects the decision information embedded in the problem data set can be measured by the correlation (or consistency degree) between the relative degrees of influence of criteria obtained by
sensitivity analysis for the method (as shown in Figure 1) and the relative det
grees of influence of cαn岫
Table 均2) . The results for TS , SA W, WP and TOPSIS using Pearson's correlation coefficients (Spearrnan's rank correlation coeflicients) are 0.77 (0.62) , 0.75
(0.62) , 0.69 (0 .4 5), and 0.94 (0.85) respectively. Clearly, the decision outcome
produced by TOPSIS can best match the decision information content embedded in the decision matrix
:::;;- 70
字的
宣
ω
11 55
..: 50
~ 45
'õ 40
:l 35
屆 30
å
25
CI
αC3
c4
C5
C6
c7
c8
Criteria
Figure I Degree of influence of individual criteria to 由e decision outcome by
four different methods
49
Ch l/ ng-Hsing Yeh and Roberl J. Wi /lis
Table 2 Values for informational
C~.
Entropy va lu e 冉
Diversification degree dj
Influence degree Ji
~
impo前ance
~
of criteria
~
~
~
~
0.9778 0.9928
0.9800 0.9902
0.9921
0.9897
0.99 16 0.9861
0.0222 0.0072
0.0200
0.0079 0.0103
0.0084 0.0139
。 2227
0.0098
0.07 18 0.2010 0.0980
0.0790 0.1034 0.0842
0.1398
6. Conclusion
There are normally a number of methods available for solving MA problerns , defined by a given decision matrix and weighting vecto r. Different methd ecαIS
釗ion outcomes 必
for ran
叫
1甘king
汝
all the alternaods often produce incωonslst紀en叫It 岱
t加
Ive
臼s. Despite the importance of the validity of decision outcomes, very few
studies ha ve been conducted to help 也e DM make valid decisions. In this paper,
we have presented an empirical validity procedure to the selection of the most
appropriate method for a given problem data se t. The most appropriate method
is the one that best reflects the decision information content, indicated by the
relative con甘ast intensity of altematives ' performance ratings on each criterion
based on Shannon' s en甘opy concep t. An empirical study of a scholarship s個­
dent selection problem has been conducted to illustrate how the procedure can
be used to help select the most valid method for a given data se t. Different
problem data sets rnay result in different methods being selected. With its simplicity in both concept and computatio口 ， the procedure can be applied in general
decision problerns solvable by compensatory MA methods. It is particularly
suited to large - sca le 恥1A problerns where the decision outcomes produced by
different methods differ significantly
References
[1]
Bose U , Davey AM and Olson DL (1 997) Multi-attribute utility methods
in group decision m且king : past applications and potential for inclusion in
GDSS. Omeg且， lnternational Journal of Management Science , 25(6) ,
[2]
Davey A, Olson D and Wallenius J ( 1994) The process of multiattribute
decision making: a case study of selecting applicants for a Ph.D. program
European Journal ofOperational R釘earch ， 72, 469-484
Deng H , Yeh C-H and Willis , R) (2000) In ter-company comparison using
modi白ed TOPSIS w岫 ob
句'jecl1ve wel屯
ght臼S. CO Illψ
'put仿
er.
叮
's & Operat
search , 27( 10) , 963-973
Diakoulaki D, Mavrotas G and Papayannakis L (1 995) Determining objective weights in multiple criteria problerns: the CRITIC method. Computer
& Operations Reseat叫 ， 22( 7)， 763 -770
691-706
[3J
[4J
50
Clwng-Hsing Yeh and Roberl J. Willis
[5]
[6]
[7]
[8]
[9]
[1 0]
[11]
[1 月
[13]
[14]
[15]
[16]
[17]
[18]
[19]
Doyle JR, Green RH and Bottomley PA (1 997) Judging relative importance: direct rating and point allocation are not equivalent. Organizational
Behaνior and Human Decisioll Process凹， 70 ， 6$-72
Dyer JS , Fishburn PC, Steuer RE , Wallenius J and Zionts S (1992) Multiple criteria decision making, multiattribute utility theory: the next ten
years. Managelllenl Science , 38(5) ,645-653
Guitouni A and Martel JM (1998) Tentative guidelines to help choosing
an appropriate MCDA method. E lI ropean JOllrnal of Operalional Research , 109, 501 -521
Hobbs BF, Chakong V, Hamadeh W and Sta岫iv EZ (1992) Does choice
of multicriteria method matter? An experiment in water resources planning. Warer Resollrces Research , 28 , 1767-1 780
Hwang CL and Yoon K (1981) Mulliple altribule decision lIl aking: melhods and applicatio肘， a slale呼 Ih e-arl survey. Springer-Verlag, New
York
Keeney R and Raiffa H (1993) Decision with Multiple 0句ectiν缸， Pr可er­
ences and Va lue Trade Offs. Cambridge University Press , New York
Olson DL, Moshkovich HM , Schellenberger R and Mechitov AI (1995)
Consistency and accuracy in decision aids: Experiments with four multiattribute systems. Decision Sciences , 26 , 723-748.
Oze rno y VM (1 992) Choosing the best multiple criteria decision-making
method . ln戶川1O tion Systellls and Operalional Research , 30(2) , 159-171
Parkan C and Wu ML (1999) Measurement of the perfonnance of an inveslrnent bank using the operationa1 competitiveness rating procedure
Omega, lnternational Journal of Management Science, 27 , 201-217
Saaty TL (1994) How to make a decision: the analytic hierarchy process
Inlerfaces , 24 , 19-43
Shannon CE and Weaver W (1947) Th e Mathematical Theory ofCommunication. The University of lllinois Press, Urbana
Shipley MF , de Korvin A and Obid R (1991) A decision making model for
multi-attribute problems incorporating uncertainty and bias measures
Compulers & Operations Research , 18 , 335-342
Siskos Y and Spyridakos A (1999). Intelligent multicriteria decision support: Overview and perspectives. European Journal of Operational Research , 113 , 236-246
Starr MK and Greenwood LH (1977) Norrnative generation of alternatives
with multip1e criteria eva1uation, in: M .K. Starr and 恥1 . Ze1eny (Eds.) ,
Multiple Criteria Decision Making, North Holland, New York , 111-128
Stewart TJ (1992) A critical survey on the status of multip1e criteria decision making: 出eory and practice. 0
51
Chung-Hsing Yeh and Robert J. Wi l/ is
[20] Voogd H (1 983) Multicriteria Evaluation for Urban and Regional Planning. Pion , London
[21] Weber M and 80rcherding K (1993) 8ehaviora1 intluences on weight
judgments in mu1tiattribute decision making. European Journal of Operational Research , 67 , 1-12
[22] Wi1son T (1999) A student se1ection method and predictors of success in a
graduate nursing program. Journal ofNursing Educatio月， 38(4) ， 183-187
[23] Wo1ming S (1 999) Validity issues in higher education se1ection: a Swedish examp1e. Studies in Educational Evaluation , 25 , 335-351
[24] Yeh C-H , Willis RJ , Deng H and Pan H (1 999) Task oriented weighting in
mu1ti-criteria ana1ysis. European Journal of Operational Research , 119( 1),
130-146
[2月 Yeh C-H , Deng H and Chang Y-H (2000) Fuzzy mu1ticriteria analysis for
performance evaluation of bus companies. European Journal of Operational Research , 126(3) , 1-15
[26] Zanakis SH , Solomon A, Wishart N and Dublish S (1998) Mu1t卜attribute
decision making: A simu1ation comparison of se1ect methods. European
Journal ofOperational Research , 107 , 507位9
[27] Zeleny M (1 982) Multiple criteria decision making. McGraw-Hi l1, New
York
52