Methods of Psychological Research Online 1998, Vol.3, No.1
Internet: http://www.pabst-publishers.de/mpr/
c 1998 Pabst Science Publishers
Utility Analysis Of Personnel Selection
An Overview And Empirical Study Based
On Objective Performance Measures
Heinz Holling
Westfalische Wilhelms-Universitat Munster, Germany
Abstract
In this study, we will rst give an overview of utility analysis of personnel selection. The development of the utility models used most often will be briey
outlined. Furthermore, we will summarize empirical studies reported in literature. These studies show considerable nancial benets due to an introduction
or improvement of personnel selection programs. But, nearly all studies suer
from the often discussed problems of estimating the parameters, especially the
standard deviation of job performance. Consequently, the distributional assumptions and the requirements for regression analysis cannot be proved. Our
own study is based on sales gures representing an objective job performance
measure. These sales gures allow a simple and more valid computation of the
standard deviation of performance, the Achilles' heel of utility analysis. Also
the validity, the correlation of predictor scores with sales gures, is estimated
more precisely. Furthermore, we are able to prove distributional prerequisites.
Presupposing a normal distribution for our performance data would lead to
severe violations of the necessary assumptions for regressing performance on
assessment center scores. As a consequence, the utility would be strongly
overestimated. The results of our study, based on more reliable assumptions,
show a more realistic nancial benet compared to previous studies.
1 Introduction
Utility analysis has become an established quantitative method of evaluating human
resource programs. It can make valuable contributions to judgements and decisions
about investment in human resource. Utility analysis has been applied to personnel
recruiting, selection management, training, as well as to turnover or human resource
planning. The most important eld, however, is personnel selection. The classical
contributions and most of the recent developments focus on this area. Since the
classical studies by Brogden (1949), Brogden & Taylor (1950), and Cronbach &
Gleser (1965), two main directions in research can be identied.
The rst main emphasis in research has been the attempt to further develop
utility models. This has particularly concerned concepts of economy (Boudreau,
1983a; Cronshaw & Alexander, 1985) and a realistic representation of personnel
recruitment and turnover (Boudreau, 1983b; Murphy, 1986). The second direction
was stimulated by the work of Schmidt, Hunter, McKenzie and Muldrow (1979) and
concentrates on the problems of parameter estimation. In particular, the estimation
of the parameter SD , representing the standard deviation of the monetary value
of performance, was adjudged by Cronbach and Gleser(1965) to present diculties.
Schmidt et al. (1979) developed the rst easy-to-use method for estimating SD . A
y
y
6
H. Holling: Utility Analysis Of Personnel Selection
series of alternative methods for estimating SD and empirical studies in this eld
have been published since 1979 (e.g. Bobko, Karren & Parkington, 1983; Judiesch,
Schmidt & Hunter, 1993; Raju, Burke & Normand, 1990).
y
2 Development of utility models
The classical means of measuring the goodness of a selection test x to predict a
criterion y is the correlation coecient r . Several indices have been derived from
this measure, e.g. the coecient of determination r2 or the index of forecasting efciency 1 , (1 , r2 )1 2 , which indicates the proportionate reduction in the standard
error of criterion scores by the test.
In addition to the correlation coecient, Taylor and Russel (1939) introduced
two further concepts: (1) the base rate, i.e. the proportion of able persons in the
population of applicants and (2) the selection ratio, the proportion of applicants to
be selected. Taylor & Russel (1939) dene the success ratio as the proportion of
selected applicants who will succeed. Total utility in their model is the dierence
between the success ratio given a specic combination of validity, base rate and
selection ratio minus the success ratio which results without the knowledge of the
test result.
The Taylor-Russel model does not include important parameters like costs or
time periods during which the eects will last. The measure of the base rate and
success of selection is also troublesome. A further approach, the CAPER model
(Cost of Attaining Personnel Requirements) by Sands (1973), proposing a strategy
to minimize total costs of hiring and recruiting, suers from most of the weaknesses
of the Taylor-Russel model.
Modern utility models are mainly based on the work by Brogden (1946, 1949),
Cronbach and Gleser (1965) and Naylor and Shine (1965). The main foundation is
the linear model.
Y = Z + + ;
(1)
with
Y : job performance measured in monetary value
:
linear regression coecient
Z : standardized test score
: mean job performance (in monetary value)
:
error of prediction.
Assuming, E() = 0, the expected job performance is given by
E(Y ) = E(Z ) + :
(2)
Furthermore, for any subgroup s the expected performance becomes
E(Y ) = E(Z s ) + (3)
or
Y =Z s + :
(4)
Usually those applicants surpassing a certain test score are selected. It should
be noted that the only assumptions made are linearity and E() = 0.
We estimate by the best linear unbiased estimator r (SD =SD ), where r
is the Pearson correlation coecient and SD ; SD the standard deviations of y, x
resp. Because of the standardized test score SD is 1 and thus
y = r SD z s + :
(5)
xy
xy
xy
=
x
y
x
y
x
s
y
x
s
x
y
y
xy
y
y
x
xy
x
x
s
MPR{online 1998, Vol.3, No.1
xy
y
x
y
c 1998 Pabst Science Publishers
7
H. Holling: Utility Analysis Of Personnel Selection
y gives the absolute monetary value of mean job performance for one individual
in the selected group. However, the increase in the monetary value of average
performance in the selected group, due to applying the test, is given by the marginal
utility y , for one selectee, which is usually denoted by U /selectee. Thus
s
y
s
U=selectee = r SD z s :
xy
y
(6)
x
The total utility gain U depends on the number of persons selected, denoted by
N :
S
(7)
U = N r SD z s :
S
xy
y
x
In the following we omit the subscript s for z s and use z as the mean of those
applicants selected by using the selection procedure.
z usually cannot be measured since the performance of the rejected applicants
is unknown. But it can be estimated by the conditional expectation value given
(1) the selection rate is known and (2) the candidates have been selected in a top
down strategy. In nearly all empirical studies normally distributed test scores are
assumed. In this case the conditional expectation value becomes (SR)/SR, where
(SR) is the ordinate of the normal distribution at SR. As a consequence of this
assumption, the criterion is normally distributed, too. By summarizing the above
arguments we arrive at:
x
x
x
U = N r SD (SR)=SR:
S
xy
(8)
y
Brogden (1949), as well as Cronbach and Gleser (1965), were the rst researchers
who also included the costs of personnel selection C . Furthermore, they extended
their model to the time of the expected average tenure, T , for the group of N
selected applicants. The expected benet U is computed as follows including
these parameters:
s
U = N T SD r z , C:
S
y
xy
(9)
x
This equation has come to be known as the Brogden-Cronbach-Gleser (B-C-G)
model.
Boudreau (1983a; 1983b) and Cronshaw and Alexander (1985) suggest that
human resource management programs should be considered as a type of investment
decision. Further basic nancial and economic concepts that have been incorporated
include variable costs, taxes and discounting.
First, the costs C may be separated into (1) xed costs, C , which occur once,
e.g. for the development and implementation of the test procedure, and (2) variable
costs per applicant, C , e.g. for testing each applicant. Thus C = C + N C ,
where N refers to the number of applicants. Since N = N =SR, we may write
C = C + C N =SR.
A second important concept to be included is the proportion of variable costs of
performance denoted by V . Job performance, y, here is understood as the sales value
of the product or service of an employee (Greer & Cascio, 1987). The contribution
margin of job performance y is computed by y(1+ V ). Thus, the standard deviation
of the contribution margin of job performance is calculated by SD (1 + V ).
Incorporating, furthermore, taxes, denoted by TAX , we arrive at
f
v
f
a
a
f
v
a
v
s
s
y
U = N r SD (1 + V )z (1 , TAX ) , C (1 , TAX ) , C N =SR(1 , TAX ):
(10)
S
xy
y
x
f
v
s
We include discounting as a further concept. Since money can be invested to
earn interest, nancial analysis discounts future earning and costs to consider these
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
8
H. Holling: Utility Analysis Of Personnel Selection
potential investment returns. Given the interest rate i discounting U over a period
of T time units (mostly years) yields:
U =
X
1 N r SD (1 + V )z (1 , TAX )
(1
+
i)
=1
T
t
X
s
xy
y
x
t
,
1 C (1 , TAX )
(1
+
i) ,1
=1
T
X
(11)
f
t
t
,
1 C N =SR(1 , TAX )
,1
=1 (1 + i)
T
v
t
s
t
In the rst line of (11) the returns from the personnel program are estimated. In
the second line the variable costs, and in the third the xed costs of the program
are given.
The impact of personnel selection programs is usually of long duration. Since
productivity is often variable during this time, Boudreau (1983b) suggested dividing
the duration of the intervention program into a number of intervals. These are time
periods in which productivity changes occur. Furthermore, the model parameters
may have dierent values in each of the time periods. This is an essential feature
in cases where, for example, a predictor has a variable predictive validity.
By dividing the duration into dierent periods, it is also possible to encompass
the ow of employees. Assuming that N persons will be selected at the beginning
of period t and N employees selected by the program in a former period will leave
the organization after time period t, the valid number of employees during this time
period, N , amounts to
X(N , N ):
N =
st
lt
t
t
t
j
=1
sj
lj
When we include the above suggestions, the following utility model results:
U =
X X(N
T
t
=1 j =1
X
, (N
sj
, N )r
lj
xyt
SDyt (1 + Vt ) z xt (1=(1 + it )t )(1 , TAX t )
t
T
=1
st
=Qt )Cvt (1=(1 + it )t,1 )(1 , TAX t )
ft
(1=(1 + i ) ,1 )(1 , TAX )
X(C
(12)
t
,
T
=1
t
t
t
t
Summarizing of all parameters of (12) yields:
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
9
H. Holling: Utility Analysis Of Personnel Selection
U
Utility of the personnel selection program in time period 1 : : : T
Time period. This could be a duration of up to a complete year and indicates
the particular t-th year after the commencement of the program.
T
Impact duration of the program.
Nt
The number of employees in the organization in time period t who have been
selected through the program.
Nst
The number of employees selected in time period t.
Nlt
The number of employees who left the organization in time period t.
rxyt
Validity of the selection instrument in time period t. This is the productmoment correlation of the predictor x and the sales value of the job performance y within the applicant's population.
zt
mean predictor score within the selected group in time period t
SRt
Selection rate in time period t.
(SRt ) Ordinate of the normal distribution of the selection rate in time period t.
SDyt
Standard variation of the sales value of the job performance in the applicant
population in time period t.
Vt
Proportion of the variable costs of the job performance for the organization
in time period t.
it
Interest rate in time period t.
TAX t Tax rate in time period t.
Cvt
Variable costs of the personnel selection program per applicant in time period
t.
Cft
Fixed costs of the program including those incurred for the development,
implementation and evaluation of the program in time period t.
t
The above model (12) refers to the utility of an introduced intervention compared
to a situation without an intervention. Considering personnel selection we mostly
have to compare dierent treatments, i.e. selection strategies because random selection occurs very seldom. In this case r , C and C refer to the dierence
of the validity, variable costs, xed costs, resp. of a certain selection program in
relation to an alternative selection. Then U illustrates the incremental value of a
personnel selection program over an alternative program. This is the incremental
utility produced when the utility of a personnel selection program is compared to
an alternative program.
Model (12) still represents state of the art. However, some further complements
have been proposed. Murphy (1986) derived formulas for correcting the average test
score z if selectees decline oers and lower scoring candidates must be accepted.
Three cases representing realistic circumstances are considered: (a) oers are declined at random, (b) oers are declined by the highest scoring applicants, and (c)
test scores are related to the probability of accepting an oer.
Tziner, Meir, Dahan & Birati (1994) take the ination rate as a further parameter into account. This parameter behaves completely analogous to discounting. If
the ination rate is denoted by f , the discounting factor 1=(1 + i) has to be multiplied by 1=(1 + f ) . It is not necessary to include the ination rate as a separate
factor. We can combine both factors to an adjusted interest i = i + f + if , since
1=(1 + i) 1=(1 + f ) = 1=(1 + i + f + if ) . If the ination rate is included we will
speak of an adjusted interest rate i .
xyt
ft
vt
x
t
t
a
t
t
t
a
3 Estimation of the standard deviation of job performance
The concept of job performance (y) is the most critical component in utility analysis.
It has been dened in various ways, e.g. as: \... the yearly value to your agency
... [to] ... consider what the cost would be of having an outside rm provide these
products or services" (Schmidt et al., 1979, p. 621), \... dollar value as sold"
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
10
H. Holling: Utility Analysis Of Personnel Selection
(Hunter, Schmidt & Coggin, 1988, p. 526) and \the total amount [in dollars and
cents] contributed toward the coverage of xed costs, and then prot ..." (Greer &
Cascio, 1987, p. 590).
Boudreau (1991) distinguishes three concepts of payo in utility analysis: (1)
payo as cost reduction, e.g. reductions in trainings costs, replacement costs or
accidents, (2) payo as the value of output as sold reecting the sales value of productivity, and (3) payo as increased prots reecting the sales revenue generated
when an production unit is sold less any costs in producing that unit. Many authors
(e.g. Cronbach & Gleser, 1965; Hunter et al, 1988) endorse the last concept as the
most important one. Indeed, it is most consistent with the concept of prot the
utility approach is based upon and includes the two other approaches. Considering
variable costs, xed costs, taxes and discounting given in equation (12) focus on
increased prots. But, in some cases, e.g. non-prot organisations, one of the other
concepts may be more appropriate, especially if there is not enough information to
specify the additional parameters.
An important result of the previous discussion about performance is that, even
for experts, it is a dicult task to specify an adequate concept of payo. But this
specication is an implicit part of most estimation procedures which demand such
a performance by so-called experts or coworkers within relatively short time. The
dierent methods for estimating the standard deviation of job performance may
be classied in (1) cost accounting procedures, (2) global estimation methods, (3)
proportional rules methods, and (4) individualized estimation procedures.
Cost accounting methods attach a monetary value to production units according
to their contribution to organizational prot. The number of units produced by each
individual in a sample over a certain period of time is observed. Multiplying each
unit by its contribution to prot yields a productivity score for each individual. The
standard deviation of these scores is used as an estimation of SD .
Schmidt et al. (1979) rst proposed global estimation of standard deviation.
Experts, mostly supervisors, have to estimate the monetary value of dierent points
on presumed normally distributed job performance. Usually the 15th, 50th and 85th
percentiles have to be specied. The mean of the dierences between the 50th and
15th and between the 85th and 50th percentile is used as the SD estimate, provided
the dierences do not dier signicantly. The 50th percentile is sometimes provided
as an anchor.
Proportional rules were developed by Hunter and Schmidt (1982). They reviewed empirical studies providing estimates of SD and concluded that the average
for SD falls in the range of 40% to 70% of average salary. This simple proportional
rule has been used in many utility studies since it is an easy to use method.
According to individualized estimation, a monetary value is attached to the
output of each individual in a sample. This procedure is closely connected to cost
accounting methods. The mostly used method within this class is the CascioRamos estimate of performance in dollars (CREPID) (Cascio & Ramos, 1986).
This method determines the important principal activities of a job and rates each
activity with respect to the attributes time/frequency and importance. Multiplying
both ratings provides an overall weight to each activity. The proportion of total
weight represents the nal weight of each activity. Dividing the average salary for
the job among the activities according to these nal weights leads to a monetary
scale. In a further step supervisors have to rate a sample of co-workers in terms of
their performance on each principal activity. Multiplying the monetary value with
these ratings provides estimates of SD .
Several modications of this method have been proposed (e.g. Janz & Dunnette,
1977; Edwards, Frederick & Burke, 1988). A further individualized estimation
method consists of direct assigning dollar values to individual employees based
mainly on sales. The individualized methods are more costly than the simpler
y
y
y
y
y
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
11
H. Holling: Utility Analysis Of Personnel Selection
global and proportional rules. Nevertheless even these individualized estimation
procedures are subjective ratings.
The amount of knowledge about the psychometric properties of all subjective
estimation methods, especially validity, is small. Bobko, Karren and Kerkar (1987)
show many shortcomings in these methods for estimating SD and, thus, demand
for systematic research for understanding estimates of SD and analyzing psychometric properties. Furthermore, many empirical studies comparing dierent estimation methods clearly demonstrate considerable dierences between the results
of dierent estimation procedures (e.g. Hakstian, Wolley, Woolsey & Kryger, 1991;
Barthel & Schuler, 1989). Cost accounting methods providing more objective measures have been used very seldom, as we will show in the next section.
y
y
4 Overview of published utility analyses
A literature search yielded 20 studies dealing with utility analysis of personnel
selection. All 20 studies are summarized in gure 1. Boudreau (1991) also gives
an overview of utility studies which have been published before 1988. We include
only those studies in our overview dealing with the impact of personnel selection
on the monetary value of overall performance. Thus, the included studies will be
more comparable.
Three studies briey outlined in the following have been excluded. Van Naersson
(1963) estimated a monetary value of $ 16 per selectee for reducing training costs
when drivers in the Dutch Army were selected by a driving experience questionnaire. In the study of Schmidt and Homann (1973) the impact of using weighted
application blanks for nurses in a hospital on turnover costs amounted to $ 262 per
year and selected nurse. Lee & Booth (1974), using the same selection procedure,
yielded cost reductions for recruiting, hiring and training of $ 490 for each of 245
clerical employees per year.
A few of the articles included in gure 1 provide more than one utility analysis
by varying one or more parameters. In this case we selected one utility analysis
which we assumed to be the most representative.
The amount of empirical studies investigating the monetary utility of personnel
selection since the beginning in 1960 is relatively small. Especially in the nineties
more articles dealing with theoretical and statistical problems rather than empirical
studies have been published.
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
12
H. Holling: Utility Analysis Of Personnel Selection
Figure 1: Overview of utility analyses of personnel selection
a
Data have been taken from Boudreau (1991)
No of study/ Sample/Predictor/
Reference Utility Estimation
Model for computing U
U=sy (U per selectee and per
year)
1) Roche
(1961)
Radial drill operators
in a manufacturing
organization/ Test
battery/Cost
accounting
U /per hour = r SD z , C
= 0.313 $0.585 1.11 , $0.0006
U=sy = $405 (assuming 2000 working
hours)
2) Cascio &
Silbey
(1979)
Food and beverage
sales managers/
Assessment center
minus interview/
Estimation of
percentiles
U = T N r SD z , N C
= 5 50 0.10 $ 8.281.9 0.798
,100 $113.28
U=sy = $616
3) Schmidt,
Hunter,
McKenzie &
Muldrow
(1979)
U.S. government
computer
programmers/
Programmer Aptitude
Test Estimation of
percentiles
U = T N r SD z , N C
= 10 618 0.76 $ 10,413 0,8
,1; 236 $10
U=sy = $6,329
4) Arnold,
Rauschenberger,
Soubel &
Guion (1982)
Steelworkers/Strength U = N r SD z
test/Performance ratio = 1,853 0.84 3,000 1.97
U=sy = $4,964
of top worker to
bottom worker; yearly
salary
5) Dunnette
et al.
(1982)
U = r SD z , C
Nuclear power plant
= 0.30 $23,500 0.80 , $200
control room
U=sy = $5,440
operators/ Test
battery/ Estimation of
percentiles
xy
S
S
S
xy
xy
xy
xy
a
y
y
y
y
y
x
x
x
a
a
v
v
x
x
6) Ledvinka, Life insurance claim
approvers/ JEPS test
Simonet,
minus interview
Neiner &
Kruse (1983)
U = N r SD z , C
= 10 0.22 $ 5,542 1.918 , $1,104
U=sy = $2,228
7) Schmidt,
Mack, &
Hunter
(1984)
U = T N r SD z , N C
= 10 80 0,37 $4,451 1.758 ,800 $0
U=sy = $2,895
a
U.S. park rangers/
PACE test minus
interview/ Estimation
of percentiles
MPR{online 1998, Vol.3, No.1
S
xy
S
xy
y
x
y
x
a
v
c 1998 Pabst Science Publishers
13
H. Holling: Utility Analysis Of Personnel Selection
8) Wroten
(1984)a
Pump
operators/Various
selection tests/ 12
dierent estimation
variants of percentiles
U = T r SD z , C
= 3 0.30 $14,205 1.55 , $1,000
U=sy = $6,272
9) Weekley,
O`Connor,
Frank &
Peters
(1985)
Convenience store
managers/test
battery/ estimations
of
percentiles/CREPID
method
U = N r SD z , N C
= 1,000 0.45 $7,701 1.102 ,
3,000 $10
U=sy = $3,789
10) Cascio & Telephone company
oce managers/
Ramos
Assessment center
(1986)
minus interview/
CREPID method
xy
S
y
xy
x
y
x
a
v
U = T N r SD z , N C
= 4.4 1,116 0.258 $10,250 1.116 ,
3,488 $ 388
U=sy = $2,676
S
xy
y
x
a
v
11) Schmidt,
Hunter,
Outerbridge
& Trattner
(1986)
U.S. government
employees/ Test of
cognitive abilities/
40% of lowest salary
U = T N r SD z
= 13 22,5731 0.31 $5,429 1.55
U=sy = $2,609
12) Burke &
Frederick
(1986)
Midlevel sales
managers/ Assessment
center minus
interview/ Yearly sales
volume, annual
salaries, estimation of
percentiles
U = =1 1=(1 + i) N r SD (1 + V )
z (1
P, TAX ) ,C (1 , TAX )
= 4=1 1=(1 + 0:178) 29 0:43
$32; 323(1 , 0:048) 0:872 (1 , 0:49) ,
$213; 151 (1 , 0:49)
U=sy = $3,036
13)
Cronshaw,
Alexander,
Wiesner &
Barrick
(1987)
Clerical/
administrative
employees for the
Canadian military
Cognitive ability test /
40% of average salary
UP= =1 1=(1+ i) N r SD z , N C
= 18=1 1=(1 + 0:1125) 470 0:52 $10; 680 1:09 , 1; 410 $205
U=sy = $2,570
14) Rich &
Boudreau
(1987)
Computer
programmers/
Programmer Aptitude
Test minus interview/
Estimation of
percentiles
U =
S
xy
y
x
P
T
t
S
t
xy
y
x
t
t
P
T
t
S
t
xy
y
x
a
t
P =1 P =1(N
, N )1=(1 + i)
r SD (1 + V )z (1 , TAX )
,P
=(1 + i) ,1 N =SR C (1 , TAX )
=1 1P
P
11
= =1 =1 (N , N )1=(1 + 0:15)
0:P
59 $15; 888(1 , 0) (1 , 0:39)
, 5=1 N =0:40 $10(1 , 0:39)
xy
T
t
t
j
y
t
lt
x
T
t
st
t
st
t
t
j
t
st
st
lt
t
U=sy = $2,876
(N1 : : : N11 ) =
(25; 79; 129; 167; 190; 168; 110; 60; 26; 8; 1)
MPR{online 1998, Vol.3, No.1
v
t
c 1998 Pabst Science Publishers
14
H. Holling: Utility Analysis Of Personnel Selection
Insurance sales
people/ Biographical
questionnaires/
Computation of SD
from sales data
provided by an
insurance company
PU =P
16) Gerpott
(1990)
Managers in
chemistral industry/
Assessment center
minus unstructured
interviews/ 33% of
average salary
U = =1 1=(1 + i) N r SD z , C;
T = 5; i = :1
= 3:79 30 0:22 DM 26; 000 1:55 , 507; 276
U=sy = DM 3; 339
17)
Hakstian,
Woolley,
Woolsey &
Kryger
(1991)
Managers in an
industrial setting/
management selection
battery/ estimation of
percentiles
U = N r SD z , N C
= 50 0:44 $55; 880 1:271 , 200 $1; 050
U=sy = $27; 093
18) Tziner,
Meir, Dahan
& Birati
(1994)
Managers in an Israeli
corporation/
Assessment center/
SD was provided by
accounting
department, no
further details are
reported
U = N =1 1=(1 + i ) r SD z
(1 P
+ V )(1 , TAX ) ,(C1 + C2 )(1 + i)
, =1 C1 =t TAX 1=(1 + i )
,(C2 TAX , (C1 + C2 ) i TAX
with C1 : cost which are amortized over
the life of the selection program
C2 : cost which
P are not amortized
U = 276 2=133 1=(1 + 0:32) 0:22 $8; 321(1 , 0:28)(1 , 0P:45)
,$111; 954(1+0:10), 2=133 $55; 977=t0:45
1=(1 + 0; 32) , $55; 977 0:45 ,
$111; 594 0:10 0:45
U=sy = $506
15) Barthel
& Schuler
(1989)
y
y
t
t
=1 j =1 (Nst , Nlt )1=(1 + i) rxy SDy z x
T
, t=1 Ct 1=(1 + i)t,1
= 6t=1 tj=1 (Nst , Nlt )1=(1+0:07)t 0:18 DM 4; 800 , DM 85; 000 , 3t=2 DM 20; 000
1=(1 + 0:07)t,1
U=sy = $448
(N1 : : : N6 ) = (150; 235; 299; 202; 117; 53)
P
P P
T
t
P
P
T
t
s
t
s
xy
y
P
x
y
x
a
T
a
t
xy
t
xy
y
T
a
t
:
x
t
t
t
:
t
t
19) Funke,
Schuler &
Moser
(1995)
Research and
development
managers/
combination of work
probes, personality
tests and cognitive
ability tests/
estimation of
percentiles; estimation
of monetary value of
each person's
performance;
CREPID-method
MPR{online 1998, Vol.3, No.1
P
U = =1 (N (1=(1 + i) r SD (1 + V )
z (1 , TAX )
, P =1 C (1 , TAX )1=(1 + i) ,1 ,
C
)
P14(1=1,(NTAX
(1=(1 + 0:0267 0:20 DM
100; 000 (1 , 0:26) 1:5496(1 , 0; 666)
, P10=1 85; 000 (1 , 0:666) 1=(1 + 0:0267) ,1
,109; 000 (1 , 0:666)
U=sy = DM 6; 008
(N1 : : : N14 ) =
(25; 50; 75; 100; 125; 125; 125; 125; 125; 125;
100; 75; 50; 25)
T
t
t
t
xy
y
x
T
vt
t
t
o
t
t
t
t
c 1998 Pabst Science Publishers
15
H. Holling: Utility Analysis Of Personnel Selection
U = r SD z , C=SR
= 0:05 $20; 000 0:965 , $950=0:4
U=sy = $3; 340
(without considering adverse impact)
Incorporating adverse impact:
U=sy = ,$1; 394
with SR for assessment center = 0.3 and
SR for test = 0.8
Furthermore some of these studies are hypothetical studies only aiming at a
demonstration of the utility of a valid personnel selection, e.g. Cascio & Silbey
(1979, p. 109) state: \The utility model was applied to a second-level manager
selection and placement process of a hypothetical rm."
The empirical studies refer to quite dierent jobs and selection procedures.
While in the beginning the utility of more simply structured jobs was analyzed, all
studies conducted in the nineties refer to more complex management tasks. Nearly
all dierent selection procedures ranging from simple strength tests to assessment
centers have been subject of utility measurement. Dierent estimation methods
have been used. However, as already mentioned, in 17 of the 20 studies subjective
estimation methods have been used. Estimation of percentiles is especially popular.
Only Barthel and Schuler (1989) computed the standard deviation from real sales
data.
The BCG-model is the dominant method for calculating utilities. This model
was applied in 13 studies. In the remaining studies at least one of the nancial
and economic concepts was included. But all parameters apart from sample sizes
of applicants, selection ratios and costs were held constant for the dierent time
intervals. Thus the Boudreau-model has not been applied in its full generality up
to now.
It is dicult to compare the yearly benet for one selected applicant because
of the variety of included jobs or selection procedures. Furthermore, including
discounting, taxes and/or variable costs leads to smaller gains compared to the
BCG-model. Referring only to those studies focusing on management tasks, the
lowest utility in terms of yearly benet per selected applicant amounted to DM 429
in the study by Funke et al. (1995). Hakstian et al. (1991) reported a yearly benet
per selectee amounting to $ 27,093 which represents the maximum of all studies.
But all studies based on the Boudreau-model provide much less utility than those
using the BCG-model.
A shortcoming in many studies is the estimation of the validity. Often coecients are derived from meta-analytic studies. But these coecients overestimate
empirical relationships if corrections for reliability have been made. The validity
coecients mostly reect the relationship between a predictor and a subjective
measure of performance, e.g. ratings of supervisors. Thus, these relationships may
partly be based upon an contamination error. But studies which used more objective indicators of performance report minor validity coecients. Barthel and
Schuler (1989) obtain a correlation coecient of 0.18 between biographical data
and sales gures, while Reilly and Char (1982) report a mean validity of r = 0.35
for the relationship between biographical tests and performance. Also the metaanalysis by Funke, Krauss, Schuler und Stapf (1987) provides an average coecient
of r = 0.47. The mean validity of assessment centers obtained in a metaanalysis
by Thornton, Gangler, Rosenthal and Bentson (1992) amounts to 0.37. However,
the correlation coecients in the study of Tziner et al. (1994) between an overall
assessment center score and a monetary bonus to each employee reecting the contribution to productivity of the organizational unit fall into a range from r = 0.19
20)
Homann &
Thornton
(1997)
Employees in a large
utility company/
Professional
Employment
Test-assessment
center/ 40 % salary
MPR{online 1998, Vol.3, No.1
xy
y
x
c 1998 Pabst Science Publishers
16
H. Holling: Utility Analysis Of Personnel Selection
to r = ,0:03.
In nearly all studies the parameter z has been computed by (SR)=SR assuming normally distributed predictor scores. Thus, the criterion scores have to be
distributed normally too. But this assumption is questionable. In most studies this
assumption could not be proved adequately or if possible, has not been tested. For
example, Cascio and Silbey (1979) derived the standard deviation SD from the
estimates of the 50th and 84th percentiles of the total yearly dollar value and could
not test distributional assumptions.
In most of studies the assumption of a normal distribution seems to be violated.
Already in the rst application of the percentile method by Schmidt et al (1979) a
positively skewed distribution occured. The dierence between the 80th and 50th
percentile was about 8% larger than the dierence between the 50th and 39th percentile. A similar result was obtained by Rich and Boudreau (1987), while Barthel
and Schuler (1989), as well as Hakstian et al. (1991), got a stronger positively
skewed distribution by an estimation of SD using the percentile method. Thus,
the normal distribution should not be assumed without testing its adequacy.
Summarizing our overview of the empirical studies we may state:
(1) The amount of empirical studies is not overwhelming
(2) The studies are very dierent with respect to several parameters, e.g. jobs,
selection procedures.
(3) The mostly elaborated model (12) has been seldom applicated.
(4) With exception of a very few studies the standard deviation of performance was
estimated by subjective rating methods. However, there is little known about the
psychometric properties of these procedures.
(5) Even similar studies yield very dierent utilities. Thus, the validity of the studies
is questionable.
x
y
y
5 The empirical study
Our empirical study aims at a more reliable estimation of monetary utility of personnel selection than earlier studies have. We deal with the fundamental question
whether a relatively cost-intensive selection program using assessment centers (AC)
yields a positive utility in comparison to a simpler procedure using unstructured
interviews. The human resource program in question is used by an insurance company to select their sales representatives. This study was the foundation for the
decision whether the selection program should be changed.
One of the special features of this study is that we had the opportunity to
get all data concerning the assessment center and the sales gures of a German
insurance company during three years in the early 90's as well as some biographical
variables of all sales representatives selected and leaving the company during this
time. Thus, the monetary value of performance is relatively easy to determine and
the estimation of the parameters SD and r is far simpler and more exact than
it is the case with all former studies. One exception is the study by Barthel and
Schuler (1989) also using sales gures. However, in this study the assumption of
normally distributed performance scores has not been tested. Furthermore, we have
the opportunity to test whether the assumption for the linear regression of the sales
gures on the AC-scores, such as linearity or homoscedasdicity, are met. Finally,
we do not include the further development by means of extrapolation and constrain
our analyses to the known situation. Thus our calculations are very conservative.
y
MPR{online 1998, Vol.3, No.1
xy
c 1998 Pabst Science Publishers
17
H. Holling: Utility Analysis Of Personnel Selection
Estimation values of the model parameters which determine the returns of the
personnel selection program. The values are assumed to be constant over all time periods t.
parameter
symbol
estimated unit
value
average standardized predictor value for the se- zx
0.8
lected applicants
incremental validity of the selection instrument
rxy
0.11
standard deviation of the sales value of the job SDy
105,397
DM
performance
proportion of the variable costs of job performance V
0.8
interest rate adjusted for ination
ia
0.107
tax rate
TAX
0.4
variable costs per applicant
Cvt
300
DM
xed costs
Cft
100,000
DM
Table 1:
5.1 The Assessment Center
The selection program under study was designed to select sales representatives for
a German insurance company. The core of the program is an AC developed and led
under the direction of a rm of consultants.
The ACs lasted one day. Six AC exercises were held (self-presentation, leaderless group discussion, oral presentation, exploratory interview; an exercise testing
ability to handle objections, planning task). Afterwards an assessor conference and
a feedback session with the AC participants took place. On average 5.3 applicants
participated in each AC (SD = 1.57). The assessor/applicant ratio was approximately 1:2 or better.
The performance of the participants in each exercise was measured by the assessors, each using a subset of ten performance dimensions which emerged as the
result of job analysis (persistence; resistance to stress; initiative; sociability; achievement orientation; learning and adaptation; personal appearance; independence;
self-condence; negotiating skills). The ratings are scored on a four-level scale
(1=seldom/hardly observable; 2=occasionally observable; 3=regularly observable;
4=prominently/strongly observable). Using these scores assessors built an overall
score at the observer conference which is used as the basis for the decision whether
to accept or reject candidates. The selection rate of these ACs is about 50%.
5.2 Estimated values of the model's parameters
The values of the parameters of model 6 are presented in table 1. If parameters had
to be estimated we always chose very conservative estimates. Thus our computed
U represents a lower limit of the benet.
5.3 Number of sales representatives in the time periods and
impact duration of the program
We set the length of the time period t at one year according to the usual convention.
The number of sales representatives selected during the rst three years and leaving
the company during the rst four years are represented in table 2. The number of
sales representatives treated by the personnel selection program during the rst 4
years are empirically measured with exception of those sales representatives leaving
the company during the fth year. This number has been estimated by means
of extrapolation. From table 1 it can be seen that the selection program remains
eective for a period of T = 5 years.
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
18
H. Holling: Utility Analysis Of Personnel Selection
Table 2: The number of the SR treated by the personnel selection program (Nt ) in time
periods t. The values which are marked with "*" are estimated.
time period (t)
accepted SRs (Nst ) left SRs (Nlt )
1
2
3
4
5
89
126
100
{
{
0
0
98
132
35*
number of treated
SRs in t (Nt )
89
215
217
85
50*
5.4 Average standardized predictor value of accepted applicants
As a result of data protection regulations, no AC results are available for rejected
candidates. Therefore z has to be estimated for this group using z = (SR)=SR.
The selection rate of ACs is about 50% on the average. There are no reasons to
suppose that the assumptions necessary for the estimation are seriously undermined:
z = (SR)=SR = 0.399/0.5 = 0.8.
x
x
x
5.5 Tax rate, ination rate and rate of interest
In the current utility analysis we assume that interest is paid on capital at an
average rate of 8%. The ination rate was set at 2.5%. Taking the ination rate
into account, the adjusted interest rate i amounts to 0.107. The tax rate of costs
and prots is set at a rate of TAX = 0:4 in all time periods t.
a
5.6 Standard deviation of job performance
As stated above the level of revenue generated by each sales representative in his
rst year with the company is known. These sales gures are the basis for the
calculation of the sales representatives' commission. However, the duration of each
of the policies sold has to be taken into account and, furthermore, the revenues have
to be discounted. We have estimated that all policies have an average duration of
ten years. According to expert opinion this is a reasonable estimate. If we discount
the contributions to the insurance company paid by the customers over an period
of ten years we arrive at a factor of 5.964, which has to be multiplied with the
level of revenue generated by each sales representative in the rst year. But, before
computing the standard deviation this way, we have to check the assumptions for
linear regression of sales gures in the rst year on assessment center scores. This
measure is, of course, also necessary for determining the predictive validity r .
xy
5.7 Testing the assumptions for linear regression
The basic BCG-model only assumes a linear relationship of the predictor and criterion and, furthermore, that the expectations of errors are 0. Since we estimate
the average standardized predictor value of accepted applicants by (SR)=SR, the
assumption of normally distributed predictor and criterion scores is introduced.
Since only the values of the selected sales representatives are available, the assumption of truncated normally distributed predictor and criterion scores can be tested.
This assumption is only valid for the assessment center scores using a KolmogorovSmirnov-test with Lilliefors correction (p < 0:25). Excluding four outliers even the
sales gures can be assumed to follow a truncated normal distribution (p < 0:20).
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
19
H. Holling: Utility Analysis Of Personnel Selection
The outliers also violate the assumption E() = 0, which is required by all
utility models, as stated above. Inclusion of the outliers yields a standard deviation
of DM 157,211, while the standard deviation without these four scores amounts
to DM 105,397. This is a considerable dierence which impacts the overall utility
severely, as we will show. The correlation coecient between the assessment center
scores and sales gures is also inuenced by the outliers. Neglecting the outliers,
Pearson's correlation coecient is r = 0:21 otherwise it amounts to r = 0:20. The
assumption of linearity is not violated in both cases.
5.8 Predictive validity of the predictor
We assume an incremental validity of r = 0:11 for the AC selection program
against unstructured interviews which had been applied before the introduction of
AC. The correlation of the sales gures with the overall score of the assessment
has been calculated at 0.20 (see above). But this correlation is only based upon the
accepted candidates and has to be corrected for restriction of range in the predictor.
The corrected validity amounts to 0.26. The mean validity of unstructured interviews as used in our study and also corrected for range restriction, may be estimated
as 0.15 (Hucutt & Arthur, 1984). Thus the incremental validity is determined by
0.11.
xy
5.9 Variable costs V
The ratio of variable costs to revenue generated is very high in this sector. They
consist of, for example, the commission payable to sales representatives, as well as
to managers, and the variable administration costs. The xed costs consist of xed
salaries, costs of buildings, general administrative costs, personnel programs etc.
A direct calculation of V using the internal cost calculations is very involved and
very inaccurate. Barthel (1989) used in his (albeit not explicit) estimate of V the
fact that the value (1 + V ) includes the protability. We estimate (1 + V ) = 0:2
assuming a protability of 5% and 15% of revenue for the coverage of the xed costs.
The remaining 80% of revenue is required to meet variable costs. Simplifying, we
assume that this value is constant for all time periods t.
5.10 The cost of the personnel selection program
In the case of both cost and return parameters only the increments over the simpler
selection procedure are considered. Here the advantage of handling the issue in
this way over the calculation of the absolute cost of both systems becomes very
clear. It is very dicult to estimate for example, the proportion of overhead costs
which are generated by the activities of the personnel department in the planning
and conception of a selection program. In our scheme this problem can simply be
ignored. Instead it is assumed that overhead costs are the same for both the program
actually analyzed and any alternative selection program. Thus, only the incremental
costs of the program are considered. In the case of the current AC program these
are, for example, the costs of using the potential sales representatives' immediate
superiors as observers and decision-makers in an AC. This same sta member, of
course, also participated in the interview system. In our model we only consider
the additional costs of the program in question over alternative programs, such as
observer training costs.
The xed incremental costs of the AC programs are mainly incurred at the beginning of the program. These include incremental development and implementation
costs claimed by the consulting rm, as well as the costs of training the observers
and moderators. All xed costs amounted to about DM 100,000.
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
20
H. Holling: Utility Analysis Of Personnel Selection
Utility analysis of the personal selection program. In the last line we indicate
the total amount of incremental returns, costs, and utility of the program compared to an
alternative simple selection program.
Table 3:
t
Nt
return parameters
rxy z x
1 89 0.11 0.8
2 215 0.11 0.8
3 217 0.11 0.8
4 190 0.11 0.8
5 163 0.11 0.8
total
cost parameters
Cf t
DM
100,000
0
0
0
0
total
Cvt
DM
300
300
300
300
300
returns
(1 , TAX ) 1=(1 + i ) total
DM
DM
105,397
0.2
0.6
0.90
89,245
105,397
0.2
0.6
0.82
194,754
105,397
0.2
0.6
0.74
177,566
105,397
0.2
0.6
0.67
62,831
105,397
0.2
0.6
0.60
33,387
557,782
costs
utility
U
(1 , TAX ) 1=(1 + i ) ,1 total
DM
DM
0.6
1.00
92,040
-2,795
153,778
0.6
0.90
40,976
0.6
0.82
29,377
148,189
0.6
0.74
0
62,831
0.6
0.67
0
33,387
162,393
395,389
SDy
(1 + V )
a
a
t
t
t
In addition to these xed costs it is also necessary to calculate the variable
costs (per applicant) of the selection program. We assume that no incremental
personnel costs are generated by using AC observers since they would be involved
in conducting job interviews in alternative programs.
Using the gure for the number of employees in each time period (N ) and the
selection rate (SR), it is possible to determine the total number of ACs candidates.
We set the daily cost of food, accommodation and material at DM 300 per person.
st
5.11 Results
Table 3 shows the results of the utility analysis based upon model (12) and the
above determined parameter. The selection program under scrutiny reveals, under
the conditions given above, a higher positive utility than the simpler program using
unstructured interviews. The development and implementation costs are covered
within two years and returns are greater than variable costs in every time period
after the second year.
Thus, this utility analysis endorses the conclusions of those other studies which
have conrmed the economic utility of psychological human resource programs
(e.g. Boudreau, 1991). We, however, have taken actual sales data to estimate
the standard deviation of the job performance. Furthermore, the distributional
assumptions and prerequisites for regression have been met. Thus our data is more
reliable.
But the utility in our analysis is much smaller than that provided by former
studies. The yearly benet per selectee amounts to DM 121. Neglecting distributional assumptions, i.e. computing SD while including the outliers would lead to
an utility of DM 745,235 or a yearly utility per selectee of DM 227. This would
mean a considerable over-estimation.
One might argue that the utility computed without the contributions of the
outliers is underestimated. This, of course, is true. Since each outlier reaches sales
y
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
21
H. Holling: Utility Analysis Of Personnel Selection
Table 4:
Break-even-points of the parameter values
parameter break-even-point
r
0.032
Q
87.7
V
0.942
SD
30,700 DM
C
760,000 DM
C
1,460 DM
xy
y
k
v
gures which are about four times the mean of all sales gures, we may correct
the utility. Including the benets of 16 sales representatives and costs of 4 sales
representatives yields a yearly benet per person of DM 123, which is still much
less than DM 227.
5.12 Break-even points
The break-even point of a parameter is that value that yields a utility of 0 while
holding all other parameters constant. In this manner information about the risk
involved in decisions can be derived. For example, if those parameters which are
dicult to estimate and by nature involve risk are unstable over time, or lie wellbeyond their break-even point, it can safely be assumed that the program will have
a positive utility value even if estimates are inaccurate or unfavorable conditions
should unexpectedly occur. In table 4 the break-even points of the parameters are
given. The break-even points of the parameters are considerably lower than the
estimates. Thus, our results seem to be very sound.
6 Discussion
Our study is based upon objective criteria providing a solid foundation for testing
the assumptions and estimating the parameters of the Boudreau-model. The utility
resulting from this study agrees with those of previous analyses using subjective
performance measures. In so far as a more valid selection, even if it is expensive, it
provides a more favorable return on investment. But the incremental utility is much
less than that reported in earlier studies. Our analysis clearly demonstrates the
necessity to test the assumptions underlying utility analysis. Otherwise considerable
estimation errors may occur.
Even if utility analyses yield low benets, it has to be mentioned that utility
models do not cover all components of utility of personnel intervention programs.
For example, selection of employees with higher performance usually causes less
expenses for education and training especially at the beginning of the new job. Furthermore, such employees claim less attention and time from their superiors and
colleagues. Since resources of persons with higher performance are often limited,
selection of the best forces other companies to employ persons with lower performance.
Beyond this direct benet of personnel selection programs, which are not covered
by utility models, human resource programs can produce further benets. These
side-benets only have an indirect relationship to the main aim of the programs.
Personnel selection by means of an assessment center can also contribute to the
development of management qualities in those acting as assessors (observational
skills, communication skills, the ability to handle sensitive areas following customer
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
H. Holling: Utility Analysis Of Personnel Selection
22
queries etc.). The fair and transparent selection of new members of sta also has a
positive eect on the company's culture and helps the company to project a very
condent and modern image to the public. Finally, a usual side-eect of the AC
system is the fact that superiors take part in selecting their juniors and thus share
responsibility for their selection. In this way it is more probable that the new sta
will be accepted into the rm.
References
[1] Arnold, J. D., Rauschenberger, J.M., Soubel, W., & Guion, R.M. (1982). Validation
and utility of a strength test for selecting steel workers. Journal of Applied Psychology,
67, 588-604.
[2] Barthel, E. & Schuler, H. (1989). Nutzenkalkulation eignungsdiagnostischer Verfahren
am Beispiel eines biographischen Fragebogens. Zeitschrift fur Arbeits- und Organisationspsychologie, 33, 73-83.
[3] Becker, B.E., & Huselid, M.A. (1992). Direct estimates of SDy and the implications
for utility analysis. Journal of Applied Psychology, 77, 227-233.
[4] Bobko, P, & Karren, R. (1982). The estimation of standard deviations in utility
analysis. In K.H. Chung (Ed.), Proceedings of the fourty-second annual meeting of the
Academy of Management (p. 272-276). New York: Academy of Management.
[5] Bobko, P., Karren, R., & Parkington, J.J. (1983). Estimation of standard deviations
in utility analyses: An empirical test. Journal of Applied Psychology, 68, 170-176.
[6] Boudreau, J.W. (1983a). Economic considerations in estimating the utility of human
resource productivity improvement programs. Personnel Psychology, 36, 551-576.
[7] Boudreau, J.W. (1983b). Eects of employee ows on utility analysis of human resource productivity improvement programs. Journal of Applied Psychology, 68, 396406.
[8] Boudreau, J.W. (1991). Utility analysis for decisions in human resource management.
In M.D. Dunnette & L.M. Hough (Eds.), Handbook of Industrial and Organisational
Psychology (621-745). Vol. 2. 2nd ed. Palo Alto: Consulting Psychologists Press.
[9] Boudreau, J.W., & Berger, C.J. (1985). Decision-theoretic utility analysis applied to
employee separations and aquisitions (Monograph). Journal of Applied Psychology,
70, 581-612.
[10] Brogden, H.E. (1946). On the interpretation of the correlation coecient as a measure
of predictive eciency. The Journal of Educational Psychology, 37, 65-76.
[11] Brogden, H.E. (1949). When testing pays o. Personnel Psychology, 2, 171-183.
[12] Brogden, H.E., & Taylor, E.K. (1950). The dollar criterion { Applying the cost accounting concept to criterion construction. Personnel Psychology, 3, 133-154.
[13] Burke, M.J., & Frederick, J.T. (1984). Two modied procedures for estimating standard deviations in utility analyses. Journal of Applied Psychology, 69, 482-489.
[14] Burke, M.J., & Frederick, J.T. (1986). A comparison of economic utility estimates for
alternative SDy estimation procedures. Journal of Applied Psychology, 71, 334-339.
[15] Cascio, W.F. (1982). Costing human resources: The nancial impact of behavior in
organizations. Boston (Mass.): Kent.
[16] Cascio, W.F., & Ramos, R.A. (1986). Development and application of a new method
for assessing job performance in behavioral/economic terms. Journal of Applied Psychology, 71, 20-28.
[17] Cascio, W.F., & Silbey, V. (1979). Utility of the assessment center as a selection
device. Journal of Applied Psychology, 64, 107-118.
[18] Cronbach, L.J., & Gleser, G.C. (1965). Psychological tests and personnel decisions.
2nd ed. Urbana: University of Illinois Press.
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
H. Holling: Utility Analysis Of Personnel Selection
23
[19] Cronshaw, S.F., & Alexander, R.A. (1985). One answer to the demand for accountability: Selection utility as an investment decision. Organizational Behavior and Human
Decision Processes, 35, 102-118.
[20] Cronshaw, S.F., Alexander, R.A., Wiesner, W.H., & Barrick, M.R. (1987). Incorporating risk into selection utility: Two models for sensitivity analysis and risk simulation.
Organizational Behavior and Human Decision Processes, 40, 270-286.
[21] Desimone, R.L., Alexander, R.A., & Cronshaw, S.F. (1986). Accuracy and reliability
of SDy estimates in utility analysis. Journal of Occupational Psychology, 59, 93-102.
[22] Dunette, M.D., Rosse, R.L., Houston, J.S., Hough, L.M., Toquam, J., Lammlein,
S., King, K.W., Bosshardt, M.J. & Keyes, M. (1982). Development and validation
of an industry-wide electric power plant operator selection system. Edison Electric
Institute (cited from Boudreau, J.W. (1991). Utility analysis for decisions in human
resource management. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of Industrial and Organisational Psychology (621-745). Vol. 2. 2nd ed. Palo Alto: Consulting
Psychologists Press).
[23] Eaton, N.K., Wing, H., & Mitchell, K.J. (1985). Alternative methods of estimating
the dollar value of performance. Personnel Psychology, 38, 27-40.
[24] Edwards, J.E., Frederick, J.T., & Burke, M.J. (1988). Ecacy of modied CREPID
SDy s on the basis of archival organizational data. Journal of Applied Psychology, 73,
529-535.
[25] Funke, U., Schuler, H. & Moser K. (1997). Nutzenanalyse zur okonomischen
Evaluation eines Personalauswahlprojekts fur Industriefoscher. In T.J. Gerpott &
S.H. Siemers (Eds.), Controlling von Personalprogrammen (p. 139-171). Stuttgart:
Poeschel.
[26] Gerpott, T.J. (1990). Erfolgswirkungen von Personalauswahlverfahren. Zeitschrift
Fuhrung + Organisation, 59, 37-44.
[27] Greer, O.L., & Cascio, W.F. (1987). Is cost accounting the answer? Comparison of two
behaviorally based methods for estimating the standard deviation of job performance
in dollars with a cost-accounting-based approach. Journal of Applied Psychology, 72,
588-595.
[28] Hakstian, A.R., Wooley, R.M., Woolsey, L.K., & Kryger, B. R.(1991). Management
selection by multiple-domain assessment: II. Utility to the organisation. Educational
and Psychological Measurement, 51, 899-911.
[29] Homann, C.C. & Thornton III, G.C. (1997). Examining selection utility where competing predictors dier in adverse impact. Personnel Psychology, 50, 455-469.
[30] Hukutt, A.I. & Arthur, W. (1994). Hunter and Hunter (1984) Revisited: Interview
Validity for Entry-Level Jobs. Journal of Applied Psychology, 79, 184-190
[31] Hunter, J.E., & Schmidt, F.L. (1982). Fitting people to jobs: The impact of personnel
selection on national productivity. In E.A. Fleishman & M.D. Dunnette (Eds.), Human performance and productivity: Vol. 1: Human capability assessment (p. 233-284).
Hillsdale, N.J.: Erlbaum.
[32] Hunter, J.E., Schmidt, F.L., & Coggin, T.D. (1988). Problems and pitfalls in using capital budgeting and nancial accounting techniques in assessing the utility of
personnel programs. Journal of Applied Psychology, 73, 522-528.
[33] Judiesch, M.K., Schmidt, F.L., & Hunter, J.E. (1993). Has the problem of judgment
in utility analysis been solved? Journal of Applied Psychology, 78, 903-911.
[34] Landy, F.J., Farr, J.L., & Jacobs, R.R. (1982). Utility concepts in performance measurement. Organizational Behavior and Human Performance, 30, 15-40.
[35] Ledvinka, J., Simonet, J.K., Neiner, A.G., & Kruse, B. (1983). The dollar value of
JEPS at Life of Georgia. Unpublished technical report (cited from Boudreau, J.W.
(1991). Utility analysis for decisions in human resource management. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of Industrial and Organisational Psychology
(621-745). Vol. 2. 2nd ed. Palo Alto: Consulting Psychologists Press).
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers
H. Holling: Utility Analysis Of Personnel Selection
24
[36] Lee, R., & Booth, J.M. (1974). An utility analysis of a weighted application blank
designed to predict turnover for clerical employees. Journal of Applied Psychology,
59, 516-518.
[37] Murphy, K.R. (1986). When your top choice turns you down: The eect of rejected
oers on the utility of selection tests. Psychological Bulletin, 99, 133-138.
[38] Murphy, K.R., & Cleveland, J.N. (1991). Performance appraisal. An organizational
perspective. Boston: Allyn & Bacon.
[39] Raju, N.S., Burke, M.J., & Normand, J. (1990). A new approach for utility analysis.
Journal of Applied Psychology, 75, 3-12.
[40] Rich, J.R., & Boudreau, J.W. (1987). The eects of variability and risk on selection
utility analysis: An empirical simulation and comparison. Personnel Psychology, 40,
55-84.
[41] Roche, U.F. (1961). The Cronbach-Gleser utility function in xed treatment employee
selection. Unpublished doctoral dissertation, Southern Illinois University, Carbondale.
Portions reproduced in L.J. Cronbach & G.C. Gleser (Eds.), (1965). Psychological
tests and personnel decisions. 2nd ed. Urbana: University of Illinois Press
[42] Schmidt, F.L., & Homann, B. (1973). Empirical comparison of three methods of
assessing utility of a selection device. Journal of Industrial and Organizational Psychology, 1, 13-22.
[43] Schmidt, F.L., & Hunter, J.E. (1983). Individual dierences in productivity: An
empirical test of estimates derived from studies of selection procedure utility. Journal
of Applied Psychology, 68, 407-414.
[44] Schmidt, F.L., Hunter, J.E., Outerbridge, A.N., & Trattner, M.H. (1986). The economic impact of job selection methods on size, productivity, and payroll costs of the
federal work force: An empirically based demonstration. Personnel Psychology, 39,
1-29.
[45] Schmidt, F.L., Hunter, J.E., & Pearlman, K. (1982). Assessing the economic impact
of personnel programs on workforce productivity. Personnel Psychology, 35, 333-347.
[46] Schmidt, F.L., Hunter, J.E., McKenzie, R.C., & Muldrow, T.W. (1979). Impact of
valid selection procedures on work-force productivity. Journal of Applied Psychology,
64, 609-626.
[47] Schmidt, F.L., Mack, M.J., & Hunter, J.E. (1984). Selection utility in the occupation
of U.S. park ranger for three modes of test use. Journal of Applied Psychology, 69,
490-497.
[48] Taylor, H.C., & Russel, J.T. (1939). The relationship of validity coecients to the
practical eectiveness of tests in selection: Discussion and tables. Journal of Applied
Psychology, 23, 565-578.
[49] Tziner, A., Meir, E.I., Dahan, M., & Birati, A. (1994). An investigation of the predictive validity and economic utility of the assessment center for the high-management
level. Canadian Journal of Behavioural Science, 26, 228-245.
[50] Van Naersson, R.F. (1963) Selectie van chauers. Groningen. Groningen, The Netherlands: Wolters-Noordho. Portions reproduced in L.J. Cronbach & G.C. Gleser
(Eds.). (1965). Psychological tests and personnel decisions. Urbana: University of
Illinois Press.
[51] Weekley, J.A., O'Connor, E.J., Frank, B., & Peters, L.W. (1985). A comparison of
three methods of estimating the standard deviation of performance in dollars. Journal
of Applied Psychology, 79, 122-126.
[52] Wroten, S.P. (1984, August). Overcoming the futilities of utility applications: Measures, models, and management. Paper represented at the annual meeting of the
American Psychological Association, Toronto (cited from Boudreau, J.W. (1991).
Utility analysis for decisions in human resource management. In M.D. Dunnette &
L.M. Hough (Eds.), Handbook of Industrial and Organisational Psychology (621-745).
Vol. 2. 2nd ed. Palo Alto: Consulting Psychologists Press).
MPR{online 1998, Vol.3, No.1
c 1998 Pabst Science Publishers