Journal of Scientific & Industrial Research
Vol. 58, March - April 1999, pp 149-159
Building a Process Performance Measurement System:
Some Early Experiences
P Kueng*
University of Fribourg, Rue Facigny 2, 1700 Fribourg, Switzerland
e-mail: [email protected]; Fax : ++41 263009726
and
A JW Krahn
A T Kearney (International), Bahnhofstrasse 8, 6300 Zug, Switzerland
In order to gain a competitive advantage many companies are engaging in the reorgani zation of their business processes and
implementing process-based management. However, the effects of such change programmes are often not readily apparent.
Process Performance Measurement Systems (PPMS) can be used to deal with this shortcoming. The authors argue that modern
process-oriented organizations need to establish a system which provides comprehensive and timely information on th e
performance of business processes. Based on the authors' experience with several enterpri ses, an approach to composi ng a
sound PPMS is descri bed and conclusions are reported .
1 Introduction
"Measurements are key. If you cannot measure it, you
cannot control it. If you cannot control it, you cannot
manage it. If you cannot manage it, you cannot improve
it."! And further on, the same author writes
"Measurement is fundamental to our way of life. We
measure everything. ( ...) When we were babies, the
doctor measured our height and weight to be sure that
we were healthy. When we started school, our teachers
measured us to understand our weaknesses and help us
progress."! Although these statements made by James
Harrington - one of the early proponents of business
process restructuring - are almost ten years old and
have been repeated quite often, they have not lost their
relevance. Even a cursory examination shows that many
organizations have been undergoing business process
reengineering (BPR) programmes in the last few years .2
Despite the large expenditures on reengineering
programmes, few companies are able to assess the
current level of performance of their processes.
Performance measurement at the process level still plays
a minor role. Additionally, those few enterprises which
do have a process-oriented measurement system in place
usually take only financial and time-related aspects into
consideration.
"Corresponding au thor
An enterprise which is implementing a process-based
organization, either through a radical or a stepwise
approach, should be able to answer the followin g two
questions: (I) Is the current performance of the busi ness
process better than it was yesterday? (2) To what degree
are the to-be values fulfilled? In order to answer these
questions, the so-called Proce ss Performance
Measurement System (PPMS) is needed .
The paper is structured as follows: the main
components of process management are presented in
Section 2. In Section 3 an overview of the state of the art
in the field of performance measurement is given.
Section 4 outlines the functional requirements for a
PPMS. Section 5 presents a nine-step approach - applied
in the project PROMOSYS - to compose a PPMS.
Finally, Section 6 concludes with some lessons learned.
(An earlier version of this paper has been published in
the proceedings of the BITWorld conference. 3 )
2. Three Components of Process Management
In the past, Taylorism made it possible to build up
organizations along functional lines. While this sort of
organization was an effective way to increa se
productivity during the industrialization era, it was later
recognized that core competencies lie in cross-functional
150
J SCI IND RES VOL 58
processes. The refore busin ess process manageme nt was
introduced which includes th e fo ll ow ing compone nts:
Process d esign: Since many corporations are organized
along fun ctional lin es, they us ua lly have littl e knowl edge
a bout th e ir processes . T hus , e stabli s hin g a process ori e nted organi zation starts with the ide ntifi cation of the
processes needed. In o rde r to des ig n, communicate, and
impl e ment busin es s processes, process mode ls can be
seen as an esse nti a l pre requi site . Ku e ng and Kawale k4
descr ibe a goal-based approac h to bu s in es s process
mo de llin g . It is co mpo sed o f th e fo ll ow ing ste ps : I .
De fin iti o n o f bu s in ess pro cess- re lat ed goa ls. 2.
De ri vation a nd de finiti o n o f busin ess ac tiviti e s. 3.
Descripti on and ass ignment of rol es . 4. Mode llin g of
obj ec ts.
Bllsiness case exec ut ion: Afte r compos in g and
imple me ntin g the ne w es tabli s hed processes, bu sin ess
cases (i .e. instances of a proces s suc h as ' loan appl icati on
No . 597 1') are ca rri ed o ut. T he executi on o f busin ess
cases re li es u po n th e coo rdin a ti o n , co ntro l, and
com lllunicati o n of ac tiviti es (wo rkin g steps) . As th e
capabilities of in format ion tec hno log ies have improved
co n s id e rab ly, th e de pl oy me nt of so -ca ll e d CSCW
(co mput er-s upp o rt e d coo pe rative wo rk ) too ls is
beco min g mo re and mo re com ll1 o n. Put s impl y, two
ca tegories of CSCW tools ca n be di stin gui shed: workflow
system s that ca n be effective ly used fo r hi g h Iy structured,
pre-defined processes, and g roup ware tools w hi ch are
a im e d a t supporting ra th e r un s tru c ture d , a d-h oc
processes.
Process m eaSlIreJll ent {[nd illlprovem ent: BPR proj ects
usua lly sto p afte r impleme ntin g th e new ly des ig ned
p rocesses. ]f no furth e r ac t iv it ies took pla ce, th e
e nte rpr ises a nd th e ir proce ·s ma nag e rs wou ld fa ce
~eve r LtI s ho rtcomin gs: no informati o n o n the ac tu a l
perfo rmanc e o f bus in ess processes wou ld be ava ilable,
dec is ion s about effec ti ve reso urce a ll ocat ion coul d no t
be m ad~ on a sound basi s, d iag nos in g th e wea kn esses
o f bu s in ess processes would be hard , and dec is io nmakin g rega rdin g co rrec ti ve a c ti o n s w o uld be
exceedin g ly diffi cu lt. Dea lin g wit h th ese aspects is the
aim o faPPMS.
3.
Performance Me as urement: S tate Of The A,·t
Perfo rm ance measu re me nt sy stems have been around
fo r quite so me time. Th e int e ll ectual root s of today 's
MARCH - APRIL 1999
measure me nt syste ms can be found in th e pe ri od of
move ment toward s sc ientific managem en t at the end of
the 19th century. For a long time, the most popu lar
meas ure me nt system was the so-ca ll ed D uPont sc he me,
introduced in 191 9 by the DuPont company. Even thoug h
th e DuPont sc he me (w hi c h had as it s main me asure
'return on in vestme nt' ROI) has been c ri tic ized heav i Iy,
it is still be in g tau g ht in its origi na l or in a sli g htl y
m o difi e d fOfm . Much m ore imp or tantly , in man y
e nte rpri ses, th e co ntro ll e rs still a ssess pe rforman ce
mainl y throug h finan c ia l measures. Moreove r, to the
ex te nt that managers do focus on finan c ia l parameters,
they have a stron g ince nti ve to m ~\J1ipulate the
fi g ures th ey re po rt. ) A furth e r aspec t is rai sed by
Letza: " ... tra diti o nal meas ure me nt sys te m s ha ve a
contro l bi as, that is, th ey spec ify th e parti cular acti ons
th ey want e mpl oyees to take and th e n measure to see
w hethe r o r not th e empl oyees have taken these acti o nsth ey try to control be hav iour."()
Durin g th e 1980s a nd 1990s the s ituati o n c hanged in
a s ig nifi ca nt way: se lf- assess me nt s, qua lity awards,
be nchmarkin g, a c ti v it y -b ased cos t in g, capab i li ty
maturity mode l, balan ced sco recard , workfl ow-based
monito rin g, etc. were the bu zzword s wh ic h dominated
di scuss io ns in the fie ld of pe rformance eva luation. T hese
approach es are dis c u sse d be low and th e ir m a in
characteri stics are sLlmmari zed in Tabl e I.
3. 1 A ctivity-Based Costin g
Activity-Based Cos tin g (ABC) was deve loped in the
mid 1980s within th e fram ework of Cost Manageme nt
System-Programs by Co mputer A ided M anu fac turin gInte rnati o na l, In c . The ir co nce ption came f ro m th e
fo ll ow in g co ns id e rati o ns : Mod e rn m a nufac turin g,
log isti cs a nd info rmati o n tec hn o log ies c ha nge, to a
co ns id e rab le degree, the process and cos t structures.
Instead of direc t, va lu e added ma nufac tu ri ng ac ti viti es
mo re and more plannin g, mo nito rin g and contro llin g
activ iti es, in the indirect ran ges, dom in ate in e nterprises.
As a co n ~:equen ce thi s activ ity shi ft bas substanti al effects
o n th e ope r a ti o nal co s t s tru c ture. The g row in g
impo rtan ce of fi xed overhead cos ts lea ds to the des ire
fo r bette r cos t transpare ncy by deve lo pi ng A BC. In other
word s, ABC syste ms assi g n the costs of an orga ni za ti o n's
acti viti es more acc ura te ly to its produ cts and produ c t
li nes. ABC sys tems are design ed by firs t id ent ify in g th e
act iv i ti e s pe rfo rm e d by eac h s u pport an d ope rati ng
.-'
I
KUENG & KRAHN
PROCESS PERFORMANCE MEASUREMENT SYSTEM
Table I -
151
Characteristics of selected measurement approaches
Objects measured
Frequency
Type of measures
Recipients ofresults
enterprise
recurring
financial
middle and top
management
activities and
processes
recurring
financial
finance department
Balanced Scorecard (BSC)
enterprise or
organizational units
recurring
financial and
nonfinancial,
quantitative and
qualitative
mainly top
management
Self-Assessment (e.g. by
the EFQM model)
enterprise or
organizational units
nonrecurring or
recurring
mainly nonfinancial
middle and top
management
Competitive
Benchmarking
enterprise,
organizational units,
processes
nonrecurring or
recurring
mainly nonfinancial
and quantitative
middle and top
management
Statistical Process Control
(SPC)
processes
continuously
mainly nonfinancial
and quantitative
middle management
and process actors
Workflow-based
Monitoring
processes
continuously
mainly nonfinancial
and quantitative
middle management
and process actors
software processes
recurring
nonfinancial
middle management
and process actors
processes
recurring or
continuously
financial and
nonfinancial,
quantitative and
qualitative
middle management
and process actors
Criteria
Approach
DuPont scheme
Activity-Based Costing
(ABC)
Capability Maturity Model
(SW-CMM)
Process Performance
Measurement System
(PPMS)
depa rtment a nd th e n computing th e unit costs of
performing th ese activities .7. x
3.2 Balanced Scorecard
The Balanced Scorecard (BSC), developed by Kapl an
and Norton~, was developed to describe an organi zati on's
overa ll pe rformance usin g a number of financial and
nonfinancial indi ca tors o n a reg ul a r basis. For thi s
purpose a framework with four perspectives has been
suggested : th e fin a nc ial , th e customer, th e inte rnal
bu s in ess, and th e lea rnin g and growth perspective.
Accordin g to the ori ginators, th e application of thi s too l
can be seen in three areas: for the purpose of strateg ic
perfo rmance reportin g; to link strategy with performance
measures ; to present different perspectives. An important
characteri stic ofBSC is th at the too l is concentrated upon
corporations or organizational units such as strateg ic
business units. It looks at bu sin ess processes onl y in as
far as th ey are c riti ca l for achieving cu stomer a nd
shareh o lde r obj ec ti ves. 111
3.3 Se lj~ass essfll enf
The roots of th e so-called self assess men ts ca n be
seen in the qu ality move me nt which started in Japan. In
1951 , Japan awarded th e first quality-drive n enterprise
with th e so-ca ll ed D e min g Application Prize.
Encouraged by th e Japan ese success the USA laun c hed
th e Malcolm B a ldrid ge Nation a l Quality Award
(MBNQA ) in 1988 . Finally the European Foundation
for Quality Management (EFQM) followed in 1992 with
152
J SCI IND RES VOL 58
the European Quality Award (EQA). In the American
approach the criteria used belong to the following seven
categories : leadership, strategic planning, customer and
market focus, information and analy sis, human resource
de ve lopment, process management, and business
results.11 The EFQM model on the other hand applies
the fo llowing nine categories of criteria: leadership,
people management, policy and strategy, resources,
processes, people satisfaction, customer satisfaction,
impact on society, and business results. 12 These criteria
can be used separately, i.e. without applying for an award.
From thi s it follows that a self-assessment, based on the
approaches di scussed, may help to assess process
performance , U but one has to keep in mind that the
primary focu s is upon organization s , and not on
processes.
3.4 Competitive Benchmarking
" ... we reali sed that benc hmarkin g could be applied
to all aspects of our business , and comparing ourselves
aga inst companies outside our immediate competition",
is a statement from two Xerox employees, Cross and
Iqbal. I ,) It is not accidental that a qu ote from Xerox has
bee n c ho se n, s in c e X e rox wa s th e c omp any that
intr odu ce d th e now w id e ly appli e d c onc e pt of
benchm arking. In additi on to a systematic evaluation of
business performance, benchmarkin g seeks to achieve a
seco nd goal , th e id entifi cati on of the bes t prac ti ces; but
thi s is beyond th e scope of thi s paper. W hat is the
relevance of co mpetiti ve benc hmarking within th e fi e ld
of process performance measurement? The benefits are
twofo ld. First, it can be llsed to stimul ate the di scuss ion
of perfo rman ce measure me nt. Kn ow in g the process
pe rfo rman ce leve ls of a n exce ll e nt com pa ny may
moti vate not onl y c hi ef executi ves, but also lower-leve l
ma nage rs a nd o rdin a ry s ta ff. Seco nd , co mpe titi ve
benchmark in g can be used to set targets for the process
perfo rmance leve l a compan y wants to ac hieve.
3.5 Statistica l Process COlltro l
Acco rdin g to Juran and G ryna stati sti cal process
cont ro l (SPC) can be defin ed as " .. . the applica ti on of
statisti cal meth ods to th e measurement and an alysis of
vari ati on in any process" l'i. The essence of SPC is to
take as mu c h vari ati on as poss ibl e out of the process. In
oth er wo rds, th e main obj ec ti ve of SPC li es in th e
ac hi eve me nt of stable processes throug h a reducti on of
process vari ati on. Stability in a process, i.e . a state of
s tati s ti ca l co nt ro l, ma kes it poss ibl e to predi c t th e
MARCH - APRIL 1999
behaviour of the process. To this end, making reliable
predictions regarding product quality (i .e. predicting
whether the product specifications will be met) has
become an important tool of competition. 15
3.6 Capability Maturity Model for Software
The Capability Maturity Model for Software (SWCMM), was de veloped by the Software Engineerin g
Institute (SEI) of the Carnegie Mellon Universi ty in
Pittsburgh. The underlying premise of SEI's maturity
model is that the quality of software is largely determined
by the quality of the software development process
applied to build it. By means of a questionnaire, an
organization can assess the quality (maturity leve l) of
its software process. The five stages, defined by SEI,
are as follow s: ( I) Initial, i.e . the software process is
characterized as ad hoc , and occasionally even chaotic.
Few processes are defin ed, and success depe nd s on
individual effort and heroics. (2) Repeatable . (3) Defin ed,
i.e. the software process for both managem ent and
engineering activities is documented, standardized, and
integrated into a standard software process fo r th e
organization. (4 ) Managed, and (5) Optimizing, i.e.
continuous pro cess imp ro ve m e nt is enabl e d b y
quantitative feedback from the process and from pil oting
innovati ve ideas and tec hnol ogies . 16, 17 On e of th e most
important strength s of the CMM mode l is th at eac h
maturity leve l (except leve l one) is decomposed into
several key process areas that indica te th e areas an
organi zation should focus on to move fro m one leve l to
the next.
3.7 Workf low-based Monitoring
During the past few years workfl ow systems have been
g iven co nsid erable attenti on, both in researc h and in
practi ce . Workfl ow sys tems support automatic or semiautomati c executi on of process instances, coordinati on
betw ee n process ac ti viti es, and th e co mmuni cation
between process ac tors, As a by-product of thi s sup port
masses of data are gathered . They ca n be evalu ated
a ut o mati ca ll y a nd m a y o ffe r u seful info rm a ti o n
regarding ac tivity-related cos ts, queuin g ti me of process
instances, workl oad of process paTtic ipants, etc . Whil e
tradition al measurement covers th e fitem in its ent irety,
workfl ow-based monitorin g concentrates u pon business
processes. A further difference li es in the time peri od
reported . Traditi onal co ntro l offers a pos t-h oc view,
whereas workflow-based monitoring has the character
of real-time reporting. IX The merits of wo rkflow-based
KUENG & KRAHN
PROCESS PERFORMANCE MEASUREMENT SYSTEM
153
Financial
aspects
Customer
aspects
Innovation
aspects
Societal
aspects
Employee
aspects
Figure I - Five performance relevant as pects
monitoring li e in the fast reportin g procedure as well as
in its fo cus on business processes. Its limitation s, on the
other hand, are that qu alitati ve perfo rm ance data and
perform ance data about activities carried out manu all y,
can hardly be taken into considerati on. A tec hnical view
is given by th e Workfl ow Management Coalition which
define s the term Workfl ow Monitoring as " ... the ability
to track and report on workflow events durin g workfl ow
execution . Workfl ow m onit orin g ma y be used, for
example, by process own ers to monitor the performance
of a process instance during its executi on"IY.
3.8 Comparison of Measurement Approaches
As we have see n in Sec ti o n 2 , process-o ri e nted
organi zati ons need a process management whi ch is abl e
to measure the current leve l of process perform ance.
Therefo re a measurement system is needed whi ch is
f ocused upon p rocesses - and not on organi sati onal
units such as departments. Since bu siness processes may
cross departments or even divi sions, thi s aspect is central.
Additi onall y, effecti ve process management requires a
broad spectrum of performance-relevant da la. Thu s,
fin anc ial and nonfin anc ia l data as we ll as qu antitati ve
and qualitati ve data are needed . Fro m Table I we can
see that none of the measurement approaches fulfil these
two criteri a. The aim of a PPMS is to fill thi s gap.
4. PPMS: The Concept
In general, a PPMS can be seen as an inform ati on
system which supports process actor and their colleagues
to improve the co mpetiti veness of business processes
sustainably. It is a too l to visualize and to improve
process performance continuously. Thus, it assists both
a total quality management philosophy and a processbased approach. One of the main characteri stics ofPPMS
is that it presents an integral and holistic view of the
performance of business processes. In order to guarantee
a sound view, we apply a stakeho lder-driven approach.
The stakeholders we take into account are the foll ow ing:
mon e y le nd e rslin ves to r s, e mplo yees, cu s to me rs
(suppli ers and buye rs), and soc iety. Each grou p of
stakeholders is re presented by an aspect or a dimension
of performance. Thu s, the aspects of performance we
are looking at are as foll ows: ( I ) financia l aspects (to
measure the degree of sati sfaction of the money lenders/
investors), (2) empl oyee aspects, (3) customer aspects,
and (4) soc ietal aspects. As inn ovati on is an essenti al
driver of future perfo rmance, a fifth aspect has to be
added - innovati on; (see F igure I ).
The main functi onality of a PPMS dea ls wi th the
fo ll ow ing (Figure 2):
The PPMS coll ects the current va lu es (as-is values)
of indi v idual , pro cess -s pec ifi c pe rfo rm a nce
J SCI IND RES VOL 58
154
MARCH - APRIL 1999
Process Perfonnance Measurement System
(PPMS)
gatherperfonnance-
"'' \~~ \ '\
Business process actors
Figure 2 - PPMS fro m a co nceptual view
•
s.
in d ica tors. (Th e e li c it a ti o n of th e in d icato rs is
di scussed in Secti on 5.)
T he PPMS compares current valu es again st target
valu es (to-be va lu es) a nd hi sto ri ca l va lu es. It
calc ulates the trend , i.e., it establi shes whe ther th e
gap is w idening or narrow ing .
T he PPMS calc ul a tes 'cause-effect' relati onships
be twee n th e a ppli ed pe rfo rm a nce indi ca tors . It
shows th e de pendencies between the indi ca tors and
g ives hints as to whether a certain indi cator could
be used as a lead indi cato r o r an early-warnin g
indi ca tor.
T he PPMS di ssemin ates the results (current va lues,
hi stori cal va lu es, ta rget valu es, and trend ) to the
process ac to rs . Th e y can use th e infor m a ti o n
prov ided in orde r to ide nt ify correcti ve acti ons (e .g.
process modi ficati on, stronger IT support, training,
rearra ng in g inform ati on fl ow, etc.) whic h should
lead to a hi g he r leve l of process pe rforma nce .
Composing A PPMS: 9 Steps
Takin g in to acco unt th e conce ptu al view a nd th e
fun cti onaliti es required as presented a bove, it becomes
obvious th at PPMS cann ot be bought in the form o f prepac ked, o ff-th e s he lf so ft ware. A lth o u g h p op ul a r
enterpri se resource pl anning (ERP) systems like Baan,
P eo pleS oft , or SAP h ave fac iliti es to genera te
performance reports, they have substanti al shortcomin gs:
firstl y, the measures co ll ec ted do not take qu a litati ve
aspects into co nsiderati on. Secondl y, s ince most ERP
syste ms we re built before bus in ess processes became
regarded as a central organi zati onal concept, the objects
measured are not always processes. Thirdl y, th e reports
generated co mpri se too mu ch de tail whi ch can lead to
los ing one's bearings .
In ord e r to redu ce th e s ho rtco min gs o f process
p e rfo rm a n ce meas ure m e nt , F rib o ur g U ni ve rs it y
(Sw itzerland) launched the project PROMOS YS in 1996 .
In co-ope rati on w ith four e nterpri ses, we co mp osed
e nte rpri se- and process-spec ifi c PPMS prototypes. T he
four parti c ip atin g ente rpri ses ca n be c harac te ri zed as
fo llows : The first one is a multina ti onal pharmaceuti ca l
company; the PPMS to be built has to support va rio us
processes within th e area o f 'fin a nce ' . Th e seco nd
e nterpri se is a med ium-sized Sw iss commerc ial ba nk;
the process to be supported is ca ll ed ' managing mortgage
KUENG & KRAHN
PROCESS PERFORMANCE MEASUREMENT SYSTEM
applications'. The third enterpri se is a small wholesa ler
which operates mainly in Switzerland; the process to be
supported is ca lled 'acquisition of new customers'. The
fourth enterpri se produces e lectroni c components and
has manufacturing sites in four countries; the PPMS will
support the ordering process - it starts with 'order entry'
and e nd s with "c heck ing custo me r's payment". The size
of the parti c ipatin g enterpri ses va ri es from 50 to 60,000
employees.
The approac h cons ists o f nin e steps we have c hosen
in order to compose a PPMS .
Step 1: Idelltifyi1lg busi1less process goals. Busin ess
).
processes and thei r acti viti es have to make a contributi on
to the process goa ls. Therefore process goa ls (sometimes
referred to as process objec ti ves) have to c larify w hat is
to be achieved for a business process to be co mpetiti ve
in th e lo ng te nn . In thi s first step, process goa ls are
estab li shed th roug h a co ll aborati o n between th e vari ous
process parti c ipants , e.g. process manager, th e process
actors, the manage me nt of the e nterp ri se, and th e process
custo me rs (s uppli ers and buyers). As indi cated in F igure
3, potential process goa ls are ' littl e wo rkin g cap ita l',
' hi I::>a h customer satisfac ti o n ', o r 'good working
atmosphe re' .
Step 2: Defillillg illdicators for each process goal.
T hrouoh
the use of indi cators it may be judged to w hat
I::>
ex tent the process goa ls have been fulfilled. In orde r to
find possible ind icators for a certain goa l, the foll ow in g
question ca n be asked: Which indi ca tor(s) can be used
in orde r to measure (o r to get an indication of) the exten t
to w hi c h a certain goa l is ful fill ed ? It is obvi o us that
many goa ls cannot be meas ured sufficientl y by a sin g le
aoa ls in the foreoI::> round and do not tak e furth e r aspec ts I::>
whic h are not less impo rtant - into co ns id eration. To
broade n th e scope towards the five aspects mentioned
in F igure I , we supported the d iscuss ion thro ug h a li st
of possible goa ls and indi ca tors for eac h of the five areas.
Step 4: E llsuri1lg acceptall ce . One of the crucia l
req uireme nt s fo r th e e ffe ct ive use of PPMS is the
acceptance of the chose n indicat o rs - by th e managers
as well as by the othe r process actors. He nce it is esse nti a l
to ensure that process parti c ipants can exp ress w het he r
or not they cons id e r th e proposed goa ls and indi cators
to be usefu l. Based o n thi s feedback (obtain ed throu g h a
1uesti on naire , fa ce- to- face-communicati o n , o r g roup
155
discu ss ion) goals and indicators can be changed to meet
the customer's requirements m ore effectively.
Step 5: Defining data sources and target values. For
eac h indicator one has to define where data (input) come
from and how these data can be accessed. Potenti al data
so urces are: d a ta b ases of ERP sys te m s, workflow
management system s or customer su rveys. Furthermore,
target values (to-be values) have to be dete rmined for
eac h indi cator. Omitting the definiti o n of target va lu es
would not o nl y mak e it imposs ibl e to de termin e the
deg ree of goal fulfilm e nt , it would a lso lead to a PPMS
without moti vati onal effect. In orde r to set reali st ic but
c hall e ngin g target values the concept of compet iti ve
benchmarking can serve as a he lpful instrument.
Step 6: Judging technical feasibility alld eco1lomic
efficiency . To assess the current pe rfo rmance level of
the se lected indi ca tors, differe nt data sources have to be
accessed. Throu g h the identification and definit ion of
data sources, hint s concerning feasibility and costs can
be obtained . By co mparin g the costs for gathering th e
necessary data aga in st the potential value of an ind icator,
economic effic ie ncy ca n be approx imated. If the number
o f indi cators has to be reduced, eco no mi c efficie ncy
crite ri a (po tenti a l va lu e vs. costs) can be appli ed. What
should be done if data on a certa in performance indi cator
are ex tre me ly hard to access? Unfo rtun atel y, there is no
c lear answer. As Austin 21J points out , m any important
(c riti ca l) performance indi cato rs a re difficult a nd
expensive to measure, and it is tempting no t to take them
into co ns iderat ion. On the o th e r hand it would not be
w ise to meas ure a ll c riti ca l indicators. In short, the
benefits of an indi cato r mu st exceed the costs .
Step 7: Implem elltillg th e PPMS . O nce indicators, target
va lu es and data so urces are defined, the in stmments for
c1ata co ll ecti o n, the c1ata management , and the calculati on
procedure should be de termi ned and impl e me nted . Fo r
the tec hnical part of th e impl e mentation a c li e nt/server
syste m - w ith a n und e rlyin g re lational data base
management system - is usually ap propri ate. Today 's
operati ona l information systems can de liver some input,
but they are not suitecl to give soun d, process-o ri e nted
informati o n. Therefore, add iti ona l tools ancl in struments
(e.g. to support em ployee surveys on a regul ar bas is)
have to be des ig ned to gat her the necessa ry in formation.
Step 8: Using th e PPMS . Usi ng PPMS means measurin g
th e curre nt va lues of g iven indi cators co ntinu ous ly o r
Vl
0\
Aspects
Process Goals
Financial aspects
little working capital
Performance Indicators
Cycle time between the activities "release order" and "record payment"
Proportion of orders where period allowed for payment is not exceeded
.....
Customer aspects
CIl
high customer satisfaction
-nzo
:;0
tTl
CIl
Employee aspects
<:
o
r
good working atmosphere
Vl
00
3::
:>
Proporti o6"ofprocess actors who say "My work load is too high"
:;0
n
'-------;1 Proport. of process actors who say "I'm usually suffering from a headache"
:r::,
:>
Societal aspects
~
societal responsibility
F
\0
\0
\0
Number of realized ecology measures per quarter year
Quantity of carbon dioxide emitted per man month
Innovation aspects
)-------I{
creative and stimulated process actors
Number of innovations proposed per quarter yeM
Number of innovations implemented per quarter year
Fi gure 3 - An exa mple o f goa l~ and perform ance indicators for a business process
)r
-(
KUENG & KRAHN
PROCESS PERFORMANCE MEASUREMENT SYSTEM
regularly, comparing these current values against target
values, and feeding back the information gained to the
process participants. Some of these functions can be
automated but others have to be carried out manually by
personnel who have the necessary domain knowl edge.
Step 9: Improving business processes and modifying
the indicators continuously. Bu siness processes have
•
to be competitive in the sense that the market with its
customers and its suppliers is sati sfied . A PPMS aims to
achieve this . For that purpose one has to keep in mind
that the market is dynami c: business processes which
are valid tod ay may be obsolete tomorrow. From thi s it
follows that business processes as well as indicators have
to be adapted freq ue ntl y. Throu g h thi s adaptation
hi storical comparability may suffer "( ... ) but thi s is a
minor loss. What matters is how a company is doing
compared with its current competitors, not with its own
past'" .
6. Conclusions
Having worked two years on the conceptualization
and impl e m e ntation of Process Performance
Measurement Systems, th e following conclu sions are
drawn.
•
•
The role of a bu siness process manager, or process
owner, is essenti al. He or she has to play the ro le of
a leader, an entrepreneur and a negotiator. If hi s or
her competence is narrowly limited and dec isionmaking power is restricted it will be very timeconsuming to implement PPMS , and, additionally,
PPMS ca nn o t be d e ploy ed effect ive ly if th e
necessary changes cannot be realized by the process
team.
If PPMS is perceived as a sys tem dedicated to
managers, resistance to feeding the sys tem (i.e. not
enterin g needed data) may result. In other words ,
PPMS mu s t offer a be ne fit for eac h process
participant - an incentive to do the job better and to
be more goal-direc ted .
In order to co mpose PPMS , communi cati on within
the process team and communi cation with different
organi zationa l units (e.g. the IS department) is a
key factor. To facilitate communi cati on, the business
process should be documented graphica lly.
One of the cruci al aspects in the process of building
PPMS li es in th e identifi catio n of the appropriate
performance indicato rs. Generally speaking, two
•
157
possibilities exist: ( I) pre-defined, generally valid
indicators can be applied; or (2) indicators have to
be fitted exactly to a given enterprise and its busi ness
proce sses. As mentioned above, in all four
participating enterprises, the second approach was
chosen - and we are convinced that this approach
was appropriate.
From our experience we can also conclude that
process goals were a good starting point for
gathering the right indicators. However, it turned
out that the identification of the business process
goals was often very time consuming. This may be
connected with the fact that until recently, in most
enterprises, goa ls were defined only at the enterprise
or divi sion level, but not at the business process
level. Thinking in terms of process goals helps both
the process owners and th e process actors to put
forward the idea of a 'process management '. The
definition of indicators was also time consuming.
E speci a ll y difficult was finding quantitati ve
measures for aspects w hich are rather qualitati ve
in nature (e.g . customer satisfaction , social aspects ,
etc .).
Process indi cators have to be accepted not only by
the process owner and hi s/her colleagues but also
by the next higher management level. If thi s is not
the case performance-relevant data from PPMS will
not be regarded as a solid basis, a basis to initi alize
or to promote broad correcti ve actions. In order to
impro ve acceptance , th e process actors were
involved in the task of defining the indic ators:
workshops have been organized and questionnai res
have been used to get feedback from process actors.
In contrast to the other measurement instruments,
such as Balanced Scorecard, PPMS measures the
performance of bu sin ess processes - and not of
corporati ons or organi zati onal units. As PPMS may
offer a ho li sti c view on processes, it may effect ive ly
supp o rt a process-oriented o rga ni za ti on, or an
organi zat io n which want s to beco m e processori ented.
Th e imple ment ati on of PPMS requires a lot of
resources and is therefore expens ive. While the cos ts
for hardware and software (e.g. interfaces to ex isting
software, software for presenting the results) mi ght
be moderate, the labour cost for the conceptu al part
of the impl ementati on can reac h a leve l where it
becomes difficult to claim that the benefits ofPPMS
clearly exceed the yie ld .
J SCI IND RES VOL 58
158
•
Managers and busi ness consultants quite often see
performance measurement systems as a 'cockpit'.~
Using this metaphor the process manager is seen
as somebody who has to check various instruments
(where the level of performance for selected
indicators is shown) constantly. As soon as an
unexpected signal appears, or a light switches to
red, the process manager queries his information
system to find the source of trouble . It is certainly
difficult to describe the way in which PPMS should
work, but a purely mechanistic approach would
definitely be inappropriate.
Successful deployment of PPMS requires
organizational and social infrastructure. A corporate
culture with a trusting environment is needed where
process actors participate in designing and
measuring their own tasks .
Finally it can be observed that today's performance
measurement systems still lack effective measurement
of nonfinancial aspects and that they are not focused
upon business processes. We believe that process
performance measurement is a necessity for a modern
process-oriented organization . Based on the authors'
experiences with several enterprises it seem s very
unlikely that a universal set of performance ind icators
can be applied successfully to all business processes.
Thus, performance indicators must be process-speci fic
and have to be derived mainly from process goal '. Thi s
al so implies, that it will not be possible for an off-the
shelf software to support the measurement of process
performance fu I I y.
Acknowledgement
The authors are grate ful to the CTI (Commission for
Technology and Innovation, Berne) for the ir funding .
References
I
2
H a rrington H J. Bllsin ess Pro cess Impro ve ment: Th e
breakthrollgh strat egy / o r total qllality, prodll cti vity, and
comfletitil'eness (McGraw-Hill . New York) 1991. pp.82 and 167.
of the International Conference on Business In/ormatio/1
Technology Management (Har-Anand Publications. New Delhi )
1998. 422-434.
4
Kueng P & Kawalek p. Goal-based business process model s:
creation and evaluation. Business Process Manage 1. 3 ( 1997)
17-38.
5
Eccles R G. The Performance Measurement Manifesto. Ha rvard
Business Rev. 69 (1991) 131-138.
6
Letza S R. The design and implementation of the balanced
business scorecard: an analysis of three com panies in practice.
Business Process Manage 1.2 (1996) 54-76.
7
Brimson J & Antos J. Activity-Based Man agement for Se rvice
Industries, Governm ent Entities, and Nonprofit Organizations
(Wiley, New York) 1994.
8
Glad E & Becker H. Acti vity-based Costing and Managelllent
(Wiley. New York ) 1996.
9
Kaplan R S & Norton D p. The Balanced Sco recard: measures
that drive performance. Harvard Business Rev, 70 ( 1992) 7 1-79.
10 Kaplan R S & Norton D p. The Balanced Sco recard: translatin g
strategy illlo action (Boston MA. Harvard Business School Press)
1996,92.
I I MBNQA, Malcollll Baldrige National Qllality Award Criteria
/ 01' PC/jo rman ,e Excellence. United Sta tes De partme nt of
Commerc e. NIST. Gaith e rsburg M D . USA; http: //
www.quality.nist.gov/docs/98_crit/98 crit.ht m; accessed 12 Jun e
1998.
12 EFQM, Guidelill es on Self-Assesslllent, European Foundatio n
for Quality M a na g em e nt . Bru sse ls , Be lg ium ; http: //
www.efqm.org/model.htm; accessed 12 June 1998.
13 Gadd K, Bu siness self-assess ment: strateg ic tool for build ing
process robu stness and achi ev ing integ rat ed mana ge me nt .
Bllsilless Process Re-ellg & Mallage, 1 (1 995 ) 66- 85.
14 Cross R & Iqbal A. The Rank Xerox Ex perience : benchm arkin g
ten yea rs on. in A Rol stad as (ed.). Bell c/ marking: th enr\' alld
practice (Chapm an & Hall. Lonelon) 1995. 1-10.
15 Juran J & Gryn a F. Qllalill' Plallllillg and Allalvsis:/rOllll llVdIlCl
developmellttilrollgh lise (M cGraw-Hili . New York ) 1993 .
16 Paulk M C. Curtis B, Chri ssis M B & Weber C V, Ca pabilit y
Maturity Model. Version 1.1, IEEE Soji H'are . 10, 199 3. 18-27.
i 7 SEI, SW-CMM \12.0, Orqft C. http :// www.seu: lllu.edu/ac ti vities/
cmm/draft-c/c. htmi. accessed 12 Jun e 1998.
18 McLellan M. Worktlow Metrics: one o f the great bene fit s o f
workllow, Praxis des Workflow-Managelllell i edited by H. Osteri e
& P. Vogler (Vi eweg Verlag. Braunsc hweig) 1996. pp .30 1- 3 18.
Willco cks L. Docs IT-enabled business process re-enginee ring
payoff? Recen t findin gs o n econ o mic s anel impact s , in L
Willcocks (eel. ), In vesting in In[or/llation Syst<'ms: <'l'alilation
and lI1anagement (C hapman & Hall. Lond on) 1996. 171 - 1
19 Workflow Managem ent Coa liti o n. Ten nilwiogv & Giossa rl',
http://www.aiim.org/wfmc/; accessed 12 J une 1998.
Kueng p. Supporting BPR through a Process Performance
Measurement Sys tem. in P. Banerjee et al. (Eels.). Proceedin gs
20 Austin R D. Measuring lIlId Mall agi ng Pe ljo rlllall ce i ll
Olgallizatiolls (Dorset House Publi shin g, New Yo rk ) 1996.
n.
3
MARCH - APRIL 1999
KUENG & KRAHN
PROCESS PERFORMANCE MEASUREMENT SYSTEM
Peter Kueng is a Research Assos iate of the Uni versity of Fribourg,
Switzerl and. He received hi s Ph D in Information System in 1994.
He recentl y spent two years as Visiting Researcher at Linz Uni versity
and Manchester Uni versity. Hi s research focuses on pl anning and
development of IS , process management , and worktlow system.
Adrian Krahn is a Consultant of A T. Kearney (Internati onal) AG,
Switzerl and. He received hi s Ph D in In fo rmation Systems in 1998
fro m Uni versit y of Fri bourg. Before joining theA T Kearny he worked
fo r three years as a projec t leader at F HotTmann-La Roche AG,
Sw it ze rl and . Hi s current resea rc h int eres ts includ e process
management, and process measurement.
159