BEST VALUE PROCUREMENT
RESEARCH ABOUT THE IMPACT OF BEST VALUE PROCUREMENT ON
THE PERFORMANCE OF OUTSOURCING ICT PROJECTS
Strategy track
Supervisor:
Dr. ir. J. Kraaijenbrink
University of Amsterdam
Fall term 2015
by
Rachid el Bardai
Student Number: 10499253
[email protected]
Utrecht, June 2015
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
1
Statement of originality
This document is written by Rachid el Bardai who declares to take full responsibility for the
contents of this document.
I declare that the text and the work presented in this document is original and that
no sources other than those mentioned in the text and its references have been used
in creating it.
The Faculty of Economics and Business is responsible solely for the supervision
of completion of the work, not for the contents.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
2
Content
1. INTRODUCTION ........................................................................................................................ 5
1.1 Overview........................................................................................................................................ 5
1.2 a.s.r and the application of BVP .................................................................................................... 7
1.3 Scientific relevance........................................................................................................................ 8
1.4 Practical relevance ........................................................................................................................ 9
1.5 Problem statement...................................................................................................................... 10
1.6 Research Question ...................................................................................................................... 11
1.7 Structure of the study ................................................................................................................. 12
2. THEORETICAL BACKGROUND .................................................................................................. 13
2.2 Best Value Procurement Method................................................................................................ 15
2.3 Traditional Procurement method and comparison to BVP method ........................................... 17
2.4 ICT projects and other projects ................................................................................................... 19
Conceptual framework...................................................................................................................... 24
3. METHODOLOGICAL DESIGN .................................................................................................... 26
3.1 Research methodology description............................................................................................. 26
3.2 Data-collection ............................................................................................................................ 26
3.3 Data-analyses .............................................................................................................................. 29
3.4 Strengths and limitations of the research design ....................................................................... 30
4. RESULTS ................................................................................................................................. 31
4.1 The projects compared ......................................................................................................... 31
4.2 Analysis of the results ........................................................................................................... 32
Other findings ............................................................................................................................ 37
5. DISCUSSION & CONCLUSION ................................................................................................... 41
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
3
5.1 Main findings............................................................................................................................... 41
5.2 Implications ................................................................................................................................. 45
5.3 Practical Implications................................................................................................................... 45
5.4 Limitations and further research................................................................................................. 46
6 BIBLIOGRAPHY ........................................................................................................................ 48
Appendix A - Interview Questions…....………………………………………………………………………………………….50
Appendix B - Interviews Results……....………………………………………………………………………………………….59
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
4
1. INTRODUCTION
1.1 Overview
Dutch companies and governments purchase annually for many billions of works, services
and supplies. Every year the amount is between the 100 and 130 billion euro. An important
part of this procurement process occurs through a tender. Experience shows that a vast
majority of the contracts are awarded to the lowest price. In the tender of a client, what is
also named as a request for proposal, a clear and detailed description is stated about what
the question is and the market may determine itself what price it wants. In the past,
proposals showing the lowest costs were deemed to be selected. This is also called the
traditional procurement approach. According to Cox (2006) a traditional procurement
approach is when the purchasing party specifies in detail what they exactly wants with the
assumption that it is clear to the potential providers what they are supposed to provide. The
client is seen as the expert and the supplier has only the freedom to come up with a price.
However, a trend seems to come up showing slowly that price is not the deciding factor for
all cases anymore. In 2009 and 2010 more than 5000 tenders were analyzed that took place
in the constructions industry according to a research from ‘Economisch instituut voor de
Bouw’, and the research shows us that there are over 4000 awarded to the lowest price and
1.000 tenders are based on EMVI (economisch meest voordelige inschrijving)1. EMVI stands
for "most economically advantageous tender ', or a weighted combination of price and
quality2. The contracts that are awarded based on EMVI represent in euro’s the most of all
tenders, with 60%. This award method offers clients the possibility to take also other
aspects, like quality, into account in the assessment than just the price.
In the traditional procurement approach is that the client will get what he asks, but does not
provide the maximum revenues from the market. Suppliers are limited challenged to
demonstrate their added value. They deliver what is asked from their client and will thereby
provide actually tend cost- at the required minimum level. The potential gain (meerwaarde)
for the client that could be achieved at little extra cost are excluded. This, while suppliers
often know how they can add value to the customer and the customer does not always see
1
Publiceerde het Economisch Instituut voor de Bouw in 2011
2
http://www.crow.nl/vakgebieden/aanbesteden/emvi-gunnen-op-waarde
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
5
all the solutions in the market. Internationally and in the Netherlands, this leads to a
different perspective on the procurement processes.
Another way of tendering seems to gain a strong foothold in the market. This method offers
more chances for tenders to have more influence in the purchasing process and to align it
more to the interests of the client. This new mindset is called Best Value Procurement
(BVP)3. A large number of companies in the Netherlands have been working according to
BVP and this seems to keep increasing. This method seems to provide also opportunities for
the insurance company a.s.r.
The American professor Dean Kashiwagi is the founder of Best Value Procurement (BVP).
This method was developed and refined by him and his staff at the Performance Based
Studies Research Group (PBSRG) at Arizona State University (ASU) (Kashiwagi, 1991;
Kashiwagi, 2010). Best value procurement is a process that focuses on obtaining the highest
value for the lowest price (or at least to fit in the budget) (Chan, 2004; Egan, 1998; PBSRG,
2010; Kashiwagi, 2010). Best Value Procurement became very popular in contracting
services and purchasing of large companies. Some providers are enthusiastic, because it is
not necessary anymore to produce as much paper as a traditional quotation. But many
companies experience also some difficulties. Best Value Procurement is a shift from
"monitor and control" to "let go and trust ' and from' send by the client" to "be guided by
the contractor’’. In addition, the client makes maximum use of the knowledge and expertise
of the suppliers. The client only facilitates the experts and the contractor takes the lead. The
client focuses on the expertise of the contractor and the transparency of the results of both
the client and the contractor.
According to professor Kashiwagi, the Dutch firms are one of the early adopters of this new
way of thinking, which is also called "performance procurement" (prestatie-inkoop).
Rijkswaterstaat4 has been advised by Kashiwagi personally and his ideas were
enthusiastically embraced and applied in the construction projects that took place in the
framework of the Road Widening Emergency (Spoedwet wegverbreding). Organizations that
Best Value Procurement is also known as PiPS (Performance information Procurement System) / PIRMS (Performance
Information Risk Management System)
4 The executive arm of the Dutch Ministry of Infrastructure and the Environment
3
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
6
used his method are positive about it, with over 98 percent being satisfied with the method
(Kashiwagi, 2011). Meanwhile, there are more than nine hundred projects in six different
countries performed according to the method of procurement performance of Kashiwagi.
In the approach of Kashiwagi it is not the price but the quality of the performance that is
important. The choice to win a tender is based on performance. This does not mean that the
price is less important. Best Value Procurement is specifically intended to create the highest
value: the highest quality at the lowest price (or at least fit in the budget). By tendering and
giving more space to quality and involving the knowledge and ability of the market
participants (suppliers) in the selection and award process, clients can almost always make a
better 'deal' (Kashiwagi, 2010). But this way of thinking requires a different mindset to the
procurement process and the involved stakeholders.
1.2 a.s.r and the application of BVP
A.s.r. is one of the largest Dutch insurance company in the Netherlands and offers all types
of insurance products and services. a.s.r. has more than 4,000 employees and a turnover of €
4.3 billion. A.s.r. has several brands including Ardanta, De Amersfoortse, Europeesche
Verzekeringen and Ditzo brands. The insurance company offers a wide range of financial
products in the field of non-life, Life and Disability insurance, collective and individual
pensions, health insurance, travel and leisure insurance and funeral insurance. Besides
insurance products a.s.r. also provides investment -and bank savings. Additionally the firm is
also active in the real estate industry.5
BVP is becoming more and more an important method for a.s.r. and the successful results
supports this growth within the company. The firm has completed last year an outsourced
project based on BVP. The IT-organization has outsourced last year the service "Security
Monitoring", which provides the assurance of the security guidelines for a.s.r. This project
was done based on the guidelines of BVP and was finished successfully. Now a.s.r. is
interested to apply BVP on more and different kind of projects.
5
https://www.asr.nl/en/Organization/Paginas/ASR.aspx
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
7
1.3 Scientific relevance
In the scientific literature there is an existing body of knowledge about the Best Value
Procurement methodology. BVP is a new way to deliver services, run organizations, and
optimize supply chains. It forms a leadership structure that assists inexperienced project
managers to get optimal results (Kashiwagi, 2011). BVP is now being used to successfully
deliver commodities, professional services, non-construction services, and to a lesser degree
Information Technology (IT) services (Sullivan, et. al., 2010; deyemi, et. al., 2009; Kashiwagi,
J., et. al., 2009).
In the previous research the use BVP has been tested over 700 times, delivering construction
and other services worth over $2.3B (1994-2010). Kashiwagi has been developing BVP for
the last 16 years and the lessons learned on the 700+ tests have resulted in simplifying the
process, and making the BVP method more sustainable. The method has shown to be
successful for several areas, however no studies have shown yet that this method is also
effective for ICT projects, and yet more specifically, the ICT projects in the Netherlands.
Despite the numerous studies showing that BVP is successful in several areas, it does not
necessarily mean that BVP could be as successful for ICT projects. ICT projects differ from
other kind of projects and while the studies performed have shown numerous reasons why
ICT projects fail, some have noted this relates to project management, poor planning,
unclear objectives and goals, inappropriate skills, failure in communication and other
reasons (Al Neimat, Tamour; 2005). Most of these reasons are connected to the selected
procurement method. By selecting specific procurement methods, you can choose in which
degree you would like have the supplier to be involved (room to provide expertise in
designing and planning), allowing not to be driven by cost (buy cheap and waste money) and
other factors. BVP allows this more playing room in several of these factors. Therefore, it is
an interesting question, whether BVP can be a successful method to reduce the number of
failures of ICT projects, by not only minimizing costs, but also increase the value and quality
(performance) (Al Neimat, Tamour; 2005). The outsourcing of ICT projects has been mainly
performed through traditional procurement methods, such as RFP, and several articles have
shown that what reasons underlie the failure of outsourcing the ICT projects. BVP seems to
be a method to overcome this obstacles. As stated before, BVP has shown to be an effective
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
8
procurement method in several fields, but no research conducted yet has shown this should
also be the case for the ICT industry. The ICT industry has been characterized as a struggling
industry (Kashiwagi, D. and Egbu C., 2011). Could BVP be the procurement method that
could decrease the number of failures, especially since it has proven to be a success in
several other fields? What is so different about the ICT projects that makes it struggle with
performance and how could BVP help in this?
This offers from the scientific relevance
reason to further investigate the application of Best Value Procurement on the performance
of outsourced ICT projects.
Furthermore, the Best Value Procurement theory focuses mainly on the selection phase in a
purchasing process. The steps that have to be taken in the clarification phase and during the
execution are also described well in different researches (D. T. Kashiwagi, 2013). But the
applicability and impact of BVP in ICT projects is under researched and there is insufficient
knowledge about the relation on the performance of these kind of projects (D. T. Kashiwagi,
Kashiwagi, Kashiwagi, & Sullivan, 2012). This research therefore highlights the further
development and use of Best Value Procurement in ICT projects. Main BVP projects known
in the Netherlands showing positive results are outside the ICT sector. Currently it is
unknown whether BVP would also yield to such tremendous positive results for ICT projects.
We note that there is a significant gap in the literature on this specific sector. Quint
Wellington (2004) notes this gap too as BVP has not yet been able to prove itself within the
ICT domain. As noted above, several factors affecting the reasons of the failures are related
to the selected procurement method. BVP method shows in its design some important
factors to overcome some of the mentioned reasons in relation to the failures listed. A.s.r.
has changed its procurement method for the ICT projects, abandoning its traditional method
and implementing the BVP method. By researching the results within a.s.r. some light could
be shed on this matter.
1.4 Practical relevance
The strategy of a.s.r. is strongly focused on cost reductions and quality performance. One of
the key activities to achieve these goals is through outsourcing. Currently the administrative
processes and parts of the IT landscape are outsourced to reduce costs and focus more on
the core business (Leiblein 2003, Gilley and Rasheed 2002). The coming years, a.s.r. intends
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
9
to implement Best Value Procurement approach further in their outsourcing ICT projects.
The BVP is a new way of thinking for a.s.r. and requires a different mindset. A.s.r. has the
desire to know how to implement this new development of BVP in their procurement
process, attitude and in the behaviour of their employees according to their ICT projects.
1.5 Problem statement
The above described events in section 1.3 and 1.4 lead to the following problem statement:
The evaluations of the outsourcing projects within a.s.r. shows us that there are quite some
frustrations and problems in the traditional procurement approach (internal a.s.r.
evaluations). The experience of the traditional procurement approach within a.s.r. has
taught us that the good ideas from suppliers are often not identified or appreciated. That is
because it does not fit within the predetermined specifications of the procuring
organization. In a traditional contract it cost the client a lot of time to specify their needs.
Getting acquainted and contacted with all suppliers is also a very time consuming process
that usually takes months. Moreover, a.s.r. is often not satisfied about the delivered
performance after a tender process based on the traditional procurement approach
(evaluation Service Books, BPO pension programs). Now a.s.r. wants to gain more
experience with the use of BVP on their outsourcing projects. As stated earlier in section 1.1
this might be a good alternative and BVP can provide a solution here.
A.s.r. started using BVP on its ICT projects and this has become more and more important for
the firm. In section 1.2 is described that a.s.r. completed successfully its first ICT project
(Security Monitoring) last year based on BVP instead of the traditional procurement
approach. Furthermore, the insurance company has started at the beginning of 2015 a new
project to outsource its ICT workstations (ICT werkplekken) according to BVP.
However, a.s.r. is still relatively unfamiliar with this method and wants to know when BVP is
applicable to their outsourced ICT projects and when BVP will lead to success and when it
leads to a failure in order to improve the performance by reducing project costs, shorten the
project duration (doorlooptijd) and increase the customer satisfaction. In responding to this,
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
10
the research topic is: ‘’Research about the impact of Best Value Procurement on the
performance of outsourcing ICT projects’’. The scope of this research is limited to the
purchase / tendering process, more precisely from supplier selection to the signing of the
agreement. BVP has 4 phases which is further outlined in the literature review on BVP. The
variables are also scoped to the following:

Time: duration of the process;

Cost: monetary costs related to project and effort by involved employees;

Performance: satisfaction regarding outcome for the main stakeholders.
The four phases are essential and crucial for the procurement method to be a success.
1.6 Research Question
Research goal
The goal of this research is to provide recommendations and suggestions for contractors and
a.s.r. on the performance of ICT projects based on Best Value Procurement in order to refine
or design the strategy (approach) and implement actions in their procurement process.
The research question is stated as follows:
What is the impact of the Best Value Procurement approach on the performance (time,
cost and customer satisfaction) of outsourced ICT-projects in the Netherlands?
In the research question it seems to be that there is one BVP approach. However, the
research is based on several possible approaches that could have an impact on performance.
This also means that the different elements of BVP are investigated that may have an impact
on the performance.
In order to be able to answer the question, several sub questions have been formed. BVP is a
procurement method showing benefits that can overcome some of the reasons why IT
projects fail as outlined above. In order assess the impact of the BVP method on ICT projects,
we will outline the more traditional method, showing the differences in relation to BVP. This
should get us closer as to whether BVP indeed could lead to a higher success rate and
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
11
performance on ICT projects. However, one must define how success is perceived by parties
involved in the ICT projects and moreover has the use of BVP indeed led to a higher
performance and why. Gaining insight on this matter, we should be able to have a clearer
view on what impacts the BVP method has or can have on ICT projects and how this affects
the perceived performance by the stakeholders directly involved in such a project.
Sub-questions:
1. Is BVP as applicable to ICT projects as to other projects where it has been proven to
be a successful concept?
2. What effects has the use of BVP to ICT projects in relation to previous used
methods?
3. Do the stakeholders have different expectations on the BVP and how does this
affect their assessment on the performance?
1.7 Structure of the study
The remainder of this thesis is structured as follows: a closer look to previous studies on BVP
and other main purchase methods is given in chapter 2 ending with propositions. The
methodological design of the research is centered in chapter 3, while chapter 4 will provide
the empirical results of the study. The final chapter contains the conclusions, implications
and limitations of this research.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
12
2. THEORETICAL BACKGROUND
The literature review is based on the aspects mentioned in the research question, and
describes the existing theory of best value procurement and other related theories from
different perspectives. In paragraph 2.1 a general understanding is given of the term
procurements methods within organizations, followed by an understanding of the BVP
method in paragraph 2.2, and for the traditional procurement method in paragraph 2.3. In
paragraph 2.4 an overview is provided of why ICT projects that are outsourced seem to fail
and what elements are the cause of it. The chapter concludes with a conceptual model and
propositions.
2.1 Procurement
Before outlining the several procurement methods that have been developed over the
history, one is sure to obtain a general understanding of the term procurement methods
within organizations. A procurement method is used in order to convert requirements or
requisitions into purchase orders or contracts. It is the acquisition of goods, services or
works from an external source. The goal of most firms is to procure at the best possible cost
to meet the need of the acquirer in terms of quality, quantity, time and location. Processes
are defined to promote fair and open competition while minimizing exposure to fraud and
collusion (Weele, Arjan J. van, 2010).
The methods for procurement can be competitive and non-competitive. There is a
preference for using competitive methods of procurement given that they tend to promote
transparency, economy and efficiency, and limit favoritism.
A decision to use a particular procurement method must be based primarily on the
stipulations of the procurement guidelines, manuals and procedures within the organization.
Important factors affecting the type of procurement are the complexity of the requirement,
market availability, the need to have prospective bidders to participate in finalizing the
specifications of the requirement (Nicola Dimitri, Gustavo Piga and Giancarlo Spagnolo,
2006).
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
13
Successful procurement becomes more complex as all parties involved in the project need to
comply with their respective obligations, and identify and deal with risk appropriately from
the outset. Depending on the type of requirement one can ascertain the choice of
procurement type will shift to more open competitive bidding if the complexity of the
requirement increases (Nicola Dimitri, Gustavo Piga and Giancarlo Spagnolo, 2006).
The IT projects are considered to be more customer specified and complex due to their
nature being quickly unique. The procurement strategies maintained for these projects have
not been studied in detail thoroughly. Dean Kashiwagi and Isaac Kashiwagi (2014) expressed
in their article (The best value ICT Industry) a summary of the results of the studies
performed on the success of IT projects. The results show that the failures of IT projects
were more common than successes. Money on IT projects tended to be wasted. The article
also included the results of a parliamentary inquiry that was held to address the poor
performance of IT projects. During the inquiry it was reported an amount of 1-5 Billion was
wasted with ICT-project annually. The United States has also experienced failures, spending
money on failed projects. A serious overview of reasons why projects fail are given, mostly
related project complexity or the lack of expertise. 3 solutions were given to solve the
nonperformance of the IT projects:
1. Make projects smaller;
2. Break up project into smaller milestones to get the final project deliverable;
3. Use the BVP approach to deliver the project.
The center of my research focuses on the third solution given in the article. The use of Best
Value Procurement. The Best Value Procurement is a relatively new procurement method
becoming more and more popular. The remainder of this chapter consists of the traditional
procurement method used for IT projects, comparing the procurement method to BVP.
Please note that regarding the traditional procurement method the most used method is
explored. We will also take time to describe the ICT projects in comparison to other projects.
Paragraph 4 will recap the methods and show the main differences and expectations with
respect to the ICT projects.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
14
2.2 Best Value Procurement Method
Best Value Procurement (BVP) or, as it is called in the Netherlands ‘Performance
Procurement (prestatie inkoop) is becoming very popular in contracting services and
purchasing projects. The purchasing procedure is described in detail in the book "Purchasing
Power" by Jeroen van de Rijt and Sicco Santema (2009). Best Value Procurement is a
philosophy about working together. It is an emerging method where the focus is not on
lowest price, but on the best price-value. It is a procurement, project management, and risk
management approach, based on the concepts of win-win, less management, direction and
control, and transparency between the client and the contractor. According to Kashiwagi
and Byfield (2002) the approach consists of four phases: selection, clarification, pre-award
phase and execution.
To achieve Best Value Procurement is important that clients restructure their position
regarding the contractors. With traditional procurement is the client that determines among
other all the conditions and requirements, but with BVP the contractor will determine the
requirements because he is the expert.
According to Kashiwagi and Byfield (2002) the process of Best Value Procurement exists of
the following four phases:
1. Preparation of the procurement project;
2. The selection phase: the supplier is the expert;
3. The pre-award phase (award stage): the client performs the expert role;
4. Execution: the supplier fulfils the expert role.
Below some figures are showed how the phases are followed in a BVP process.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
15
The first phase is the preparation of the project. The client defines the objectives and scope
of his inquiry. Important here is the focus on what should be reached, rather than how.
Especially within IT projects, this approach is essential since the contractor is the expert. The
client formulates the basic core of the question, based on its own internal expertise. The key
question is presented objectively and measurable, well adapted to the wider organizational
issues, so without stepping into pitfalls, technical requirements or performance indicators.
Preparation
1st filter (selection)
Sent proposition by
suppliers
Past performance
Risk file
Assessing risk and file by
team
Assessing value added by
team
Assessing files by client
Pre-award phase
Risk files
Value added file
Value added files
Interviews
Interviews
Price
contract
Execution
Figure 1. BVP process phases
The second phase according to is about the selection (selection phase). In this phase the
client starts with a meeting for contractors where they will concise and concrete their
proposals. The client selects based on the scope, the risks, and the opportunities and
planning. Within ICT projects, it remains a question whether and to what extent the
(technical) solution contributes to the objective and scope of the question. Finally, there are
interviews with key figures of the contractor to get things clearer in order to define their
proposals.
This phase is the realization phase where the chosen contractor will deepen the offer with
the client deepen. Also in this phase it is essential that the contractor must accompanied the
complex (ICT) projects and assessing the development of the proposals of the contractor.
This includes also the risks, measures management and reports that are arranged for starting
the implementation.
At fourth and last phase is the implementation phase. The contractor reports weekly the
progress regarding money, planning and 'undesirable events'. The contractor optionally
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
16
reports on performance, KPI’s or SLA’s. It is important to know that the influence of BVP
does not stop here. Indeed, thanks to the intensive coordination in the previous stages,
there is now is a good relation between the client and contractor for further cooperation
that is based on trust.
2.3 Traditional Procurement method and comparison to BVP method
The most traditional method for IT projects has been the RfP procedure. The RfP stand for
request for proposal and in its simplest form the organization posts to elicit bids from
potential vendors. It specifies what the customer is looking for and establishes criteria for
the assessment of proposals (Blake G. and Bly R.W., 1993). The RfP is especially used for a
complex IT project, the organization can then benefit from multiple bidders and perspectives
when issuing a RfP. Some organization, such as government agencies are usually even
required to issue RfPs to provide full and open competition. Other organizations can have
other incentives to use RfPs, such as driving cost down and boost the competition.
However, this illustrates immediately that the success depends on the skills of the customer
to create the RfP. For example, if the specified requirements are vague the bidder can design
the wrong solution. Overly drafted RfPs, being to detailed and restrictive, limit the bidders’
creativeness and innovative techniques the customer might not have thought of.
The RfP process can start with a draft RFP which bidders will review and give suggestions for
improvement. The customer can then create the final RfP and issue it to the bidders for
proposals. The customer can drill down the number of bidders and enter negotiations on
pricing and other details. The final stages will consist of that the remaining bidders will
submit a best and final offer and the customer can award the contract to their choice of
bidder (Blake G. and Bly R.W., 1993).
The RfPs do have advantages and disadvantages. If the organization is sure and knows
already specifically what it wants and decides to use the RfP, the RfP can require extensive
descriptions of the services needed. On the other hand, RfPs can also be written poor. In
both cases leading to possible unexpected effects further down in the process.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
17
Some variants have been developed and have a different process.
RFQ (request for quotation) is especially used when discussion with bidders is not required
and the customers is mainly interested in price. A RFQ can also be used prior to a RfP, in
these cases it is used to determine price ranges and in order to write a more fully RFP
(Phillips, J. 2010).
RFI (request for information) is used when a customer requests from a potential seller or a
service provider to determine what products and services are potentially available in the
marketplace to meet the buyer’s needs and to know the capability of the seller in terms of
offerings. The RFI is mainly used on major procurements. It is not an invitation to bid, and
may or may not lead to a RFP or RFQ (Hanna, A., Tadt, E., and Whited, G., 2012).
The BVP method is winning ground over the world including The Netherlands. Based on the
study performed by Kashiwagi, D. and Egbu C. (2011), it is showed that The Netherlands is
the country where BVP is applied on the largest scale behind the USA. Differences between
BVP and the traditional RFP procurement method have been explored and stated here
below.
BVP is less time consuming for the procurement specialists as it requires no full out written
proposal. Parties that do not want to bid or lose in the bid do also have not a significant
investment in relation to RFP or other tender methods, suggesting also less time consuming
for the suppliers. The BVP also allows the organization to make use of the expertise of the
bidders by not dictating the requirements in a RFP (Kashiwagi, D. and Egbu C., 2011). The
creativity and innovativeness of the bidders can be used. BVP also allows the bidders (the
selected bidder) to have a larger accountability as the bidder has filled in his own
assignment, the bidder is the one who has to think over the risks and cannot hide behind
“the client wanted this way, it is not my fault” (Kashiwagi, D. and Egbu C., 2011). Another
important aspect, which is probably overlooked is that procurement specialist within the
organization has to change in his role. He does not decide anymore what the company needs
or wants, but becomes a process manager. BVP also seems to fit in within the modern supply
chain management. Transparency, creating durably relationships with suppliers are more fit
within the BVP method (Kashiwagi, D. and Egbu C. 2011). Other aspects showing the RFP is
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
18
more attractive is that the customer can dictate exactly what it wants based on the literature
described above. A shift in risks does not mean it is positive for the organization. The
organization must accept to transfer the risks to an outside party and the question is
whether the bidder really wants the responsibility. One can only state that the use of BVP
also needs a mind change between the stakeholders involved (Kashiwagi, D. and Kashiwagi,
T., 2015).
The success of BVP of a project can be of course specified in detail to which factors a positive
effect will have or a negative. Considering this research focuses on the procurement process
the factors that will affect the positive assessment of the procurement specialist is the time
invested (being less) and having the perception the costs were lower than through a
traditional method. In addition, the procurement specialist would like to receive the
feedback that delivered services were a success in order to have a total positive assessment
on the process. The expectations of the customer would be in line with the procurement
officer, however, maybe not be aware of what procurement method was used.
2.4 ICT projects and other projects
The IT industry has struggled with performance since its origins. NATO held in 1968 a
conference due to the number of failing ICT projects. However studies since then still
showed that project failure was high (Grossman, 2003). Even in the Netherlands a
parliamentary inquiry was held, it was reported that 1-5 billion Euros was wasted with ICT
project annually (Kashiwagi, D. and Kashiwagi, T., 2015).
Several studies have researched by what key reasons are why projects fail. Kashiwagi, D. and
Kashiwagi, T., (2015) summarize these studies and the top reasons, which include:

lack of top management / executive commitment and support;

incomplete user requirements;

misunderstanding between parties;

lack of client involvement;

poor planning and project management;

inadequate resources and skills;
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
19

project complexity;

lack of quality control;

making an unrealistic tender or agreement;

lack of understanding of the needs of the customer.
This list is not complete as I have left out several other reasons. Kashiwagi, D. and Kashiwagi,
T., (2015) argue that ICT projects are complex and expertise is not fully utilized by parties. As
with any other project a certain level of risk is present. However, Kendrick (2009) states
while risk can be found in any project, being it an ICT project or a building project due to
always uncertainty of the occurrence of risks in a project. An IT project is more risky to
manage than other real life projects because of its unique aspects, objectives and changing
environment of the project (Kendrick, R., 2009).
Gichoya D. (2005) argues that ICT projects are complex and management often do not have
the requisite skills to make decisions for such projects as they lie on rational decision
making. This prevents them from using the expertise of the suppliers. Gichoya goes a step
further and suggest decision makers for such projects should be reskilled. Foselius P. (2005)
states that ICT projects should not be run as a single project which is appropriate for other
projects, but divide the project into smaller parts. In practice, the subdividing starts always
too late in ICT projects, after the problems already have occurred. ICT projects are often too
unique within organisations as every organisation is already unique in its own way and the
processes within that organisation. This difference with other projects as in construction for
example suggest that ICT projects also need more flexibility. Construction projects have well
and better established process, while this cannot be an objective for ICT projects. ICT
projects have a certain immaterial aspect to their projects requiring a more flexible approach
(Royal Academy of Engineering workgroup, 2004).
BVP in The Netherlands has been mainly applied in the construction industry (Kashiwagi, D.
and Egbu C., 2011). The application in the construction industry is due to the fact that the
industry has all the features of a low-bid arena with specified qualifications, standard-based
items and the visible inspection of the project (Kashiwagi, D. and Egbu C., 2011). The
construction industry also has elements making it easier to identify the project as a whole
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
20
singe projects, while ICT project the advice is mostly that it should not be run as a single
project. The research suggests based on their results that the procurement method to be
used needs to be more flexible. This allows the client more flexibility to tailor the
requirements, requests and the needs of the project to be successful which is especially
essential for ICT projects. BVP allows the bidders to use their expertise to their fullest by not
limiting their scoping.
Based on the analysis of the studies performed in the past, BVP seems the procurement
method to be available to increase the success rate of the ICT project which have been
characterized as a failing industry. ICT projects seem to be more complex as they are quickly
unique and the unavailability of sufficient expertise from the client to be able to identify
timely all risks and set out the project in detail. Suppliers however often also operate within
the boundaries set out by the client. This is quicker the case in the RfP method than the BVP
method, with the latter one allowing the supplier to provide their expertise and views to a
fuller extent and also increases their accountability (Kashiwagi, D. and Egbu C., 2011).
Due to the higher average of the complexity of ICT projects BVP can assist the clients and the
success rate in the projects as it:

Transfers the risk and responsibility to the supplier, who in turn will aim to utilize
their expertise and experiences;

Allows the contract to become a performance based contract instead of a specified
contract using standards;

Minimizes the need of the client to have technical expertise;

It can increase the value and performance of the service to be performed by the
supplier and how it is perceived by the client in terms of cost, time and quality;

Forces the suppliers to differentiate themselves from each other by utilizing their
expertise.
These factors seem to be relevant factors to increase the success rate of ICT projects taking
into account the main reasons why ICT projects fail as stated above.
Recapitulation of differences ICT projects and other projects
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
21
ICT projects
Other projects
High risk and complex
Low risk and complex
Changing environment
Stable environment
Lack of expertise within organisation
Expertise within the organisation
Several unique aspects (customisation)
Lesser unique aspects
Need for flexibility
Less need for flexibility
Immaterial aspect present requiring a more Less immaterial aspects present obligating a
flexible approach
flexible approach
Based on this assessment and the research question (including the sub questions)
expectations in this study are that BVP method allows that in the relationship between the
client and the supplier, the latter one will get a larger role in designing the project compared
to the traditional method (this also should reflect in a lesser time consuming . This should be
less time consuming for the customer and if the projects is a success, this increases the
perceived performance by the client. The client however should be able and allow the loose
of control in the project to a certain limit to the supplier. Expectations are that this is met
with a certain scepticism for the first project. Lastly, based on experience the client should
note a different attitude from the supplier who will have to contribute more to the project,
including project management, expertise, ideas and so on. Based on this factors and if
proven to be successful for the project, the client should have a positive attitude against the
use of the BVP method in relation to the traditional procurement method. The following
propositions are formed:
1. BVP method decreases the involvement of the procurement specialist in the
procurement process in relation to the traditional method as he will experience less
time is consumed for his part in the process.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
22
From most studies mentioned above with the reasoning of why IT projects fail is that
organisations seem to lack the kind of experience needed for these projects. Traditional
procurement methods leave the client specify every requirement for the solution they
created. The supplier does only have to deliver what is requested and is not required to take
responsibility on the success of the project and has no need to show its expertise. The
procurement officials within the organisation of the client is involved in making in the RFP
procurement methods, and is dependent on the business whether the requirements
specified will indeed lead to a successful project. The BVP on the other hand allows the
business not to think of the solution, but the supplier. The client only needs to state the
goals it wants to achieve with the project. Hence, the time spent on writing out the RFP with
all its requirements is decreased as the responsibility of filling out the requirements lies with
the supplier offering the solution. The procurement specialist is then only involved in
selection and managing the process. Especially, if the project is completed within time and
budget (which the procurement specialist is also involved in) it should lead to a higher
customer satisfaction.
2. The use of the BVP method will lead to a better utilization of the suppliers’ expertise
by the procurement specialist in making his decision compared to the traditional
method.
As stated above as the supplier becomes more responsible for the project, by offering the
solution, the use of the expertise role of the supplier becomes more evident.
3. The BVP method will lead to a higher involvement of the supplier in the process.
From the role of the procurement specialist, expectations are that due to a lower
involvement on the technical side and the managing of difficulties arousing during the
project, the time spent should be lower for the procurement specialist. Also, by not having
to be involved in writing the out the RFP and coming up with the solution within the
organization, the cost through the use of human resources should be lower. Especially if the
service delivered by the supplier is met with good quality, the procurement specialist should
view the BVP method as a more successful one.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
23
Conceptual framework
In this section will give an overview of the conceptual framework of this research. It shows
the link between the factors that affecting the performance of outsourced IT. This is shown
in a conceptual model as adapted for this research in figure 1.
Figure 1: Conceptual framework
+
Involvement
supplier
+
+
Use of suppliers’
expertise
+
-
Involvement
procurement
specialist
BVP
Project succes
+
Customer
satisfaction
As stated in the research question, the performance is assessed through the assessment
from the procurement specialists involved in the projects. The expectations are that the use
of BVP allows the client to make better use of the supplier’s expertise, the involvement of
the suppliers in the project will increase and this will lead to a success of the project. On the
other hand, I expect that the role of the procurement specialist to change and spend less
time throughout the process but rather manage the process. Time is saved on activities such
as writing out the requirements, specifications, the rfp, but is involved in meetings during
the supplier and the business. This decrease in time will and the higher involvement will
probably also influence the transaction costs involved within the client, to be lower. Through
the use of in-depth interviews I expect to get an overview of the variables involved in the
assessment of the BVP method to be a possible more successful one than the traditional
methods. Customer satisfaction will mainly motivated whether the time consumed in the
BVP process in relation to other procurement methods is lower (in the role as a procurement
specialist). Customer satisfaction is further increased if the customer believes the costs are
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
24
lower in relation to other methods (less time could be regarded as lower costs). Based on
the customer satisfaction the expectation is that the variable performance is positive if the
customer satisfaction is assessed to be positive.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
25
3. METHODOLOGICAL DESIGN
This chapter begins with a description of the research methodology used for the study. Then
there is an explanation and justification of the chosen research method which provides an
overview of the research approach. Finally the collection of the data is described in the last
paragraph, and what resources are used for this purpose.
3.1 Research methodology description
The nature of the research and overall design consists of a descriptive qualitative research.
The aim of this qualitative study is to understand how Best Value Procurement practices are
implemented in outsourced ICT projects and what the impact of BVP is on the performance
of ICT project. Qualitative research provides an opportunity to collect data that reflect a
thorough understanding of the context, processes, relationships and events (Yin 2009).
Furthermore, qualitative research gives the opportunity to find out the motives and opinions
of the participants, it goes deeper into the research and focuses on the depth and detail. The
goal is to gather and analyse a lot of relevant data in a relatively short time (Eisenhardt
1989). The research approach consists of a literature review (Chapter 2) and interviews with
the research group (Chapter 4). The answers from the interviews are based on practice and
the theories from the literature review will give an answer to the research question.
3.2 Data-collection
To answer the research question there is chosen to take interviews with the research groups
next to the literature review. The answers of the research group are based on practice and
give in composition with the theory from the literature study, the answer to the research
question. According to Baarda, Goede en Teunissen (2001) interviews gives the opportunity
to collect information about different themes that are made transparent. The interviews are
based on open questions. The most important themes are about the applicability of BVP,
about outsourced ICT projects and about the impact of performance.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
26
The interviews were semi-structured. This means that the order of topics and questions are
not necessarily fixed and there are opportunities to ask more and deeper questions. A semistructured questionnaire can also be used as a guide during the interviews to ensure that all
topics are discussed (Bryman and Bell, 2011). Furthermore, the choice of semi-structured
interviews is appropriate because there is a structure to the interviews while the interviewee
still has his/her freedom to convey his message.
For the interviews there is a prepared topic list. The topic list is a list of topics and possible
subtopics that are discussed during the interviews. Each topic consists of some questions
that are asked during the interview. These topics are determined for each research group as
outlined below. Based on the topic list and literature study some themes are developed.
Then the data is analysed based on these themes. This method of interviewing is appropriate
for this research, because in the scientific literature there is not much known about the use
of BVP in outsourced IT projects. Through interviews and the use of depth questions, it is
possible to gain more information in an unknown field or when some subjects or answers
are unclear.
Research groups
The research group of participants are diversified. The participants consist of three groups
which ensures an equal distribution. According to Verschuren and Doorewaard (2000), there
are three ways in which people can act as a source of information. The first one are
respondent: "someone provide information about themselves". Second one are informants:
"provides data on someone else or by his or her familiar situations, objects and processes".
And the third one are experts: "A person acts as a supplier of knowledge’’. To answer the
interview questions there will take place face-to-face interviews with respondents
(customers), with informants (procurement professionals) and with some experts (suppliers)
in the field of BVP.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
27
1) Procurement professionals
The first target group are procurement professionals who are specialized in outsourcing ITprojects and have experience in the use of Best Value Procurement approach within these
projects. Four a.s.r. procurement professionals were personally contacted to participate for
the interview. The interviewer confirmed the appointment by email and the interviews were
taken at a.s.r. Furthermore, there were two external BVP-consultants who would be
interviewed. These consultants can be seen as procurement professionals, because they
support a number of outsourcing projects for a.s.r. and have a lot of experience in the field
of BVP in the procurement fase. In total, the following five procurement professionals will be
interviewed:

(Procurement professional #1 at a.s.r.)

(Procurement professional #2 at a.s.r.)

(Procurement professional #3 at a.s.r.)

(Procurement professional #4 at a.s.r.)
2) Suppliers
The second target group of participants consists of three suppliers who also have experience
in offering IT- projects based on BVP. The suppliers were selected based on their experience
with BVP and number of IT-projects they have done in the past based on this approach. This
can be also a contracted supplier of a.s.r.

Supplier 1 (Role: Business Consultant)

Supplier 2 (Role: Customer Director and Risk Officer
3) Customers
The customers (opdrachtgever) were the final target group that will be interviewed. Three
customers were selected for the interview. It was also important to include the opinion of
the clients about the end result of their IT projects that have been done based on BVP.

Director ICT

Head of Infrastructure & Organization (I&O)
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
28
The total amount of interviews are 9 (1 interview consisted of 2 interviewees) of
approximately one hour. The interviews were conducted in enclosed spaces in the
organizations of the participants unless the participants wished otherwise. All the data is
confidential and anonymous for privacy reasons. There were no persons or organizations
named in this study if participants did not want this. Each research group required a
different approach to achieve the desired research results. The questions were tailored to
the research groups to obtain a valid and reliable answer.
3.3 Data-analyses
The semi-structured interviews were recorded on a voice recorder. All interviewees agreed
with this request and this enabled me to transcribe what was said during the interview. The
research material from the interviews were transcribed according a verbatim style (literal
word-by-word). After all transcripts were completed, the detailed interviews were compared
with each other. The questions used during the interviews are included in Appendix A.
The analysis started with the search for patterns and relationships in the statements of the
participants. A pattern is an example or answer that appears several times by different
participants (repeating answers), but a pattern may also be whether there are associations
between pulling statements. Then, all the relevant findings from the transcripts that could
be related with each other were labeled, so that they could be laid next to each other for
comparison.
After this, the results could be written into themes that were previously prepared based on
the theoretical framework and the topic list. The notes with potentially interesting findings
of the research will also be kept during the data collection. The findings in the transcripts
were labeled on relevance for the investigation. Each drawn theme that was discussed in the
transcripts was marked with a code. A code consists of a color and a number. For example,
all data defining the use of BVP in the pre-procurement stage marked with the code 'yellow
and number 2. All prepared codes were compared with each other and were described in
the results section.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
29
3.4 Strengths and limitations of the research design
There is a possible bias with interviews. The greatest danger in personal interviews is that
the respondents feel intimidated or uncomfortable during the interview. If the researcher
does not handle this, the risk exists that respondents only give answers that the interviewer
wants to hear, instead of answering the questions truthfully. This risk will be limited in this
study, because the interview is informative in nature, in which the interviewer will express
no opinion on the answers of the respondent.
The strength is that the research group of participants are diversified. This give insights from
multiple perspectives within this research. Furthermore, the relevant information from the
interviews were analysed and compared with the literature to answer the research
questions. The participants will get an overview of their answers after the data-analyse in
order to validate their results. This allows the participants to determine whether the results
are placed in the right perspective in order to reduce the risk of bias (Yin, 1981).
Other elements affecting the results include the fact that this research is limited to one
company. It is therefore not possible to generalise the findings of this company to other
companies. Considering the method used for this research consists of interviews, the
reliability of the results depend on the interviewees and is therefore a topic of discussion.
See also the first paragraph the reasoning in order to try to obtain a certain level of reliability
from the interviews. The study is confined to one organisation (a.s.r.) and to a few persons
involved in the projects which were characterised by success. This probably will affect the
overall view of the interviewees in their answers to be generally positive. Questions were
asked to gain insights in possible negative experiences and concrete examples of positive
experiences in the BVP process in relation to other procurement methods in the past to
validity of the answers.
Finally, the research depends on the willingness of the participants to participate in this
study. By making known what is the purpose is of this study and to make the participants
aware of their understanding and knowledge in their situation, this will anticipate more
cooperation.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
30
4. RESULTS
One general finding of this research is that projects are difficult to compare and are indeed
quickly unique as identified in the literature review. While a.s.r. has conducted two projects
through the BVP method, the actors involved in both projects seem to differ on opinion
between the two projects. Some background information shared by the interviewees’ shows
that the first project was relatively smaller to the second one. The smaller project seemed to
go fluent and was finished within time and budget. The procurement specialists are very
positive of the method used, but seems to be less enthusiastic with respect to the 2nd
project. However, the method is not criticised, but through more in-depth explanations it
shows that the BVP method is new and for both the customer and supplier a method to still
get accustomed to. This leads to some minor hiccups during the process. Before outlining
the results, the two projects are compared to each other, followed by the findings about the
conceptual model and its meaning. Finally, this chapter ends with a discussion about the BVP
method.
4.1 The projects compared
A.s.r. has outsourced two projects through the BVP method. The first project was the first
time a.s.r. adapted the use of the BVP method. This project is less complex, smaller, and has
a lower budget compared to the second project. Before using the BVP method a.s.r. decided
to hire a consultant who managed to assist a.s.r. in the process of applying the BVP method
in the right way. The BVP method is relatively new and especially in the ICT industry it has
not or very limited used. This assistance seemed a great help for a.s.r. to have a better grip
on the understanding of the procurement method. This was especially helpful in the first
project as the number of people involved was also less than the second project.
Procurement specialists also stated that due to the number of people involved in the second
project minor hiccups are seen, as well within a.s.r. as with the supplier. The tendency within
a.s.r. is still that employee’s act from the viewpoint they have to come up with the solution
instead of letting the supplier come with the solution. Project size seems to play a role in the
perceived satisfaction for the procurement professionals.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
31
4.2 Analysis of the results
Based on the results of the analysis I will provide below the relation with the conceptual
framework and the hypotheses as described in chapter 2.
Hypothesis 1: BVP method decreases the involvement of the procurement specialist in the
procurement process in relation to the traditional method as he will experience less time is
consumed for his part in the process.
The results confirm this hypothesis. The hypothesis is based on that the BVP method will
affect the time that the procurement specialist consumes in the process. These hypotheses
is based on the fact that no extensive RFP’s have to be written, the client has not to think of
the solution, negotiations should be decreased. The results do support this hypotheses.
As argued: “less time, will not differ much, but the feeling is present as the supplier is more
involved and does more, costs are lower as efforts are lower” (Interview 3).
It seems as expected that the supplier is more involved and this somehow creates that the
procurement specialist experiences less time spent on the project. Another interview shows
less time is spent as there is less to evaluate due to the absence of RFP’s and other aspects
related to traditional procurement method.
“Purely from procurement, we spend less time that is because we have less to evaluate”
(interview 4).
This is also confirmed in the interview with the other procurement specialists.
“Time is short than with traditional procurement method, money spent is also less as time is
less, this increases satisfaction” (interview 5).
The BVP method does indeed lessen the time for the involvement of the procurement
specialists as they require to be less involved. Negotiations are not part of the BVP method
and this is also confirmed as a positive factor which leads to less time.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
32
“You do not have negotiations with BVP, with rfp you negotiate to the last dime” (interview
7).
These negotiations are experienced as time consuming and customers experience the
absence of negotiations as positive. As noted, the 2nd project performed by a.s.r. is a bit
larger and has some hiccups. A procurement specialists experiences the feeling that the
client actually wants to negotiate which does not exist within BVP as negative:
“it looks like negotiating which does not exist within BVP” (interview 5).
As confirmed by the interviews the procurement specialist experiences that less time is
spent in the process from their sides. It seems that the absence of negotiations is seen as a
positive factor of BVP.
1. Hypothesis 2: The use of the BVP method will lead to a better utilization of the
suppliers’ expertise by the procurement specialist in making his decision compared to
the traditional method.
The hypothesis is formed based on the fact that ICT projects in the IT industry are
characterized to be failures and one of the reasons is that the lack of the expertise causes
some of these failures. The traditional procurement method involved that the customers
developed the solution for their problem and based on their solution they would search for
the supplier who was able to implement the solution. The BVP method however, shifts the
solution developer aspect to the supplier with the reasoning that the supplier has the
expertise based on experience and practice. Based on the interviews I note that this
hypothesis is indeed supported.
“Expertise of the supplier, that is a powerful element in this method” (Interview 7)
The interviewer shows that the expertise one important factor noted in the BVP method and
even argues why a large company as a.s.r. has to hire a BVP consultant, these type of firms
should have 1 in-house. The customer interviewed views quality as an important element in
service. The other customer interviewed also shares his view on this matter and suggest that
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
33
how more you are able to let the supplier do his things, how more the supplier will show his
expertise:
“how more responsibility you place on the supplier, the more expertise he will use” (interview
3).
This should be a logic consequence coming with more responsibility. From the point of the
procurement specialists, the applicability of BVP increases with complexity of projects. Due
to complexity the lack of experience becomes more visible and the more the requirement of
the expertise of the supplier is needed.
“the complexity is high and the expertise is not present within the organization but also
becomes less important as you have to lie more on the supplier for the expertise” (interview
4).
“Best value is intended for complex projects, this is because you can use the expertise of the
supplier, but the client should be able to give the supplier the room to demonstrate this and
this is really difficult for the client, there is a trade-off somewhere, but I don’t know where”
(interview 8).
The other procurement specialist notes that through the use of the supplier who comes up
with the solution the control shifts more to the supplier and this requires trust from the
customers’ side.
“Trust is important as you have less expertise, you have to trust the supplier and you can
verify it until the third phase”. (Interview 2).
In order to be able to let responsibilities go, trust has to be present. This could be an
interesting topic for future research in the use of BVP vs other methods.
All in all, the hypothesis seems to be supported and be in line as noted in chapter 2. The
supplier is able to fill in his expertise in the BVP method and this can be used by the
customer, including the procurement specialist.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
34
Hypothesis 3: The BVP method will lead to a higher involvement of the supplier in the
process.
Moving on to the final hypothesis, the BVP method allows the supplier to have a higher
involvement. The higher involvement comes with the responsibility of coming up with the
solution and implementing the solution. The supplier is responsible for the success of the
project based on the goals of the customer it wants to achieve.
The hypothesis is supported, as it is consequence of the 2nd hypothesis. Based on the
interviews, the customers experience this as a positive element.
“People involved have the feeling they are behind the wheel, which is an advantage, ….., the
supplier is in the lead” (interview 5).
However, the procurement specialist does also notes that not everybody is accustomed to
this. BVP is relatively new and this change requires a mind shift, but this will come with time
as everybody gets acquainted to the procurement method and old habits die down. It is
difficult to use the expertise of the supplier and have the client at the same time lose some
control over the project. This interesting point of view is shared by another procurement
specialist:
“Best value is intended for complex projects, this is because you can use the expertise of the
supplier, but the client should be able to give the supplier the room to demonstrate this and
this is really difficult for the client, there is a trade-off somewhere, but I don’t know where”
(interview 8).
Risk does not seem to play a role as the customer sees a split up of the risks. One could
expect that the supplier is now more the risk-carrier of the project as he has come up with
the solution and has to implement, but this is not fully confirmed in my interviews. However,
this factor is an important topic for further research.
Another viewpoint on this matter is from the suppliers. The suppliers interview do share that
the supplier is more responsible and will show more involvement, but this is necessary to
fulfil the expert role.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
35
“BVP does not transfer all risks to the supplier, some risks still are the customers’
responsibility. I do not think that the procurement specialist or the supplier have a bigger
role, this is due to the split-up in responsibilities and risks”. (Interview 6).
The supplier works out his solution from the goals and scoping determined by the customer
and does not necessarily see a larger involvement and responsibility other than the tasks
involved. However, suppliers can always point out risks that the client has missed, this is the
expertise of the supplier and his way to show the value-added of his relationship and
expertise with the customer. If the client is positive on these matters, the supplier is then
able to take the responsibility.
The other supplier interviewed does not share the view that the supplier is more responsible
for risks involved. The supplier shares the view that supplier has more responsibility, but not
more risks.
“You have more responsibility, not risks. That is a difference. This enables the supplier to
control risks better, because in the end, he is the expert”. (Interview 1).
The hypotheses are confirmed by the analysis of the results. However further analysis
shows, there are more interesting findings. This is mainly the result as the procurement
method is only used twice by a.s.r. and therefore still unfamiliar in practice how to handle
issues. The first project is characterised as a success, stated by parties, that it was delivered
in time and within budget. Regarding the second project which was a bit larger and more
complex the results are not always confirmed. The procurement specialist note that with the
2nd project they have seen that the supplier is not always acting as it should on the BVP way.
This is also noted within a.s.r. itself, the procurement specialist still has to inform his
colleagues how the method works and what is expected from them and the supplier. The
results stated above could change over time as BVP becomes more known in practice.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
36
Other findings
The BVP is a relatively new procurement method. A study performed by the BVP consultant
hired by a.s.r. shows that BVP is not the standard procurement method for IT projects,
however considering it is a relatively new procurement method this is no surprise. The
figures presented by the BVP advisor also show that the majority of the IT projects fail, the
reasons stated are in line with my reasons in chapter 2 which was based on the literature
review. The research stated a ‘mentality change’ is needed to overcome the majority of the
failed projects. While I did not have the permission the permission to fully disclose all
results, an agreement with the BVP advisor was reached to utilize some screenshots of the
study. However I note, the results of this study are to my knowledge unknown whether they
are conducted on such a way to be included in a scientific paper, but however, they do share
some insights.
Expected ICT expenditures in the Netherlands
A total over 34 billion euros was expected to be invested in 2014 on IT projects. This is an
increase of 2,1% compared to 2013.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
37
Performance of ICT projects
The amount of failed projects (both in numbers and euro amounts) only strengthen the
argument a change in procurement method is needed. Below are stated the arguments why
IT projects fail. Projects are either complex, too specified, no room for the market (supplier)
to provide his expertise. These arguments are all mentioned in my research on why BVP
could be the solution.
Matters related to traditional procurement methods
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
38
Why IT projects fail
Traditional procurement methods are not the way to go based on the research. The focus of
these procurement methods are price and suppliers are mostly not able to deliver the
required quality due to the low price. The research also shows that while the expenditures
on IT projects keep increasing, the number of projects on which the BVP method is used also
increases. However, while the number of projects on which BVP is used has increased it still
accounts
according
to
his
figures
only
4%
of
the
total
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
projects.
39
Amount of IT projects outsourced through BVP as a percentage of total outsourced projects
The BVP advisor noted some interesting point I also recognize in the interviews. He notes
that the BVP method is relatively new and the way the process is designed it requires that
organisations have to go through a ‘mind shift’ allowing more of the control of the project to
go to the expert (the supplier). This comment was confirmed in my interviews, clients have
difficulty to let control loose and this can disrupt the process as some procurement
specialists noted. The BVP advisor also commented that both the supplier and the customer
have issues on this matter. This comment was also noted in the interviews with the
procurement specialists, they shared the view that as well as employees present within a.s.r.
as the suppliers do not have full knowledge of the BVP method and tend to fall in ‘old
habits’. This issue would be resolved if a.s.r. would conduct more projects with the BVP
method to allow the method to become more a standard.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
40
5. DISCUSSION & CONCLUSION
This chapter finalizes this thesis starting with a discussion of the results and the implications.
Limitations of the research are summed up and finally suggestions for future research are
shared based on this research.
5.1 Main findings
Based on the results in chapter 4 I will summarize the main findings in relation to the
research question, the propositions and the conceptual model.
The research question is aimed to analyse the performance of the use of the BVP method of
outsourced IT projects. As noted before in chapter 1, BVP has been studied mainly in sectors
outside the IT projects. This research aims to provide some insights into whether BVP can be
applied in the IT sector which is plagued by failed projects for numerous reasons. Within
a.s.r. 2 projects went through the BVP method. Through interviewing several parties
involved insights were obtained to whether BVP is also applicable for IT projects in the
Netherlands and what effects were noted. This research provides the insights to build
further on for future research. Based on the interviews, the results show that the BVP
method is also effective for ICT projects. It is an additional procurement method that can be
used for certain projects. However, measuring performance is difficult, but this research
does indicate there are positive elements favoured within this procurement method in
relation to the traditional procurement methods. For instance, the first project was
applauded as it was delivered by the supplier within time and budget. These factors are
mainly named when performance is discussed. Other factors included were that the client
was able to make use of the expertise of the supplier to come up with the solution rather
than have the client develop their own solution. This element is consistent with the model
prescribed by Kashiwagi and Byfield (2002). Based on the literature and the propositions
which were derived from the literature review, the results were positive that the client was
able to make better use of the expertise of the supplier and that the supplier’s involvement
was higher. The clients also experienced a higher degree of responsibility shifted to the
supplier, this is in line with what is noted by Kashiwagi and Kashiwagi (2015). However,
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
41
whether the BVP method decreases the involvement of the procurement specialists are
somewhat ambiguous. Not all procurement specialists agree on this and have difficulty to
quantify this matter. They note however that the role of the procurement specialist does
shift in the BVP method as they become more a process manager. The difficulty to quantify
and the hiccups experienced in applying the BVP method are the result that BVP is a new
procurement method. These hiccups will disappear in time as more parties will become
familiar with the procurement method.
The findings also confirm the conceptual framework which is based on the hypotheses.
However I note it is difficult to have a quantification of the variable performance, but the
positive wordings in the interviews do indicate that it is clearly a positive experience. If we
move further to whether the BVP method is the way to go for ICT projects, I note some side
notes. The use of BVP as a procurement method is not a prescription for success. The
procurement specialist indicates that the method was successful for the first project, stating
mainly the factors, the use of the suppliers’ expertise and finishing within budget and time.
However, as for the second project the opinions are, whilst positive, not as enthusiastic as
the first project. One important factor seeming to affect this is that the second project is
perceived as being more complex and bigger than the first project. Complexity is noted by
several researchers including Dimitri, Piga and Spagnola (2006) and Kashiwagi (2014) as a
reason why IT projects tend to fail. Kashiwagi offers BVP as a solution, but BVP does not
handle complexity of a project easily, however it offers the expertise of the supplier to deal
better with the complexity. Another reason stated was that the mind shift of not dictating
the solution, but rather to think in goals to be achieved by the client and have the supplier
come up with the solution was hard to let go, especially within the business from the client
points of view. The unfamiliarity regarding the use of BVP was also a reason why this process
seemed sometimes not to go through the concept of BVP as it should. This unfamiliarity
should disappear with the number of the projects performed through the BVP method. This
mind shift is an interesting field for further research to see it effects on the use of new
procurement methods. To my knowledge no research is performed on this matter and this
study shows that ‘people’ do have difficulty to adapt to new things with no resistance.
Another interesting point of view is that the BVP method should not be used as the sole
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
42
procurement method for outsourcing ICT projects. Complexity seemed to play an important
role as procurement specialists indicated that the BVP method was an additional
procurement tool that can be used under some circumstances. It is not necessary to use the
BVP method when purchases are standard, non-complex, rather it should be used for
projects where the lack of expertise within the organisation is evident and the use of the
expertise of the supplier is needed for the ‘best’ solution. This findings confirm my literature
review in chapter 2. The use of the expert’s role from the supplier is in their eyes more
evident than in the traditional procurement methods. Suppliers have to come up with
solutions and the client views this as a positive element within the BVP method and is
recognised within both projects. This element shows the advantage of the use of BVP in
relation to traditional procurement method and is clearly in line with the view of Kashiwagi
and Egbu (2011). Projects differ from each other, characterised by complexity, magnitude,
the number of people involved and probably more other factors which could make it difficult
to apply the BVP method in its purest form as developed by Kashiwagi. The benefits of the
use of the BVP method do weigh up against the use of the traditional procurement methods
for projects, also in the ICT sector. However, there are more factors in the playing field that
need to be managed, which are probably present if using the traditional procurement
method, but due to unfamiliarity with the BVP method some stakeholders fell prey to handle
things the ‘old way’. This habit will take time to loose and factors such as hiring a BVP
consultant could help make the process easier when the procurement method selected for
the project is BVP.
As argued the BVP method has several advantages over the traditional procurement method
that can help overcome several aspects. It is difficult to generalise results whether BVP is a
more effective method than the traditional methods, as it has not been used to an extent
that it can provide same results in several different settings. It is also the question whether
the performance for the procurement specialist should be researched based on time and
budget. They seem to link positive factors of the project performed through BVP to these
factors but are not able to quantify the differences with the traditional procurement
method, this is also difficult, as it technically means how it would be if both projects would
be performed through both procurement methods. The BVP has its strengths, it works for
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
43
example with a maximal amount of budget on which a supplier is selected. Costs seems to
be more controlled by this aspect which is a goal for of most firms according to Weele
(2010). Another interesting finding is that whether BVP can be used for all projects,
according to the procurement specialists, this differs on factors such as what should be
procured, is it standard, complex or non-complex. Does the investment to use the BVP
method pay off against the project? Some of these factors were included in the table in
chapter 2 to show the differences between IT projects and other projects. It does show that
within IT projects complexity is not always present and therefore BVP should not be used for
every IT projects.
However, it is clear that BVP is an additional procurement method that should be seriously
considered when deciding which procurement method should be selected. The applicability
of BVP to ICT projects has shown to be a success for a.s.r., especially with the first project
(unanimously confirmed by all parties), but the second project experiences some hiccups,
showing that the BVP method is new and when something does not seem to go through the
process, employees fall back in old habits. The use of BVP is still in young feet and it is
interesting how the results should be when a.s.r. has more experience in the application of
the method, but also the suppliers.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
44
5.2 Implications
As outlined in the literature review, the research on BVP method has been studied several
times, however not in the field of ICT projects. The BVP method has shown to be successful
in several other fields, such as construction. Studies on ICT projects has shown that these
projects are mainly characterized as failures. This study tries to offer some insight in whether
the BVP method can be a solution for the IT industry. Results indicate that the BVP method
has some strengths that can overcome some of the reasons of why ICT projects fail. Factors
such as complexity, the lack of expertise on the side of the clients, are as risk for the failure
of the project lowered by utilizing the expertise of suppliers by having them design the
solution rather than design self the solution. The BVP method seems to be an effective
procurement method for these type of projects. Clients also experience through the use of
the BVP method that the role of the supplier is larger as they have to come up with the
solution and there are more often consultations between the client and the supplier. The
role of the procurement specialist also changes as he becomes more a process manager
stated by the procurement specialists. To my knowledge there are no researches present
that have studied the roles of the procurement specialists when using different procurement
methods. The abovementioned effects of the use of BVP in ICT projects show some new
findings, but probably are limited in this research. This research also shows that stakeholders
can have different expectations, as they are not familiar with BVP and tend to fall in old
habits. The BVP method is of course a relatively new procurement method that needs time
to be fully accepted by all stakeholders as they are unfamiliar with it. These findings of
course offer room further research and exploration.
5.3 Practical Implications
Based on the research the BVP method seems to be an effective procurement method for
the ICT industry. As one procurement specialist noted, it is one of the few procurement
methods that can be used based on some factors, such as the complexity of the project and
the need of expertise that is not present within the organisation. BVP seems to be then a
more effective method than the traditional methods. As stated above in the literature
review, ICT projects are characterised by failures for numerous reasons. Reasons include that
ICT projects are complex and in a changing environment and organisations have often a lack
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
45
of expertise to come up with the right solutions. The BVP method offers some solutions for
these reasons, as it allows the supplier (the expert) to provide the solutions. The need for
this element increases as the complexity of the purchase of the service is higher. By having
the solution provided by the expert, the role of the supplier increases by not only delivering
the specifications required, but a success service, the project. The research also shows that
the BVP method is applicable in ICT projects. However, it does not necessarily mean that the
BVP method is applicable for all ICT projects. As the procurement specialists indicate, it is an
additional procurement method, not a replacing method for all other procurement methods.
For the procurement of standard services and goods, the procurement specialists indicate
that the BVP method is not the method to be used. Another interesting point the research
shows is that the BVP method is a new procurement method. While the theory is outlined,
the practice is different. Procurement specialists indicate that they often note things that
people tend to fall in old habits and the behaviour is not in line with what BVP should be
about. An advice could be that both the supplier and the customer make use of a consultant
to help them both with the key essentials of the project to help that it goes the BVP way.
However, as BVP will probably be used more in the future, this problem should be less
present in time and maybe absent at all.
5.4 Limitations and further research
The research performed is a qualitative research. While qualitative research can produce
compelling results, it is not without limitations. The quality of the research depends on the
researcher, such as developing the type of questions and interpreting the results through his
own beliefs which could be affected by bias. It is also hard to repeat the results in other
studies as the results are characterised by the beliefs of the researcher and the
interpretation of the researcher. It is hard to confirm or deny the results, but also an
additional factor in this research is that is performed within one organisation. Organisations
differ and the use of BVP can be easier or more difficult in other organisations, depending on
the type of the organisation, the culture and competence of the organisation which are all
excluded in this research. Another factor is that the use of BVP in this organisation is limited
to two projects, generalisation is extra difficult. However, this study does show some factors
which provide insights into how BVP is experienced that can be understood from a certain
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
46
perspective. Whether this perspective is right or wrong depends on the interpretation for
the reader, but more importantly it does provide food for future research in probably other
settings for further scientific research.
The results of this study show that there are several factors affecting the BVP method in a
positive or negative way, such as complexity of the project, the number of people involved in
the project, the familiarity with BVP, the possible role shifting of the procurement
specialists, but also the measurement of the performance of BVP in relation to other
procurement methods. More scientific research can be conducted on this matter and on a
larger scale in order also to have more generalising results. BVP has proven to be a
procurement method that can be used next to other procurement methods. It has its
advantages that can make it useful for certain procurements, but not all procurements. The
ICT sector seems to be characterized by failures as noted in the literature review. BVP can
offer solutions for several factors causing these failures, but whether it is more used
nowadays is unknown to me. In case if it is used more often, research on the success of the
ICT projects since using the BVP method can shed more light whether this procurement
method does indeed offer the solution. For measuring performance from the point of view
of the procurement specialists an in depth study can be performed on which factors they
relate to when assessing the performance of the procurement method. My research shows
that time and budget play a key role, the delivery of the service by the supplier seems to be
less important as it is the business who should form an opinion on the quality and not the
procurement specialists. All in all, there are several areas that need future research.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
47
6 BIBLIOGRAPHY
Weele, Arjan J. van (2010) Purchasing and Supply Chain Management: Analysis, Strategy, Planning
and Practice (5th ed. ed.).
Dimitri N., Piga G. and G. Spagnolo, (2006) Handbook of procurement
Blake G. and Bly R.W. (1993), The Elements of Technical Writing, pg. 100. New York: Macmillan
Publishers.
Phillips, J. (2010) "Procurement Management in Project Management - Taking Out a Contract".
Hanna, A., Tadt, E., and Whited, G. (2012). ”Request for Information: Benchmarks and Metrics for
Major Highway Projects.” J. Constr. Eng. Manage., 138(12), 1347–1352.
Kashiwagi, D.T. (2011) Case Study: Best Value Procurement/Performance Information Procurement
System Development (p. 13-35)
Kashiwagi, D. and Egbu C. (2011) Best Value Procurement / PIPS in the Netherlands, Journal for the
advancement of Performance Information and value, Vol. 3, No. 1.
Kashiwagi, D.T. (2008) Best Value Procurement, 3rd edition, Performance Based Studies Research
Group, Arizona State University
Kashiwagi, D. and Savicky, J. (2003) The cost of 'best value' construction, Journal of Facilities
Management 2 (3) 285-295
Kashiwagi, D., Sullivan, K., Greenwood, D., Kovell, J. and Egbu, C. (2007) Source of construction
industry instability and performance problems, Construction Research Congress 2005: Broadening
Perspectives.
Weele, A.J. van (2008) Inkoop in strategisch perspectief, 6th edition, Kluwer bv, Alphen aan den Rijn
Andrew Cox Article first published online: 5 APR 2006: Understanding Buyer and Supplier Power: A
Framework for Procurement and Supply Competence.
Douglas D. Gransberg and Michael A. Ellicott. "Best-value contracting criteria" Best-value contracting
criteria 39.6 (1997): 31-34.
Kashiwagi, D. and Kashiwagi, T., (2015) The Best Value ICT Industry, Journal for the Advancement of
Performance Information and Value, Vol. 6, No.1.
Kashiwagi, D. and Byfield, R. (2002). ”Testing of Minimization of Subjectivity in Best Value
Procurement by Using Artificial Intelligence Systems in State of Utah Procurement.” J. Constr. Eng.
Manage., 128(6), 496–502.
Park, J., Ojiako, U., Williams, T., Chipulu, M., and Marshall, A. (2014). "Practical Tool for Assessing
Best Value at the Procurement Stage of Public Building Projects in Korea." Journal of Management in
Engineering, 10.1061/(ASCE)ME.1943-5479.0000337, 06014005.
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
48
Jeong, M. and Lee, G. (2010). "A Study on the Prediction of Referral Intension based on Customer
Satisfaction in Construction Management." Korean Journal of Construction Engineering and
Management, 10.6106/KJCEM.2010.11.6.100, 100-110.
Kashiwagi, D., Parmar, D., and Savicky, J. (2003). "The impact of minimising specifications and
management at the University of Hawaii." Journal of Facilities Management,
10.1108/14725960410808168, 131-141. Online publication date: 1-Apr-2003.
Kashiwagi, D. and Savicky, J. (2003). "The cost of ‘best value’ construction." Journal of Facilities
Management, 10.1108/14725960410808267, 285-297. Online publication date: 1-Jan-2003.
Best-value Procurement Methods for Highway Construction Projects, Nummer 561 Door Sidney Scot
Implementing ‘Best Value’: Local Public Services in Transition // Steve Martin // Article first published
online: 17 DEC 2002
Best Value in Construction // John Kelly, Roy Morledge, Sara J. Wilkinson
Procurement Strategies: A Relationship-based Approach // Derek Walker,Keith Hampson
Geoffrey E. Segal; Adrian T. Moore (2002) Weighing the Watchman: Evaluating the Costs and Benefits
of Outsourcing Correctional Services, Part I: Employing a Best-value Approach to Procurement
LEIBLEIN (2003): The Choice of Organizational Governance Form and Performance: Predictions from
Transaction Cost, Resource-based, and Real Options Theories, Journal of Management, 2003, No. 29,
Vo. 6
Verschuren and Doorewaard. ‘’Het ontwerpen van een onderzoek’’ (2000):p. 118
Baarda, Goede en Teunissen Basisboek Kwalitatief Onderzoek (2001): p. 34-55)
Yin (2009): Introduction to case study research. Chapter 1 & 2
S. J. Tracy Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research (2010)
Eisenhardt, Kathleen M. ‘’Building Theories from Case Study’’ Source: The Academy of Management
Review, Vol. 14, No. 4, (Oct., 1989), pp. 532-550
Why IT Projects Fail Al Neimat, Taimour (2005), Project Perfect White Paper Collection l. (online) 2-5
http://sct.emu.edu.tr/courses/mtit/itec580/userfiles/files/info_it_projects_fail.pdf
Research about the impact of Best Value Procurement on the performance of outsourcing ICT projects | Master Thesis
49