Contract Choice and Product Quality
Outcomes in Outsourcing:
Empirical Evidence from Software Development
Sandra Slaughter
Donald E. Harter
Soon Ang
Jonathan Whitaker
11/01/2005
American University
1
Software Quality

Problem



In April, a software glitch resulted in the loss of
thousands of dollars for US Airways Group Inc. when
some tickets were mistakenly priced at $1.86.
(ComputerWorld 7/25/05)
A software bug apparently caused the largest power
outage in North America, the Northeast blackout of
August 2003, which threw millions of people into
darkness (ComputerWorld 7/25/05)
Flawed software cost the U.S. economy $60 billion in
2002 (NIST 2002)
11/01/2005
American University
2
Research Motivation

Solutions

Process maturity is key to higher quality, lower costs, shorter
development time


Questions remain:


Harter, Krishnan, Slaughter, Management Science 2000
How to encourage higher quality?
Research question


Can contract selection be a vehicle to encourage software
quality?
If so, what factors drive contract selection?
11/01/2005
American University
3
Contract Selection

Contract theory & agency theory


Choice of contract structure is crucial to ensure that
agent’s goals are aligned with principal (Crocker &
Reynolds, 1993; Grossman & Hart, 1983; Milgrom &
Roberts, 1992)
Issues:



Hidden information leads to adverse selection
Information asymmetry leads to moral hazard
Type of contract can serve as an effective governance
mechanism
11/01/2005
American University
4
Information Asymmetry

Sources of information asymmetry

uncertainty in product specifications and
uncertainty about the vendor’s ability to
develop quality products
Artz and Norman 2002
 Stump and Heide 1996
 Kalnins and Mayer 2004


High uncertainty increases costs of writing
specific contract terms
11/01/2005
American University
5
Research Model
Uncertainty of Product Specifications:
Specification
Uncertainty
Design
Complexity
H1, H2
Uncertainty of Vendor Quality:
Prior Contracting
Experience
H3, H4
Contract Choice
Time & Material,
Hybrid, Fixed Price
H5a,
Verification &
H5b Validation Quality
Software
Process Maturity
11/01/2005
American University
6
Product Uncertainty

Issues





Client requirements can be ambiguous (Nidumolu
1995)
Software products are innovations and innovations
embody uncertainties (Ang & Beath 1993)
Software development is frequently exploratory
(MacCormack 2001)
Client understanding is evolutionary (Richmond 1992)
Hypothesis

H1: Time & Materials contracts more likely when
specification uncertainty is high
11/01/2005
American University
7
Product Complexity

Issues





Complex designs are more difficult to develop (Brooks
1995)
Effort required for testing complex designs is highly
variable (Banker 2002)
Higher software complexity increases technical risk
(Barki 1993)
Development cost estimation is more uncertain
Hypothesis

H2: Time & Materials contracts more likely when
design complexity is high
11/01/2005
American University
8
Vendor Uncertainty

Issues




Inability to determine vendor quality can create problems of adverse
selection and moral hazard (Artz & Norman 2002)
Repeated interaction and long-term relationships mitigate adverse
selection and moral hazard (Baker 1994)
Repeated transactions provide incentives that decrease likelihood of
opportunism (MacNeil 1978; Granovetter 1985)
Corts & Singh (2004)




Repeated interactions reduce contracting costs, leading to fixed price
Interaction reduces opportunism, leading to time & material
Variance of these costs affects contract choice (Kalnins & Mayer 2004)
Hypothesis

H3: Hybrid contracts more likely when contracting experience between
vendor and client is low
11/01/2005
American University
9
Vendor Uncertainty

Issues




Adverse selection can be addressed using signals
designed to reveal private information (Milgrom &
Roberts 1992; Mishra 1998)
Qualification process can identify vendors with
necessary skills (Stump & Heide 1996)
Process maturity can be used to signal quality (Arora
& Asundi 1999)
Hypothesis

H4: Fixed price contracts more likely when software
process maturity is high
11/01/2005
American University
10
Effect of Contract Choice
on Quality Outcomes

Issues


Opportunity for ex post opportunism by both parties (Williamson
1979)
Vendor has financial incentive to freeze the specification in fixed
price contract


Clients may change are articulate new requirements as they
discover what they truly need


Incentives are to develop the software right the first time, according
to the specification
Vendor profits from new requirements under Time & Materials, and
may accommodate client’s requirements
Hypotheses


H5a: Fixed price contracts have higher development and production verification
quality
H5b: Time & Materials contracts have higher acceptance validation quality
11/01/2005
American University
11
Research Site &
Data Collection


Data collected on software projects
developed from 1987 to 2004
78 contracts were negotiated



26 time and material
38 fixed price
14 hybrid
11/01/2005
American University
12
Contract Types

Time & Material



Fixed Price




Vendor reimbursed through hourly rate
Technical and financial risks on client
Vendor agrees to fixed contract value
Technical and financial risks on vendor
Costly to negotiate – requires detailed specifications ex ante
Hybrid


Agreement on cost estimate, but client pays all costs; profit
based on initial estimate and performance
Financial risk primarily on client
11/01/2005
American University
13
Measures

Contract choice

Categorical variables


Quality



Verification (development & production) – technical issues of whether
the software has been developed correctly and performs correctly
Validation (acceptance) – whether the right software has been
developed that satisfies the users
Antecedents of contract choice





1-T&M, 2-hybrid, 3-Fixed price
Specification uncertainty
Design complexity
Prior contracting experience
Software process maturity
Controls

Product size
11/01/2005
American University
14
Regression Models: Choice

Stage 1: Contract Choice



multinomial regression using Newton-Raphson maximum
likelihood estimation
Prob(yi=j) = e jXi / Σe kXi
Corrections

Non-independence of disturbances across different contract
segments


Huber (1967)/White (1982) sandwich estimator
Results



Antecedents significant in predicting choice (p<.001)
Explain significant variance (pseudo R2 = 0.751)
Correlation between predicted and actual contract is 88.5%
11/01/2005
American University
15
Uncertainty & Complexity
Likelihood of Contract Choice
Given Levels of Design Complexity
1
1
0.9
0.9
0.8
0.8
0.7
0.6
low
high
0.5
0.4
0.3
likelihood of choosing
likelihood of choosing
Likelihood of Contract Choice
Given Levels of Specification Uncertainty
0.7
0.6
low
high
0.5
0.4
0.3
0.2
0.2
0.1
0.1
0
0
Fixed Price
Hybrid
Time & Materials
Fixed Price
11/01/2005
Hybrid
Time & Materials
contract choice
contract choice
American University
16
Hypotheses Summary

H1: Time & Materials preferred over fixed price
for higher levels of specification uncertainty



66% likelihood for high specification uncertainty
10% likelihood for low specification uncertainty
H2: Time & Materials preferred over fixed price
and hybrid when there is higher design
complexity


71% likelihood for high design complexity
8% likelihood for low design complexity
11/01/2005
American University
17
Experience & Process
Likelihood of Contract Choice
Given Levels of Software Process Maturity
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
low
high
0.5
0.4
0.3
likelihood of choosing
likelihood of choosing
Likelihood of Contract Choice
Given Levels of Prior Contracting Experience
0.6
0.4
0.3
0.2
0.2
0.1
0.1
0
low
high
0.5
0
Fixed Price
Hybrid
Time & Materials
Fixed Price
contract choice
11/01/2005
Hybrid
Time & Materials
contract choice
American University
18
Hypotheses Summary

H3: hybrid contracts preferred over fixed
price when prior contracting experience is
lower



84% likelihood of hybrid for low experience
88% likelihood of fixed price for high
experience
H4: fixed price preferred for higher levels
of process maturity
11/01/2005
American University
19
Regression Models: Quality

Stage 2: Quality Outcomes of Contract Choice



Models




Multivariate general linear modeling (GLM)
Two-step multinomial selection bias correction method of Lee (1983)
Development Verification Quality = f(Contract-Choice, Specification-Uncertainty,
Design-Complexity, Prior-Contracting-Experience, Software-Process-Maturity,
Product-Size)
Production Verification Quality = f(Contract-Choice, Specification-Uncertainty,
Design-Complexity, Prior-Contracting-Experience, Software-Process-Maturity,
Product-Size)
Acceptance Validation Quality = f(Contract-Choice, Specification-Uncertainty,
Design-Complexity, Prior-Contracting-Experience, Software-Process-Maturity,
Product-Size)
Results


Hotelling’s T2 test of contract choice significant (p<.001)
Post hoc calculation of power is 0.97
11/01/2005
American University
20
Quality Outcomes
2.25
2
Errors per KLOC
1.75
1.5
Devel-Verif
Prod-Verif
Accept-Val
1.25
1
0.75
0.5
0.25
0
Time & Materials
Hybrid
Fixed Price
Contract Type
11/01/2005
American University
21
Discussion


Information asymmetry arising from
product uncertainties (specification
uncertainty and design complexity) shifts
contract choice to Time & Material
Uncertainty of vendor quality is a strong
motivator of contract choice

Vendor quality (30.2%) explains eight times
the variance of product uncertainty (3.7%)
11/01/2005
American University
22
Discussion

Prior contracting experience is a critical mitigator
of information asymmetry

Hybrid contracts more likely when experience
between client and vendor is low


Reducing contracting and shirking costs
Vendor quality certification explains highest
variance in contract choice (20%)

Quality certification engenders greater confidence in
the vendor’s abilities to estimate and deliver software
products to specifications
11/01/2005
American University
23
Thank You!
Questions?
11/01/2005
American University
24