Value-Based Software Engineering II:
Theory, Process, and Case Study
LiGuo Huang
Computer Science and Engineering
Southern Methodist University
1
Outline
• VBSE Theory Motivation and Context
• “Theory” Definitions and Criteria
• VBSE Process and Case Study
– Elements and Contributions
– Theory-driven process
– Supply chain case study
• Conclusions and Future Research
2
“Theory” Definitions
• 1960 definition: System of general laws
– Spatially and temporally unrestricted; nonaccidental
• 1994 definition: System for explaining a set of
phenomena
– Specifies key concepts, laws relating concepts
– Not spatially and temporally unrestricted
– Better for people-intensive activities
• A system for explaining a set of phenomena that
specifies the key concepts that are operative in the
phenomena and the laws that relate the concepts
to each other
3
Criteria for a Good Theory
• Utility: Addresses critical success factors vs. trivial
• Generality: Covers a wide range of situations and
concerns
– Procedural, technical, economic, human
• Practicality: Helps address user’s needs
– Prediction, diagnosis, solution synthesis, best-practice
generation
• Preciseness: Situation-specific, accurate guidance
• Parsimony: Avoids excess complexity; simple to
learn and use
• Falsifiability: Coherent enough to be empirically
refuted
4
Theory W: Enterprise Success Theorem
– An informal proof
Theorem: Your enterprise will succeed
if and only if
it makes winners of your success-critical stakeholders
• Proof of “if”:
Everyone that counts is a winner.
Nobody significant is left to complain.
• Proof of “only if”:
Nobody wants to lose.
Prospective losers will refuse to participate, or will
counterattack.
The usual result is lose-lose.
5
Theory W: WinWin Achievement Theorem
Making winners of your success-critical
stakeholders requires:
i. Identifying all of the success-critical
stakeholders (SCSs).
ii. Understanding how the SCSs want to win.
iii. Having the SCSs negotiate a win-win set of
product and process plans.
iv. Controlling progress toward SCS win-win
realization, including adaptation to change.
6
VBSE Theory 4+1 Structure
Dependency
Theory
How do dependencies
affect value realization?
Utility Theory
What values are important?
How is success assured?
How important are the
values?
Theory W:
SCS Win-Win
How to adapt to change and
control value realization?
Control Theory
How do values determine
decision choices?
Decision Theory
7
VBSE Component Theories
• Theory W (Stakeholder win-win)
– Enterprise Success Theorem, Win-Win Achievement
Theorem
• Dependency Theory (Product, process, people
interdependencies)
– Systems architecture/performance theory, costing and
scheduling theory; organization theory
• Utility Theory
– Utility functions, bounded rationality, Maslow need
hierarchy, multi-attribute utility theory
• Decision Theory
– Statistical decision theory, game theory, negotiation theory,
theory of Justice
• Control Theory
– Observability, predictability, controllability, stability theory
8
Dependency Theory - Example
9
Utility Theory - Example
10
Decision Theory - Example
11
Decision Theory – Example (2)
RE = P(L) * S(L)
- RE due to inadequate plans
- RE due to market share erosion
- Sum of risk exposures
high P(L): inadequate plans
high S(L): major problems
(oversights, delays, rework)
high P(L): plan
breakage, delay
high S(L): value
capture delays
Sweet Spot
low P(L): few plan delays
low S(L): early value capture
low P(L): thorough plans
low S(L): minor problems
Time and Effort Invested in Plans
12
Control Theory - Example
Yes
Develop/update
business case;
time-phased cost,
benefit flows; plans
Perform
to plans
Value
being
realized?
Yes
Assumptions
still valid?
No
No
Determine corrective actions
Value Realization Feedback Control
13
VBSE Process Mapping Onto Spiral/RUP/MBASE Milestones:
I – Inception Readiness Phase
Theory W,
Dependency Theory
… Concurrent Spirals
Identify Success-Critical Stakeholders (SCSHs);
Value Propositions (Win Conditions);
Existing Capabilities; Issues; Risk Assessments
Utility, Dependency
Theory
Do more assessment,
expectations
management
Yes
Fail
Inception Phase
risk negligible?
Yes; Evaluate
Yes; proceed

No
Inception Readiness
Review (IRR)
(DoD Milestone CD)
Decision Theory
Worth
continuing?
Inception Phase
risk too high?
No; Terminate
No

Proceed into Inception
Phase
Proceed to Life Cycle
Objectives (LCO) Review
.
.
.
.
.
.
Yes
14
VBSE Process Mapping Onto Spiral/RUP/MBASE Milestones:
II – Inception Phase
Theory W,
Dependency Theory
… Concurrent Spirals
- Develop top-level Operational Concepts, Requirements,
Solution Options, Life Cycle Plans, LCO Feasibility Rationale
- Business case; evidence of feasibility; SCSH commitments
- Develop, execute, monitor risk management plans
- Monitor new environment risks, opportunities
Utility, Dependency
Theory
Do more assessment,
expectations
management
Control
Theory
Yes; Evaluate
Elaboration
Phase risk too
high?
No
No; Terminate

Yes
Back to mini-IRR
No
Yes
Worth
continuing?
Major changes
in above?
.
.
.
Elaboration
Phase risk
negligible?
Yes; proceed

Fail
Life Cycle Objective (LCO)
Review
(DoD Milestone A)
Decision Theory
No
Yes
Proceed into Elaboration
Phase
Proceed to Life Cycle
Architecture (LCA)
Review
.
.
.
.
.
.
15
VBSE Process Mapping Onto Spiral/RUP/MBASE Milestones:
III – Elaboration, Construction, Transition Phases
Theory W,
Dependency Theory
… Concurrent Spirals
- Develop Life Cycle Commitment level Operational Concepts,
Requirements, Solutions, Plans, LCA Feasibility Rationale
- Business case; evidence of feasibility; SCSH commitments
- Develop, execute, monitor risk management plans
- Monitor new environment risks, opportunities
Utility, Dependency
Theory
Do more assessment,
expectations
management
Yes
.
.
.
Back to mini-IRR
Control
Theory
Major
Criticality of
changes in
above?
Moderate
Back to mini-LCO
Yes; Evaluate
Minimal
Worth
continuing?
Life Cycle
Feasibility risk
too high?
No; Terminate
No

.
.
.
Yes; proceed

Life Cycle Architecture (LCA)
Review (DoD Milestone B)
Decision Theory
Agile preparation for next increments
Pro-active V&V
Proceed into incremental Plan-Driven
Construction, Transition with appropriate
risk mitigation plans
  
…
…
...
16
Outline
• VBSE Theory Motivation and Context
• “Theory” Definitions and Criteria
• VBSE Theory Elements and Process
– Elements and Contributions
– Theory-driven process
– Supply chain example
• Conclusions and Future research
17
Initial VBSE Theory: 4+1 Process
– With a great deal of concurrency and backtracking
5a, 7b. Option, solution
development & analysis
Dependency
Theory
Utility Theory
2a. Results Chains
2. Identify SCSs
3b, 5a, 7b. Cost/schedule/
performance tradeoffs
3b, 7a. Solution Analysis
3. SCS Value
Propositions
(Win conditions)
Theory W:
SCS Win-Win
6, 7c. Refine, Execute,
Monitor & Control Plans
Control Theory
6a, 7c. State measurement,
prediction, correction;
Milestone synchronization
4. SCS expectations
management
5a, 7b. Prototyping
5. SCS Win-Win
Negotiation
1. Protagonist goals
3a. Solution exploration
7. Risk, opportunity, change
Decision Theory
management
5a. Investment analysis,
Risk analysis
SCS: Success-Critical Stakeholder
18
Example Project: Sierra Mountainbikes
– Based on what would have worked on a
similar project
• Quality leader in specialty area
• Competitively priced
• Major problems with order processing
– Delivery delays and mistakes
– Poor synchronization of order entry,
confirmation, fulfillment
– Disorganized responses to problem
situations
– Excess costs; low distributor satisfaction
19
Order Processing Project Goals
Goals: Improve profits, market share,
customer satisfaction via improved order
processing
Questions: Current state? Root causes of
problems? Keys to improvement?
Metrics: Balanced Scorecard of benefits
realized, proxies
– Customer satisfaction ratings; key elements
(ITV: in-transit visibility)
– Overhead cost reduction
– Actual vs. expected benefit and cost flows, ROI
20
Initial VBSE Theory: 4+1 Process, Step 1
– With a great deal of concurrency and backtracking
Dependency
Theory
Utility Theory
Theory W:
SCS Win-Win
1. Protagonist goals
Decision Theory
Control Theory
SCS: Success-Critical Stakeholder
21
Frequent Protagonist Classes
Protagonist Class
Goals
Authority
Ideas
Resources
Leader with Goals, Baseline Agenda
X
X
X
X
Leader with Goals, Open Agenda
X
X
Entrepreneur with Goals, Baseline
Agenda
X
Entrepreneur with Goals, Open Agenda
X
Inventor with Goals, Ideas
X
Consortium with Shared Goals
X
X
X
X
X
X
(X)
(X)
•Sierra Moutainbikes: Susan Swanson, new CEO
– Bicycle champion, MBA, 15 years’ experience
– Leads with goals, open agenda
22
Initial VBSE Theory: 4+1 Process, Step 2
– With a great deal of concurrency and backtracking
Dependency
Theory
Utility Theory
2. Identify SCSs
2a. Results Chains
Theory W:
SCS Win-Win
1. Protagonist goals
Decision Theory
Control Theory
SCS: Success-Critical Stakeholder
23
DMR/BRA* Results Chain
Order to delivery time is
an important buying criterion
INITIATIVE
Contribution
Implement a new order
entry system
ASSUMPTION
OUTCOME
Contribution
OUTCOME
Reduced order processing cycle
(intermediate outcome)
Increased sales
Reduce time to process
order
Reduce time to deliver product
*DMR Consulting Group’s Benefits Realization Approach
24
Expanded Order Processing System Benefits Chain
Distributors, retailers,
customers
Assumptions
- Increasing market size
- Continuing consumer satisfaction with product
- Relatively stable e-commerce infrastructure
- Continued high staff performance
New order-entry
system
Developers
Less time,
fewer
errors per
order
entry
system
New order fulfillment
processes,
outreach, training
New order fulfillment
system
Safety, fairness
inputs
Less time, fewer
errors in order
processing
Faster,
better
order
entry
system
Interoperability
inputs
Increased
customer
satisfaction,
decreased
operations costs
Faster order-entry steps, errors
On-time assembly
New order-entry
processes,
outreach, training
Improved supplier
coordination
Sales personnel,
distributors
Increased
sales,
profitability,
customer
satisfaction
Increased profits,
growth
Suppliers
25
Initial VBSE Theory: 4+1 Process, Step 3
– With a great deal of concurrency and backtracking
2a. Results Chains
3b. Cost/schedule/
performance tradeoffs
Utility Theory
3. SCS Value
Propositions
(Win conditions)
Dependency
Theory
2. Identify SCSs
3b. Solution Analysis
Theory W:
SCS Win-Win
1. Protagonist goals
3a. Solution exploration
Decision Theory
Control Theory
SCS: Success-Critical Stakeholder
26
Initial VBSE Theory: 4+1 Process, Step 4
– With a great deal of concurrency and backtracking
2a. Results Chains
3b. Cost/schedule/
performance tradeoffs
Utility Theory
3. SCS Value
Propositions
(Win conditions)
Dependency
Theory
2. Identify SCSs
4. SCS expectations
management
3b. Solution Analysis
Theory W:
SCS Win-Win
1. Protagonist goals
3a. Solution exploration
Decision Theory
Control Theory
SCS: Success-Critical Stakeholder
27
The Model-Clash Spider Web: Master Net
- Stakeholder value propositions (win conditions)
28
EasyWinWin OnLine Negotiation Steps
29
Red cells indicate lack of
consensus.
Oral discussion of cell
graph reveals unshared
information, unnoticed
assumptions, hidden
issues, constraints, etc.
30
Tradeoffs among Cost, Schedule, and Reliability
– 100K-SLOC Project
(RELY, MTBF (hours))
9
(VL, 1)
8
Cost ($M)
7
(L, 10)
6
5
(N, 300)
4
3
2
(H, 10K)
•For 100-KSLOC set of features
•Can “pick all three” with 77-KSLOC set of features
1
0
(VH, 300K)
0
10
20
30
Development Time (Months)
40
50
-- Cost/Schedule/RELY:
pick any two” points
Initial VBSE Theory: 4+1 Process, Step 5
– With a great deal of concurrency and backtracking
5a. Option, solution
development & analysis
Utility Theory
3. SCS Value
Propositions
(Win conditions)
Dependency
Theory
2. Identify SCSs
4. SCS expectations
management
Theory W:
SCS Win-Win
5a. Prototyping
5. SCS Win-Win
Negotiation
3b. Solution Analysis
2a. Results Chains
3b, 5a. Cost/schedule/
performance tradeoffs
1. Protagonist goals
3a. Solution exploration
Decision Theory
Control Theory
5a. Investment analysis,
Risk analysis
SCS: Success-Critical Stakeholder
32
Business Case Analysis
• Estimate costs and schedules
– COCOMO II and/or alternative for software
– PRICE H or alternative for hardware
– COSYSMO for systems engineering
• Estimate financial benefits
– Increased profits
– Reduced operating costs
• Compute Return on Investment
– ROI = (Benefits – Costs) / Costs
– Normalized to present value
• Identify quantitative metrics for other goals
– Customer satisfaction ratings
• Ease of use; In-transit visibility; overall
– Late delivery percentage
33
Order Processing System Schedules and Budgets
Milestone
Due Date
Budget ($K)
Cumulative Budget ($K)
Inception Readiness
1/1/2004
0
0
Life Cycle Objectives
1/31/2004
120
120
Life Cycle Architecture
3/31/2004
280
400
Core Capability Drivethrough
7/31/2004
650
1050
Initial Oper. Capability: SW
9/30/2004
350
1400
Initial Oper. Capability: HW
9/30/2004
2100
3500
Developed IOC
12/31/2004
500
4000
Responsive IOC
3/31/2005
500
4500
Full Oper. Cap’y CCD
7/31/2005
700
5200
FOC Beta
9/30/2005
400
5600
FOC Deployed
12/31/2005
400
6000
Annual Oper. & Maintenance
3800
Annual O&M; Old System
7600
34
Order Processing System: Expected Benefits
and Business Case
New System
Current System
Market Share %
Sales
Profits
Market Share %
Sales
Profits
Cost Savings
Change in Profits
Cum. Change in Profits
Cum. Cost
ROI
Late Delivery %
Customer Satisfaction (0-5)
In-Transit Visibility (0-5)
Ease of Use (0-5)
Customers
Market Size ($M)
Financial
12/31/03
12/31/04
360
400
20
20
72
80
7
8
20
20
72
80
7
8
0
0
0
0
0
0
0
4
0
-1
12.4
11.4
1.7
3.0
1.0
2.5
1.8
3.0
12/31/05
12/31/06
440
480
20
20
88
96
9
10
22
25
97
120
10
13
2.2
3.2
3.2
6.2
3.2
9.4
6
6.5
-.47
.45
7.0
4.0
4.0
4.3
3.5
4.0
4.0
4.3
12/31/07
520
20
104
11
28
146
16
4.0
9.0
18.4
7
1.63
3.0
4.5
4.3
4.5
12/31/08
560
20
112
12
30
168
19
4.4
11.4 29.8
7.5
2.97
2.5
4.6
4.6
4.6
Date
35
Project Strategy and Partnerships
• Partner with eServices, Inc. for order processing and
fulfillment system
– Profit sharing using jointly-developed business case
• Partner with key distributors to provide user feedback
– Evaluate prototypes, beta-test early versions, provide
satisfaction ratings
• Incremental development using MBASE/RUP anchor points
– Life Cycle Objectives; Architecture (LCO; LCA)
– Core Capability Drivethrough (CCD)
– Initial; Full Operational Capability (IOC; FOC)
• Architect for later supply chain extensions
36
Initial VBSE Theory: 4+1 Process, Step 7
– With a great deal of concurrency and backtracking
5a, 7b. Option, solution
development & analysis
Utility Theory
3. SCS Value
Propositions
(Win conditions)
Dependency
Theory
2. Identify SCSs
4. SCS expectations
management
5a, 7b. Prototyping
3b, 7a. Solution Analysis
2a. Results Chains
3b, 5a, 7b. Cost/schedule/
performance tradeoffs
Theory W:
SCS Win-Win
6, 7c. Refine, Execute,
5. SCS Win-Win
Monitor & Control Plans
1.
Protagonist
goals
Negotiation
3a. Solution exploration
7. Risk, opportunity, change
management
Decision Theory
Control Theory
5a. Investment analysis,
Risk analysis
6a, 7c. State measurement,
prediction, correction;
Milestone synchronization
SCS: Success-Critical Stakeholder
37
Earned Value System
Budgeted Cost of
Work Scheduled
Cost
Specs
Plans
Analyses
Prototypes
Time
38
Earned Value System
Budgeted Cost of
Work Scheduled
Cost
Specs
Budgeted Cost of
Work Performed
Plans
Project Expenditures
Analyses
Prototypes
Time
39
“Earned Value” Tracks Cost, Not Value
Yes
Develop/update
plans, BCWS
Perform
to plans
BCWP>
BCWS?
Yes
BCWP>
cost ?
No
No
Determine corrective actions
•BCWS: Budgeted Cost of Work Scheduled
•BCWP: Budgeted Cost of Work Performed
40
A Real Earned Value System
• Current “earned value” systems monitor cost and
schedule, not business value
– Budgeted cost of work performed (“earned”)
– Budgeted cost of work scheduled (“yearned”)
– Actual costs vs. schedule (“burned”)
• A real earned value system monitors benefits
realized
– Financial benefits realized vs. cost (ROI)
– Benefits realized vs. schedule
- Including non-financial metrics
– Actual costs vs. schedule
41
Benefits Realization Feedback Process
Yes
Develop/update
business case;
time-phased cost,
benefit flows;plans
Perform
to plans
Benefits
being
realized?
Yes
Assumptions
still valid?
No
No
Determine corrective actions
42
Value-Based Expected/Actual Outcome Tracking Capability
Milestone
Schedule Cost ($K) Op-Cost Market
Annual
Annual
CumΔ ROI Late Cust. ITV Ease
Savings Share % Sales ($M) Profits ($M) Profits
Life Cycle
Architecture
3/31/04
3/31/04
400
427
Core Capability Demo
(CCD)
7/31/04
7/20/04
1050
1096
Software
9/30/04
Init. Op. Cap'y 9/30/04
(IOC)
1400
1532
20
20
72
72
7.0
7.0
Deliv % Sat.
12.4
12.4
1.7 1.0
1.7 1.0
Risks/Opportunities
of Use
1.8 Increased COTS ITV Risk.
1.8
Fallback identified.
Using COTS ITV Fallback.
2.4* 1.0* 2.7*
New HW Competitor;
renegotiating HW
2.7* 1.4* 2.8*
Hardware
IOC
9/30/04
10/11/04
3500
3432
Deployed
12/31/04
4000
20
80
8.0
0.0 -1.0 11.4
3.0 2.5
3.0
IOC
12/20/04
4041
22
88
8.6
0.6 -.85 10.8
2.8 1.6
3.2 identified, being protoyped
Responsive
3/31/05
4500
300
9.0
3.5 3.0
3.5
IOC
3/30/05
4604
324
7.4
3.3 1.6
3.8
Full Op.
Cap'y CCD
7/31/05
7/28/05
5200
5328
1000
946
3.5* 2.5* 3.8*
Full Op.
Cap'y Beta
9/30/05
9/30/05
5600
5689
1700
1851
3.8* 3.1* 4.1*
Full Op.
12/31/05
6000
2200
22
106
12.2
3.2 -.47
7.0
4.0 3.5
4.0
Cap'y Deployed 12/20/05
5977
2483
24
115
13.5
5.1 -.15
4.8
4.1 3.3
4.2
Release 2.1
6/30/06
$200K savings from
renegotiated HW
New COTS ITV source
New COTS ITV source
initially integrated
6250
43
Planning and Control Summary
• Good planning and control requires
- Framework of techniques
- Much up-front work on plans
- Commitment to control
• Good P&C can produce self-fulfilling estimates
• Project plans are living entities
• P&C techniques aren’t personnel evaluation devices
• P&C techniques aren’t necessarily routinizing
11/26/2003
44
Conclusions and Future Research
• VBSE Theory applied well to the supply
chain example.
– Application to other domains, situations should
further uncover its underlying capabilities,
shortcomings and assumptions.
• It satisfies the main criteria for a good
theory (utility, generality, practicality,
preciseness, parsimony, and falsifiability)
reasonably well so far.
• Future work is geared towards testing the
theory per se and improvising it.
45
References - I
C. Baldwin & K. Clark, Design Rules: The Power of Modularity, MIT Press, 1999.
S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Gruenbacher (eds.), Value-Based Software
Engineering, Springer, 2005 (to appear).
D. Blackwell and M. Girshick, Theory of Games and Statistical Decisions, Wiley, 1954.
B. Boehm, C. Abts, A.W. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and
B. Steece, Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
B. Boehm and L. Huang, “Value-Based Software Engineering: A Case Study, Computer,
March 2003, pp. 33-41.
B. Boehm, and R. Ross, Theory-W Software Project Management: Principles and Examples,
IEEE Trans. SW Engineering., July 1989, pp. 902-916.
W. Brogan, Modern Control Theory, Prentice Hall, 1974 (3rd ed., 1991).
P. Checkland, Systems Thinking, Systems Practice, Wiley, 1981.
C. W. Churchman, R. Ackoff, and E. Arnoff, An Introduction to Operations Research, Wiley,
1957.
R. M. Cyert and J.G. March, A Behavioral Theory of the Firm, Prentice Hall, 1963.
C. G. Hempel and P. Oppenheim, Problems of the Concept of General Law, in (eds.) A. Danto
and S. Mogenbesser, Philosophy of Science, Meridian Books, 1960.
46
References - II
R. Kaplan & D. Norton, The Balanced Scorecard: Translating Strategy into Action,
Harvard Business School Press, 1996.
R. L. Keeney and H. Raiffa, Decisions with Multiple Objectives: Preferences and Value
Tradeoffs, Cambridge University Press, 1976.
A. Maslow, Motivation and Personality, Harper, 1954.
J. Rawls, A Theory of Justice, Belknap/Harvard U. Press, 1971, 1999.
J. Thorp and DMR, The Information Paradox, McGraw Hill, 1998.
R. J. Torraco, Theory-building research methods, in R. A. Swanson & E. F. Holton III
(eds.), Human resource development handbook: Linking research and practice pp.
114–137, Berrett-Koehler, 1997.
S. Toulmin, Cosmopolis: The Hidden Agenda of Modernity, U. of Chicago Press, 1992
reprint edition.
J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior,
Princeton University Press, 1944.
A. W. Wymore, A Mathematical Theory of Systems Engineering: The Elements, Wiley,
New York, 1967.
47
VBSE-Q&A
48