University of Southern California
Center for Systems and Software Engineering
Using the Incremental Commitment Model
Process Decision Table for Integrated
Systems and Software Engineering
----Tutorial---Barry Boehm and Jo Ann Lane
University of Southern California
Center for Systems and Software Engineering
http://csse.usc.edu
Presented at
Systems
Syste
sa
and
d So
Software
t a e Technology
ec o ogy Co
Conference
e e ce
April 2008
University of Southern California
Center for Systems and Software Engineering
G l off Tutorial
Goals
T t i l
1. Provide an overview the Incremental
Commitment Model (ICM) and its capabilities
2. Provide a detailed explanation of the ICM
process decision table
3. Provide guidance on using the ICM process
decision table
4 Review
4.
R i
ad
detailed
t il d case study
t d illustrating
ill t ti the
th
power of the ICM process decision table
5 Conduct exercise to develop ICM model for
5.
various special cases
April 2008
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Tutorial Outline
• Motivation
• Overview of ICM
• ICM process decision table
• Guidance and examples for using the ICM
process decision table
• Group Exercise: Case study using ICM
process decision table
p
• Team Exercise: Applying ICM to 10 cases
April 2008
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
What are Current Lifecycle Concerns?
April 2008
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
Lifecycle Evolution Motivators as Identified
by CSSE Sponsors and Affiliates
• Current lifecycle models and associated practices
– Based on outdated assumptions
– Don’t always address today’s system development
challenges, especially as systems become more complex
• Need better integration of
–
–
–
–
Systems engineering
Software engineering
g
g
Human factors engineering
Specialty engineering (e.g. information assurance, safety)
• N
Need
d tto b
better
tt id
identify
tif and
d manage critical
iti l issues
i
and risks up front (Young memo)
Additional details found in Backup Charts…
April 2008
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
Example: SoSE Synchronization Points
FCR1
SoS-Level Exploration
Valuation
Candidate Supplier/
Strategic Partner n
DCR1
OCR1
Architecting
Develop
OCR2
•••
Operation
LCO-type
LCO
Proposal &
Feasibility
Info
●
●
Source
Selection
Rebaseline/
Adjustment FCR1
●
Candidate Supplier/
Strategic Partner 1
OCRx2
OCRx1
System
y
x
•••
Develop
OCRx3
Operation
Operation
OCRx5
OCRx4
Operation
Operation
DCRC
OCRC1
•••
●
●
FCRC
●
System C • • •
Exploration
Valuation
Architecting
FCRB
System B • • •
Exploration
Valuation
April 2008
•••
Exploration
Valuation
DCRB
Architecting
FCRA
System A
Develop
Operation
OCRB1
Develop
©USC-CSSE
•••
OCRB2
Operation
•••
OCRA1
DCRA
Architecting
OCRC2
Develop
Operation
6
•••
University of Southern California
Center for Systems and Software Engineering
Tutorial Outline
• Motivation
• Overview of ICM
• ICM process decision table
• Guidance and examples for using the ICM
process decision table
• Group Exercise: Case study using ICM
process decision table
• Team Exercise: Applying ICM to 10 cases
April 2008
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Wh t is
What
i the
th ICM?
• Risk-driven framework for tailoring system lifecycle
l processes
• Integrates the strengths of phased and riskdriven spiral process models
• Synthesizes together principles critical to
y
development
p
successful system
– Commitment and accountability of system sponsors
– Success-critical stakeholder satisficing
– Incremental growth of system definition and
stakeholder commitment
– Concurrent engineering
– Iterative development cycles
– Risk-based activity levels and anchor point milestones
Principles
P
i i l
trump
diagrams…
Used by 60-80% of CrossTalk Top-5 projects, 2002-2005
April 2008
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
Future Process Challenges-I
• Multi-owner,
M lti
multi-mission
lti i i systems
t
off systems
t
– Integrated supply chain: strategic planning, marketing,
merchandising, outsourcing, just
just-in-time
in time manufacturing,
logistics, finance, customer relations management
– Over 50 separately evolving external systems
– Need to satisfice among multiple stakeholders
• Rapid pace of change
– IIn competition,
titi
mission
i i priorities,
i iti
technology,
t h l
Commercial
C
i l
Off-the-Shelf (COTS), environment
– Need incremental development to avoid obsolescence
– Need concurrent vs. sequential processes
– Need both prescience and rapid adaptability
• Software important; humans more important
April 2008
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
Priorities for Rapid Adaptation to Change
• Software is more adaptable
than hardware
• People are generally more
adaptable
d
bl than
h software
f
• Need systems and software
that better help
p people
p p to
adapt
April 2008
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
Future Process Challenges-II
• Emergence and human-intensiveness
– Requirements
R
i
t nott pre-specifiable
ifi bl
– Need for evolutionary growth
– Need to manage uncertainty and risk
• Always-on, never-fail systems
– Need well-controlled, high-assurance processes
– Need to synchronize and stabilize concurrency
April 2008
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
ICM Nature and Origins
• Integrates hardware, software, and human factors
elements
l
t off systems
t
engineering
i
i
– Concurrent exploration of needs and opportunities
– Concurrent engineering of hardware
hardware, software
software, human aspects
– Concurrency stabilized via anchor point milestones
• Developed
p in response
p
to DoD-related issues
– Clarify “spiral development” usage in DoD Instruction 5000.2
• Initial phased version (2005)
– Explain Future Combat System of systems spiral usage to GAO
• Underlying process principles (2006)
– Provide framework for human
human-systems
systems integration
• National Research Council report (2007)
• Integrates strengths of current process models
– But not their weaknesses
April 2008
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
ICM Integrates Strengths of Current Process
Models—But not their Weaknesses
• V-Model:
V M d l Emphasis
E h i on early
l verification
ifi ti and
d validation
lid ti
– But not ease of sequential, single-increment interpretation
• Spiral Model: Risk-driven
Risk driven activity prioritization
– But not lack of well-defined in-process milestones
• RUP and MBASE: Concurrent engineering stabilized by
anchor point milestones
– But not software orientation
• Lean Development: Emphasis on value-adding
activities
– But not repeatable manufacturing orientation
• Agile Methods: Adaptability to unexpected change
– But not software orientation, lack of scalability
April 2008
©USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
Incremental Commitment in
Gambling
g
• Total Commitment: Roulette
–P
Putt your chips
hi on a number
b
• E.g., a value of a key performance parameter
– Wait and see if you win or lose
• Incremental Commitment: Poker, Blackjack
– Put some chips in
– See your cards, some of others’ cards
– Decide whether, how much to commit to
proceed
April 2008
12/31/2007
©USC-CSSE
14
14
University of Southern California
Center for Systems and Software Engineering
S l bl Remotely
Scalable
R
t l Controlled
C t ll d
Operations
April 2008
12/31/2007
©USC-CSSE
15
15
University of Southern California
Center for Systems and Software Engineering
Total vs. Incremental Commitment – 4:1 RPV
• Total Commitment
–
–
–
–
–
Agent technology demo and PR: Can do 4:1 for $1B
Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months
PDR: many outstanding risks, undefined interfaces
$800M, 40 months: “halfway” through integration and test
1:1 IOC after $3B, 80 months
• Incremental Commitment [with a number of competing
teams]
–
–
–
–
–
$25M, 6 mo. to VCR [4]: may beat 1:2 with agent technology, but not 4:1
$75M, 8 mo. to FCR [[3]:
] agent
g
technology
gy may
y do 1:1; some risks
$225M, 10 mo. to DCR [2]: validated architecture, high-risk elements
$675M, 18 mo. to IOC [1]: viable 1:1 capability
1:1 IOC after $1B,
$1B 42 months
April 2008
12/31/2007
©USC-CSSE
16
16
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Synchronize stabilize concurrency via FEDs
Synchronize,
Risk patterns
determine life
y
p
process
cycle
April 2008
03/19/2008
©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
A h Point
Anchor
P i t Feasibility
F
ibilit Evidence
E id
Descriptions
D
i ti
• Evidence provided by developer and validated by
i d
independent
d t experts
t that:
th t
If the system is built to the specified architecture, it will
– Satisfy the requirements: capability,
capability interfaces,
interfaces level of service,
service and
evolution
– Support the operational concept
– Be
B b
buildable
ild bl within
ithi the
th budgets
b d t and
d schedules
h d l in
i the
th plan
l
– Generate a viable return on investment
– Generate satisfactory
y outcomes for all of the success-critical
stakeholders
• All major risks resolved or covered by risk management
plans
• Serves as basis for stakeholders’ commitment to proceed
C b
Can
be used
d to
t strengthen
t
th currentt scheduleh d l or event-based
tb
d reviews
i
April 2008
©USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Concurrently engr
engr.
OpCon, rqts, arch,
plans, prototypes
April 2008
©USC-CSSE
Concurrently engr.
Incr.N (ops), N+1
(devel) N+2 (arch)
(devel),
19
University of Southern California
Center for Systems and Software Engineering
ICM HSI Levels of Activity for Complex Systems
April 2008
©USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
Different Risk Patterns Yield Different Processes
April 2008
03/19/2008
©USC-CSSE
21
21
University of Southern California
Center for Systems and Software Engineering
Example ICM Commercial Application:
Symbiq Medical Infusion Pump
Winner of 2006 HFES Best New Design
g Award
Described in NRC HSI Report, Chapter 5
April 2008
©USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
S bi IV Pump
Symbiq
P
ICM Process
P
-I
• Exploration Phase
–
–
–
–
Stakeholder needs interviews, field observations
Initial user interface prototypes
Competitive analysis, system scoping
Commitment to proceed
• Valuation Phase
–
–
–
–
–
Feature analysis and prioritization
Display vendor option prototyping and analysis
T
Top-level
l
l life
lif cycle
l plan,
l
business
b i
case analysis
l i
Safety and business risk assessment
C
Commitment
it
t tto proceed
d while
hil addressing
dd
i risks
i k
April 2008
©USC-CSSE
23
University of Southern California
Center for Systems and Software Engineering
S bi IV Pump
Symbiq
P
ICM Process
P
- II
• Architecting
g Phase
–
–
–
–
–
–
Modularity of pumping channels
Safety feature and alarms prototyping and iteration
Programmable therapy types, touchscreen analysis
Failure modes and effects analyses (FMEAs)
Prototype usage in teaching hospital
Commitment to proceed into development
• Development Phase
–
–
–
–
Extensive usability criteria and testing
Iterated FMEAs and safety analyses
Patient-simulator testing; adaptation to concerns
Commitment to production and business plans
April 2008
©USC-CSSE
24
University of Southern California
Center for Systems and Software Engineering
Incremental Commitment In Systems and Life:
Anchor Point Milestones
• Common System/Software stakeholder commitment
points
– D
Defined
fi d in
i concertt with
ith Government,
G
t industry
i d t
organizations
– Initially coordinated with Rational’s Unified Software
D
Development
l
t Process
P
• Exploration Commitment Review (ECR)
– Stakeholders’
Stakeholders commitment to support initial system
scoping
– Like dating
• Validation Commitment Review (VCR)
– Stakeholders’ commitment to support system concept
definition and investment analysis
– Like going steady
April 2008
©USC-CSSE
25
University of Southern California
Center for Systems and Software Engineering
Incremental Commitment In Systems and Life:
Anchor Point Milestones (continued)
• Foundations Commitment Review (FCR)
– Stakeholders
Stakeholders’ commitment to support system architecting
– Like getting engaged
• Development
p
Commitment Review (DCR)
(
)
– Stakeholders’ commitment to support system
development
– Like
Lik getting
tti married
i d
• Incremental Operational Capabilities (OCs)
– Stakeholders
Stakeholders’ commitment to support operations
– Like having children
April 2008
©USC-CSSE
26
University of Southern California
Center for Systems and Software Engineering
ICM Anchor Point Milestone Content (1)
(Risk-driven level of detail for each element)
Milestone Element
Foundations Commitment Review
Development Commitment Review
Definition of
Operational
Concept
• Top-level system objectives and scope
– System boundary
– Environment parameters and
assumptions
– Evolution parameters
• Operational concept
– Operations and maintenance
scenarios and parameters
– Organizational life-cycle
responsibilities (stakeholders)
• Elaboration of system objectives and
scope of increment
• Elaboration of operational concept by
increment
System
P t t
Prototype(s)
( )
• Exercise key usage scenarios
• Resolve
R
l critical
iti l risks
i k
• Exercise range of usage scenarios
• Resolve
R
l major
j outstanding
t t di
risks
i k
Definition of System
Requirements
• Top-level functions, interfaces, quality
attribute levels, including
– Growth vectors and priorities
– Prototypes
• Stakeholders’ concurrence on
essentials
• Elaboration of functions, interfaces,
quality attributes, and prototypes by
increment
• Identification of TBD’s (to-bedetermined items)
• Stakeholders’ concurrence on their
priority concerns
April 2008
©USC-CSSE
27
University of Southern California
Center for Systems and Software Engineering
ICM Anchor Point Milestone Content (2)
(Risk-driven level of detail for each element)
Milestone Element
Foundations Commitment Review
Development Commitment Review
Definition of
System and
Software
Architecture
• Top-level definition of at least one feasible
architecture
– Physical and logical elements and
relationships
– Choices
Ch i
off COTS and
d reusable
bl software
ft
elements
• Identification of infeasible architecture
options
• Choice of architecture and elaboration by
increment
– Physical and logical components,
connectors, configurations,
constraints
– COTS, reuse choices
– Domain-architecture and architectural
style choices
• Architecture evolution parameters
Definition of LifeCycle Plan
• Identification of life-cycle stakeholders
– Users, customer, developers,
maintainers, interoperators, general
public others
public,
• Identification of life-cycle process model
– Top-level stages, increments
• Top-level WWWWWHH* by stage
• Elaboration of WWWWWHH* for Initial
Operational Capability (IOC)
– Partial elaboration, identification of
key TBD
TBD’s
s for later increments
Feasibility
Evidence
• Assurance of consistency among elements
above
– Via analysis, measurement, prototyping,
simulation, etc.
– Business case analysis for
requirements, feasible architectures
• Assurance of consistency among elements
above
• All major risks resolved or covered by risk
management plan
April 2008
©USC-CSSE
*WWWWWHH: Why, What,
When, Who, Where, How, How Much
28
University of Southern California
Center for Systems and Software Engineering
Risk-Driven
Risk
Driven Scalable Spiral Model:
Increment View
Rapid
Ch
Change
Foreseeable
Change
(Plan)
Increment N Baseline
High
Assurance
April 2008
Short Development
Increments
Short, S
Sh
Stabilized
bili d
Development
Of Increment N
Increment N Transition/O&M
Stable Development
Increments
©USC-CSSE
29
University of Southern California
Center for Systems and Software Engineering
Risk-Driven Scalable Spiral Model: Increment View
Unforeseeable Change (Adapt)
Future Increment Baselines
Agile
Rebaselining for
Future Increments
Rapid
Ch
Change
Foreseeable
Change
(Plan)
Short
D
Development
l
t
Increments
Increment N Baseline
Stable Development
Increments
High
Assurance
Current V&V
Resources
Continuous V&V
April 2008
Deferrals
Short, Stabilized
Development
p
of Increment N
Artifacts
O
Operations
ti
and
d Maintenance
M i t
Concerns
Verification and
Validation (V&V)
off Increment N
©USC-CSSE
Increment N Transition/
Future V&V
Resources
30
University of Southern California
Center for Systems and Software Engineering
Risk-Driven Scalable ICM:
Life Cycle View
System DCR
System
Inception
System DI1 DCR
System,
System
Elaboration
DI2 B/L DCR
Changes
Agile DI2 (OO&D)
Rebaselining
Plan-Driven DI1
Construction (A)
DI1 V&V
Plan-Driven DI2
Construction (A)
DCR: Development Commitment Review
OO&D: Observe, Orient and Decide
V&V: Verification and Validation
DI:
Development Increment
B/L:
Baselined
April 2008
DI2 DCR
DI2 V&V
©USC-CSSE
31
University of Southern California
Center for Systems and Software Engineering
Risk-Driven Scalable ICM:
Life Cycle View
System
y
DCR System,
y
DI1 DCR
System
Inception
System
Elaboration
DI2 B/L DCR
DI3 B/L DCR
DI4 B/L DCR
Changes
Agile DI2 (OO&D)
R b
Rebaselining
li i
Plan-Driven DI1
Construction (A)
DI1 V&V
Changes
Update
Update
DI1 OCR
DI1
Trans’n
DI1
Usage
DI2 DCR
Agile DI3 (OO&D)
Rebaselining
Plan-Driven DI2
Construction (A)
DI2 V&V
Changes
Update
DI2 OCR
DI2
Trans’n
DI2
Usage
DI3 DCR
Agile DI4 (OO&D)
Rebaselining
DCR: Development Commitment Review
OCR: Operational Commitment Review
OO&D: Observe, Orient and Decide
V&V: Verification and Validation
DI:
Development Increment
B/L:
Baselined
April 2008
DI3 OCR
Plan-Driven DI3
Construction (A)
DI3
Trans’n
DI3 V&V
DI4 DCR
DI3
Usage . . .
...
©USC-CSSE
32
University of Southern California
Center for Systems and Software Engineering
Acquisition Via Spiral OODA Loops
Observe new/updated objectives,
Orient with respect to stakeholders
• Usage monitoring
• Risk/Opportunity analysis
Competition technology,
technology
• Competition,
marketplace ISR
• Business case/mission analysis
constraints alternatives
constraints,
priorities feasibility,
priorities,
feasibility risks
• Prototypes, models, simulations
Operate as current system
Accept new system
Decide on next-cycle
y
capabilities,
p
Act on plans, specifications
architecture upgrades, plans
• Keep development stabilized
• Stable specifications, COTS
upgrades
• Change
g impact
p
analysis,
y ,
preparation for next cycle (miniOODA loop)
• Development, integration, V&V, risk
management plans
• Feasibility evidence
Life Cycle Architecture Milestone for Cycle
April 2008
©USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
A il Change
Agile
Ch
Processing
P
i
and
d Rebaselining
R b
li i
Stabilized
I
Increment-N
tN
Development Team
Agile FutureI
Increment
t
Rebaselining Team
Future
F
t
Increment
I
t
Managers
Change
Ch
Proposers
Proposed changes
Propose
Changes
Defer some Increment-N capabilities
Recommend handling
in current increment
Negotiate
N
i change
h
disposition
Accept changes
Handle
Accepted
Increment-N
changes
g
Assess Changes,
Propose Handling
Discuss, revise,
defer, or drop
Handle in current
rebaseline
Formulate, analyze options
i context off other
in
h changes
h
Recommend deferrals to future increments
Discuss, resolve deferrals to
future increments
Rebaseline
future increment
future-increment
Foundations packages
April 2008
Recommend no action,
provide rationale
©USC-CSSE
Prepare for rebaselined
future-increment
development
34
University of Southern California
Center for Systems and Software Engineering
S k h ld Commitment
Stakeholder
C
i
Ranges
R
vs. Phase
Ph
Drivers
System Size
Length of Production Run
System Criticality
System Understanding Level
Stakeholder Compatibility
Personnel Capability
Amount of New Modeling/Infrastructure
April 2008
©USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
ICM Summary
S
• Current processes not well matched to future challenges
– Emergent, rapidly changing requirements
– High assurance of scalable performance and qualities
• Incremental Commitment Model addresses challenges
– Assurance via evidence-based milestone commitment reviews,
stabilized incremental builds with concurrent V&V
• Evidence shortfalls treated as risks
– Adaptability via concurrent agile team handling change traffic and
providing evidence-based
evidence based rebaselining of next-increment
next increment specifications
and plans
– Use of critical success factor principles: stakeholder satisficing,
i
incremental
t l growth,
th concurrentt engineering,
i
i
iterative
it
ti development,
d
l
t riski k
based activities and milestones
• Major implications for funding, contracting, career paths
April 2008
©USC-CSSE
36
University of Southern California
Center for Systems and Software Engineering
Tutorial Outline
• Motivation
• Overview of ICM
• ICM process decision table
• Guidance and examples for using the ICM
process decision table
• Group Exercise: Case study using ICM
process decision table
• Team Exercise: Applying ICM to 10 cases
April 2008
©USC-CSSE
37
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Synchronize stabilize concurrency via FEDs
Synchronize,
Risk patterns
determine life
y
p
process
cycle
April 2008
03/19/2008
©USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
The ICM as Risk
Risk-Driven
Driven Process
Generator
• Stage I of the ICM has 3 decision nodes with 4 options/node
– Culminating with incremental development in Stage II
– Some options involve go-backs
– Results in many possible process paths
• Can use ICM risk patterns to generate frequently-used
frequently used
processes
– With confidence that they fit the situation
• Can generally determine this in the Exploration phase
– Develop as proposed plan with risk-based evidence at VCR
milestone
– Adjustable in later phases
April 2008
©USC-CSSE
39
University of Southern California
Center for Systems and Software Engineering
Th ICM Process
The
P
Decision
D i i Table:
T bl
Key Decision Inputs
•
•
•
•
Product and project size and complexity
R
Requirements
i
t volatility
l tilit
Mission criticality
Nature of Non-Developmental Item (NDI)* support
– Commercial, open-source, reused components
• Organizational and Personnel Capability
* NDI Definition [DFARS]: a) any product that is available in the commercial marketplace; b) any
previously developed product in use by a U.S. agency (federal, state, or local) or a foreign government that
has a mutual defense agreement with the U.S.; c) any product described in the first two points above that
requires only modifications to meet requirements; d) any product that is being produced,
produced but not yet in the
commercial marketplace, that satisfies the above criteria.
April 2008
©USC-CSSE
40
University of Southern California
Center for Systems and Software Engineering
The ICM Process Decision Table:
Key Decision Outputs
• K
Key St
Stage I activities:
ti iti
incremental
i
t l definition
d fi iti
• Key Stage II activities: incremental
development and operations
gg
calendar time per
p build,, per
p
• Suggested
deliverable increment
April 2008
©USC-CSSE
41
University of Southern California
Center for Systems and Software Engineering
Common Risk-Driven
Risk Driven Special Cases of the ICM
Key Stage I Activities : Incremental
Definition
Key Stage II Activities: Incremental
Development, Operations
Acquire NDI
Use NDI
Agile-ready
Med-high
Skip Valuation , Architecting
phases
Scrum plus agile methods of
choice
<= 1 day;
2-6 weeks
Good;
most in place
Agile-ready
Med-high
Combine Valuation, Architecting
phases. Complete NDI
preparation
Architecture-based Scrum of
Scrums
2-4 weeks;
2-6 months
M d
MedVery
High
G d
Good;
In place
Experienced;
E
i
d
med-high
Concurrentt HW/SW engineering.
C
i
i
CDR-level ICM DCR
IOC Development,
D
l
t LRIP,
LRIP FRP.
FRP
Concurrent Version N+1
engineering
SW: 1-5
SW
15d
days;
Market-driven
0.3 – 1
HighVery
High
Some in place
Experienced;
med-high
Determine minimum-IOC likely,
conservative cost. Add
deferrable SW features as risk
reserve
Drop deferrable features to meet
conservative cost. Strong award
fee for features not dropped
SW: 2-6
weeks;
Platform: 6-18
months
Med –
High
0.3 – 3
MedVery
High
NDI-driven
architecture
NDIexperienced;
Med-high
Thorough NDI-suite life cycle
cost-benefit analysis, selection,
concurrent requirements/
architecture definition
Pro-active NDI evolution
influencing, NDI upgrade
synchronization
SW: 1-4
weeks;
System: 6-18
months
C4ISR
Med –
Very
High
Mixed
parts:
1 – 10
Mixed
parts;
Med
MedVery
High
Mixed parts
Mixed parts
Full ICM; encapsulated agile in
high change, low-medium
criticality parts (Often HMI,
HMI
external interfaces)
Full ICM ,three-team incremental
development, concurrent V&V,
next increment rebaselining
next-increment
1-2 months;
9-18 months
8. Multi-owner
system of
systems
Net-centric
military
operations
Very
High
Mixed
parts:
1 – 10
Very
High
Many NDIs;
some in place
Related
experience,
med-high
Full ICM; extensive multi-owner
team building, negotiation
Full ICM; large ongoing
system/software engineering
effort
2-4 months;
18-24 months
9. Family of
systems
Medical Device
Product Line
Med –
Very
High
1–3
Med –
Very
High
Some in place
Related
experience,
med – high
Full ICM; Full stakeholder
participation in product line
scoping. Strong business case
Full ICM. Extra resources for
first system, version control,
multi-stakeholder support
1-2
1
2 months;
9-18 months
9. Multi-owner
system of
systems
Net-centric
military
operations
Very
high
Mixed
parts; 1-10
Very
High
Many NDIs;
some in place
Related
experience,
med-high
E Full ICM; extensive multiowner team building, negotiation
Full ICM: large ongoing
system/software engineering
effort
2-4 months;
18-24 months
10. Family of
systems
Medical device
product line
MedVery
High
1-3
MedVery
High
Some in place
Related
experience,
med-high
Full ICM; Full stakeholder
participation in product line
scoping. Strong business case.
Full ICM. Extra resources for
first system, version control,
multi-stakeholder support
1-2 months;
9-18 months
Size,
Complexi
ty
Change
Rate %
/Month
Criticality
NDI Support
Special Case
Example
1. Use NDI
Small
Accounting
2. Agile
E-services
Low
1 – 30
LowMed
Good;
in place
3. Architected
Agile
Business data
processing
Med
1 – 10
MedHigh
4 HW
4.
component with
embedded SW
Multisensor
M
lti
control device
L
Low
03–1
0.3
5. Indivisible
IOC
Complete
vehicle platform
Med –
High
6. NDIIntensive
Supply Chain
Management
7. Hybrid agile /
plan-driven
system
Org, Personnel
Capability
Complete
C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review.
DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware.
April 2008
©USC-CSSE
IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software
Time per Build;
per Increment
42
University of Southern California
Center for Systems and Software Engineering
Tutorial Outline
• Motivation
• Overview of ICM
• ICM process decision table
•G
Guidance and examples for
f using
the ICM process decision table
• Group Exercise: Case study using ICM
process decision table
• Team Exercise: Applying ICM to 10 cases
April 2008
©USC-CSSE
43
University of Southern California
Center for Systems and Software Engineering
Case 1: Use NDI
•
•
Exploration phase identifies NDI opportunities
NDI risk/opportunity analysis indicates risks acceptable
– Product growth envelope fits within NDI capability
– Compatible NDI and product evolution paths
– Acceptable NDI volatility
• Some open-source components highly volatile
– Acceptable usability, dependability, interoperability
– NDI available or affordable Example: Small accounting system
•
•
•
•
•
•
•
•
•
Size/complexity: Low
Anticipated change rate (% per month): Low
Criticality: Low
NDI support: Complete
Organization and personnel capability: NDI-experienced
Key Stage I activities: Acquire NDI
Key Stage II activities: Use NDI
Time/build: Driven by time to initialize/tailor NDI
Time/increment: Driven by NDI upgrades
April 2008
©USC-CSSE
44
University of Southern California
Center for Systems and Software Engineering
Case 2: Pure Agile Methods
•
Exploration phase determines
–
–
–
–
–
•
•
•
•
•
•
•
•
•
•
Low product and project size and complexity
Fixing increment defects in next increment acceptable
Existing hardware and NDI support of growth envelope
Sufficient agile-capable personnel
Need to accommodate rapid change, emergent requirements, early user capability
Example: E-services
p
y Low
Size/complexity:
Anticipated change rate (% per month): 1-30%
Criticality: Low to medium
NDI support: Good; in place
Organization and personnel capability: Agile-ready, medium to high
capability
Key Stage I activities: Skip Valuation and Architecting phases
Key Stage II activities: Scrum plus agile methods of choice
Time/build: Daily
y
Time/increment: 2-6 weeks
April 2008
©USC-CSSE
45
University of Southern California
Center for Systems and Software Engineering
Case 3: Architected Agile
•
Exploration phase determines
– Need to accommodate fairly rapid change, emergent requirements, early user
capability
– Low risk of scalability up to 100 people
– NDI support
pp
of growth
g
envelope
p
– Nucleus of highly agile-capable personnel
– Moderate to high loss due to increment defects
•
•
•
•
•
•
•
•
•
Example: Business data processing
Size/complexity: Medium
Anticipated change rate (% per month): 1-10%
C iti lit Medium
Criticality:
M di
to
t high
hi h
NDI support: Good, most in place
g
and personnel
p
capability:
p
y Agile-ready,
g
y med-high
g capability
p
y
Organization
Key Stage I activities: Combined Valuation and Architecting phase,
complete NDI preparation
Key Stage II activities: Architecture-based
Architecture based scrum of scrums
Time/build: 2-4 weeks
Time/increment: 2-6 months
April 2008
©USC-CSSE
46
University of Southern California
Center for Systems and Software Engineering
C
Case
4
4: F
Formall Methods
M th d
•
•
•
•
•
•
•
•
•
•
•
Biggest
gg
risks: Software/hardware does not accurately
y implement
p
required algorithm precision, security, safety mechanisms, or
critical timing
p
Security
y kernel or safety-critical
y
LSI chip
p
Example:
Size/complexity: Low
Anticipated change rate (% per month): 0.3%
Criticality: Extra high
NDI support: None
Organization and personnel capability: Strong formal methods
experience
Key Stage I activities: Precise formal specification
y Stage
g II activities: Formally-based
y
programming
p
g
g language;
g g ;
Key
formal verification
Time/build: 1-5 days
Time/increment: 1-4 weeks
April 2008
©USC-CSSE
47
University of Southern California
Center for Systems and Software Engineering
Case 5: Hardware Component with
Embedded Software
• Biggest risks: Device recall, lawsuits, production line rework,
hardware-software integration
– DCR carried to Critical Design Review level
– Concurrent hardware-software design
• Criticality makes Agile too risky
– Continuous hardware-software integration
• Initially with simulated hardware
• Low risk off overrun
– Low complexity, stable requirements and NDI
– Little need for risk reserve
– Likely single-supplier software
April 2008
©USC-CSSE
48
University of Southern California
Center for Systems and Software Engineering
Case 5: Hardware Component with
Embedded Software (continued)
•
•
•
•
•
•
•
•
•
•
Example: Multi-sensor control device
Size/complexity: Low
Anticipated change rate (% per month): 0.3-1%
Criticality: Medium to very high
NDI support: Good, in place
Organization and personnel capability: Experienced; medium to high
capability
Key Stage I activities: Concurrent hardware and software engineering;
CDR-level ICM DCR
Key Stage II activities: IOC Development, LRIP,FRP, concurrent version
N+1 engineering
Time/build: 1-5 days (software)
Time/increment: Market-driven
April 2008
©USC-CSSE
49
University of Southern California
Center for Systems and Software Engineering
C
Case
6
6: IIndivisible
di i ibl IOC
• Biggest risk: Complexity
Complexity, NDI uncertainties cause costschedule overrun
– Similar strategies to case 5 for criticality (CDR, concurrent HWSW design,
design continuous integration)
– Add deferrable software features as risk reserve
• Adopt conservative (90% sure) cost and schedule
• Drop software features to meet cost and schedule
• Strong award fee for features not dropped
– Likely multiple-supplier software makes longer (multi-weekly)
builds more necessary
April 2008
©USC-CSSE
50
University of Southern California
Center for Systems and Software Engineering
C
Case
6
6: IIndivisible
di i ibl IOC (continued)
•
•
•
•
•
•
•
•
•
•
Example: Complete vehicle platform
Size/complexity: Medium to high
Anticipated change rate (% per month): 0.3-1%
Criticality: High to very high
NDI support: Some in place
O
Organization
i ti and
d personnell capability:
bilit Experienced,
E
i
d
medium to high capability
Key
y Stage
g I activities: Determine minimum-IOC likely,
y,
conservative cost; Add deferrable software features as risk
reserve
Key Stage II activities: Drop deferrable features to meet
conservative cost; Strong award fee for features not dropped
Time/build: 2-6 weeks (software)
Time/increment: 6-18 months (platform)
April 2008
©USC-CSSE
51
University of Southern California
Center for Systems and Software Engineering
Case 7: NDI-Intensive
•
•
•
•
•
•
•
•
•
•
•
Biggest risks: incompatible NDI; rapid change, business/mission
criticality; low NDI assessment and integration experience; supply chain
stakeholder incompatibilities
Example: Supply chain management
Size/complexity: Medium to high
Anticipated change rate (% per month): 0.3-3%
Criticality: Medium to very high
NDI support: NDI-driven architecture
Organization and personnel capability: NDI-experienced; medium to
g capability
p
y
high
Key Stage I activities: Thorough NDI-suite life cycle cost-benefit
analysis, selection, concurrent requirements and architecture definition
Key Stage II activities: Pro-active
Pro active NDI evolution influencing, NDI upgrade
synchronization
Time/build: 1-4 weeks (software)
Time/increment: 6
6-18
18 months (systems)
April 2008
©USC-CSSE
52
University of Southern California
Center for Systems and Software Engineering
Case 8: Hybrid Agile/Plan
Agile/Plan-Driven
Driven System
•
•
•
•
•
•
•
•
•
•
•
Biggest
gg
risks: large
g scale, high
g complexity,
p
y rapid
p change,
g mixed
high/low criticality, partial NDI support, mixed personnel capability
Example: C4ISR system
Size/complexity: Medium to very high
Anticipated change rate (% per month): Mixed parts; 1-10%
Criticality: Mixed parts; medium to very high
NDI support: Mixed
Mi d parts
Organization and personnel capability: Mixed parts
y Stage
g I activities: Full ICM;; encapsulated
p
agile
g in high
g
Key
changed; low-medium criticality parts (often HMI, external
interfaces)
y Stage
g II activities: Full ICM,, three-team incremental
Key
development, concurrent V&V, next-increment rebaselining
Time/build: 1-2 months
Time/increment: 9-18 months
April 2008
©USC-CSSE
53
University of Southern California
Center for Systems and Software Engineering
Case 9: Multi-Owner System of Systems
•
Biggest risks: all those of Case 8 plus
– Need to synchronize, integrate separately-managed, independently-evolving
systems
t
– Extremely large-scale; deep supplier hierarchies
– Rapid adaptation to change extremely difficult
•
•
•
•
•
•
•
•
•
Example: Net-centric military operations or global supply chain
management
Size/complexity: Very high
Anticipated change rate (% per month): Mixed parts; 1-10%
Criticality: Very high
NDI support: Many NDIs; some in place
Organization and personnel capability: Related experience, medium to
high
K St
Key
Stage I activities:
ti iti
Full
F ll ICM;
ICM extensive
t
i multi-owner
lti
t
teambuilding,
b ildi
negotiation
Key Stage II activities: Full ICM; large ongoing system/software
engineering
i
i effort
ff t
Time/build: 2-4 months
Time/increment:18-24 months
April 2008
©USC-CSSE
54
University of Southern California
Center for Systems and Software Engineering
Case 10: Family of Systems
•
Biggest risks: all those of Case 8 plus
– Need to synchronize,
y
integrate
g
separately-managed,
p
y
g
independentlyp
y
evolving systems
– Extremely large-scale; deep supplier hierarchies
– Rapid adaptation to change extremely difficult
•
•
•
•
•
•
•
•
•
Example: Medical device product line
Size/complexity: Medium to very high
Anticipated change rate (% per month): 1-3%
Criticality: Medium to very high
NDI support: Some in place
Organization and personnel capability: Related experience, medium
to high capability
Key Stage I activities: Full ICM; full stakeholder participation in
product line scoping; strong business case
Key Stage II activities: Full ICM; extra resources for first system,
version control, multi-stakeholder support
pp
Time/build: 1-2 months
Time/increment: 9-18 months
April 2008
©USC-CSSE
55
University of Southern California
Center for Systems and Software Engineering
Frequently Asked Question
• Q: Having all that ICM generality and then using the
decision table to come back to a simple model seems
like an overkill.
– If my risk patterns are stable, can’t I just use the special case
indicated by the decision table?
• A: Yes,, you
y can and should – as long
g as your
y
risk
patterns stay stable. But as you encounter change,
the ICM helps you adapt to it.
– And it helps you collaborate with other organizations that
may use different special cases.
April 2008
©USC-CSSE
56
University of Southern California
Center for Systems and Software Engineering
Tutorial Outline
•
•
•
•
Motivation
Overview of ICM
ICM process decision table
G id
Guidance
and
d examples
l for
f using
i
the
th ICM
process decision table
• Group Exercise: Case study using
ICM p
process decision table
• Team Exercise: Applying ICM to 10 cases
April 2008
©USC-CSSE
57
University of Southern California
Center for Systems and Software Engineering
Case Study: Regional Area Crisis
Response System (RACRS)
Net-Centric
- Centric SoS
Connectivity
Net
April 2008
©USC-CSSE
58
University of Southern California
Center for Systems and Software Engineering
RACRS Key Features
• Goals:
– Minimize impacts of area crises
– Contain potential losses
• Ability to coordinate responses to regional area crises
–
–
–
–
Classify type of crisis
Alert appropriate organizations
Alert/evacuate public
Id tif and
Identify
d manage needed
d d resources
•
•
•
•
•
Fire trucks
Airplanes
Helicopters
Robots/remotely controlled vehicles
Medical supplies/special treatment or isolation facilities
• Request and coordinate support from other agencies: state,
Federal, or other regional areas
• Strategic partnership with local news stations for live video feeds
• Support crisis management activities in other regions
April 2008
©USC-CSSE
59
University of Southern California
Center for Systems and Software Engineering
RACRS Issues and Risks
•
•
•
•
•
Incompatible interfaces between existing systems
COTS products available to support interconnectivity,
interconnectivity but have
not been used at this level (potential scalability issues)
Police and fire departments currently have on-going projects to
i t
integrate
t the
th police,
li
fire
fi department,
d
t
t and
d 911 systems
t
Limited local budgets to modify other existing systems
p
for related State and Federal
Little or no modifications expected
systems but expectations that these will evolve
– Potential impacts with interfaces to other regional area systems
•
•
•
•
Federal funds available if system implemented within the next 5
years
Both County Board of Supervisors and City Council need to
approve plans
l
and
d budgets
b d t
Citizen privacy and security issues
pp g authorities during
g crises: local,, state,, and
Potential overlapping
Federal
April 2008
©USC-CSSE
60
University of Southern California
Center for Systems and Software Engineering
RACRS Desired Characteristics
•
•
•
•
•
Integrate existing legacy systems together using a net
net-centric
centric
architecture that includes wireless, mobile networks for mobile units
and existing networks for fixed Control Center connectivity
Must work across coastal plain, intermediate mountain range, and
low-lying desert area on far side of mountain range
As part of this effort, the city and county planning and land use
organizations would like to replace their location tracking systems
with
ith a new system
t
that
th t is
i based
b
d on city/county
it /
t records
d and
d nott the
th
more general purpose map programs/ databases typically provide by
Geographic Information System (GIS) vendors
No other new system components planned for the early versions of
this SoS
Build on existing connectivity
– Some sort of connectivity exists between
• City police, sheriff’s, 911, and ambulance systems
• Jail information system and state and Federal agencies
– Most other system components are relatively closed
closed, independent
systems
April 2008
©USC-CSSE
61
University of Southern California
Center for Systems and Software Engineering
Elements to Think About
• Key mission scenarios
– Earthquake in Southern California
– Hazardous material spill on freeway during rush hour
• Feature, service, or crisis p
priorities—how to define early
y
increments
• Candidate architecture(s) and increment definitions: What can
be defined as “independent
p
projects”?
p j
How does this impact
p
cost and schedule?
• What elements require early simulation/prototyping/evaluation?
• Risk management: What key risks should be addressed first?
• Where to be agile? Where to be plan-driven?
• Types of oversight for various types of component system
providers
id
– Strategic partners
– Vendors
− Suppliers
− Developers
• Any assumptions?
April 2008
©USC-CSSE
62
University of Southern California
Center for Systems and Software Engineering
E
Exercise
i 1
1 Develop
1.
D
l ICM model
d l for
f RACRS ((as a group))
–
–
–
Initial increments
Planned prototypes,
protot pes simulations,
sim lations and models
Reviews
2 Identify elements to review in initial feasibility
2.
assessments (integrate lists from tutorial
pa t c pa ts)
participants)
3. List candidate key risks to monitor in early
increments ((integrate
g
lists from tutorial
participants)
April 2008
©USC-CSSE
63
University of Southern California
Center for Systems and Software Engineering
RACRS ICM Characteristics
•
•
•
•
•
•
•
•
Size/complexity:
Anticipated change rate (% per month):
Criticality:
NDI support:
Key Stage I activities:
Key Stage II activities:
Time/build:
Time/increment:
April 2008
©USC-CSSE
64
University of Southern California
Center for Systems and Software Engineering
RACRS Initial Increment
April 2008
©USC-CSSE
65
University of Southern California
Center for Systems and Software Engineering
RACRS Planned Prototypes
Prototypes,
Simulations, and Models
April 2008
©USC-CSSE
66
University of Southern California
Center for Systems and Software Engineering
RACRS Planned Reviews
April 2008
©USC-CSSE
67
University of Southern California
Center for Systems and Software Engineering
Tutorial Outline
•
•
•
•
Motivation
Overview of ICM
ICM process decision table
G id
Guidance
and
d examples
l for
f using
i
the
th ICM
process decision table
• Group Exercise: Case study using ICM
process decision table
• Team Exercise: Applying ICM to 10
cases
April 2008
©USC-CSSE
68
University of Southern California
Center for Systems and Software Engineering
E
Exercise
i 2 (Time
(Ti
Permitting)
P
itti )
1 Break
1.
B
k iinto
t groups off 4
4-5
5 people
l
2. Select ICM Special Case
3. Develop an example system description that
would use the selected case
4 For
4.
F example
l system,
t
identify
id tif
–
–
–
–
Initial increments
Planned prototypes,
protot pes simulations,
sim lations and models
Reviews
Key risks to monitor in early increments
5. Each team presents brief summary of
decisions to group
April 2008
©USC-CSSE
69
University of Southern California
Center for Systems and Software Engineering
Backup Charts
Motivation
ot at o
Special Cases of the ICM—Table Details
University of Southern California
Center for Systems and Software Engineering
2007 Study Scope and Context
• Statement of Work
– Evaluate current DoD SysE, SwE processes and standards
• Primarily SEP Guide, DAG Ch. 4 (SysE), DoDI 5000.2
• Identify deltas,
deltas issue areas,
areas recommendations
– Evaluate ICM for integration applicability
– Evaluate relevant DoD g
guidance, education, and training
g
• Provide recommendations based on evaluations
• Context: current and future DoD S&SE challenges
– Multi-owner, multi-mission systems of systems
– Rapid pace of change
– Always-on,
Always-on never-fail systems
– Need to turn within adversaries’ OODA loop
• Observe, orient, decide, act
– Within asymmetric adversary constraints
April 2008
©USC-CSSE
71
University of Southern California
Center for Systems and Software Engineering
Current SysE Guidance: Outdated Assumptions
Example: SysE Plan Preparation Guide
Sometimes OK for hardware; generally not for software
•
•
•
•
•
•
•
•
•
An omniscient authority pre-establishes the requirements, preferred system
concept etc
concept,
etc. (A4.1,
(A4 1 4
4.2,
2 4
4.4)
4)
Technical solutions are mapped and verified with respect to these (A4.1,
4.2, 4.4)
Th and
They
d technology
t h l
do
d nott change
h
very often
ft (A4.2,
(A4 2 4
4.5,
5 6
6.2)
2)
– Emphasis on rigor vs. adaptability
The p
program
g
has stable and controllable external interfaces ((A4.5))
Project organization is function-hierarchical and WBS-oriented (A3.1, 3.2)
All requirements are equally important (A4.2)
Reviews event/product
event/product-based
based vs.
vs evidence-based
evidence based (A5.2)
(A5 2)
The program’s critical path can be defined up front (A6.1)
Can achieve optimum readiness at minimum life-cycle cost (A6.5)
– No slack for contingencies
April 2008
©USC-CSSE
72
University of Southern California
Center for Systems and Software Engineering
System/Software Architecture Mismatches
- Maier, 2006
• System Hierarchy
• Software Hierarchy
– Part-of relationships;
no shared parts
– Uses relationships;
layered multi-access
– Function-centric; single
data dictionary
– Data-centric; classobject data relations
– Interface dataflows
– Interface p
protocols;
concurrency challenges
– Dynamic functionalphysical migration
– Static functionalphysical allocation
April 2008
©USC-CSSE
73
University of Southern California
Center for Systems and Software Engineering
Examples of Architecture Mismatches
• Fractionated,
Fractionated incompatible sensor data management
…
…
Sensor 1
SDMS1
…
Sensor 2
Sensor 3
Sensor n
SDMS2
SDMS3
SDMSn
• “Touch Football” interface definition earned value
– Full earned value taken for defining interface dataflow
– No earned value left for defining interface dynamics
• Joining/leaving network, publish-subscribe, interrupt
handling, security protocols, exception handling, mode
transitions
– Result: all green EVMS turns red in integration
April 2008
©USC-CSSE
74
University of Southern California
Center for Systems and Software Engineering
Effect of Software Underrepresentation
•Software risks discovered too late
•Slow, buggy change management
•Recent large project reorganization
Original (WBS-based)
New
PM
PM
C4ISR
Sys Engr
Platforms
C4ISR
Software
Sys Engr
SW
Sensors
Networks
WMI
Sensors
SW
SW
SW
SW
April 2008
Networks
©USC-CSSE
SW
75
University of Southern California
Center for Systems and Software Engineering
Young Memo: Prototyping and Competition
• Discover issues before costly SDD phase
– Producing detailed designs in SDD
– Not
N t solving
l i myriad
i d technical
t h i l issues
i
• Services and Agencies to produce competitive
prototypes through Milestone B
– Reduce technical risk, validate designs and cost estimates,
evaluate manufacturing processes, refine requirements
• Will reduce time to fielding
– And enhance govt.-industry teambuilding, SysE skills,
attractiveness
tt
ti
to
t nextt generation
ti off technologists
t h l i t
• Applies to all programs requiring USD(AT&L) approval
– Should be extended to appropriate programs below ACAT I
April 2008
©USC-CSSE
76
University of Southern California
Center for Systems and Software Engineering
I
Implementing
l
ti the
th Young
Y
Memo
M
• Need way of assuring continuity of funding, but with off
offramps
– Prototyping and Competition Plan
– Incremental off-ramp milestones
• Need criteria for assessing feasibility to proceed
– Avoid
A id overfocus
f
on prototyping
t t i
• Feasible, scalable technology and management infrastructure
q
budget,
g schedule, critical skills
• Adequate
• Key stakeholders committed to proceed
– Shortfalls in evidence are risks, need risk management plans
• ICM provides
id desired
d i d processes and
d criteria
it i
– Anchor Point milestones and Feasibility Evidence deliverable
– Incremental phase entry and exit criteria
April 2008
©USC-CSSE
77
University of Southern California
Center for Systems and Software Engineering
Backup Charts
Motivation
Special Cases of the ICM—Table Details
University of Southern California
Center for Systems and Software Engineering
Case 1: Use NDI
•
•
•
•
•
•
•
•
•
•
Example: Small accounting system
Size/complexity: Low
Anticipated change rate (% per month): Low
Criticality: Low
NDI support: Complete
Organization and personnel capability: NDI-experienced
Key Stage I activities: Acquire NDI
Key Stage II activities: Use NDI
Time/build: Driven by time to initialize/tailor NDI
Time/increment: Driven by NDI upgrades
April 2008
©USC-CSSE
79
University of Southern California
Center for Systems and Software Engineering
Case 2: Pure Agile Methods
•
•
•
•
•
•
•
•
•
•
Example: E-services
Size/complexity: Low
Anticipated change rate (% per month): 1
1-30%
30%
Criticality: Low to medium
pp
Good; in place
p
NDI support:
Organization and personnel capability: Agile-ready, medium
to high capability
K St
Key
Stage I activities:
ti iti
Skip
Ski Valuation
V l ti and
d Architecting
A hit ti
phases
Key
y Stage
g II activities: Scrum plus
p
agile
g methods of choice
Time/build: Daily
Time/increment: 2-6 weeks
April 2008
©USC-CSSE
80
University of Southern California
Center for Systems and Software Engineering
Case 3: Architected Agile
•
•
•
•
•
•
•
•
•
•
Example: Business data processing
Size/complexity: Medium
Anticipated change rate (% per month): 1-10%
Criticality: Medium to high
NDI support: Good
Good, most in place
Organization and personnel capability: Agile-ready, medium
to high capability
Key Stage I activities: Combined Valuation and Architecting
phase, complete NDI preparation
Key Stage II activities: Architecture-based scrum of scrums
Time/build: 2-4 weeks
Time/increment: 2-6 months
April 2008
©USC-CSSE
81
University of Southern California
Center for Systems and Software Engineering
Case 4: Formal Methods
•
•
•
•
•
•
•
•
•
•
Example: Security kernel or safety-critical LSI chip
Size/complexity: Low
Anticipated change rate (% per month): 0.3%
Criticality: Extra high
NDI support: None
Organization and personnel capability: Strong formal
methods experience
Key Stage I activities: Precise formal specification
Key Stage II activities: Formally-based programming
language; formal verification
Time/build: 1-5 days
Time/increment: 1-4 weeks
April 2008
©USC-CSSE
82
University of Southern California
Center for Systems and Software Engineering
Case 5: Hardware Component with
Embedded Software
•
•
•
•
•
•
•
•
•
•
Example: Multi-sensor control device
Size/complexity: Low
Anticipated change rate (% per month): 0.3-1%
Criticality: Medium to very high
NDI support: Good,
Good in place
Organization and personnel capability: Experienced;
medium to high capability
Key Stage I activities: Concurrent hardware and software
engineering; CDR-level ICM DCR
Key Stage II activities: IOC Development,
Development LRIP
LRIP,FRP,
FRP
concurrent version N+1 engineering
Time/build: 1-5 days (software)
Time/increment: Market-driven
April 2008
©USC-CSSE
83
University of Southern California
Center for Systems and Software Engineering
C
Case
6
6: IIndivisible
di i ibl IOC
•
•
•
•
•
•
•
•
•
•
Example: Complete vehicle platform
Size/complexity: Medium to high
Anticipated change rate (% per month): 0.3-1%
Criticality: High to very high
NDI support: Some in place
O
Organization
i ti and
d personnell capability:
bilit Experienced,
E
i
d
medium to high capability
Key
y Stage
g I activities: Determine minimum-IOC likely,
y,
conservative cost; Add deferrable software features as risk
reserve
Key Stage II activities: Drop deferrable features to meet
conservative cost; Strong award fee for features not dropped
Time/build: 2-6 weeks (software)
Time/increment: 6-18 months (platform)
April 2008
©USC-CSSE
84
University of Southern California
Center for Systems and Software Engineering
C
Case
7
7: NDI
NDI-Intensive
I t
i
•
•
•
•
•
•
•
•
•
•
Example: Supply chain management
Size/complexity: Medium to high
Anticipated change rate (% per month): 0.3-3%
Criticality: Medium to very high
NDI support: NDI-driven architecture
O
Organization
i ti and
d personnell capability:
bilit NDI-experienced;
NDI
i
d
medium to high capability
Key
y Stage
g I activities: Thorough
g NDI-suite life cycle
y
costbenefit analysis, selection, concurrent requirements and
architecture definition
Key Stage II activities: Pro
Pro-active
active NDI evolution influencing
influencing,
NDI upgrade synchronization
Time/build: 1-4 weeks (software)
Time/increment: 6-18 months (systems)
April 2008
©USC-CSSE
85
University of Southern California
Center for Systems and Software Engineering
Case 8: Hybrid Agile/Plan-Driven System
•
•
•
•
•
•
•
Example:
E
ample C4ISR s
system
stem
Size/complexity: Medium to very high
Anticipated change rate (% per month): Mixed parts; 1-10%
1 10%
Criticality: Mixed parts; medium to very high
NDI support: Mixed parts
Organization and personnel capability: Mixed parts
Key Stage I activities: Full ICM; encapsulated agile in high
changed; low-medium criticality parts (often HMI,
HMI external
interfaces)
• Key Stage II activities: Full ICM, three-team incremental
d
development,
l
concurrent V&V,
V&V next-increment
i
rebaselining
b
li i
• Time/build: 1-2 months
• Time/increment: 9-18 months
April 2008
©USC-CSSE
86
University of Southern California
Center for Systems and Software Engineering
Case 9: Multi-Owner System of Systems
•
•
•
•
•
•
•
•
•
•
Example: Net-centric military operations
Size/complexity: Very high
Anticipated change rate (% per month): Mixed parts; 1
1-10%
10%
Criticality: Very high
pp
Many
y NDIs; some in place
p
NDI support:
Organization and personnel capability: Related experience,
medium to high
K St
Key
Stage I activities:
ti iti
Full
F ll ICM;
ICM extensive
t
i multi-owner
lti
teambuilding, negotiation
Key
y Stage
g II activities: Full ICM; large
g ongoing
g g
system/software engineering effort
Time/build: 2-4 months
Time/increment:18 24 months
Time/increment:18-24
April 2008
©USC-CSSE
87
University of Southern California
Center for Systems and Software Engineering
Case 10: Family of Systems
•
•
•
•
•
•
•
•
•
•
Example: Medical device product line
Size/complexity: Medium to very high
Anticipated change rate (% per month): 1
1-3%
3%
Criticality: Medium to very high
pp
Some in place
p
NDI support:
Organization and personnel capability: Related experience,
medium to high capability
K St
Key
Stage I activities:
ti iti
Full
F ll ICM;
ICM full
f ll stakeholder
t k h ld participation
ti i ti
in product line scoping; strong business case
Key
y Stage
g II activities: Full ICM;; extra resources for first
system, version control, multi-stakeholder support
Time/build: 1-2 months
Ti /i
Time/increment:
t 9-18
9 18 months
th
April 2008
©USC-CSSE
88
University of Southern California
Center for Systems and Software Engineering
References - I
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Beck, K., Extreme Programming Explained, Addison Wesley, 1999.
Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”,
S t
Systems
E
Engineering
i
i
9(1),
9(1) pp. 1-19,
1 19 2006
2006.
Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of
Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004.
Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive
Systems of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006.
B h B.,
Boehm,
B and
d Lane,
L
J.,
J “Using
“U i
th
the ICM tto IIntegrate
t
t S
System
t
A
Acquisition,
i iti
Systems
S t
Engineering,
E i
i
and
d
Software Engineering,” CrossTalk, October 2007, pp. 4-9.
Boehm, B., and Lane, J., “A Process Decision Table for Integrated Systems and Software
Engineering,” Proceedings, CSER 2008, April 2008.
Boehm, B., “Future Challenges and Rewards for Software Engineers,” DoD Software Tech News,
October 2007
2007, pp
pp. 6-12
6-12.
Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
Boehm, B., Software Engineering Economics, Prentice Hall, 2000.
Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive
Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-26, 2001.
Carlock P
Carlock,
P., and J
J. Lane
Lane, “System
System of Systems Enterprise Systems Engineering,
Engineering the Enterprise
Architecture Management Framework, and System of Systems Cost Estimation”, 21st International
Forum on COCOMO and Systems/Software Cost Modeling, 2006.
Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2nd ed., 1999).
Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/,
006
2006.
Department of Defense (DoD), Instruction 5000.2, Operation of the Defense Acquisition System, May
2003.
Department of Defense (DoD), Systems Engineering Plan Preparation Guide, USD(AT&L), 2004.
Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System
Galorath, D., and Evans, M., Software Sizing, Estimation, and Risk Management, Auerbach, 2006.
Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976.
April 2008
©USC-CSSE
89
University of Southern California
Center for Systems and Software Engineering
References -II
II
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Highsmith, J., Adaptive Software Development, Dorset House, 2000.
I t
International
ti
l Standards
St d d Organization,
O
i ti
Information
I f
ti
Technology
T h l
Software
S ft
Life
Lif Cycle
C l Processes,
P
ISO/IEC
12207, 1995
ISO, Systems Engineering – System Life Cycle Processes, ISO/IEC 15288, 2002.
Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,”
Proceedings, ISPA 5, April 1983, pp. 88-92.
K
Krygiel,
i l A.,
A Behind
B hi d the
th Wizard’s
Wi d’ Curtain;
C t i CCRP P
Publication
bli ti
S
Series,
i
J
July,
l 1999
1999, p. 33
Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator
Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007.
Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of
IEEE Systems, Man, and Cybernetics Conference, 2005.
Lientz B.,
Lientz,
B and Swanson,
Swanson E.B.,
E B Software Maintenance Management,
Management Addison Wesley
Wesley, 1980
1980.
Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development",
USC CSSE Technical Report USC-CSSE-2006-623, 2006.
Maier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp
267-284).
Maier M
Maier,
M., “System
System and Software Architecture Reconciliation,
Reconciliation ” Systems Engineering 9 (2)
(2), 2006
2006, pp
pp.
146-159.
Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software
Engineering Institute, 2006.
Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New
Look, National Academy Press, 2007.
Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,”
IEEE Trans SW Engr., July 1978, pp. 345-361.
Rechtin, E. Systems Architecting, Prentice Hall, 1991.
Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC
g
g Systems
y
and Software Engineering
g
g Workshop,
p,
Integrating
http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007.
April 2008
©USC-CSSE
90
University of Southern California
Center for Systems and Software Engineering
List of Acronyms
B/L
C4ISR
CD
CDR
COTS
DCR
DI
DoD
ECR
EVMS
FCR
FED
FMEA
FRP
GAO
GUI
April 2008
Baselined
Command, Control, Computing, Communications, Intelligence, Surveillance,
Reconnaissance
Concept Development
Critical Design Review
Commercial Off-the-Shelf
Development Commitment Review
Development Increment
Department of Defense
Exploration Commitment Review
Earned Value Management System
Foundations Commitment Review
p
Feasibilityy Evidence Description
Failure Modes and Effects Analysis
Full-Rate Production
y Office
Government Accountability
Graphical User Interface
©USC-CSSE
91
University of Southern California
Center for Systems and Software Engineering
List of Acronyms
HMI
HSI
HW
ICM
IOC
IRR
IS&SE
LCO
LRIP
MBASE
NDI
NRC
OC
OCR
OO&D
OODA
O&M
April 2008
(continued)
Human-Machine Interface
y
Interface
Human-System
Hardware
Incremental Commitment Model
Initial Operational Capability
Inception Readiness Review
Integrating Systems and Software Engineering
Life Cycle Objectives
Low-Rate Initial Production
Model-Based Architecting and Software Engineering
Non-Developmental Item
National Research Council
Operational Capability
Operations Commitment Review
Observe, Orient and Decide
Observe, Orient, Decide, Act
Operations and Maintenance
©USC-CSSE
92
University of Southern California
Center for Systems and Software Engineering
List of Acronyms
PDR
PM
PR
PRR
RUP
SoS
SoSE
SSE
SW
SwE
y
SysE
Sys Engr
S&SE
USD (AT&L)
VCR
V&V
WBS
WMI
April 2008
(
(continued)
ti
d)
Preliminary Design Review
Program Manager
Public Relations
Product Release Review
Rational Unified Process
System of Systems
System of Systems Engineering
y
and Software Engineering
g
g
Systems
Software
Software Engineering
Systems
y
Engineering
g
g
Systems Engineer
Systems and Software Engineering
Under Secretary of Defense for Acquisition, Technology, and Logistics
Validation Commitment Review
Verification and Validation
Work Breakdown Structure
Warfighter-Machine Interface
©USC-CSSE
93