Controlling Reqts Risk
Identifying & Controlling
Reqts Risk
1. Reqts can be a Problem
2. Identifying Reqts Hazards
A Survey of Safety Tactics
3. Reqts Risk Mgmt
4. Safety Tactics
David Gelperin
ClearSpecs Enterprises
[email protected]
ClearSpecs
Education
ClearSpecs
Education
1
1
2
Why Bother with Reqts?
Reqts Scope can be Vague
We find
– reqts receivers, both present and
future (e.g., designers, developers,
testers, lawyers, writers, maintainers)
– reqts providers (e.g., customers,
developers, testers, regulators)
among project stakeholders.
1. Customer Needs
System Reqts
Architecture and Interface Specs
(Design Constraints)
Project Reqts
Primary purpose of reqts information
is to inform reqts receivers about needs
and constraints so they can do their jobs
effectively and efficiently.
efficiently.
2. Musts or
Musts & Wants or
Musts, Wants, & Likes
Reqts are unnecessary
unnecessary,, if reqts receivers
already know the details of needs and
constraints.
Your reqts process needs improvement
improvement,,
if your reqts receivers don’t get accurate or
sufficient information or are receiving far
too much.
ClearSpecs
Education
ClearSpecs
Education
3
2
4
We are analog beings trapped in a
digital world [and we did it to ourselves]
Yin Yang of Reqts Dev
Technical Aspects
We have constructed a set of artificial devices
that are very much NOT in our own image
-- Don Norman
(30%) – Neat
Analyzing, Specifying
Imprecise
Inconsistent
Incomplete
Human Aspects
Needs, Desires,
& Preferences
(70%) – Messy
Teaming, Discovering, Understanding
Communicating, Negotiating
Implemented
Systems
[Psychology, Politics, Sociology]
Precise
Consistent
Complete
ClearSpecs
Education
ClearSpecs
Education
5
3
6
Then a Miracle Occurs (Rarely)
Sooner or Later
Fuzzy Concepts
(from the customer)
Precision always appears in
systems development, but it
can be painful
System Development
The issue is not if it will appear,
but who provides it, when
when,,
and who understands it
Acceptable Precision
(electronics, data, tests)
Poor definition of “acceptable” is a
leading cause of unacceptable systems
ClearSpecs
Education
ClearSpecs
Education
7
4
8
Reqts Problems
Stakeholders differ in --
Poor reqts and inadequate verification
and validation
Missing architecturallyarchitecturally-significant reqts
Inappropriate constraints
Over emphasis on simplistic modeling
Excessive volatility
Inadequate reqts mgmt
including scope creep
Untraced sources and allocations
Inadequate process and tool support
Unprepared reqts analysts
Don Firesmith 2007
Inadequate user involvement
Inadequate senior mgmt support
Unrealistic expectations
Level of commitment &
constructive involvement
Understanding, assumptions, &
expectations (domain & technical)
Ability to learn & make decisions
Willingness to explore
Tolerance for details &
compromise
Styles of communication
Team building and negotiation
may be the critical challenges
i.e. you may be herding cats
Standish Group
(Chaos Report ) 1995
ClearSpecs
Education
ClearSpecs
Education
9
5
10
Info can be Unreliable
Customers may not know exactly what
they want
Customers can not tell all they know
Customers make mistakes
Few have experience with precise
human--understandable detail
human
We can’t fully understand abundant
detail
Some details can only be discovered
during build
We make mistakes
Things change
Change may be rapid
Root Cause of
Many Reqts Problems
LOU – Lack Of Understanding
Recognized vs. Unrecognized LOU
U LOU is very dangerous
ClearSpecs
Education
ClearSpecs
Education
11
6
12
Reqts can be a Problem
Biggest Reqts Problem
Communication is hard:
Imprecision is the rule,
rule, not the
exception. Devil’s in the many
many details.
Checking is hard: “Best practice”
reviews miss 1/2 the defects
(Gilb
Gilb))
Deep understanding is acquired
in layers (no eureka moment -it takes time)
Human factors get in the way
None of this will change
ClearSpecs
Education
ClearSpecs
Education
13
7
14
Hazard
Controlling Reqts Risk
1. Reqts can be a Problem
Hazard
2. Identifying Reqts Hazards
3. Reqts Risk Mgmt
Adverse
Event
Harm(s)
Adverse
Event
Harm(s)
Adverse
Event
Harm(s)
4. Safety Tactics
Synonyms for “harm”:
“harm”: loss, injury, damage
Hazard
(n):
potential cause of
significant harm
Synonyms:: danger, peril
Synonyms
ClearSpecs
Education
ClearSpecs
Education
15
8
16
Reqts Hazards
Sid & Lou
Reqts Hazards
Hazards:: hazardous to reqts
and hazards in reqts
Application
Defect-free
DefectFailures
Stakeholder
Process
Resource
Complexity
Reqts
Defect-based
DefectFailures
Change
“raising” children includes “protecting”
from dangers to them & dangers from them
ClearSpecs
Education
ClearSpecs
Education
17
9
18
Hazardous to Reqts
Application Hazards (to)
Potential
otential for seriously defective
reqts info rooted in:
Application is:
application characteristics –
Application Hazards
– unfamiliar
incorrect information (assumptions),
inadequate understanding or
involvement, inappropriate behavior –
Stakeholder Hazards
inadequate reqts dev or mgmt
processes – Process Hazards
inadequate time or tools –
Resource Hazards
confusing project, problem, or
solution – Complexity Hazards
rapidly changing info –
Change Hazards
ClearSpecs
Education
– complex
– missionmission-critical
ClearSpecs
Education
19
10
20
Spec Stakeholders
Stakeholder Hazards (to)
Providers (& Reviewers)
Customers
Outsourcers
Users
Application Experts
Quality Experts (Safety, Security, Usability)
System Testers
Quality Assurance
Auditors
Lawyers
Both direct and indirect
(bosses, life partners) stakeholders
behavior
+
1
2
3
Specifiers
Business Analysts
Product Managers
System Architects
Requirements Engineers
Technical Writers
• information
• understanding
• involvement
-
+
4
Receivers
System Houses
Project Managers
System Designers
System Developers
System Testers
Documenters
It’s not what we don’t know that’s the problem,
it’s what we know that ain’t so.
Josh Billings
ClearSpecs
Education
21
11
ClearSpecs
Education
22
Process Hazards (to)
scope too narrow or too broad
inadequate discovery
incorrect derivation
inadequate attention to nonfunctional reqts
confusing responsibilities
context independent (i.e. rigid) tasks
ineffective communication or negotiation
too much or too little specification
inadequate verification and validation
creeping scope
ineffective change or info management
inadequate defect data collection
ClearSpecs
Education
Ambiguity
Two hunters are out in the woods when
one of them falls to the ground. He doesn't
seem to be breathing and his eyes are
rolled back. The other hunter pulls out his
cell phone and calls 911. He gasps to the
operator: “My friend is dead! What can I
do?”
The operator, in a calm voice says: “Just
take it easy. I can help. First, let's make
sure he's dead.” There is silence and then
a shot.
The hunter’s voice comes back on the line.
“OK, now what?“
ClearSpecs
Education
23
12
24
Context
Resource Hazards (to)
A woman sues a man for defamation of
character charging that he called her a pig.
The man is found guilty and made to pay
damages.
critical stakeholders overlooked,
unavailable, or underinvolved
After the trial, he asks the judge, “Does this
mean that I can no longer call Ms. Harding
a pig?”
The judge says, “That is correct.”
“And does this mean that I can’t call a pig
Ms. Harding?”
“No,” says the judge, “you are free to call a
pig Ms. Harding. There is no crime in that.”
arbitrary schedule as hard
requirement
inadequate time for analysis and
learning
insufficient tool support
The man look Ms. Harding in the eye and
says, “Good afternoon, Ms. Harding.”
ClearSpecs
Education
ClearSpecs
Education
25
13
26
Change Hazards (to)
Complexity Hazards (to)
social (stakeholder group)
individual reqts unstable
application (problem)
scope expands or refocuses
design (solution)
rate of change decreases too
slowly
significant change comes late
Expect change due to
learning and human error
ClearSpecs
Education
ClearSpecs
Education
27
14
28
Hazards in Reqts
Who’s in Peril “from”?
Potential
otential for
significant problems in
implementing a suite of defect
defect--free
reqts – Defect
Defect--free Hazards
significant harm or loss rooted in
defective reqts – Defect
Defect--based
Hazards
Danger
Will Robinson
ClearSpecs
Education
Project Estimator
Product Architect
Functional Designer
Product Tester
Product Developer
Documentation Writer
Product User
Bystander
Product Customer
ClearSpecs
Education
29
15
30
Defect-free Hazards
Task-adequate Reqts
overwhelming volume,
complexity, or unfamiliarity
misunderstanding
design challenges
e.g. due to integration
schedule constraints
budget constraints
ClearSpecs
Education
Task-adequate means reqts
Taskreceivers can perform their tasks
without having to cope with
major defects i.e., unclear,
unnecessary, inaccurate or
insufficient information.
Task-inadequate requirements
Tasklead to uncoordinated coping
behavior and inevitable
misunderstandings..
misunderstandings
ClearSpecs
Education
31
16
32
Defect-based Hazards
Single - alone
–
–
–
–
–
–
–
–
–
–
–
unnecessary
overover-specified
compound
incorrect (e.g., incompatible)
incomplete
ambiguous
imprecise
unclear
infeasible
unverifiable
nonconformant
Every reqts process has a footprint
Defects and failures are:
(caused by process or project
constraints [too little time])
– natural
(caused by inexperience
or human limitations)
duplicated
conflicting
inconsistent
unlabelled
untraced
– accidental
(caused by human error)
1. What are your major risk factors?
2. What is your unmanaged
footprint likely to be?
Suite
–
–
–
–
– systemic
Single – in a suite
–
–
–
–
–
Footprints
insufficient
infeasible
disorganized
nonconformant
ClearSpecs
Education
ClearSpecs
Education
33
17
34
Reqts-Related Failures
Controlling Reqts Risk
budget and schedule overruns
system failures
poor interfaces
inappropriate architecture
unnecessary complexity
unnecessary functionality
project cancellation
system abandonment
lost opportunities
ClearSpecs
Education
1. Reqts can be a Problem
2. Identifying Reqts Hazards
3. Reqts Risk Mgmt
4. Safety Tactics
ClearSpecs
Education
35
18
36
Risk
Hazard
Risk Reduction
Adverse
Event
Harm(s)
Adverse
Event
Harm(s)
Adverse
Event
Harm(s)
Since risk can be measured by:
Σ ( cLhazard
X
cLevent X Eharm )
Risk can be reduced by reducing:
1. expected harms i.e., Eharm
Risk
(n):
potential harm
2. likelihoods of hazards causing
adverse events i.e., cLevent
especially those associated
with the largest expected harms
Synonyms:: jeopardy
Synonyms
Risk can be measured by:
Σ ( cLhazard
X
cLevent X Eharm )
3. likelihoods of hazards i.e., cLhazard
especially those associated
with the largest expected harms
Expected harm can be expressed as:
# of deaths, lost $, # of serious defects
ClearSpecs
Education
ClearSpecs
Education
37
19
38
Risk Mgmt is
Project Mgmt for Adults
Risk Mgmt
An almost defining characteristic of
adulthood is a willingness to confront
the unpleasantness of life,
life, from the
niggling to the cataclysmic. … You
have to face up to unpleasant
realities. That’s what it means to be
a growngrown-up.
Risk Mgmt entails minimizing
harm, by accurately forecasting
danger,, assessing its degree,
danger
and taking steps to eliminate or
reduce it.
Effective risk management is
inherently iterative i.e., you
monitor, learn, and adapt your
strategy as you get smarter.
DeMarco & Lister
ClearSpecs
Education
ClearSpecs
Education
39
20
40
How (Not) To Forecast Danger
System Safety Engineering
I can see clearly now.
Mr. Magoo
Systems safety
Most projects run in Magoo mode
◊ emphasizes building in safety
starting with requirements
◊ deals with systems as a whole
◊ takes a larger view of hazards
◊ emphasizes analysis of new
situations
◊ emphasizes qualitative approaches
◊ recognizes tradeoffs and conflicts
The sky is falling!
The sky is falling!
Henny Penny
Worry smart
I’ve seen every possible ending.
None of them is good for you.
Cris Johnson
We apply similar principles to
requirements development
Develop 20/20 Foresight
ClearSpecs
Education
ClearSpecs
Education
41
21
42
SEI Continuous Risk Mgmt
What Can Go Wrong?
For safety critical products, the first questions to
ask are:
“How many ways can we harm product users?”
users?”
“How do we convince ourselves (and others) that
this either can’t happen or is very unlikely when
our “simple” directions are followed
followed?”
?”
Process includes:
• identify risk factors i.e., risks, indicators, and root
causes
These are good questions to ask on most projects
• classify, assess and prioritize factors
along with:
• develop and execute a risk management strategy
“What should I be afraid of on this project?”
project?” and
• monitor factors and strategy
• adapt strategy, journaling causes and
question?”
“Who can help me answer this question?”
adjustments
• communicate activities and results
ClearSpecs
Education
ClearSpecs
Education
43
22
44
Especially in IT
Context of Risk Mgmt
Risk management is the
least practiced of all project
management disciplines across
all industry sectors, and nowhere
less applied than the IT industry.
Risky Culture or Risky Project Mgmt
Culture or Mgmt may be:
• risk denying
• neutral
• safety-oriented
Robert Charette
quoting a PMI study
Risk Mgmt may be:
not/poorly/well planned and
not/poorly performed
or
not/poorly/well planned and
well performed
ClearSpecs
Education
Some cultures reward firefighting,
even when the fighters caused the fires
ClearSpecs
Education
45
23
46
Project Risk Factors
Balanced Risk Mgmt
People
Sponsors
Management
Support
Skills
Head count
Under--performance
Under
Politics
Conflict
Some risks
– can’t be reduced
– aren‘t worth reducing
– are worth reducing
System
Requirements
Development
Interaction with environment
Innovation
No reqts. risk mgmt is not balanced
Project
Schedule
Budget
Change
Staff
Goals
Implementation
ClearSpecs
Education
ClearSpecs
Education
47
24
48
Who’s Responsible?
Reqts Risk Mgmt
RRM leadership candidates:
– Product Mgr
– Development Director
– Project Mgr or Lead
– Reqts Mgr or Analyst
– Quality Assurance Staff
– Process Improvement Lead
– Verification and Test Lead
Reqts Risk Mgmt ((RRM
RRM)) entails
minimizing harm to and from
reqts,, by accurately
reqts
forecasting danger, assessing
its degree, and taking steps to
eliminate or reduce it.
it.
ClearSpecs
Education
ClearSpecs
Education
49
25
50
Categories of Safety Tactics
Controlling Reqts Risk
1. Reqts can be a Problem
2. Identifying Reqts Hazards
3. Reqts Risk Mgmt
4. Safety Tactics
1.
Monitor reqts and project feasibility
2.
Avoid reqts
reqts--related mistakes
3.
Minimize reqts
reqts--based problems
4.
Detect reqts defects
5.
Monitor reqts and risk status
6.
Mitigate impact of reqtsreqts-based problems
7.
Staff for safer development
8.
Plan for reqts risk mgmt
9.
Prepare for reqts risk mgmt
10.
Compensate for reqtsreqts-based problems
Safety Strategies are Inelegant
ClearSpecs
Education
ClearSpecs
Education
51
26
52
1. Monitor Reqts and
Project Feasibility
Reconfirm reality and
satisfiability of stated needs
Triage and prioritize reqts
Manage customer expectations
Reconfirm project feasibility or
reduce scope
Identify and resolve conflicts
early
Commit incrementally
Triage and Prioritize
Triage (create 3 groups for)
reqts to separate those needing
attention from those that don’t
either because they are OK or
because nothing can be done
e.g. infeasible.
Prioritize selected reqts by
– customer and user needs
– feature clusters
– architectural and functional
dependencies
– risks e.g. satisfiability
– costs
Monitor feasibility to assure you
aren’t rearranging deck chairs.
chairs.
ClearSpecs
Education
ClearSpecs
Education
53
27
54
Identify Conflicts
Project Feasibility
Between:
sufficient development resources?
(staff, schedule, budget)
• stakeholders
• requirements – nonnon-functional
sufficient knowledge resources?
(sources, time)
• priorities – differ by stakeholder
or too many highs
rate of development greater than
rate of reqts change?
ClearSpecs
Education
• implementations – undesired
interactions
ClearSpecs
Education
55
28
56
Look for Failure
But, If You See It
Signs of potential project failure
lack of senior mgmt support
under--involved customers
under
cross--functional stakeholders
cross
– conflict
– fear
– ignorance
– incompetence
Some messages are hard to
deliver
People’s capacity for denial
is infinite
The truth may set you free –
from your job
Early termination of a
project headed for failure
is a good thing
“A strange game. The only
winning move is not to play”
Joshua
ClearSpecs
Education
ClearSpecs
Education
57
29
58
2. Avoid Reqts-related Mistakes
Personal Safety and Respect
part 1
Maintain a climate of personal
safety and respect
Acknowledge personality types
Include stakeholders with a deep
understanding of opportunities or
solutions
No fear of disrespect or reprisal
Study solutions to similar problems
Have developers maintain similar
applications
Immerse stakeholders i.e.,
customers in system dev and
developers in customer activities
Provide training in domain concepts
and terminology
ClearSpecs
Education
Enables:
– truth telling
– voicing concerns
– “don’t knows”
ClearSpecs
Education
59
30
60
2. Avoid Reqts-related Mistakes
Intentional Imprecision is OK
part 2
Intentional Imprecision –
For vague requirements,
requirements,
identify alternative solutions
and their costs
Prototype unfamiliar functions
and interfaces
Have experienced requirements
engineers or technical writers
author larger specs
Promote collaborative research
and cooperative learning
Elicit metameta-requirements
Customize a “just enough”
reqts strategy
ClearSpecs
Education
Reqts provider states high
high--level needs with a
marker and then expects supplier to:
(1) lead the discovery of detailed choices
with costs
(2) request opinions on choices
Example: Product manager writes –
Example:
Our next generation chromatograph
must have a faster <IOK
IOK>
> cycle time
than our fastest current model
Intentional imprecision is appropriate when:
(1) supplier knows as many or more details
than provider
(2) neither supplier nor provider know details
Intentional imprecision should be marked
marked..
ClearSpecs
Education
61
31
62
“Just Enough” Strategy
Elicit Meta-requirements
Ask downstream stakeholders:
project estimators
product architects
component designers
product testers
product documenters
Exactly what types of information
in what formats do they need to do
their jobs?
ClearSpecs
Education
Focuses primarily on the risk of
“inadequate understanding of the
higher priority reqts “ by reqts users
Considers initial understanding by
stakeholders and remaining info that
should be discovered and communicated
to enable reqts users to do their jobs
Tries to maximize and enhance initial
understanding of stakeholders
Tries to effectively discover and
communicate remaining info
Results in a contextcontext-sensitive reqts
process
ClearSpecs
Education
63
32
64
Risk Depends on Initial Understanding
Areas to Understand
Domain concepts and terminology
Shallow PU
Opportunity/Problem including
Opportunity/Problem
customer needs and user populations
System constraints, technologies, and
designs
– alternatives
– consequences (environmental
impact)
Communication and learning
Reqts Development (RD)
Medium PU
Deep PU
Deep RU
Understanding varies by aspect
as well as by stakeholder
Medium RU Shallow RU
Initial Provider Understanding (PU) of problem
domain and specific needs, wants, and preferences
Initial Receiver Understanding (RU) of solution
domain and relevant solutions
ClearSpecs
Education
ClearSpecs
Education
65
33
66
Factors Influencing Choices
RD Strategy Choices
types of info (~50)
forms of info (~15)
discovery techniques (~20)
1.
Impact of system failure (due to faulty reqts)
reqts)
2.
Depth of customer understanding (of needs)
3.
Depth of system supplier understanding (of
needs and solutions)
4.
Depth of reqts. developer understanding of RD
techniques and tactics
checking techniques (~50)
5.
Number and diversity of customer stakeholders
risk mgmt tactics (~60)
6.
Understanding, learning and communication
styles of critical stakeholders
50
More than 10
RD strategies
7.
RD schedule
8.
Relative cost of tactic introduction and use
- influenced by availability and knowledge
of support tools
9.
Organization, industry, and oversight standards
and guidelines
10. Government regs and legal considerations
ClearSpecs
Education
ClearSpecs
Education
67
34
68
Dialog Mapping
3a. Minimize Comm Problems –
part 1
Use dialogue mapping
Isolate well known
details
Use rich definitions
Replace “magic numbers”
i.e., constants, with
named constants or
formulas
Group facilitation process that creates a
diagram capturing and connecting
participant comments as a meeting unfolds.
The diagram serves as "group memory
The icons represent the basic elements of
the Dialogue Mapping grammar: Questions,
Ideas, Pros and Cons.
Cons.
ClearSpecs
Education
ClearSpecs
Education
69
35
70
Plain Definitions
are Imprecise
Isolate Well Known Details
If legal, oversight, or testing
considerations require the
recording of system details
already known to reqts users,
isolate these from the useful
details to avoid obscuring the
unknown.
Hazard: a potential cause
Hazard:
of significant harm or loss
Plain definitions define words using
phrases and sentences.
If recording can be delayed,
wait until the system is built
i.e., document asas-built
Most plain definitions do not enable
precise calculations or decisions.
decisions.
Consider: Accurately identify
potential customers for our new
blood analyzer
ClearSpecs
Education
ClearSpecs
Education
71
36
72
Need Rich Definitions
Derived Values
(modified nouns)
Rich definitions precisely define words (nouns),
phrases, and sentences (events) using 7 patterns
Patterns
Objects
Examples
Nouns, unmodified
customer or order
Nouns, modified
VAR controller
Derived
value
Nouns, modified
employee bonus
Derived
condition
Nouns, modified
active customer
Verbs, premodified
rarely fail
Nouns, modified
satisfied customer
Verbs, premodified
accurately identify
Verbs,
postmodified
display complaints
Verbs, premodified
tentatively identify
Derived action
Verbs,
postmodified
plan vacation
Event profile
Events
accurately identify
potential
customers for our
new system
Entity profile
Quality profile
Action contract
Define noun phrases using a cascade of
algebraic formulas
Weekly income =
yearly income / 52
Days after sale =
current date – sale date
Total of orders for
salesperson--id in year
salesperson
year--id =
SUM OF values FROM orders
WHERE
(sales contact = salesperson
salesperson--id)
AND (year = year
year--id)
id)
Rich definitions are contextual, precise, and
focus on defining imprecise phrases
ClearSpecs
Education
ClearSpecs
Education
73
37
74
Derived Conditions
(modified nouns)
Value of Rich Definitions
Derived conditions name collections of
one or more logical expressions (joined
by AND
ANDs
s and Or
Ors)
s)
Rich definitions
catalyze crucial conversations
active customer =
(bought many services) or
(bought services A and B) or
(bought a lot of service C)
bought many services =
(total invoiced service types) > 5
bought services A and B =
(invoiced for service A)
and (invoiced for service B)
bought a lot of service C =
(invoiced amount for service C)
> $500,000.00
ClearSpecs
Education
When derived by consensus,
their value is directly related to
the difficulty of their creation
i.e. easy = low value
hard = high value
ClearSpecs
Education
75
38
76
Algebraic Formulas
Avoid “Magic Numbers”
Example:
Example:
TCAS shall provide collision avoidance
protection for any two commercial aircraft
(excluding the Concorde) closing horizontally
at a rate up to
max horizontal closing
[= 2 (max air speed);
speed);
currently (max air speed) =
600 knots, but
expected in 2010 to be 750 knots]
and vertically at a rate up to
max vertical closing
[= (max
(max control descent)
+ (max climb);
climb);
currently (max control descent)
= 5000 fpm
and (max climb) = 4000 fpm]
TCAS shall provide collision
avoidance protection for any
two commercial aircraft closing
horizontally at a rate up to
1200 knots and vertically at a
rate up to 9000 fpm
The term magic number refers to the poor
programming practice of using numbers
directly in source code without explanation.
In most cases, this makes programs harder
to understand and maintain.
Wikipedia
(max horiz closing) and (max vert closing) are
defined in a rich glossary as derived values
ClearSpecs
Education
ClearSpecs
Education
77
39
78
3a. Minimize Comm Problems –
part 2
Just Enough Precision
Reqts are part of interactive
communication between reqts
providers and solution suppliers
Identify user types and create
user personas
Clarify with examples and
measures
Structure information with
spec patterns – avoid text blocks
Many Details
Up Front
Just Enough Details
Up Front
Few Details
Up Front
Picture reqts info
<waterfall>
<smart>
<agile>
Capture context
ClearSpecs
Education
ClearSpecs
Education
79
40
80
Structure Info with Models
Picture Reqts Info
Context
Diagrams
–
–
–
–
–
–
–
–
–
–
–
–
• Inter
Inter--actors
• Boundary data flow
• Interfaces
Usage
• User Stories
• Use Cases
• Usage Scenarios
• Acceptance Tests
entity
entityentity-relationship
data flow
concept maps
reqts graph
usage context
usage
scenario
collaboration
sequence
activity
state
Mockups
– screens
– data displays
– reports
Behavior
•
Decision Tables
•
Trigger-Response Tables
•
State Transition Tables
ClearSpecs
Education
Simulations
– process flow (scenario)
– screen flow
ClearSpecs
Education
81
41
82
(Logical) Screen Mockup
Usage Diagram
request
access id
supply
id
verify
id
No
No
ok
> 3 tries
Yes
Request
data
Phrase:
Context:
Request
action
Create a derived-value conditional
Response
message
Create a derived-value conditional
Response
data
Abort
employee bonus
in 2008
If
Then
Yes
Suppy
id
provide
functions
Comments:
request
[withdrawal]
verify
request
ok
No
Suppy
Response
actions
display
problem
Save
Clear
Exit
Yes
Suppy
id
perform
[withdrawal]
ClearSpecs
Education
ClearSpecs
Education
83
42
84
Derived Requirements
Capture Context
Result of interpretation of a
source reqt based on:
Discuss and document
• facts and assumptions about:
rationales
derivations
satisfaction arguments
assumptions
expectations
goals
intentions
alternatives
•
•
•
the application domain
the application environment
interfacing systems and
components
• external constraints
(standards, laws, policies)
• source reqt decomposition
• reqts conflict resolution
• design choices
Link to sources and satisfaction argument
ClearSpecs
Education
ClearSpecs
Education
85
43
86
Satisfaction Arguments
Satisfaction Argument
may cover multiple derived requirements
grouped using propositional logic (i.e.
ANDs & ORs).
ORs).
For Example – Provide status reporting Provide multiday, multihour,
multihour, AND
multiminute status reporting.
Multiminute status reporting AND human
factors automatic monitoring AND outoutof
of--range notification.
may incorporate facts, assumptions,
external constraints, conflicts, or other
requirements as well as static (e.g. timed
swim lanes) or dynamic (e.g. performance
simulations or prototypes) modeling
should address both the soundness and
completeness of the derivation
apply to system design and test design
as well as requirements
Must interface
with Comm System
source reqt
<satisfaction
argument>
current fact
Currently,
Comm System
interface only
uses TCP
Must implement
Transmission
Control Protocol
derived reqt
ClearSpecs
Education
ClearSpecs
Education
87
44
88
Quality Requirements
3b. Minimize Other Problems
Identify minimal sets of
marketable features
Use elicitation workshops to find
issues early
Focus on quality and
environmental requirements early
Identify early aspects
Develop verification
strategies for negative reqts
Identify impacts and mitigation
strategies
Analyze benefits, risks, priority,
and cost of proposed reqts and
changes
ClearSpecs
Education
Usability – Ease of learning and
operating, relevance of functions to
needs, and sufficiency of
performance such as response and
throughput
Reliability – Degree for failurefailure-free
usage
Verifiability – Ability to determine
if a system satisfies its reqts
Maintainability – Potential for lowlowcost repair, adaptation, and
extension
Portability – Potential for minimal
cost transfer to other execution
platforms
ClearSpecs
Education
89
45
90
Environmental Reqts
Early Focus on Quality Reqts
A recent analysis of 15 publicly
available software requirements
specifications showed an almost
universal lack of any requirements
describing product qualities.
qualities.
interfaces
integration
timing
platforms
internationalization
Cleland--Huang
Cleland
CFP -- Quality Reqts
Quality and environmental reqts
drive architectural choices
ClearSpecs
Education
ClearSpecs
Education
91
46
92
Impacts & Mitigation
4. Detect Reqts Defects
Identify potential impacts using
impact (effects) analysis
impacts on:
* systems/products
Employ a copy editor
Incrementally review long specs
Identify
- vague
- incorrect
- missing, and
- unnecessary info
Verify satisfaction arguments
for derived requirements
* processes
* customers
* staff
* finances
* schedules
Include estimates of likelihood
Develop and document mitigation
strategies for negative outcomes
ClearSpecs
Education
ClearSpecs
Education
93
47
94
Incremental Reviews 1
Incremental Reviews 2
What:
•
•
How:
Long documents (≥ ~ 30 pages of new
content) are developed in segments
•
Provide two segments for author learning
and then balance the number of reviews
and remaining pages based on risk.
•
Create more segments based on content
criticality, content density (e.g. equations
and tables are denser), and author
inexperience with technical writing.
•
Sample segmentations –
Each segment is thoroughly reviewed
Why:
•
Lots of intense reviewing is a daunting
task
•
Later parts and hard parts are poorly
reviewed [Gilb estimates “best practice”
30 pages: 15, 30
reviews miss 2/3 of the defects]
defects]
•
Repeated defects frustrate reviewers
and obscure new defects
•
Lots of defects leave no time for
learning and frustrate authors,
especially when avoidable
ClearSpecs
Education
50 pages: 15, 30, 50
or
20, 50
100 pages: 20, 40, 70, 100
or
20, 50, 100
ClearSpecs
Education
95
48
96
Review Strategies
Incremental Reviews 3
Reviewers
Review strategies –
•
involved from the start
–
perceiving
•
advise on segment content
–
using
•
prefer highesthighest-risk first
•
answer questions during
development
•
•
– ~ 80% of defects missed by each
individual (Schneider)
– ~ 65% of major defects missed by a
group (Gilb)
detect and resolve
disagreements among
reviewers
share expectations, in early
segment reviews
Authors
•
•
Effectiveness of perceiving
ask for guidance, rather than
guess
–
test design finds faulty reqts
–
function point analysis uncovers
reqts defects
–
modeling reqts reveals problems
The proof of the pudding is in the eating
The proof of the reqts is in the using
share intentions, in early
segment reviews
ClearSpecs
Education
Effectiveness of using
ClearSpecs
Education
97
49
98
5. Monitor Reqts and Risk
Status – part 1
Basic Monitoring Process
Monitor reqts instability and
growth along with their major
causes and project impacts
Monitor accuracy and feasibility of
- assumptions
benefits, costs, risks
- expectations
1.
Identify aspects to monitor
2.
Develop expectations
3.
Monitor reality
4.
Identify and explain
“major” differences
5.
Adjust development process
accordingly
- priorities
Monitor stakeholder participation
ClearSpecs
Education
ClearSpecs
Education
99
50
100
5. Monitor Reqts and Risk
Status – part 2
Instability & Growth
Between 2 points in time, t1
t1 & t2
t2:
Require frequent demos
of understanding
Trace requirements
derivations and links to
resulting work products
Estimate defect risk
Instability % = (Dt2 + Ct2)/ Rt1 x 100
Growth % = (Rt2 – Rt1)/ Rt1 x 100
Rt1 is number of Reqts at t1
t1
Dt2 is number of reqts
t2
Deleted from Rt1 at t2
Ct2 is number of reqts
t2
Changed in, but not deleted from, Rt1 at t2
Rt2 is number of Reqts at t2
t2
If Rt2 = 14
14,, Rt1 = 8, Dt2 =3, & Ct2 = 1
Then instability of t1
t1 reqts at t2
t2 = 50
50%
%
and growth of t1
t1 reqts at t2
t2 = 75
75%
%
It’s the rate of change that matters
ClearSpecs
Education
ClearSpecs
Education
101
51
102
Estimate Defect Risk
Demos of Understanding
Estimate Density of Major Defects
Design partial tests (pre
and post conditions) early
Design tests (not plans or
procedures) in reverse
order of execution
i.e., acceptance first,
then system,
then integration,
then build
OR create use cases,
usage scenarios,
screen mockups,
state charts, or
decision tables
EARLY
Assemble 2 to 4 reviewers
Select small set (5 to 8) of spec writing rules
to check
Select 1 to 3 typical pages from the spec
Each reviewer checks sample for
conformance to spec writing rules
Identify and report major defects (potential
to loss time or product quality)
Estimate total majors per page using:
Total Majors Present per page =
3 x (Total Unique Majors Found per page)
Estimate Total Spec Majors by multiplying
Total Majors Present per page by page count
Take action based on estimates
Tom Gilb
Spec QC
ClearSpecs
Education
ClearSpecs
Education
103
52
104
6. Mitigate Impact
of Reqts-based Problems
5. Monitor Reqts and Risk
Status – part 3
Track reqts defects
Decouple vision from build
Analyze and classify reqts
defects and root causes
Subdivide project scope
Build in multilane cycles
Use technologies that match
solution
Request external, midmid-project
audit of hazards for a fresh
perspective
Include requirements risk
reassessments in your project
status reviews
Mitigation saves money (by definition)
ClearSpecs
Education
ClearSpecs
Education
105
53
106
Decouple Vision from Build
Build in Multilane Cycles
Envision (Evolve)
Develop Reqts
Model Domain
Design Acceptance Tests
Design Architecture
<Insight can not be scheduled>
scheduled>
<Reqts is not a phase,
but a continual process>
process>
Build
Build
Build
Build (Cycle)
Scope Build
Design Build Tests
Design Build Details
Implement Build & Tests
Test Build
Integrate Build
Build
Build
…
…
Build
…
Build production quality components
foundation
familiar parts
unfamiliar parts
Deliver (Increment)
ClearSpecs
Education
…
Envision
ClearSpecs
Education
107
54
108
Cross-Functional Teams
7. Staff for Safer Dev
Build crosscross-functional team for
both development and verification
Supplement with outside
knowledge, skill, and experience
(domain, technology, reqts,
reqts, test,
tech writing)
writing)
A crosscross-functional team of
stakeholders should be
responsible for reqts dev from
user reqts to system specs
The
Replace underperformers and
disruptors
ClearSpecs
Education
team should include:
customer
perspective user
business analyst
systems analyst
development lead
test lead
technical writer
ClearSpecs
Education
109
55
110
Team Success Factors
8. Plan for RRM
Proper stakeholders
Effective leadership
Brainstorm risks and tactics
Involvement, communication,
collaborative behavior
Identify risk indicators and root
causes
Clear charter, goals, and
deliverables (reqts., priority,
stability, & risk)
Customize, carry out, and
monitor RRM strategy
Consensus groundground-rules (forming,
storming, norming,
performing)
Embrace stakeholder diversity,
but acknowledge their differences
ClearSpecs
Education
ClearSpecs
Education
111
56
112
Likely Risks 1
Likely Risks 2
Stakeholders who may not
Communication & Analysis
participate
know enough
share information
be comfortable with details
be realistic
negotiate
what may interfere with
understanding?
Reqts or stakeholder change
which changes are probable?
which are hard to manage?
Domain, scope, objectives,
reqts, & scenarios
what do we NOT know,
understand, or remember
(e.g. usage)?
what can we NOT describe
(tacit knowledge)?
what do we know that isn't so?
what do we over or under
emphasize (biases)?
ClearSpecs
Education
What are the indicators and
root causes of the identified
risks?
ClearSpecs
Education
113
57
114
Reduce Risk of Change
Reduce Impact of Change
Internal causes
External causes
incorrect assumptions
inappropriate biases
incorrect guessing <Rigid change
control (and BRUF) encourages
guessing>
gold plating
human error (mistakes) and poor
judgment
miscommunication
learning (deeper understanding)
about needs, system and
environment during (and after)
the project
stating solution rather than
problem
undetected or unresolved
disagreements
ClearSpecs
Education
new needs
new users
new interfaces
new platforms
new technology
new enterprise organization
regulatory or legal change
competitive change
(new features & products)
paradigm shifts
ClearSpecs
Education
115
58
116
9. Prepare for RRM
Predictable or Preventable Change
…in all the cases I’ve studied,
the amount of change that was not
predictable or preventable was less
than 20% of all changes
Systematize change mgmt and
reqts defect and failure tracking
Acquire tools supporting safer
requirements
Develop reqts defect and root
cause profiles (occurrence and
impact)
Conduct reqts retrospectives
Richard Zultner
Anticipate change
sources
types
impacts (conflicts)
likelihoods
Prepare by knowing
your past and present
problems
ClearSpecs
Education
ClearSpecs
Education
117
59
118
Tools Can Support
Develop Problem Profiles
Monitor feasibility
• prioritizing reqts (reqts mgmt)
• identifying conflicting nonnon-functional
reqts
Avoid mistakes
reqts defects
root causes
• prototyping functions and interfaces
Minimize comm problems
•
•
•
•
•
using rich definitions
using dialog maps
using spec patterns
picturing reqts info
capture context (reqts mgmt)
Monitor reqts
• monitoring instability and growth
(reqts mgmt)
• identifying defective reqts
• trace reqts (reqts mgmt)
• tracking reqts defects
• classifying defects and root causes
Analyze defect reports
and change requests
2.
Analyze root causes
Each project has unique profiles
as a function of -stakeholder understanding
and behavior
reqts practices
project constraints
Prepare
• develop reqts defect and root cause
profiles
• systematize change mgmt
ClearSpecs
Education
1.
ClearSpecs
Education
119
60
120
Occurrence Profile
Dimensions of
Causal Analysis
Requirements defects found in
the International Space Station spec
had the following distribution:
Cause
Type
Missing
Occurrence %
What might cause –
Effect
33 %
What might result from –
Incorrect
24 %
Incomplete
21 %
Over--specified
Over
6%
Ambiguous
6%
Inconsistent
5%
ClearSpecs
Education
Likelihood
What are the chances of –
121
61
Causal Chains
Analyze Causal Chains
Proximate Cause / Effect
A direct or immediate cause / effect
(relative to granularity of analysis)
e.g., Having the gasoline vapor near
the open flame caused (proximate) the
explosion. Jim caused (not proximate)
the explosion by putting the leaking
gas can near the heater.
Causes
of a sequence
Effects
of a sequence
Events or
Conditions
Events or
Conditions
Causal Chain
A sequence of “linked” causecause-effect
pairs in which each effect is the
subsequent cause
Events or
Conditions
Defect
Events or
Conditions
Events or
Conditions
Events or
Conditions
C
E (C) E(C) Events or
Conditions
Reqts
E
Likelihood
of a sequence
62
Dimensions of Reqts
Root Cause Analysis
Reqts. Process Failures
Stakeholder Root Causes
What human ignorance, knowledge,
mental process or behavior initiated?
Every defect in a reqts spec
results from at least two process
failures:
Process Root Causes
Which reqts activities failed?
Reqts Defects
Failure in reqts.
reqts.
discovery and/or
communication
What defective info resulted?
Project Problems
What budget or schedule impacts
resulted?
Failure in reqts.
reqts.
verification
Product Defects
What component defects resulted?
Product Failure
What usage failures resulted?
Likelihood
What are the chances of?
ClearSpecs
Education
125
63
10. Compensate for
Reqts-based Problems
Problems To Analyze
Root causes of:
expensive reqtsreqts-based
failures
unused features
expensive, late, or infeasible
reqts changes
critical reqts defects
systemic reqts defects
ClearSpecs
Education
Estimate & track reqts
reqts--based
rework
Adjust for surprises, both good
and bad e.g. replace dysfunctional
stakeholder, reduce scope
ClearSpecs
Education
127
64
128
Conclusion
Selecting Your RRM Tactics
Identify tactics already a part of your
requirements development, system
development, or project management process.
Identify tactics that do not apply to your
application domain or are incompatible with
your culture.
Identify tactics for permanent integration into
your requirements development, system
development, or project management process.
Add optional tactics you have used effectively
for controlling requirements risk.
Consider a few of the remaining tactics for
inclusion in your requirements risk
management strategy at the beginning of and
during each project after specific hazards have
been recognized.
Goal: Reduce reqts
reqts--related risk
1. Rapidly and continuously identify hazards
a. Accurately anticipate hazards
b. Recognize indicators and hazards
during the project
2. Assess risk
a. likelihood of adverse events
b. degree of harm
3. Select a few useful risk control tactics
4. Implement control tactics
5. Continuously monitor risk
6. Communicate findings, assumptions, and
strategy adjustments
ClearSpecs
Education
ClearSpecs
Education
129
65
130
It Won’t Be Easy
And, then there’s change
Intro of Seat Belts
1910
1960
2007
Commercial
Production of
Automobiles
Seatbelts
Click It
or Ticket
It’s not so much that we’re
afraid of change or so in love
with the old ways, but it’s that
place in between that we fear.
fear.
..... It’s like being between
trapezes. It’s Linus when his
blanket is in the dryer.
There’s nothing to hold on to.
Optional safety measures
can be a hard sell
Marilyn Ferguson
ClearSpecs
Education
ClearSpecs
Education
131
66
132
Reckless Endangerment
of
Project Success
Manage Risk
to be lucky
ClearSpecs
Education
133
67