Fraunhofer Institute for Open Communication Systems | Kaiserin-Augusta-Allee 31 | 10589 Berlin, Germany
From textual requirements to test models
- Coping with natural language ambiguities for testing
Marc-Florian Wendland | RCIS 2013 Industrial Day | 31st May, 2013 | Paris, France
Agenda
 Motivation
 Requirements and model-based testing
 Conclusion
Agenda
 Motivation
 Requirements and model-based testing
 Conclusion
From textual requirements to test models
Some facts about requirements
Costs of fixing software defects:
•
logical architecture
1.000 €
•
technical architecture
4.000 €
•
realisation
12.000 €
•
acceptance / rollout
48.000 €
•
in field
90.000 €
[2]
[1]
Costs per bug found
As much as 60% of all defects in
a systems lifetime originate from
deficient requirements
40%
55%
[3]
30%
5%
req. spec.
10%
design
45%
- bugs introduced
- bugs found
15%
impl.
in field
[4]
From textual requirements to test models
Impact network of requirements
Req. Engineering
System/Software
Requirements
Specification (SRS)
Test Analysis
<<organizes>>
Requirements
<<verifies>>
Test
Requirements
<<realizes>>
<<interprets>>
System Dev.
Test Design
<<realizes>>
<<covers>>
System/Software
Specification
System/Software
Test Specification
1..*
<<satisfies>>
<<satisfies>>
Test Exec.
Implementation
<<executes against>>
Test Case
From textual requirements to test models
Requirements specification techniques
Unrestricted NL
Restricted NL
Visual Notation
Formal language
Formalism and precision
Granted freedom to engineers
Suitable for automated transformation
Acceptance in industry
We need an approach that copes with natural language ambiguity
and imprecision.
From textual requirements to test models
Satisfaction with requirements engineering tasks
Requirements elicitation
Requirements analysis
Requirements specification
Requirements validation
Requirements management
[5]
high
50% of all respondents sees unsufficiently specified
requirements as biggest challenge for testing
medium
low
Agenda
 Motivation
 Requirements and model-based testing
 Conclusion
From textual requirements to test models
State of the art in automated test design – Model-Based Testing
Requirements
Test Plan
Formalisation
Test Model
Implicit knowledge
•
•
•
•
•
•
Implicit/imperfect knowledge is made explicit
and (semi-)perfect
Test design is done on the model
Repeatable, comprehensible and systematic
Prevents loss of knowledge
Model is self-documented
Quality of test model depends on experiences and
ingenuity of the tester
Intellectual task
TC
TC
TC
SUT
SUT
SUT
Automated
clerical task
From textual requirements to test models
Requirements models as starting point
 Introduce a dedicated requirements model
 Formalize the externally observable
behavior of the system based on its
requirements
 Requirements model can be exploited to
partially facilitate activities both system
and testing side
 Shared source of information for further
development
We need a methodology, which ensures the requirements model is
complete, unambiguous, consistent, correct and testable (IEEE 830)
From textual requirements to test models
What is Behavior Engineering?
Model-centric
Document-centric
Requirements Engineering (RE)
• Requirements Elicitation
• Requirements Identification
• Requirements Specification
Behavior Engineering (BE)
• Requirements Formalisation
• Requirements Integration
• Requirements Refinement/Completion
• Requirements Validation & Verification
Phases of BE
Formalisation (RBT)
Integration (pIBT)
Validation (IBT)
From textual requirements to test models
Requirements Formalisation – Translation
Requirement 1
There is a single control button available for the use
of the oven. If the oven door is closed and the user RBT
push the button, the oven will start cooking (that is,
energize the power-tube) for one minute.
CT
User
Translate each single requirement independently
From textual requirements to test models
Requirements Integration
Precondition Axiom
Every constructive, implementable, individual
functional requirement of a system, expressed as
a behavior tree, has associated with it a
precondition that needs to be satisfied in order
for the behavior encapsulated in the functional
requirement to be applicable."
Interaction Axiom
For each individual functional requirement of a
system, expressed as a behavior tree, the
precondition it needs to have satisfied in order to
exhibit its encapsulated behavior, must be
established by the behavior tree of at least one
other functional requirement that belongs to the
set of functional requirements of the system."
From textual requirements to test models
Requirements Integration
Precondition Axiom
Every constructive, implementable, individual
functional requirement of a system, expressed as
a behavior tree, has associated with it a
precondition that needs to be satisfied in order
for the behavior encapsulated in the functional
requirement to be applicable."
Interaction Axiom
For each individual functional requirement of a
system, expressed as a behavior tree, the
precondition it needs to have satisfied in order to
exhibit its encapsulated behavior, must be
established by the behavior tree of at least one
other functional requirement that belongs to the
set of functional requirements of the system."
From textual requirements to test models
(Test) Analysis model as result of BE
IBT
CT
Fokus!BE
BE with UML
Agenda
 Motivation
 Requirements and model-based testing
 Conclusion
From textual requirements to test models
Conclusion
 Requirements are still mostly specified textually with unrestricted natural language
 Requirements validation techniques need to cope with unrestricted natural language
 BE is a methodology that is aiming at validating the requirements by integration
 BE alleviates the transition from a requirements model to further models
 Testing is highly depending on the quality of the requirements
 MBT is highly depending on the quality of the model that model the requirements
 BE is predestined to tackle current model-based testing challenges
 BE with UML seems wise since system/test models are often represented in UML/UTP
Thanks for your attention.
Questions?
References
[1] Boehm, Berry: Software Engineering Economics, Englewood Cliffs, NJ :
Prentice-Hall, 1981. ISBN 0-13-822122-7
[2] Spillner, A: SQS Empirische Daten aus 3000 IT-Projekten. 1st Testing Day
Franken, Nürnberg, 10-March-2009
[3] Berry, D. M.: Formal methods: the very idea - some thoughts about why they
work when they work. Sci. Comput. Program., 42(1):11–27, 2002
[4] Standish Group: ChAOS Reports. URL: http://www.standishgroup.com. Last
visit: January 05, 2012
[5] SwissQ, Trends und Benchmark Report Schweiz, Wo stehen wir, wo geht
es hin? Testing 2013, http://de.slideshare.net/swissq/testing-trends-undbenchmarks-2013-web