SOFTWARE TEST DESIGN TECHNIQUES
Sigist meeting, 13th December 2011
CONNECTING
BUSINESS & TECHNOLOGY
PRESENTATIONS
PRINCIPLES OF TECHNIQUES
USAGE OF TECHNIQUES IN TESTING
EXAMPLES OF USAGE OF TECHNIQUES
GO OUT AND DO IT
2
14/12/2011
COPYRIGHT©
ROADMAP
DEVOTEAM – 10+ YEARS OF INTERNATIONAL
EXPANSION
Sweden
Norway
Denmark
EMEA Consultancy
United Kingdom
Devoteam Group founded in 1995
More than 4500 employees
The Netherlands
Luxembourg
Belgium
Germany
Poland
Czech Republic
Austria
Switzerland
France
Offices in 23 countries in EMEA
Italy
Spain
Russia
Turkey
Long-term annual growth (>30%)
Listed - stock exchange in Paris
Tunisia
Algeria
Morocco
Jordan
Saudi Arabia
United Arab Emirates
Business & Process Development
• Strategy & Innovation
• Business Process Optimization & Change
• Risk & Security Management
Information Systems & Management
• Enterprise Architecture
• Information Management
• IT Governance
ICT Infrastructure
Governance
• IT Service Management
• Workstations, Servers & Storage
• Enterprise Networks
Telecom
• Core Networks
• Service Platforms
• Devices
13-3-2009
COPYRIGHT©
CUSTOMER VALUE PROPOSITION
COPYRIGHT©
ANNE METTE HASS
Background
•
•
•
•
M. Sc. 1979; in IT since 1980
Since 2007 in ISO 29119 working group
Since 2007 external lecturer at ITU
Software Testing Excellence Award 2009
Experience
• All areas in software development with focus
on requirements, test, and configuration mgt.
• Worked in Denmark, Norway, England,
France, and Italy
• Numerous branches
Books
• ”Guide to Advanced Software Testing”
• ”Configuration Management”
• ”Requirements Handling”
[email protected]
PRESENTATIONS
PRINCIPLES OF TECHNIQUES
USAGE OF TECHNIQUES IN TESTING
EXAMPLES OF USAGE OF TECHNIQUES
GO OUT AND DO IT
6
14/12/2011
COPYRIGHT©
ROADMAP
Technique = A method of achieving something or carrying
something out, especially one requiring some skill or
knowledge.
From Ancient Greek
τεχνικός (technikos, “of or pertaining to art, artistic, skillful”)
τέχνη (techne, “art, handicraft”)
τίκτειν (tiktein, “to bring forth, produce”)
7
14-12-2011
COPYRIGHT©
TECHNIQUE
of jobs well done using the right technique
How many test design techniques do you know?
8
12/14/2011
COPYRIGHT©
EXAMPLES …
COPYRIGHT©
OVERVIEW OF TEST DESIGN TECHNIQUES IN ISO 29119
Software test design techniques
9
Specification-Based
Structure-Based
Equivalence Partitioning
Classification Tree Method Boundary Value Analysis State Transition Testing Decision Table Testing Cause‐Effect Graphing Syntax Testing Combinatorial Test Techniques Scenario Testing Error Guessing Random Testing Statement Testing
Branch Testing Decision Testing Condition Testing Branch Condition Testing Branch Condition Combination Testing Modified Condition Decision Coverage (MCDC) Testing Data Flow Testing 12/14/2011
Support systematic and meticulous work
Expression of ’best practice’ – not science
Effective in finding possible failures
Can prevent redundant testing
May be repeated by others with similar result
Work can be explained
Based on models of the system
Provide coverage items
10
COPYRIGHT©
STATEMENTS ABOUT TEST DESIGN TECHNIQUES
PRESENTATIONS
PRINCIPLES OF TECHNIQUES
USAGE OF TECHNIQUES IN TESTING
EXAMPLES OF USAGE OF TECHNIQUES
GO OUT AND DO IT
11
14/12/2011
COPYRIGHT©
ROADMAP
The purpose of testing is to provide information and hence
facilitate improvement of the quality of:
the product (find defects)
decisions (quantify risks)
the development process (identify root causes)
The most important data are
incidents found
obtained coverage
12
12/14/2011
COPYRIGHT©
THE PURPOSE OF TESTING
13
12/14/2011
COPYRIGHT©
TEST DESIGN RESULTS IN ISO 29119 – PRINCIPLE
A test condition
is a testable aspect of a test object, e.g. a function,
transaction, feature, quality attribute, or structural element
identified as a basis for testing. (from ISO 29119)
starts with: It can be tested that ….
can be broken into individual coverage items
Coverage items can be turned into test cases
This can be fully documented or not documented at all
14
12/14/2011
COPYRIGHT©
TEST CONDITION
COPYRIGHT©
THE GOOD TEST CASE
Find the right balance between:
Effective: Reasonable probability of detecting errors
Exemplary: Practical, low redundancy
Economic:
Reasonable development cost,
return on investment
Evolvable:
Flexible,
structured, maintainable
Effective
Economic
Evolvable
Exemplary
Source: Grove Consult and John Fodeh
15
Identify feature sets
Go through the specification and find out what to test
Derive test conditions
Use appropriate technique to find out if there is enough information
in the test basis
Derive test coverage items
Use appropriate technique to identify all possible coverage items
Derive test cases
Select coverage items to cover and derive info for the test cases
16
12/14/2011
COPYRIGHT©
THE TEST DESIGN ACTIVITIES
17
Technique
Coverage item
Classification tree
Leaf
Boundary value analysis
Boundary value
Combinations
Combination
Decision tables
Combination
Flow graphs
Path
State transitions
Transition (or transitions)
12/14/2011
COPYRIGHT©
THE MOST COMMON TEST DESIGN TECHNIQUES
PRESENTATIONS
PRINCIPLES OF TECHNIQUES
USAGE OF TECHNIQUES IN TESTING
EXAMPLES OF USAGE OF TECHNIQUES
GO OUT AND DO IT
18
14/12/2011
COPYRIGHT©
ROADMAP
Req. 13a
The product is expected to provide the booker the ability to
finish a booking by accepting it.
Technique: Classification tree
Coverage item: Leaf, 1
Test condition: It can be tested that the system provides …
Test case: input = valid new booking + accept
expected result = the booking is finished
19
12/14/2011
COPYRIGHT©
EXAMPLE 1
COPYRIGHT©
EXAMPLE 2
Requirement 78
If the new estimate is higher the system shall DOTHIS (and
vise-versa if smaller).
Technique: Classification tree
All new
estimates
Compare with (x)
New < (x)
= leaf
20
12/14/2011
New = (x)
New > (x)
COPYRIGHT©
Coverage item: Leaf, 3
Test conditions: It can be tested that…
1. if the new estimate is smaller than (x) the system DOTHIS reversed
2. if the new estimate is equal to (x) ?
3. if the new estimate is greater than (x) the system DOTHIS
Test case: input = new estimate greater than (x)
expected result = DOTHIS
Remember: If testers miss information, so do developers
21
12/14/2011
4.13.1 [581]
One of the variants of the product shall be equipped with a lid to
protect the technicians performing the analyses.
The lid covers the carousel when it is moving. The lid has to be
locked before the carousel is started and it is not possible to
open it before the carousel has stopped completely. Two
sensors are in place to detect if the lid is locked and if the
carousel is moving.
As long as the lid is locked it is possible to start the carousel
moving either forwards or backwards. To change the direction it
is necessary to stop the carousel first, but it is not necessary to
open the lid.
The manoeuvering panel has the following buttons: ‘Lock’, ‘Open’,
‘Forward’, ‘Backward’, ‘Stop’
22
14-12-2011
COPYRIGHT©
EXAMPLE 3
COPYRIGHT©
Technique: State-transition testing
S4
P ‘For’
T5
P ‘Lock’
C moves
T1
Start
P ‘Stop’
T6
C stops
Lid locks
S1
S2
P ‘Open’
T2
Lid opens
P ‘Back’
P ‘Stop’
T4
T3
C moves
C stops
S3
Coverage item: Single transition, 6
23
12/14/2011
COPYRIGHT©
Test conditions: It can be tested that the transitions
T1, T2, T3, T4, T5, and T6 are performed correctly
Test case: input = start state S1, press ‘Lock’
expected result = lid locks, end state S2
Test conditions could also be longer sequences of transitions
24
12/14/2011
PRESENTATIONS
PRINCIPLES OF TECHNIQUES
USAGE OF TECHNIQUES IN TESTING
EXAMPLES OF USAGE OF TECHNIQUES
GO OUT AND DO IT
25
14/12/2011
COPYRIGHT©
ROADMAP
Which testing technique(s) should we use?
The big answer:
It depends!
There is no ‘best’ technique
the ‘best’ depends on the nature of the product/requirement
some technique is better than none
a combination of techniques is better than just one
26
COPYRIGHT©
THE BIG QUESTION:
The choices may be based on:
Risk analysis
the risks we face
the risks we are willing to run
Customer requirements (contract)
Regulatory standards
Experience
which techniques does the tester know, and how well
what techniques has the tester used on similar systems
11c 27
COPYRIGHT©
WE HAVE TO CHOOSE
Remember
Test is difficult
Test requires overview
Test requires creativity
Test requires systematic work
Test requires imagination
Test requires courage
Test is fun
28
COPYRIGHT©
THANK YOU!
COPYRIGHT©
CONTACT:
Anne Mette Hass
PHONE:
+45 23 44 02 20
EMAIL:
[email protected]
COUNTRY:
Denmark
WWW.DEVOTEAM.DK
AUTHOR:
Anne Mette Hass
DATE:
4 November 2011
FURTHER
INFORMATION:
© DEVOTEAM GROUP
THIS DOCUMENT IS NOT TO BE COPIED OR REPRODUCED IN ANY WAY WITHOUT
DEVOTEAM EXPRESS PERMISSION. COPIES OF THIS DOCUMENT MUST BE
ACCOMPANIED BY TITLE, DATE AND THIS COPYRIGHT NOTICE.
PICTURES:
DREAMSTIME.COM OG BIGSTOCKPHOTO.COM
29
12/14/2011