Test Case Design
and Testing Techniques
Factors to Consider
A RapidValue Solutions Whitepaper
Content
Executive Summary...................................................................................................................... 03
Test Case Design and Testing Techniques..................................................................................... 04
Test Analysis............................................................................................................................... 04
Phases of Test Development........................................................................................................ 04
Quality or Product Risk and Risk Based Testing............................................................................. 06
Types of Test Design Techniques................................................................................................. 06
Choosing the Techniques - Factors to Consider............................................................................. 09
Test Case Design Checklist.......................................................................................................... 10
Conclusion................................................................................................................................. 12
A Whitepaper by RapidValue Solutions - July 2016
02
Executive Summary
Software testing is an essential and important technique for assessing the quality of a particular software
product/service. In software testing, test cases and scenarios play an inevitable and a pivotal role. A
good strategic design and technique help to improve the quality of the software testing process. The
process improves the quality of the product/service and ensures effectiveness. Software testing is the
process of analyzing a software item to know the differences between the existing and required
conditions (bugs).Testing helps to evaluate the features of the software, to ensure it is free of bug. It is
an activity that is carried out in co-ordinance with the development cycle and before the deployment.
This paper provides information about test case design activities, test analysis, quality risks, testing
techniques, phases of test development. The paper also, explains the factors that need to be considered
while choosing the right testing techniques and provides a checklist of test cases based on our rich
experience of testing mobile apps.
A Whitepaper by RapidValue Solutions - July 2016
03
Test Case Design and Testing Techniques
Test design is an art that has interesting techniques associated with it, which make it effective and
useful for testing. Many people, when creating a test plan or writing the test case, fail to decipher what
to test in a given cycle or what not to test in a given project. When you are checking the parameters
that are not be tested, one must ensure that the requirement which is not testable by testers must be
addressed here.
Test Analysis
Once the test planning is completed, the test analysis and design in software testing life cycle need to
be carried out. As the first step, you need to schedule all the test-basis. Test-basis is all the
requirements and design specification such as network architecture, system architecture. All the
documents that help in testing must be reviewed when performing the test analysis. Based on the
review of test basis, you need to evaluate testability of the test basis and objects. Identify and prioritize
test conditions, test requirements, or test objectives and required test data, based on analysis of test
items. Once the test condition has been ascertained, you can start prioritizing high-level test cases.
Writing high-level test cases is kind of a pseudo test case design that means the test cases that do not
have any test data. For example, while writing a test case for a login screen in an application or
entering valid username and password, the user must go to the login screen, not writing the test data
for username and password. In analysis and design phase, you have to focus on high level test cases,
which are logical test cases. In this phase, you have to set the test environment that is based on the
application domain (healthcare, social, banking etc.). Another important aspect is identifying
infrastructure and testing tools. In case of mobile application testing, you need to select the device
model, version and screen resolutions. Once you have identified the infrastructure and tools, create
bi-directional traceability between test cases and test basis.
Phases of Test Development
Test development takes place in the following phases:
QUALITY RISK ANALYSIS
HIGH-LEVEL TEST DESIGN
A Whitepaper by RapidValue Solutions - July 2016
04
LOW-LEVEL TEST DESIGN
Once the quality risk analysis is completed, you need to work on a high-level test design and
eventually, move on to a low-level test design. The input to risk analysis can be functional specification,
system specification and output can be high-level test plan and risk analysis document. All the output
from risk analysis goes as input to the high-level test design. Output from high-level test design is
high-level test design document, logical test cases; all the documents serve as input to low-level test
design. Low-level test design is characterized by a lot of test cases, test suites and test design
documents.
Quality risk analysis. In quality risk analysis, identify all risks in the system, using risk analysis
techniques, like informal risk analysis technique (functionality, data quality, error handling and
recovery, performance, localization, usability etc.). All risks must be noted and covered in high-level
and low-level test design. Also, you can use formal risk analysis techniques like ISO9126. (It explains
the major six categories of system risk and sub-categories).
High-level test Design. In high-level test design, create test suites for all quality risk categories that
are identified. During risk analysis, make sure to cover every risk category, with at least one test suite.
You must trace every risk with the test suite.
Functional Test Cases
Sl. Test Case
Module
no.
ID
Priority
1
P1
TC_001
Login
Scenario/
Test Case
Description
Verify the
successful
login with
google
account
Expected
Result
Test Steps
1. Install the
application from
app store.
Once after
successful
validation of
email and
password the
user should be
able to view
the mail in the
inbox.
2. Tap icon from
the list.
3. Tap to sign in
with Google
button.
Actual
Result
Same as
expected
4. Enter Valid email
id and password.
5. Tap to sign in.
Fig.1: Sample test case design template
A Whitepaper by RapidValue Solutions - July 2016
03
05
Status
Pass
Defect
ID
Raised
Build
Version
v1.27.18
1454
Remarks
Quality or Product Risk and Risk Based Testing
Risk is something that can result into undesirable consequences. The level of risk can be determined by
likelihood and impact. With experience, some of the risk can be mitigated, but not all. In quality or
product risk, there is a possibility that the system will fail to satisfy the customers, users, or other
stakeholders. A set of possible bugs are behind the quality risk.
Risk based testing reduces quality risk throughout the project, when identified, and assesses the risk,
and guides the test process, using risk. More knowledge about risk helps to answer key testing
questions. Ideally, risk based testing is part of a larger risk management approach. Once you identify
the risk then assign the level of risk; only separate risk items, when necessary, to distinguish between
different levels of risk. You need to think about technical and business risk; impact of technical risk on
system and business risk on users. Testing, follow-up and re-alignment of risk analysis are mandatory
with regard to key project milestone.
Types of Test Design Techniques
There are three main types of test design techniques.
SPECIFICATION-BASED TECHNIQUE
STRUCTURE-BASED TECHNIQUE
EXPERIENCE-BASED TECHNIQUE
Specification-based technique is, also, called black-box technique. In this technique, you create
tests, primarily, by analysis of the test basis and tracing the bugs in order to know how the system
behaves. One of the basic specification-based techniques used is equivalence partitioning technique. In
equivalence partitioning, divide the inputs, outputs, behaviors and environments into classes. Define, at
least, one test case in each partition, or use boundary values in partitions that are in ranges. Tests can
be designed to cover all valid and invalid partitions.
Second technique in specification-based is boundary value analysis. Boundary value analysis is the
refinement of equivalence partitioning that selects the edges or end-points of each partition for testing.
Equivalence partitioning looks for bugs in the code that handles each equivalent class. Boundary values
are members of equivalence classes that, also, look for bugs in the definition of the edges. Boundary
value technique can be applied when the elements of the equivalence partition are ordered. In
non-functional testing, you can use non-functional boundaries.
A Whitepaper by RapidValue Solutions - July 2016
06
03
Use case testing is, typically, used when you are about to enter UAT stage, at the end of the system
testing. In this testing, you use the use cases or business scenarios for end to end systems’ testing.
Use case has preconditions and post conditions to be met. Use cases have main flows, alternative flows
and sometimes, exceptional flows. Use case testing uncovers defects in process flows during real world
use of system.
Structure-based technique is, also, called white-box technique. The key concepts include code
coverage, statement and decision coverage, and control-flow test design technique. Structure-based
tests are based on how the system works inside; which helps to determine and achieve a level of
coverage of control flows based on code analysis. Data flows, based on code and data analysis, also
determine and achieve a level of coverage of interfaces, classes, call flows, and the like, based on APIs,
system design etc.
Different levels of code coverage include:
1.
2.
3.
4.
5.
Statement coverage: every statement executed.
Branch coverage or decision coverage: every decision taken.
Condition coverage: each condition evaluated, both true and false.
Multiple condition coverage: every combination of true and false conditions evaluated.
Loop coverage: all loop paths taken zero, once, and multiple times.
Experience-based technique is based on the tester’s skill,
perception, experience with similar applications, and
experience with similar technologies. In this technique, tests
are often, created during the test execution, that is, test
strategy
is dynamic. Examples include error guessing, bug
Sl.
Description
no.
hunting,
breaking applications based on checklists, and
Pilot Release
exploratory testing. Testing experience provides much more
Appropriate client
is displayed.
to1 understand
the icon
scenario
that helps to improve the
experience-based testing on different applications.
To be Implemented
Release I
Release II
Release III
*
*
*
*
*
*
2
Click on application from device launches application.
3
Alignment of pages in both the views (portrait and landscape).
*
*
4
Images are properly displayed/present/in the contents view.
*
*
5
Zooming functionality of the pages/images.
*
*
6
Search functionality.
*
*
7
Validate fields in 'Sign In'/'Sign Up' through 'Settings' option.
*
*
8
Advertisements images/videos are displayed properly in app.
*
*
*
9
Forward and backward swiping is consistently working.
*
*
*
10
Crash encountered during swiping, zooming and searching.
*
*
*
Text in the
properly
displayed. - July 2016
A 11
Whitepaper
by buttons
RapidValue
Solutions
12
Check for duplication of contents in the bookmark page.
*
*
*
*
07
*
*
*
*
Open
Closed
Urgent (P1)
5
7
High (P2)
3
2
Medium (P3)
7
3
Low (P4)
2
5
Critical
3
9
Major
5
1
Moderate
9
5
Minor
1
3
Testing Status
Testing Status
UI Test Cases
Functional Test Cases
NFR Test Cases
2
5
2
Failed
3
3
3
Not Tested
5
7
5
On Hold/Blocked
9
2
9
1
3
1
Pass
Total
Priority
Severity
Defects Based on Priority
Defects Based on Status
1
1
0.9
0.8
0.8
0.6
0.7
0.4
0.5
0.6
0.4
0.2
0.3
0.2
UI Test Cases
Fig. 2: Sample test execution dashboard
08
08
lo
ck
l
ld
/B
To
ta
ed
d
te
es
Ho
Closed
A Whitepaper by RapidValue Solutions - July 2016
tT
Pa
ss
Lo
w
No
Open
Fa
ile
d
0
(P
(P
4)
3)
0.1
M
ed
iu
m
(P
h
Hi
g
Ur
ge
nt
(
P1
)
2)
0
Functional
Test Cases
NFR
Test Cases
Choosing the Techniques - Factors to Consider
Choosing the right test technique depends on the following factors:
1. Type of system or software application. The design techniques, that are chosen, will depend
on the type of software application or system. Basically, techniques are chosen related to the
application type, mainly domain of the application, like finance, social media, healthcare etc. Also,
you need to categorize mobile or web applications.
2. Regulatory standards. Regulatory standards are common testing standards approved,
internationally. Selection of techniques, also, depends on the standard of testing that is followed
in the IT industries.
3. Customer requirements. Selection of techniques is also, based on the customer requirements
(sometimes they do not provide any requirement, in such a scenario you have to choose the
experience based approaches). The techniques may vary, based on the customer requirements.
4. Level and type of risk. Based on the risk during the quality analysis phase, you have to choose
the right techniques. Risks may include lack of requirement, equipment or anything similar that
affects the quality. There can be both, high-level and low-level design techniques.
5. Test objectives. Test objectives are important aspects that give a clear idea about the scope of
the testing activities. Based on that, you can select appropriate techniques.
6. Documentation or requirement. Selection of techniques depends on the availability of test
documents like requirement document, analysis report, design document etc.
7. Tester’s skill and knowledge. This factor is quite significant and plays a pivotal role in the
selection of experience-based technique. The knowledge of the tester, his/her perception of the
application and experience on test execution, to figure-out the defects, help to make the product
a quality one.
8. Time and budget. Both, time and budget, are critical factors while choosing the right design
technique. Some projects are short-term and some are long-term. Based on the project you need
to choose techniques. You need to also consider how much budget is allotted for the project. For
small budgets, cost-effective approach should be taken.
9. Application development life cycle. The application development life cycle has different
stages, parallel to testing stages. Different stages of development and testing require different
techniques.
A Whitepaper by RapidValue Solutions - July 2016
03
03
09
10. Previous experience on types of defects tracked. This is kind of user experience on defects
that he/she had faced in testing life cycle. Further, he/she can replicate the same situation and
catch the defects, encountered previously, from his/her experience. This is, also, a type of
experience based technique.
Test Case Design Checklist
Creating an exhaustive checklist having test cases covering all the possible scenarios and tracking it
throughout the project lifecycle would help you achieve the optimal software quality. Here is a sample
checklist of test cases, which could be considered while testing a mobile app.
To be Implemented
Sl.
no.
Description
Pilot Release
Release I
Release II
Release III
*
*
*
*
*
*
1
Appropriate client icon is displayed.
2
Click on application from device launches application.
3
Alignment of pages in both the views (portrait and landscape).
*
*
4
Images are properly displayed/present/in the contents view.
*
*
5
Zooming functionality of the pages/images.
*
*
6
Search functionality.
*
*
7
Validate fields in 'Sign In'/'Sign Up' through 'Settings' option.
*
*
8
Advertisements images/videos are displayed properly in app.
*
*
*
9
Forward and backward swiping is consistently working.
*
*
*
10
Crash encountered during swiping, zooming and searching.
*
*
*
11
Text in the buttons properly displayed.
12
Check for duplication of contents in the bookmark page.
13
*
*
*
*
*
*
*
Crash encountered while clicking on images simultaneously.
*
*
*
14
Jittering of the images/contents while scrolling.
*
*
*
15
Check for the 'Application Name'(Style/Font/Color/Size).
*
*
*
16
Type of animation implemented upon click on the buttons.
*
*
*
17
Virtual keyboard displayed when the user click on any text fields.
*
*
*
18
Validation in the password field.
*
*
*
*
*
*
A Whitepaper by RapidValue Solutions - July 2016
19
Check that the URL displayed in any page should display as link.
20
Check that any website link should be easy to click.
*
*
03
03
10
*
14
Jittering of the images/contents while scrolling.
*
*
*
15
Check for the 'Application Name'(Style/Font/Color/Size).
*
*
*
16
Sl.
no.
17
Type of animation implemented upon click on the buttons.
Description
To be Implemented
*
*
*
1
18
Pilot Release
Release I
*
Release II
*
Release III
*
Appropriate
icon is displayed.
Validation inclient
the password
field.
*
*
*
*
*
*
*
2
19
Click
application
device
Checkonthat
the URL from
displayed
in launches
any page application.
should display as link.
*
*
*
*
*
*
*
3
20
Alignment
pages
in both
views
and landscape).
Check thatof
any
website
link the
should
be(portrait
easy to click.
*
*
*
4
21
Images
properly
displayed/present/in the contents view.
Validate are
register
properties.
*
*
*
*
5
22
Zooming
functionality
of the pages/images.
Call interrupt
functionality.
*
*
*
*
*
6
23
Search
functionality.
Check the
record time for Audio/Video/Voice recorder if any.
*
*
*
*
*
7
24
Validate
fields
in 'Sign In'/'Sign
through
'Settings'
option.
Download
functionality
if exists,Up'
works
without
any interruption.
*
*
*
*
*
8
25
Advertisements
are displayed
properly in app.
No crash during images/videos
or after download
functionality.
*
*
*
*
*
*
9
26
Forward
and backward
swiping
is consistently
Login/Logout
functionality
in Settings
page. working.
*
*
*
*
*
*
*
10
27
Crash
swiping,
searching.
Correctencountered
username isduring
displayed
in thezooming
settings,and
after
user login.
*
*
*
*
*
*
*
*
11
28
Text
in the buttons
properlywhile
displayed.
Connectivity
error message
downloading without Wi-Fi.
*
*
*
*
*
12
29
Check
duplication
of contents
in the
bookmark
page. time limit.
Sessionfor
expiry
while the
application
is idle
for a particular
*
*
*
*
*
*
*
*
13
30
Crash
encountered
while
clicking
on playing
images simultaneously.
Progress/
duration bar
display
while
audio or video.
*
*
*
*
*
*
14
Jittering of the images/contents while scrolling.
*
*
*
15
Check for the 'Application Name'(Style/Font/Color/Size).
*
*
*
16
Type of animation implemented upon click on the buttons.
*
*
*
17
Virtual keyboard displayed when the user click on any text fields.
*
*
*
18
Validation in the password field.
*
*
*
19
Check that the URL displayed in any page should display as link.
*
*
*
20
Check that any website link should be easy to click.
21
Validate register properties.
*
*
22
Call interrupt functionality.
*
*
23
Check the record time for Audio/Video/Voice recorder if any.
*
*
*
24
Download functionality if exists, works without any interruption.
*
*
*
25
No crash during or after download functionality.
*
*
*
26
Login/Logout functionality in Settings page.
*
*
*
*
*
*
*
*
*
*
*
*
Virtual keyboard displayed when the user click on any text fields.
A Whitepaper by RapidValue Solutions - July 2016
27
Correct username is displayed in the settings, after user login.
28
Connectivity error message while downloading without Wi-Fi.
*
*
03
03
11
Conclusion
The process of test designing is of high priority. A poorly designed test will lead to improper testing of
an application and thereby, yield test wrong and harmful results. This, in turn, will lead to the failure in
identifying defects. As a consequence, an application, containing errors, may be released.
There are various types of designing techniques and the challenge lies in selecting the right set of
relevant test design techniques for the particular application. The different types of testing techniques
have their own unique benefits. The use of any particular technique is considered, only, after much
contemplation and by giving maximum emphasis on the type of application.
This whitepaper is written by Sanoj Swaminathan, Technical Lead - Quality Assurance,
RapidValue Solutions.
If you’d like to learn more on test case designs and testing techniques, please reach out to us
at [email protected] . We’d be happy to hear from you.
A Whitepaper by RapidValue Solutions - July 2016
03
03
12
About RapidValue
A global leader in digital transformation for enterprise providing end-to-end mobility,
omni-channel, IoT and cloud solutions. Armed with a large team of experts in consulting, UX design, application development, integration and testing, along with experience
delivering projects worldwide, in mobility and cloud, we offer a wide range of services
across industry verticals. We deliver services to the world’s top brands, fortune 1000
companies, Multinational companies and emerging start-ups. We have offices in the
United States, UK and India.
www.rapidvaluesolutions.com
www.rapidvaluesolutions.com/blog
+1 877.643.1850
[email protected]