Software testing and development for
intended quality
Tero Pesonen
Objective



To show how software development, software testing and
intended quality depend on one another.
The research area includes, in addition to traditional
software development and service models, also the
emerging XaaS (Everything as a Service) architectures,
technologies and service models.
The project results help the participating companies in
improving the efficiency of their quality management
and software testing and hence the efficiency of their
software development as a whole.
Testing techniques
 Testing as a service

Research Problem
Assessment framework through which each OU is evaluated
Intended
Software Quality
Software
Development
•Software
Products
ISO/IEC 12207
ISO/IEC 15504
•New Services
SaaS
ISO/IEC 25010,
Software
Product Quality
Software Testing
ISO/IEC 29119, Testing
Spice, IEEE Std 1012
StaaS
Preliminary Findings

In the software developing OU, requirements
management plays an important role in achieving
intended quality.
Requirements act as the basic communication medium
through which the customer/user, developers and testers
concretize different aspects of quality.
 Requirements are never throughout, and different
stakeholders interpret them differently. Requirements
change.
 Successful requirements management contributes positively
to achieving the targeted quality by reducing overhead
caused by doing wrong work.

Preliminary Findings




Building ”good” quality is different to building targeted
type and level of quality as efficiently as possible.
For many organizations, concretizizing intended quality
attributes (also non-functional) is not yet as well
implemented as actual development and testing
activities are.
Many organizations are also developer centric, and
here QA emphasizes functional quality.
Intended quality appears therefore to be also a
knowledge transfer issue within the development
process, not just a testing or software development
challenge, as often perceived.
Preliminary Findings

Agile methods favour direct, short development cycles
that produce immediate feedback on quality for testers,
developers and users.
Agility establishes an efficient communication channel.
 Non-agile methods seem to require stricter, more formal
quality management approaches for similar level of success.


Large, complex, multistakeholder projects do not benefit
from agility to the same extend, as the communication
overhead is rather increased than reduced in such a
context.

Well-defined waterfall-like processes may fare better.
Observations from the data



Developers and testers need to be able to communicate directly,
face-to-face, without effort; physical co-location is seen very
important for mutual productivity.
Developers should not review their own code; one becomes blind
to one’s own mistakes. That time is better spent reviewing a
colleague’s work.
Requirements management solutions (e.g. Shared management
centers) seem a good starting point in improving the efficiency of
the overall development process.


Note: How is non-functional quality communicated in the solution?
The same organization may need to apply different processes to
different types of development work, not just have all
development governed by one procedure.


Rapid, explorative development work benefits from agility.
Maintenance work can be more strictly phased and documented.
Observations from the data

In projects were testing is done as a service, it is paramount
that no communication obstacles are created between
testers, managers and developers, despite them being part
of different organizations.
Knowledge transfer between stakeholders should be considred
already in project planning.
 Develoeprs and testers should be communicated the formal
aspects and be inquired for feedback; they observe, notice and
draw conclusions – and frustrate when things do not work.


Cloud architectures support very quick development for
targeted quality with module re-use and well-defined API’s
But force vendor lock-in.
 Offer poor portability to non-cloud platforms if software grows.

Publication 1 / STX

Research question: How do developing organizations build intended
quality? Sub questions:



Do different size and type of OU’s build quality differently?
Do cloud environments affect how quality is built?
What is the role of testing methods and tools for quality?
Conference paper.
Contents:
1. Introduction
2. Related research
3. Research process




Qualitative, Grounded Theory method.
18 STX interviews + 10 interviews of supplementary data.
xx sheets of transcripted data.
Publication 1 / STX
4. Results
4.1 Quality is communicated as requirements, which are translated into
architecture and then into software code. Testing ensures the
requirements are, first, correct, and, second, being addressed correctly.
4.2 For most organizations, the potential in improved requirements
management seems currently greater than that provided by better
testing techniques or other kinds of testing tools.
4.3 Process improvement seems to follow improved requirements
management.
4.4 Service-oriented OU’s view quality less formally than product-oriented
ones. Knowledge transfer plays a role in organization’s growth, and has
to be reckoned with.
4.5 Cloud environments facilitate certain kinds of development work and
aid in achieving targeted quality efficiently, but not without major
compromises.
5. Discussion and Summary.
Research: Next Steps
Assessment of testing
 On-going work:

Master’s Thesis by Aoke Li on the status of standardization in
China.
 Due to February 2012.

A new Master of Science Thesis study for STX will be started
this spring.
 Research topics:

Software product lines (ISO/IEC 2655x.)
 Distributed global software development.
 Cloud laboratory (external project with China.)
 Quantitative survey.
