INTRODUCTION TO CQA
– principles for quality control in complexity.
INTRODUCTION TO CQA
We are migrating towards complexity in how software is built and maintained. We need to develop multiple tools and techniques to help us stay in
business and be competitive. This makes it hard to find the balance between
quality and innovation, structure and chaos.
IT IS TIME TO TAKE A STEP forward and bring it all together. Today
working with quality assurance, it is necessary to build quality into the DNA
of software development. We introduce Continuous Quality Assurance
CQA™, to support companies to more efficiently put all knowledge together
and increase structured efficiency.
SHORT DESCRIPTION OF CQA
Continuous Quality Assurance embraces uncertainty and does not try to describe simple linear models for taking control. Instead it postulates principles and practices to function efficiently in a complex context. Same ideas
that have been promoted a long time ago within Lean but with additional
focus how to incorporate what we have learned from how test have evolved
in modern time. *
It is important to understand that these principles require more training,
some additional infrastructure and a disciplined culture. There is no need to
make additional investment to apply CQA, unless the context requires this.
If you are currently investing a lot in approaches within QA where linear
assumptions will give poor results in complex environments, it is probably
worth moving to CQA as soon as possible.
TIME-TO-MARKET BECOMES A MUST
“Soon is not enough”, the life time of products has decreased over the years.
The expectation of innovation has increased. The competitive and fast moving technology industry makes cost of delay one of the large risk factors.
Looking back in the past it took 100 year after the invention of the camera
before it became a commercial product. For cars it was drasti-cally reduced
to only 50 years. Nowadays it is a matter of a year or less until competition
has something similar on the market (e.g. Iphone and Android).
Continuing along this path, we can expect even shorter lead times in the
future. So investments that are waiting to be completed are a risk. We need to
change ourselves to be efficient in the new world of shorter cycles.
Others have said the same for years, we have seen Agile and Context Driven School of Testing promoting change for some time now. With new continuous techniques there is a need to take a more holistic perspective and put
things together. This is where CQA fits into the picture, integrating Agile,
Continuous, DevOps and Test together. Combined the different principles in
CQA help each other to reduce leadtimes.
”If you are currently investing
a lot in approaches within QA
where linear assumptions will
give poor results in complex
environments. It is probably
worth moving to CQA as soon
as possible”
* THE FIVE PRINCIPLES ARE
1. Business Value Centric
2. Holistic
3. Continuous
4. Team Adaptability
5. Single Source of truth
IT as core business
Many companies have used IT to support the business. Innovation within the
business happened without any significant changes to IT. Companies that
have had IT as a business, such as larger software companies, have been coupled with the innovation velocity of IT for a long time.
2
INTRODUCTION TO CQA
What is currently happening is that more and more companies can no
longer view their IT as merely supporting the business, they have to understand that it is a very active part of the business. New innovation is no longer
possible without adapting the IT services. Moore’s Law predicts a continued
linear growth of technology, which means that we can never stand still if we
want to stay competitive.
When we depend on other services that do not stand still, our overall environment becomes complex. To control and work with innovation in complex
environment, new approaches and
principles apply for efficiency. With
the high coupling of business and IT,
even companies that did not previously regard themselves as IT companies
must now improve their pace, their
patterns and their disciplines.
If focus on quality is not managed
properly, it can become too slow or
give too little valid information about
the emergent environment. The traditional approach has been to take
control with linear processes and slow
down the pace. This is not only less
efficient in a complex world, there is
also a risk of losing out to the competition. All this requires a new paradigm
shift to evolve.
The paradigm shift
For a long time software has been
treated as a physical object. Core ideas
on how to best create physical objects
came from the industrial factory model and economic efficiency theories
came from the American mechanical engineer Frederick Taylor (18561915). In today’s IT industry we have
“If focus on quality is not managed properly, it can become too slow
gone from discontinued to continued
or give too little valid information about the emergent environment.”
software development. The core of this
model has much more resemblance to
biological factors, and the evolutionary theory of “Survival of the fittest”. “Fit”
in this context refers to “most well adapted to the current environment”.
The transition from industrial factory model to a services oriented model
has been going on for a considerable period of time. The service oriented architecture and decreased lead times have put IT into an environment that is
continuously changed. Building something today is never done in isolation,
components are reused, customized and put together to create new products.
The dynamic character of this higher complexity has been identified and
also been embraced. New methods and techniques have been launched to be
able to work with software development in this environment. As with earlier
paradigm shifts, it has been the development discipline that has been driving
3
INTRODUCTION TO CQA
this change. The continuous methods approach solves many problems that
the the traditional system development life cycle had.
But without taking a more holistic perspective and still working with traditional project paradigms derived from the industrial factory production, it
is hard to truly make full use of these new methods. Quality Assurance has
been in the back seat of this change, but it should be in the front seat, driving
the change towards the requirement. We want to change this focus by taking
a more active part in developing the perspectives one must have to meet,
rather than fear, those changes.
MODERN TECHNIQUES TO RESOLVE
THE CHALLENGES
Specification by Example (SbE) is a method to define requirements by writing functional tests that illustrate realistic scenarios in readable format. It
focus on the business value for end users and encourages a continuous dialog
on how to achieve the goals of the software. As the results are executable
tests, they will verify the requirements for working with existing systems, and
they can also be used for compliance transparency.
Continuous Integration (CI) is a method that focuses on working with
the original codebase and modifying and integrating those modifications frequently. It embraces the fact that integration debt can be costly and aims to
mitigate this by make small deposits often. Continuous Integration is intended to work with Unit tests that are executed when building the system. The
unit tests can be written in advance of producing the code, as proposed by the
method TDD (Test Driven Development). **
** THE CORE TENETS OF
CONTINUOUS INTEGRATION
• Version Control
• Automatic Builds
• Discipline
• Collective Team Responsibility
4
INTRODUCTION TO CQA
There is a technical part of CI that focuses on automatic builds, but the
ideas behind the method are equally much about the culture of dividing work
tasks into smaller items. Sharing these changes with others for fast feedback
reduces unnecessary work that might go in the wrong direction. In order to
encourage knowledge sharing even further, CI principles are very often combined with Code Reviews.
Continuous Delivery (CD) is a method that goes beyond compilation to
include how software is released. Verifying installed software binaries with
automated business tests in the same way as customers would use it is a core
part of CD. Everything in the process should be traced and also be possible to
undo, in order to easily revert to a version that functioned. ***
The result of CD is to get to a stage where changes to the system, both
functional and in regard to infrastructure, are rapidly performed, easy to
trace and quick to undo. Because every change is noted in an audit trail, this
method is much favoured by compliance regulatory disciplines.
DevOps is derived from the two words Development and Operations.
It is an approach that reinforces communication and automations within operations. It is based on Continuous Delivery in order to reduce any manual
modifications to the configuration of the operation and to maintain control.
Thus it also focuses on how to work with Programmable Infrastructure and
Infrastructure as Code.
Difficulties with full implementation
The problem with these techniques is that they are mostly derived from the
Software Development discipline and focus on doing things right. But they do
not take a holistic approach and ignore the question “Are we doing the right
things?”. In addition, they are very often put into a timeboxed process that
can be hard to work with the continuous demands, so a lot of the value gets
lost with the exception of Continuous Integration.
“Having a more holistic
view and going beyond just
optimizing one step of the
cycle is necessary to create
efficiency.”
*** THE CORE TENETS OF
CONTINUOUS DELIVERY
• Configuration Management
• Version Control
• Dependency Management
• Build and packaging
• Test Strategy
Continuous quality assurance
To achieve full use of different techniques we need to make them possible
to cultural transmit according to the memetic theory. Ideas and beliefs that
works together and can steer efficiently, even when the context is complex
systems. A continuous, scalable process where quality focuses on business
value and adaption to the current environment.
Such a process utilizes modern methods and techniques from multiple
disciplines. It needs to be receptive to new methods and tools as they are developed. It aims to expedite the process of going from idea to customer usage
down to a few days, or even hours, without building quality debt.
5
INTRODUCTION TO CQA
THE PRINCIPLES IN CQA
Continuous Quality Assurance (CQA) is a concept and consists of 5 principles
and a process.
Business Value Centric
Every work starts from a business need. When the object was physical and
expected to be fin-ished once developed, with no or little changes happening
after being completed, it made sense to invest upfront to ensure completeness.
Today we need to consider the cost of delay as one of the major risks
to capitalizing on the investment. In a rapidly changing environment, it is
important to adapt often and make sure we deliver business value. In other words, that we are doing the right things, at the right time, i.e. when it
is asked for and that we minimize spending time on other things. The CQA
perspective invokes evidence based reasoning to provide a feedback loop on
business values as part of the creation software cycle.
Handling the queue and prioritizing among the jobs to be done is essential. With a short queue and small sized jobs focusing on lead time and improvements to reduce expensive parts. The new economics of IT are modified
to allow adaption to changes in the environment. The job is done not when
something has passed through an assembly line but when it has added value
to the enduser and has fulfilled objectives from the user perspective.
Infrastructure is just as equal a part of changes to the environment as
functional change. Infrastructure can be used to enable business value. Synchronizing development priorities to handle technical changes rapidly in order to adapt to the environment will be a success factor.
Holistic
The principles of CQA are not derived from just one discipline and it the
concept will not have no effect if one does not takeing a holistic approach.
to the organization. The marketing strategy must be in line with the release
of business value packages. The way requirements are created defined,and
transformed into development packages and queued must be in line with the
new steeringgovernance.
The overall architecture of the system must build in a quality perspective.
Incentive models in the company must encourage the right conduct,help and
not negatively reinforce wrong behaviors. Companies that succeed with this
will take control and use the changes in the environment to its their advantage.
Continuous
Working in a continuous mode is another game play for the organization and
new problems arise. For example, should we use a green-blue production
environment to direct users to the new version?
With the larger set of information that exist in complex system, quantities of information and results are a challenge in itself. It is hard to fully
halt activities on system level. How do emergency stops work? How can one
6
INTRODUCTION TO CQA
analyze and explore the data? The relevance of these kinds of topics increases
as complexity increases.
Every step that is expensive can be improved. If the most expensive activity is compilation, then focus on improving this, if producing the correct
marketing materials takes a very long time, consider how this can be improved. Having a more holistic view and going beyond just optimizing one
step of the cycle is necessary to create efficiency.
Team adaptability
Focusing on CQA challenges the traditional ideas derived from Taylorism
that efficiency is created by individuals optimizing their own work tasks and
being given incentives to do so. Instead, the focus must be on collaboration,
communication and strength in diversified teams. It is not advantageous
that everyone should know everything, rather that the collective competence
grows and be used more actively.
The same person can have multiple roles to complete tasks. Developing
expert competence and at the same time working collectively in cross functional teams is a change for many people and it is important to work with this
change. Team adaptability focus on cultural changes and cognitive challenges, competences and skills with individuals collaborating.
Single source of truth
It is important to keep the actual service or product in sync with the collective competence. Traditionally, multiple sources have been used, such as
7
INTRODUCTION TO CQA
requirements specifications, design documents, working software, test specifications, marketing materials and so forth.
Production of any artefact that delivers business valuable packages is
part of the developement cycle, and the idea is to minimize lead times. This
principle is about keeping a lean perspective on the cycle and avoiding duplica-tion, yet maintain enough information and keep an audit trail of changes
for future references. “Specification by Example” is a technique that comes in
handy deriving require-ments and test from a single source.
THE PROCESS IN CQA
The starting point in this process is an idea that is intended be used by the
end-user. Feedback from usage may lead to improvements. In the end of the
life cycle the feature or system is removed when they are no longer needed.
This in itself is not new, the same concept was used for creating cars with
Taylor’s theory of labor productivity and economics.
Very few have put together the collaboration and interaction of the different disciplines to create a modern version of productivity economics,
which embraces the evolution of a system in an environment that is continuously changing and that is very complex. In the future of Quality Assurance, it is necessary to connect maintian the focus continuously throughout the lifecycle of the product or service lifecycle, not as a single phase
but adaptingve with to the evolution of the both the software, it needs to
emerge together with the path of evolution that and the follows the system.
There are five steps in the CQA process:
1. Create Minimum Viable Product (MVP)
Ideas come in all sizes. There are ways to encourage the dialog between users
and the delivery organization in order to transform the idea into something
that can be implemented. One is LeanUX, which creates Minimum Viable
Products (MVPs), whilst keeping a mindset of reducing wasteful efforts in
order to improve profit and reduce loss. During this step, prioritization is key
as well as embracing failure as a strategy to explore multiple paths.
THE FIVE STEPS
1. Create minimum viable
product (MVP)
2. Creating development
package (DP)
3. Develop and release (DP)
4. Integrate DP to MVP release
5. Feedback the usage
2. Create Develop Packages (DP)
This step breaks down the business value package into smaller increments
that can be placed in the development queue. It is also covers considering
marketing strategies. The idea is to decouple the deployment of a development package from the release of the business value package. To keep the
integration debt low, the newly created package should be possible to add to
production in order to start exploring its value and, if needed, stop investment if we can prove that future profits are not worth the investment.
3. Develop and release DPs
This step is about development and release. Design, architecture and continuous delivery are all methods and techniques that are used in this step to
keep the quality debt low.
4. Integrate DPs into MVPs
As mentioned in step 2, it is important to decouple deployment from release.
A continuous process changes the way we view marketing of our changes.
8
INTRODUCTION TO CQA
It also challenges the cognitive abilities of users. How should we avoid the
cognitive stress of handling changes?
5. Monitoring usage for feedback
Feedback is key in all steps of this process. Today’s lifestyle expects feedback,
and it can be controlled or it can be left to flow on its own in social media and
similar. Some techniques that have been used for this are A/B testing and
other production usage analyses.
A service oriented environment combines everything in the dialog between the service provider and the end user. This means that evaluation of
ideas goes beyond implementation, it must also address the full experience of
the changes. Testing aims to guide the service provider by informing them of
the gaps between expectation and results. CQA must therefore must work in
a broader way and adapt to the ongoing changes in the environment during
evaluating.
Scaling the cycle
The results from step 5 are input to step 1. The steps can be run in parallel
with multiple ongoing cycles. Queueing theory principles state that ideas belong to the arrival process (A) and the treatment of them is the server process
(C). In the modern economics of software development we can use mathematics from Queueing theories to optimize labor efficiency. Briefly stated,
that means small sized jobs never push utilization above 80% and focus on
Mean Time of Delivery.
Antifragility to handle failures
Nassim Nicholas Taleb coined the word ’anti-fragile’ to describe the situation
where there is a convex response, from a stressor or source of harm. In contrast to fragile systems that easily breaks from stressor, antifragile systems
strengthen themselves. These stressors are leading to a positive sensitivity to
volatility for the system. It is the same principles as an evolutionary theory
of the species. The natural selection that contributes to preservation of the
race is based on variance in the individuals and their struggle to survive. Put
another way, the risks to the individuals are not transferred to the risk of the
race. In software development, you similarly desire to minimize the systematic risk by allowing the components to be stressed and fail fast on their own.
You need to focus on isolation and Mean Time to Recover rather than Mean
Time between failures.
THE FUTURE
Our vision is that it is possible to go from idea to a delivered business value
package within an hour without building quality debt. The value of the feedback migratws internal interaction to customer centric conversations. We do
not see CQA as a phase but the beginning of a journey we are already on.
IT becomes part of our day-to-day business, quality demands increase with
leadtime pressure and with competition. By leading this process change, the
experience of this journey is promising and full of new possibilities.
9
INTRODUCTION TO CQA
ABOUT THE AUTHORS
Kennet Osbjer, Viktor Laszlo, and Anna Almén has collectively very broad
competence from everything of Product Management, Test Leaders, Development, Testing. All from multiple types of industry sectors from large software companies, exchange trading, governmental from more than 15 years
of experience each. As a cross functional team lots of types of challanges has
been brought into the development of the principles and practices presented
in this whitepaper.
ABOUT ADDQ
ADDQ Consulting specializes in quality assurance and improvement of system effectiveness. We make every day for our customers a little easier by focusing on quality and efficiency. ADDQ Consulting has approx. 100 quality
assurance specialists. We function as advisors in both the public and private
sectors and as hands-on specialists to make business critical systems work as
well as they can and as economically as possible.
10