Taking the Mystery out of QoS Requirements
Solving the mystery: How to write a good Quality of Service
(QoS) requirement, also known as a Non-functional
Requirement?
QoS requirements must comply with all the requirement best
practices like feasible and unambiguous; But how?
Requirement Checklist
Requirement statements should conform to the following properties. References: [IEEE][SEI]
1. Correct [IEEE]. Each requirement must accurately describe the functionality to be delivered.
The reference for correctness is the source of the requirement, such as an actual customer
or a higher-level system requirements specification. A software requirement that conflicts
with a corresponding system requirement is not correct.
2. Unambiguous [IEEE]. The reader of a requirement statement should be able to draw only
one interpretation of it. Also, multiple readers of a requirement should arrive at the same
interpretation. Natural language is highly prone to ambiguity. Avoid subjective words like
user-friendly, easy, simple, rapid, efficient, several, state-of-the-art, improved, maximize,
and minimize.
3. Complete [IEEE]. No requirements or necessary information should be missing.
Completeness is also a desired characteristic of an individual requirement. IT is hard to spot
missing requirements because they aren’t there. Organize the requirements hierarchically to
help reviewers understand the structure of the functionality described, so it will be easier for
them to tell if something is missing.
4. Consistent [IEEE]. Consistent requirements do not conflict with other requirements or with
higher level (system or business) requirements.
5. Verifiable [IEEE]. See whether you can devise tests or use other verification approaches,
such as inspection or demonstration, to determine whether each requirement is properly
implemented in the product. If a requirement is not verifiable, determining whether it was
correctly implemented is a matter of opinion. Requirements that are not consistent, feasible,
or unambiguous also are not verifiable. Any requirement that says the product shall
“support” something is not verifiable.
6. Modifiable [IEEE]. You must be able to revise the requirements when necessary and
maintain a history of changes made to each requirement. This means that each requirement
be uniquely labeled and expressed separately from other requirements so you can refer to it
unambiguously.
Page: 1
Taking the Mystery out of QoS Requirements
7. Traceable [IEEE]. You should be able to link each software requirement to its source, which
could be a higher-level requirement, a use case, or a voice-of-the-customer statement. Also
link each software requirement to the design elements, source code, and test cases that are
constructed to implement and verify the requirement.
8. Necessary. Each requirement should document something the customers really need or
something that is required for conformance to an external requirement, and external
interface, or a standard. Another way to think of “necessary” is that each requirement
originated from a source you recognize as having the authority to specify requirements.
9. Prioritized. Assign an implementation priority to each requirement, feature, or use case to
indicate how essential it is to include it in a particular product release.
10. Requirements shouldn’t aggregate multiple requirements. Traceable requirements are
uniquely labeled and are written in a structured, fine-grained way, as opposed to large,
narrative paragraphs or bullet lists.
11. No Implementation Basis [SEI]. Functional requirements should be business-driven and
not include assumptions regarding the technical implementation. During the architecture and
design process, design and software constraints are considered.
Quality Attributes
Quality attributes are the overall factors that affect run-time behavior, system design, and user
experience. They represent areas of concern that have the potential for application wide impact
across layers and tiers. Some of the attributes are related to the overall system design, while
others are specific to run time, design time, or user centric issues.
To the extent to which the application possesses a desired
combination of quality attributes such as usability, performance,
reliability, and security indicates the success of the design and
the overall quality of the software application. [SEI Quality
Attribute Workshop]
The following table contains a list of candidate quality attributes. Not all are important to every
system/solution.
Quality Attributes
Accessibility
Measurable characteristics that indicate the degree to which a system is
available to, and usable by, individuals with disabilities. The most
common disabilities include those associated with vision, hearing, and
mobility, as well as cognitive disabilities. [NIST500-291]
Accuracy
The accuracy of any transaction or calculation performed
Page: 2
Taking the Mystery out of QoS Requirements
Quality Attributes
Adaptability
The ability of a system to adapt itself efficiently and its responsiveness to
changed circumstances
Agility
The ability of a system to both be flexible and undergo change rapidly.
[MIT ESD 2001]
Auditability
The transparency of a system with regard to external review
Availability
The degree to which a system or component is operational and
accessible when required for use. [IEEE 1990]
Compatibility
The ability of a system to integrate with earlier versions or with other
systems
Configurability
The ease of a system to be configured, customized, and modified given
the possibility of adjustment input parameters
Cultural Adaptability Cultural Adaptability is the ability to communicate with users in different
languages to handle different cultural and regional differences. It refers to
support for internalization.
Efficiency
Efficiency refers to how well the system utilizes processor capacity, disk
space and memory. An architect needs to assure that the systems make
efficient use of given resources as, for instance, 100% resource usage
can severely affect the performance of the system.
Extendability
The ease with which a system or component can be modified to increase
its storage or functional capacity. [IEEE 1990]
Extensibility
The solution is extensible if new features can be added without impacting
system entities already using the solution.
Flexibility
The ease with which a system or component can be modified for use in
applications or environments, other than those for which it was
specifically designed. [Barbacci, 1995]
Installability
The ease of system installation and the support for incremental system
updates / auto updates
Interoperability



The ability of two or more systems or components to exchange
information and use the information that has been exchanged. [IEEE
1990]
The capabilities to communicate, execute programs, or transfer data
among various functional units under specified conditions. [American
National Standard Dictionary of Information Technology (ANSDIT)]
The ability to share information and services. [TOGAF]
Localizability
The system features to support local context such as local languages,
currencies and number formats.
Maintainability
The ease with which a software system or component can be modified to
correct faults, improve performance or other attributes, or adapt to a
changed environment. [IEEE]
Page: 3
Taking the Mystery out of QoS Requirements
Quality Attributes
Manageability


Performance


Performance






Portability


Portability
The ability to manage a resource, or the ability of a resource to be
managed. [OASIS]
How efficiently and easily a software system can be monitored and
maintained in order to keep the system performing, secure, and
running smoothly.
The responsiveness of the system—that is, the time required to
respond to stimuli (events) or the number of events processed in
some interval of time. Performance qualities are often expressed by
the number of transactions per unit time, or by the amount of time that
it takes to complete a transaction with the system. [Bass, 1998]
The ability to track service and resource usage levels and to provide
feedback on the responsiveness and reliability of the network. [ETSI
and 3GPP Dictionary]
Recovery Time – The time required to recover from a failure
Response Time – The waiting time when a system receives a process
request
Shutdown Time – The time required for a system to shut down under
normal conditions
Startup Time – The time required for a system start up
Throughput – The success rate of message delivery over a
communication channel
Latency - The delayed response time experienced in a system
The ease with which a system or component can be transferred from
one hardware or software environment to another.[IEEE]
The capability of a program to be executed on various types of data
processing systems with little or no modification and without
converting the program to a different language. [American National
Standard Dictionary of Information Technology (ANSDIT)]
1) The ability to transfer data from one system to another without being
required to recreate or reenter data descriptions or to modify significantly
the application being transported.
2) The ability of software or of a system to run on more than one type or
size of computer under more than one operating system.
[Federal Standard 1037C, Glossary of Telecommunication Terms, 1996]
Privacy
Information privacy is the assured, proper, and consistent collection,
processing, communication, use, and disposition of personal information
(PI) and personally identifiable information (PII) throughout its life cycle.
[NIST Cloud Computing Reference Architecture and Taxonomy Working
Group]
Recoverability
The ability to restore a system, program, database, or other system
resource to a state in which it can perform required functions.
Page: 4
Taking the Mystery out of QoS Requirements
Quality Attributes
Reliability



The ability of a system to perform a required function under stated
conditions for a specified period of time. [IEEE]
Reliability is accuracy, availability, and recoverability.
The ability to reduce the magnitude and/or duration of disruptive
events to critical infrastructure. The effectiveness of a resilient
infrastructure or enterprise depends upon its ability to anticipate,
absorb, adapt to, and/or rapidly recover from a potentially disruptive
event. [Critical Infrastructure Resilience Final Report and
Recommendations, National Infrastructure Advisory Council,
September 8, 2009]
Resilience
The ability to reduce the magnitude and/or duration of disruptive events to
critical infrastructure. The effectiveness of a resilient infrastructure or
enterprise depends upon its ability to anticipate, absorb, adapt to, and/or
rapidly recover from a potentially disruptive event. [Critical Infrastructure
Resilience Final Report and Recommendations, National Infrastructure
Advisory Council, September 8, 2009]
Resiliency
Resiliency is an ability to withstand failures or disaster and mitigate them.
Resiliency = Availability + Recoverability (DTO, DPO) + Monitoring +
Security + Survivability + more
Reusability
The degree to which a software module or other work product can be
used in more than one computer program or software system. [IEEE]
Robustness
The degree to which a system or component can function correctly in the
presence of invalid inputs or stressful environmental conditions. [IEEE]
Scalability
Scalability is a desirable property of a system, a network, or a process,
which indicates its ability to either handle growing amounts of work in a
graceful manner, or to be readily enlarged. For example, it can refer to
the capability of a system to increase total throughput under an increased
load when resources (typically hardware) are added. [NASCIO]
Security
A measure of the system's ability to resist unauthorized attempts at usage
and denial of service, while still providing its services to legitimate users.
Security is categorized in terms of the types of threats that might be made
to the system. [Bass, 1998]
Simplicity
The degree to which a system or component has a design and
implementation that is straightforward and easy to understand. [IEEE
1990]
Supportability


Page: 5
Those actions related to the reliability, maintainability, and affordability
of component implementations, and the integrated logistics support
and configuration management required. [SEI]
Supportability is a combination of adaptability, auditability,
compatibility, configurability, instability, localizability, maintainability,
scalability, and testability.
Taking the Mystery out of QoS Requirements
Quality Attributes
Survivability
The ability of a system to remain in operation or existence despite
adverse conditions, including both natural occurrences, accidental
actions, and attacks on the system.
Testability
The degree to which a requirement is stated in terms that permit
establishment of test criteria and performance of tests to determine
whether those criteria have been met. [IEEE 1990]
Upgradeability
Upgradeability refers to how easy it is to upgrade (redeploy) the system.
Usability




Usability




The measure of a user's ability to utilize a system effectively.
[Clements, 2002]
The ease with which a user can learn to operate, prepare inputs for,
and interpret outputs of a system or component. [IEEE Std. 610.12]
A measure of how well users can take advantage of some system
functionality. Usability is different from utility, which is a measure of
whether that functionality does what is needed. [Barbacci, 2003]
The extent to which a product can be used by specified users to
achieve specified goals with effectiveness, efficiency, and satisfaction
in a specified context of use.
Accessibility – The ability to access functionalities of an IT system
Aesthetics – The beauty and quality of the user interface
Intuitiveness – The familiar and natural way of using the system
Consistency – The consistent use of methods to enhance user
experiences
SEI best practices recommend that a quality attribute trade-off analysis be performed. With
quality attributes you definitely “can’t have your cake and eat it too”. For example, security
constraints typically have a negative impact on performance.
The table provides a prioritization of the non-functional requirements that will influence the
design. It illustrates the trade-offs. For example, when flexibility (row name) is increased, then
efficiency (column name) may be reduced (down arrow) and maintainability may be increased
(up arrow), etc.
Page: 6
Taking the Mystery out of QoS Requirements
Quality Attribute Trade-off Analysis
Haught Technology Solutions, LLC
Quality of Service (QoS) Requirements
Quality of Service requirements capture conditions that do not
directly relate to the behavior or functionality of the solution
Quality of Service requirements describe environmental conditions
under which the solution must remain effective or qualities that the
systems must have.
They are also known as non-functional or supplementary
requirements.
Page: 7
Taking the Mystery out of QoS Requirements
Using Requirement Patterns
Using requirement patterns (Source: [Withall]) for developing good
QoS requirements is very helpful. Requirement patterns:
• Facilitates well-formed requirement structures
•
Provides a template for creating a requirement to address the
Quality of Service requirements
•
Provides at least one example per pattern
Sample 1: Average Transaction Time
Template: Average time of <transaction completion> shall not be greater than <time
period>.
Sample: Average time of returning a list of flights shall not be greater than ten seconds.
Sample 2: Throughput by Transaction Type
Template: The system shall accommodate <number> <transaction type> per <time period>.
Sample: The system shall accommodate 1,000 booked flights per minute.
Scalability is a desirable property of a system, a network, or a
process, which indicates its ability to either handle growing
amounts of work in a graceful manner, or to be readily
enlarged.
Scalability is the ability to still meet all of the other QoS
requirements when the load on the system increases (e.g.,
ability to handle expected peak load of 200 concurrent users)
The capacity of the system is the maximum number of
transactions and/or users that a system can handle and still
meet the QoS requirements
Template: Total number of defined users (all users defined in
Active Directory).
Example: The solution shall support at least 1000 defined users.
Page: 8
Taking the Mystery out of QoS Requirements
Template: Total number of active users (all users interacting with
the systems on a daily and/or weekly basis)
Example: The solution shall support at least 800 active users.
Template: Total number of concurrent users (Total number of user using the system
simultaneously during the peak window)
Example: The solution shall support at least 500 concurrent users.
Template: Total number of users by type.
Example: The solution shall support at least 1410 active Physicians.
Using Change Cases
But how do we handle those vague future requirements? For example, flexibility, extensibility,
etc?
•
•
•
•
Page: 9
Change cases (Bennett 1997) are used to
describe new potential requirements for a
system or modifications to existing
requirements
Change cases are modeled in a simple
manner
You describe the potential change to your
existing requirements, indicate the likeliness of
that change occurring, and indicate the
potential impact of that change
The name of a change case should describe
the potential change itself
Taking the Mystery out of QoS Requirements
Change Case Template Topics
• Name (One sentence describing the change)
• Identifier (A unique identifier for this change case, e.g.
CC17)
• Description (A couple of sentences or a paragraph
describing the basic idea)
• Likelihood (Rating of how likely the change case is to
occur (e.g. High, Medium, Low or %))
• Timeframe (Indication of when the change is likely to
occur)
• Potential Impact (Indication of how drastic the impact will
be if the change occurs)
• Source (Indication of where the requirement came from)
• Notes
Reference: [Ambler CUC]
In order to put some specification behind those goals of “extensibility” or “flexibility” consider:
1. How can the business change?
2. What is the long-term vision for our organization?
3. What technology can change?
4. What legislation can change?
5. What is your competition doing?
6. What systems will we need to interact with?
7. Who else might use the system and how?
Example Change Cases
a. A marketing campaign solution must
support any number of marketing
campaigns running concurrently.
b. An insurance policy application must
support new eligibility rules.
c. A medical scheduling system must
support new procedure types and
physician types.
Page: 10