TQS - Teste e Qualidade de Software
(Software Testing and Quality)
Course Positioning
João Pascoal Faria
[email protected]
www.fe.up.pt/~jpf
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Index
Course positioning with respect to ...

Capability Maturity Model Integration for Software Engineering
(CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon
University (CMU)
• To help organize quality related activities and terminology

Software Engineering Education Knowledge, Computing Curricula Software Engineering (CC-SE), IEEE Computer Society / ACM
• To help organize and put in context this (educational) knowledge area

Guide to the Software Engineering Body of Knowledge (SWEBOK),
IEEE Computer Society
• To help organize and put in context this (4 years professional) knowledge area

Certified Software Quality Engineer (CSQE) Body of Knowledge,
American Society for Quality (ASQ)
• To help organize and put in context this (8 years professional) knowledge area
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Capability Maturity Model Integration for Software
Engineering (CMMI-SW), Version 1.1


Developed by the Software Engineering Institute, Carnegie Mellon University
Designed to help organizations improve their product and service
development, acquisition, and maintenance processes
• Judge the maturity of the software processes of an organization
• Identify the key practices required to increase the maturity of these processes

Staged representation - measures process-improvement achievement across
an organizational unit using maturity levels (as in traditional SW-CMM)
• At maturity level n, an organization has achieved all the specific goals of the
process areas assigned to maturity levels  n (and generic goals assigned to
levels  n)
• To achieve each goal a set of (best) practices must be implemented


Continuous representation - measures process-improvement achievement
within individual process areas (e.g. configuration management) using
capability levels
The CMMI models are the result of an effort of integration of the
"traditional" CMM models (SW-CMM, etc.)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1, staged version
Maturity Levels
1) Initial. The software process is characterized as ad hoc, and occasionally even
chaotic. Few processes are defined, and success depends on individual effort and
heroics.
2) Managed. Basic project management processes are established to track cost,
schedule, and functionality. The necessary process discipline is in place to repeat
earlier successes on projects with similar applications.
3) Defined. The organization’s set of standard processes is established and improved
over time. Projects and organizational units establish their defined processes by
tailoring the organization’s set of standard processes according to tailoring
guidelines.
4) Quantitatively Managed. Detailed measures of the software process and product
quality are collected. Both the software process and products are quantitatively
understood and controlled.
5) Optimizing. Continuous process improvement is enabled by quantitative feedback
from the process and from piloting innovative ideas and technologies.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1, staged version
Process Areas by Maturity Levels and Process Area Categories
Level 2)
Managed
Level 3)
Defined (*)
Organizational Process Focus
Organizational Process Definition
Project Planning
Integrated Project Management
Project Monitoring and Control
for Integrated Product and Process
Development (IPPD)
Supplier Agreement Management
Risk Management
Requirements Management (REQM)
Requirements Development (RD)
Organizational
Process
Performance
Level 5)
Optimizing
Organizational
Innovation and
Deployment
Quantitative
Project
Management
Technical Solution (TS)
Product Integration (PI)
Verification (VER)
Validation (VAL)
Configuration Management (CM)
Support
Level 4)
Quantitatively
Managed
Organizational Training
Engineering
Project
Managem.
Process
Managem.
Level 1)
Initial
Decision Analysis and Resolution
(DAR)
Measurement and Analysis (MA)
Causal Analysis
and Resolution
(CAR)
Process and Product Quality Assurance
(PPQA)
(*) also includes practices to institutionalize a defined process for each process area of level 2
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1
Verification and Validation (V&V)

Verification - ensures that the specified requirements are satisfied
• Confirms that (intermediate and final) work products properly reflect the
requirements specified for them
• Ensures that “you built it right”
• Occurs throughout the development of the product and work products, beginning
with verification of the requirements, progressing through the verification of the
evolving work products, and culminating in the verification of the completed
product

Validation - ensures that the product will fulfill its intended use
• Confirms that the (final) product, as provided, will fulfill its intended use
• Ensures that “you built the right thing”
• Applied mainly to the product and product components in its intended
environment, but can also be applied to work products (e.g., requirements,
designs, prototypes) selected on the basis of which are the best predictors of
how well the product and product component will satisfy user needs

Both validation and verification activities often run concurrently and
may use portions of the same environment
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1
Interactions Among V&V and other Engineering Process
Areas
REQM
Requirements
Product and product
component requirements
Alternative
solutions
Product
components
TS
RD
Product
PI
Customer
Requirements
Product components,
verification and
work products,
validation reports
VER
VAL
Customer needs
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1
Process and Product Quality Assurance (PPQA)

Quality Assurance - ensures that planned processes are implemented
• Involves objectively evaluating performed processes and work products against
applicable process descriptions, standards, and procedures, and communicating
and ensuring resolution of noncompliance issues
• Usually performed by a QA group that is independent of the project, or by peers
not directly involved in the development or maintenance of the work product
subject to evaluation
• Should begin in the early phases of a project (with the participation of the QA
group) to establish plans, processes, standards, and procedures that will add value
to the project and satisfy its requirements and the organizational policies and that
will be useable for performing QA evaluations, and to designate the specific
processes and work products that will be evaluated

V&V and PPQA may on occasion address the same work product but
from different perspectives; care should be taken to minimize
duplication of effort
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1
Interactions Among PPQA and other Process Areas
Organization
IPPD
Infrastructure
organizational
environment for
integration
OEI
Ability to develop
and deploy IPPD processes
and supporting assets
CAR
Process improvement
proposals
IPPD
knowledge
and skill
needs
Defects and
other problems
Quality and
noncompliance
issues
Measurements,
analyses
MA
Information
needs
Integrated work
environment and
people practices
Project
Management
process areas
Process
Management
process areas
PPQA
Processes and work products,
standards and procedures
All process areas
Selected
issues
Configuration
items,
change
requests
Baselines,
audit reports
Formal
evaluations
DAR
CM
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1
Terminology - artifacts

Product - any tangible output or service that is a result of a process and
that is intended for delivery to a customer or end user
• A product is a work product that is delivered to the customer

Product components - lower level components of the product, that are
integrated to “build” the product
• There may be multiple levels of product components
• A product component is any work product that must be engineered (requirements defined
and designs developed and implemented) to achieve the intended use of the product
throughout its life

Work product - any artifact produced by a process, including files,
documents, parts of the product, services, processes, specifications, and
invoices
• Examples of processes to be considered as work products include a manufacturing process, a
training process, and a disposal process for the product
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CMMI-SW, v1.1
Terminology – actors and activities

Customer - the party (individual, project, or organization) responsible for
accepting the product or for authorizing payment
• The customer is external to the project, but not necessarily external to the organization
• The customer may be a higher level project
• Customers are a subset of stakeholders

Stakeholder - a group or individual that is affected by or in some way
accountable for the outcome of an undertaking
• Stakeholders may include project members, suppliers, customers, end users, and others


Project manager - the person responsible for planning, directing,
controlling, structuring, and motivating the project and satisfying the
customer
Project - a managed set of interrelated resources that delivers one or more
products to a customer or end user
• This set of resources has a definite beginning and end and typically operates according to a
plan
• Such a plan is frequently documented and specifies the product to be delivered or
implemented, the resources and funds used, the work to be done, and a schedule for doing
the work
• A project can be composed of projects
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Index




Capability Maturity Model Integration for Software Engineering
(CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon
University (CMU)
Computing Curricula - Software Engineering (CC-SE), IEEE Computer
Society / ACM
Guide to the Software Engineering Body of Knowledge (SWEBOK),
IEEE Computer Society
Certified Software Quality Engineer (CSQE) Body of Knowledge,
American Society for Quality (ASQ)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Software Engineering Education Knowledge (SEEK),
Computing Curriculum – Software Engineering (CC-SE), 2004
Knowledge areas and knowledge units (1/2)

Computing Essentials (172 hours)
•
•
•
•


Computer Science foundations
Construction technologies
Construction tools
Formal construction methods
•
•
•
•
•
•
Modeling foundations
Types of models
Analysis fundamentals
Requirements fundamentals
Eliciting requirements
Requirements specification &
documentation
• Requirements validation
Mathematical & Engineering
Fundamentals (89 hours)
• Mathematical foundations
• Engineering foundations for software
• Engineering economics for software

Professional Practice (35 hours)
• Group dynamics / psychology
• Communications skills (specific to SE)
• Professionalism
Software Modeling & Analysis (53
hours)

Software Design (45 hours)
•
•
•
•
•
•
Design concepts
Design strategies
Architectural design
Human computer interface design
Detailed design
Design support tools and evaluation
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE
Knowledge areas and knowledge units (2/2)
Process and Product

Software Verification &
Validation (42 hours)

• Software quality standards (2 h)
• Reviews (6 h)
• Software quality processes (4 h)
• Testing (21 h)
• Process assurance (4 h)
• Human computer UI testing and
evaluation (6 h)  IPC
• Product assurance (4 h)
• Problem analysis and reporting (4 h)

Software Quality (16 hours)
• Software quality concepts and culture
(2 h)
• V&V terminology and foundations (5
h)

Assurance
Software Evolution (10 hours)

Software Management (19
hours)
• Management concepts
• Evolution processes
• Project planning
• Evolution activities
• Project personnel and organization
Software Process (13 hours)
• Process concepts
• Project control
• Software configuration management
• Process implementation
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE / SEEK
Software Verification & Validation (1/2)

Software Verification & Validation (42 hours) -> 20 h
"Software verification and validation uses both static and dynamic techniques of
system checking to ensure that the resulting program satisfies its specification
and that the program as implemented meets the expectations of the
stakeholders. Static techniques are concerned with the analysis and checking of
system representations throughout all stages of the software life cycle, while
dynamic techniques involve only the implemented system."

V&V terminology and foundations (5 h )  2h
•
•
•
•
•
Objectives and constraints of V&V
Planning the V&V effort
Documenting V&V strategy, including tests and other artifacts
Metrics & Measurement (e.g. reliability, usability, performance, etc.)
V&V involvement at different points in the lifecycle
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE / SEEK
Software Verification & Validation (2/2)

Testing (21 h )  14h
• Unit testing
• Exception handling (writing test cases to
trigger exception handling; designing
good handling)
• Coverage analysis (e.g. statement,
branch, basis path, multi-condition,
dataflow, etc.)
• Black-box functional testing techniques
• Integration Testing
• Developing test cases based on use cases
and/or customer stories
• Operational profile-based testing
• System and acceptance testing
• Testing across quality attributes (e.g.
usability, security, compatibility,
accessibility, etc.)
• Regression Testing
• Testing tools
• Deployment process (D)



Reviews (6 h )  1 h
• Desk checking
• Walkthroughs
• Inspections
Human computer UI testing and evaluation (6
h)  IPC
• The variety of aspects of usefulness and
usability
• Heuristic evaluation
• Cognitive walkthroughs
• User testing approaches (observation
sessions etc.)
• Web usability; testing techniques for web
sites
• Formal experiments to test hypotheses
about specific HCI controls (D)
Problem analysis and reporting (4 h)  3 h
• Analyzing failure reports
• Debugging/fault isolation techniques
• Defect analysis
• Problem tracking
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE / SEEK
Software Quality

Software Quality (16 hours) -> 14 h
"Software quality is a pervasive concept that affects, and is affected by all aspects of
software development, support, revision, and maintenance. It encompasses the
quality of work products developed and/or modified (both intermediate and
deliverable work products) and the quality of the work processes used to develop
and/or modify the work products. Quality work product attributes include
functionality, usability, reliability, safety, security, maintainability, portability,
efficiency, performance, and availability."

Software quality concepts and culture (2 h) 1h
•
•
•
•
•
•
•
Definitions of quality
Society's concern for quality
The costs and impacts of bad quality
A cost of quality model
Quality attributes for software (e.g. dependability, usability, etc.)
The dimensions of quality engineering
Roles of people, processes, methods, tools, and technology
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE / SEEK
Software Quality

Software quality standards (2 h)  2

• The nature of process assurance
• Quality planning
• Organizing and reporting for process
assurance
• Techniques of process assurance
• Quality Management Systems
• ISO/IEEE Standard 12207 Software Life Cycle
Processes
• Organizational implementation of standards
• IEEE software quality-related standards (D)

Software quality processes (4 h)  4
• Software quality models and metrics
• Quality-related aspects of software process
models
• Introduction/overview of ISO 15504 and the
SEI CMMs
• Quality-related process areas of ISO 15504
• Quality-related process areas of the SW-CMM
and the CMMIs
• The Baldrige Award criteria as applied to
software engineering (O)
• Quality aspects of other process models (O)
Process assurance (4 h)  4

Product assurance (4 h)  3
• The nature of product assurance
• Distinctions between assurance and
V&V
• Quality product models
• Root cause analysis and defect
prevention
• Quality product metrics and
measurement
• Assessment of product quality
attributes (e.g. usability, reliability,
availability, etc.)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE / SEEK
A proposed course on Software Quality Assurance (1/2)

Software Quality Assurance (37 hours)

34 h
• Broad coverage of software quality and testing.
• Course Description:
- Quality: how to assure it and verify it, and the need for a culture of quality.
Avoidance of errors and other quality problems. Inspections and reviews.
Testing, verification and validation techniques. Process assurance vs. Product
assurance. Quality process standards. Product and process assurance.
Problem analysis and reporting. Statistical approaches to quality control.
• Learning objectives
Upon completion of this course, students will have the ability to:
- Conduct effective and efficient inspections
- Design and implement comprehensive test plans
- Apply a wide variety of testing techniques in an effective and efficient
manner
- Compute test coverage and yield, according to a variety of criteria
- Use statistical techniques to evaluate the defect density and the likelihood of
faults
- Assess a software process to evaluate how effective it is at promoting quality
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CC-SE / SEEK
A proposed course on Software Quality Assurance (2/2)
• Suggested sequence of teaching modules:
-
Introduction to software quality assurance
Inspections and reviews
Principles of software validation
Software verification
Software testing
Specification based test construction techniques
White-box and grey-box testing
Control flow oriented test construction techniques
Data flow oriented test construction techniques
Cleanroom approach to quality assurance
Software process certification
• Sample labs and assignments
-
Use of automated testing tools
Testing of a wide variety of software
Application of a wide variety of testing techniques
Inspecting of software in teams; comparison and analysis of results
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Index




Capability Maturity Model Integration for Software Engineering
(CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon
University (CMU)
Computing Curricula - Software Engineering (CC-SE), IEEE Computer
Society / ACM
Guide to the Software Engineering Body of Knowledge (SWEBOK),
IEEE Computer Society
Certified Software Quality Engineer (CSQE) Body of Knowledge,
American Society for Quality (ASQ)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Guide to the Software Engineering Body of Knowledge
(SWEBOK), Trial Version 1.0, 2001

Developed by the IEEE Computer Society

Established with the following objectives:
1.
2.
3.
4.
5.
•
Promote a consistent view of software engineering worldwide.
Clarify the place—and set the boundary—of software engineering with respect to other
disciplines such as computer science, project management, computer engineering, and
mathematics.
Characterize the contents of the software engineering discipline.
Provide a topical access to the Software Engineering Body of Knowledge.
Provide a foundation for curriculum development and individual certification and
licensing material.
Describes knowledge expected after 4 years of experience as
software engineer
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
SEWBOK - Knowledge areas (1/2)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
SEWBOK - Knowledge areas (2/2)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
SEWBOK
Software
Testing
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
SEWBOK
Software Quality
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
SWEBOK
Terminology

Verification & Validation versus Software Quality Assurance
• SQA governs the procedures meant to build the desired quality into the products
by assuring that the process is well-planned and then applied as prescribed
• V&V is aimed more directly at product quality, in that it is based on testing that
can locate deviations and fix them. But it also validates the intermediate
products and therefore the intermediate steps of the software engineering
process
• SQA ensures that appropriate types of tests are planned, developed, and
implemented, and V&V develops test plans, strategies, cases and procedures


Testing is a technique for evaluating product quality and also for
indirectly improving it, by identifying defects and problems
Software testing consists of the dynamic verification of the behavior
of a program on a finite set of test cases, suitably selected from the
usually infinite executions domain, against the specified expected
behavior
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Index




Capability Maturity Model Integration for Software Engineering
(CMMI-SW), Software Engineering Institute (SEI), Carnegie Mellon
University (CMU)
Computing Curricula - Software Engineering (CC-SE), IEEE Computer
Society / ACM
Guide to the Software Engineering Body of Knowledge (SWEBOK),
IEEE Computer Society
Certified Software Quality Engineer (CSQE) Body of Knowledge,
American Society for Quality (ASQ)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
Certified Software Quality Engineer (CSQE) Body of
Knowledge, American Society for Quality (ASQ)


The CSQE is a professional who has comprehensive understanding of
software quality development and implementation, a thorough
understanding of software inspection, testing, verification, and validation,
and can implement software development and maintenance processes and
methods.
Requires 8 years of experience (3 if master's or doctorate)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
I. GENERAL KNOWLEDGE, CONDUCT, and ETHICS
A. Quality philosophy and principles
1. Benefits of software quality (C)
Describe how software quality
engineering can benefit an
organization.
2. Prevention vs. detection (C)
Describe how quality engineering
methodologies can reduce the
length of time for testing and can
influence other defect detection
methods.
3. Organizational and process
benchmarking (An)
Identify, analyze, and model best
practices at the macro
(organizational) and micro
(process and project) levels.
Identify and develop business
objectives, use metrics to monitor
their achievement, and provide
feedback to close the process
improvement loop.
B. Standards, specifications, and models
(Ap)
Identify and use software process and
assessment models, including ISO 9001,
ISO 15504, IEEE software standards,
IEEE/EIA 12207, SEI Capability Maturity
Model Integrated (CMMI), etc., in a
variety of situations.
C. Leadership tools and skills (Ap)
1. Organizational leadership
2. Team management
3. Team tools
4. Facilitation skills
5. Communication skills
D. Ethical conduct and professional
development
1. ASQ Code of Ethics (E)
2. Software liability and safety issues
(Ap)
3. Professional training and
development (Ap)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
II. SOFTWARE QUALITY MANAGEMENT - A
A. Goals and objectives
1. Quality goals and objectives (E)
Describe, analyze, and evaluate quality goals and objectives for programs,
projects, and products.
2. Outsourced services (E)
Define, analyze, and evaluate the impact of acquisitions, subcontractor
services, and other external resources on the organization's goals and
objectives.
3. Planning (E)
Identify, apply, and evaluate scheduling and resource requirements necessary to
achieve quality goals and objectives.
4. Software quality management (SQM) systems documentation (C)
Identify and describe various elements related to SQM system documentation.
5. Customer requirements (E)
Analyze and evaluate customer requirements and their effect on programs,
projects, and products.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
II. SOFTWARE QUALITY MANAGEMENT - B
B. Methodologies
1. Review, inspection, and testing (E)
Define, describe, evaluate, and differentiate between these defect detection
methods.
2. Change management methods (E)
Identify and apply various methods appropriate for responding to changes in
technology, organizations, environment, human performance, etc.
3. Cost of quality (COQ) (An)
Define, differentiate, and analyze COQ categories (prevention, appraisal,
internal failure, external failure) and their impact on products and processes.
4. Quality data tracking (E)
Define, describe, select, and implement information systems and models used
to track quality data in various situations.
5. Problem reporting and corrective action procedures (E)
Define, describe, analyze, and distinguish between these procedures for
software defects, process nonconformances, and other quality system
deficiencies.
6. Quality improvement processes (E)
Define, describe, analyze and distinguish between various defect prevention,
detection, and removal processes, and evaluate process improvement
opportunities in relation to these tools.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
II. SOFTWARE QUALITY MANAGEMENT - C
C. Audits
1. Program development and administration (C)
Identify roles and responsibilities for various audit participants, including team
leader, team members, auditee, auditor, etc.
2. Audit preparation and execution (C)
Define and distinguish between various audit types, including process,
compliance, supplier, system, etc. Define and describe various steps in the
audit process, from scheduling the audit through the closing meeting and
subsequent follow-up activities. Define and identify various tools and
procedures used in conducting audits.
3. Audit reporting and follow up (Ap)
Identify, describe, and apply the steps of audit reporting and follow up,
including the need for and verification of corrective action.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
III. SOFTWARE ENGINEERING PROCESSES
A. Environmental conditions
1. Life cycles
2. Systems architecture
B. Requirements management
1. Requirements prioritization and
evaluation
2. Requirements change management
3. Bi-directional requirements
traceability
C. Requirements engineering
1. Requirement types
2. Requirements elicitation
3. Requirements analysis and modeling
4. System and software requirements
specifications
D. Analysis, design, and
development methods and tools
1. Software design methods
2. Types of software reuse
3. Clean room and other formal
methods
4. Software development tools
E. Maintenance management
1. Maintenance types
2. Operational maintenance
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
IV. PROGRAM AND PROJECT MANAGEMENT
A. Planning
1. Project planning elements
2. Goal-setting and deployment
3. Project planning tools
4. Cost and value data
B. Tracking and controlling
1. Phase transition control techniques
2. Interpreting and reporting cost of quality (COQ) data
3. Tracking elements and methods
4. Project reviews
C. Risk management
1. Risk management planning methods
2. Risk probability
3. Product release decisions
4. Software security, safety, and hazard analysis issues
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
V. SOFTWARE METRICS, MEASUREMENT, AND ANALYTICAL METHODS –A,B
A. Metrics and measurement theory
1. Definitions
2. Basic measurement theory and techniques
3. Psychology of metrics
B. Process and product measurement
1. Process, product, and resource metrics (Ap)
2. Commonly used metrics (Ap)
Define and use metrics to measure various aspects of software, including
software complexity, lines of code (LOC), non-commented lines of code
(NCLOC), design defects, requirements volatility, system performance, etc.
3. Software quality attributes (C)
Identify and describe various criteria for measuring attributes such as
maintainability, verifiability, reliability, usability, reusability, testability,
expandability, etc.
4. Defect detection effectiveness measures (Ap)
Define, describe, and use defect detection measures such as cost, yield,
customer impact, etc., and track their effectiveness.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
V. SOFTWARE METRICS, MEASUREMENT, AND ANALYTICAL METHODS - C
C. Analytical techniques
1. Data integrity (S)
Define, use, and interpret various techniques to ensure the quality of metrics
data, its accuracy, completeness, timeliness, etc.
2. Quality tools (An)
Define, select, and use quality analysis and problem-solving tools such as flow
charts, Pareto charts, cause and effect diagrams, check sheets, scatter
diagrams, control (run) charts, histograms, root cause analysis, affinity
diagrams, tree diagrams, process decision program charts (PDPCs), matrix
diagrams, interrelationship digraphs, prioritization matrices, activity network
diagrams.
3. Sampling theory and techniques (An)
Describe, differentiate, and analyze various sampling techniques for use in
auditing, testing, product acceptance, etc.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - A
A. Theory
1. V&V planning procedures and tasks (S)
Identify and select various methods for verification and validation, including
static analysis, structural analysis, mathematical proof, simulation, etc.
Identify and analyze which tasks should be iterated as a result of proposed or
completed modifications.
2. V&V program (An)
Describe and analyze methods for managing and reviewing a V&V program,
including technical accomplishments, resource utilization, program status, etc.
3. Evaluating software products and processes (S)
Analyze and select various ways of evaluating documentation, source code, test
and audit results, etc., to determine whether user needs and project objectives
have been satisfied.
4. Interfaces (C)
Identify various interfaces used with hardware, user, operator, and software
applications.
5. Program performance and process effectiveness (An)
Identify and use various methods of examining performance and effectiveness.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - B
B. Reviews and inspections
1. Types (Ap)
Define, describe, and use various types of reviews and inspections, including
desk-checking, walk-throughs, Fagan and Gilb inspections, technical
accomplishments, resource utilization, future planning, etc.
2. Items (Ap)
Identify, describe, and use various review and inspection items, including
proposals, project charters, specifications, code, tests, etc.
3. Processes (Ap)
Define, describe, and use various review and inspection processes to examine
objectives, criteria, techniques, methods, etc.
4. Data collection, reports, and summaries (Ap)
Define, describe, and use terms related to data collection, including
preparation rates, defect density yield, phase containment, etc.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - C
C. Test planning and design
1. Types of tests (S)
Select, apply, and develop various types of test, including functional,
performance, regression, certification, environmental load, stress, worst case,
perfective, exploratory, etc.
2. Test tools (C)
Define and describe the application and capabilities of commonly used test
tools such as acceptance test suites, utilities (for memory, screen capture,
string-finding, file viewer, file comparison, etc.), and diagnostics (for
hardware, software, configuration, etc.).
3. Test strategies (S)
Identify, analyze, and apply various test strategies, including top-down,
bottom-up, black-box, white-box, simulation, automation, etc.
4. Test design (Ap)
Identify, describe, and apply various types of test design including fault
insertion, fault-error handling, equivalence class partitioning, boundary value,
etc.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - C
C. Test planning and design (cont.)
5. Test coverage of specifications (S)
Identify, apply, and develop various test coverage specifications, including
functions, states, data and time domains, etc.
6. Test environments (S)
Identify various environments and use tools such as test libraries, drivers,
stubs, harnesses, etc., in those environments, and describe how simulations
can be used in test environments.
7. Supplier components and products (Ap)
Identify the common risks and benefits of incorporating purchased software
into other software products. Use various methods to test supplier components
and products in the larger system.
8. Test plans (Ap)
Identify, describe, and apply methods for creating and evaluating test plans
including system, acceptance, validation, etc., to determine whether project
objectives are being met.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
VI. SOFTWARE VERIFICATION AND VALIDATION (V&V) - D
D. Test execution and evaluation
1. Test implementation (Ap)
Define, describe, and use various implementation elements, including
scheduling, freezing, dependencies, V-model, error repair models, acceptance
testing, etc.
2. Test documentation (Ap)
Define, describe, and use various documentation procedures, including defect
recording and tracking, test report completion metrics, trouble reports,
input/output specifications, etc.
3. Test reviews (S)
Describe, develop, and analyze various methods of reviewing test efforts,
including technical accomplishments, future planning, risk management, etc.
4. Code coverage metrics (Ap)
Define and apply various metrics including branch-to-branch, condition, domain,
McCabe's cyclomatic complexity, boundary, etc.
5. Customer deliverables (S)
Identify and select various methods for testing the accuracy of customer
deliverables, including packaged or downloaded products, license keys, user
documentation, marketing and training materials, etc.
6. Severity of anomalies (E)
Identify and select various methods for evaluating severity of anomalies in
software operations.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
VII. SOFTWARE CONFIGURATION MANAGEMENT
A. Configuration infrastructure
1.Configuration management
2.Library/repository processes
3.Defect tracking and library tools
B. Configuration identification
1.Configuration items
2.Baselines
3.Configuration identification methods
4.Software builds
C. Configuration control
1.Item and baseline control
2.Proposed modifications
3.Review and configuration control
boards (CCBs)
4.Concurrent development
5.Traceability
6.Version control
7.Configuration item interfaces
D. Configuration status accounting
1. Status reporting
2. Changes to configuration
items and baselines
3. Documentation control
E. Configuration audits
1. Functional configuration audit
2. Physical configuration audit
F. Release and distribution issues
1. Product release process issues
2. Packaging, production, and
distribution
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
CSQE Body of Knowledge
Levels of Cognition (Bloom's Taxonomy)






Knowledge Level (K) - Being able to remember or recognize terminology,
definitions, facts, ideas, materials, patterns, sequences, methodologies,
principles, etc.
Comprehension Level (C) - Being able to read and understand descriptions,
communications, reports, tables, diagrams, directions, regulations, etc.
Application Level (Ap) - Being able to apply ideas, procedures, methods,
formulas, principles, theories, etc., in job-related situations.
Analysis (An) - Being able to break down information into its constituent parts
and recognize the parts’ relationship to one another and how they are
organized; identify sublevel factors or salient data from a complex scenario.
Synthesis (S) - Being able to put parts or elements together in such a way as to
show a pattern or structure not clearly there before; identify which data or
information from a complex set is appropriate to examine further or from which
supported conclusions can be drawn.
Evaluation (E) - Being able to make judgments regarding the value of proposed
ideas, solutions, methodologies, etc., by using appropriate criteria or standards
to estimate accuracy, effectiveness, economic benefits, etc.
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›
References and further reading

http://www.sei.cmu.edu/cmmi/
• Capability Maturity Model Integration for Software Engineering (CMMI-SW),
Software Engineering Institute (SEI), Carnegie Mellon University (CMU)

http://sites.computer.org/ccse/
• Computing Curricula - Software Engineering (CC-SE), IEEE Computer Society /
ACM

http://www.swebok.org/
• Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE Computer
Society

http://www.asq.org/cert/types/csqe/index.html
• Certified Software Quality Engineer (CSQE) Body of Knowledge, American Society
for Quality (ASQ)
Teste e Qualidade de Software, Mestrado em Engenharia Informática, João Pascoal Faria, 2004
‹#›