Cmpe 589
Spring 2008
Software Quality Metrics
•
Product  product attributes
–
•
Process  Used to improve development and
maintenance processes
–
•
Size, complexity, design features, performance,
quality level
Effectiveness of defect removal, the response time
of the fix process
Project  Describe project
attributes/characteristics and execution
–
The number of developers, the staffing pattern,
cost, schedule and productivity
Software Quality Engineering
Seeks relationships among in-process
metrics, project characteristics, and endproduct quality (then use this to improve)
Product Quality Metrics
•
•
•
•
Mean time to failure (MTTF)
Defect density
Customer problems
Customer satisfaction
Product Quality Metrics
• Measured by
– the number of “bugs” (functional defects)
– How long the software can run before it
“crashes”
• MTTF is used in safety critical systems
– Measures the time between failures
• Defect density is used in commercial apps
– Defects relative to the software size
Defect vs. Failures
• Human mistakes: resulting in incorrect
software operation
• Failure- software module no longer carries
out intended function or performance level
Defect Density Metrics
•
Time frame – L.O.P – Life of Product
– Four years (?)
•
•
•
LOC – Lines of Code
KLOC – Thousand lines of code
Time Frame/ LOC or KLOC
Lines of Code
• Count only executable lines
• Count executable lines plus data definitions
• Count executable lines, data definitions, and
comments
• Count executable lines, data definitions,
comments, and job control language
• Count lines as physical lines on an input screen
• Count lines as terminated by logical delimiters
Lines of Code
• In the context of defect rate calculation
• Productivity studies
– The amount of LOC is negatively correlated
with design efficiency
• Enhancements and new versions
– LOC count for the entire product and the
changed code
– Defect tracking
Maintenance Process
•
•
Answer: compute defect rates for new
and old code
Change flagging- (comments) – ID
number linked to specific requirement,
version release number
Function Points
• A collection of executable statements that
performs a certain task together with
declarations of the formal parameters and
local variables manipulated by those
statements
• Originated by Albrecht at IBM in mid 70s
Function Points
• Weighted total of 5 major components:
– Number of external inputs (transaction types)X4
– Number of external outputs (report types)X5
– Number of logical internal files (the ones that user
may concieve, not the physical files)X10
– Number of external interface files (files accessed by
the apps but not maintained by it)X7
– Number of external inquiries (types of online inquiries
supported) X4
• These are average weighting factors
Function Points
• There are low and high weighting factors
depending on the complexity assessment of the
app.:
– External input: low complexity,3; high complexity, 6
– External output: low complexity, 4; high complexity, 7
– Logical internal file:low complexity, 7; high complexity,
15
– External interface file: low complexity, 5; high
complexity, 10
– External inquiry: low complexity, 3; high complexity, 6
Function Points
• Complexity is defined as set of standards
according to objective guidelines
– e.g. For the external output: if the no of data
element types is 20 or more and the no of file
types referenced is 2 or more, than
complexity is high.
• Calculate Function Counts (FC):
5
3
FC = Σ Σ wij  x ij
i=1 j=1
Function Points
• Scale from 0 to 5 to assess the impact of 14 general system
characteristics:
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Data communications
Distributed functions
Performance
Heavily used configuration
Transaction rate
Online data entry
End-user efficiency
Online update
Complex processing
Reusability
Installation ease
Operational ease
Multiple sites
Facilitation of change
Function Points
• Sum up the scores and calculate Value
Adjustment Factor (VAF)
VAF = 0.65 + 0.01Σc¡
14
c¡ = the score for general system characteristic i.
i=1
• The number of function points
is obtained:
FP = FC x VAF
International Function Point User’s Group
Standard (IFPUG, 1999)
Function Points
• Issues related to the function point metric:
– More research is needed on: meaning of FP
and the derivation algorithm
– Various other standards than IFPUG
– Time consuming and expensive
– Accurate calculation need certified FP
specialists
Function Points: example
• From Jones (2000). Software Assessments,
Benchmarks, and Best Practices
– The average no of software defects in US is approx. 5
per function point during the entire life cycle
– Defect removal efficiency is calculated by the level of
CMM
– The estimated defect rates per function point:
•
•
•
•
•
Level 1: 0.75
Level 2: 0.44
Level 3: 0.27
Level 4: 0.14
Level 5: 0.05
Customer Satisfaction
• Customer problems metric- problems using product
• Problems per user month (PUM) = (number of valid
defects/time period)
• Compute this every month if you want to lower PUM
• Ex. Number of installed licenses times the number
sold per month to reduce PUM
– Improve development process and reduce product defects
– Reduce non-defects oriented problems- improve, support,
usability, documentation, communication, training
– Increase sales of installed licenses
Scopes of Three Quality Metrics
Customer
Satisfaction
Defects
Customer
Problems