Formal Descriptions of Software Measurement and Evaluation
A Short Overview and Evaluation
Reiner R. Dumke, Andreas Schmietendorf1, Horst Zuse2
Otto-von-Guericke-Universität Magdeburg, Postfach 4120, 39016 Magdeburg,
[email protected]
1
Fachhochschule Harz, Friedrichstr. 57-59, 38855 Wernigerode
[email protected]
2
Technische Universität Berlin, Franklinstr. 28/29, 10587 Berlin
[email protected]
Contents
1 Introduction ……………………………………………………….……….…………………………..
2
2 The Software Measurement Background …………………………………..……………………….. 2
2.1 The Software Engineering …………………………………………………………………………… 2
2.2 The Software Product ………………………………………………………………………………… 4
2.3 The Software Development Resources …………………………………………………………………. 6
2.4 The Software Development Process ………………………..…………………………………………. 7
2.5 The Use of the Software Product ……………………………………………………………………… 10
2.6 The Software Maintenance ……………………………………………………………………………. 11
2.7 Software Probabilities and Causalities ………………………………………………………………… 12
3 Some of the Existing Formal Approaches of Software Measurement ……….…………………….
3.1 Algebraic Approaches of Software Measurement ……………………………………………..……
3.2 Axiomatic Approaches of Software Measurement ……………………………………………………
3.3 Functional Approaches of Software Measurement ……………………………………………………
3.4 Rule-Based Approaches of Software Measurement ……………………………………………………
3.5 Structure-Based Approaches of Software Measurement ……………………………………………...
3.6 Information-Theoretic Approaches of Software Measurement ………….………………………..……
3.7 Methods of Statistical Analysis ……………………………………………………………………..…
3.8 A Short Summary ………………………………………………………………………………………
14
14
16
17
20
20
22
23
24
4 Software Measurement Systems and Approaches …………………………….……………………… 26
4.1 Aspects and Trends in Software Measurement ………………………………………………………… 26
4.2 The Formal Description of the Software Measurement System ……………………………….……… 31
4.2.1 The Measurement Goals ……………………………………………………………………………... 31
4.2.2 The Measurement Artefacts ………………………………………………………………………….. 33
4.2.3 The Measurement Methods ………………………………………………….………………………. 34
4.2.4 The Measurement Quantities ………………………………………………………………………… 42
4.2.5 The Measurement Values ……………………………………………………………………………. 43
4.2.6 The Measurement Units ……………………………………………………………………………… 44
4.2.7 The Measurement Experiences …………………………………………………………………..….. 45
4.2.8 The Measurement Tools …………………………………………………………………………..… 47
4.2.9 The Measurement Personnel ………………………………………………………………………… 49
4.3 Software Measurement Frameworks …………………………………………………………………… 50
4.3.1 A Formal Characterization of the ISO 15939 ……………………………………………..………… 50
4.3.2 The Zuse Measurement Framework ………………………………………………………..……….. 52
4.3.3 The CAME Approach ……………………………………………………………………..………… 54
5 Conclusions …………………………………………………………………………………………….. 55
6 References ……………………………………………………………………………………………….. 56
1
Abstract: The following paper describes the area of software measurement from a formal point of view as a
basic for application of software metrics in the different fields of software engineering. At first, we discuss the
measurement background addressing the different parts of software engineering as software product, software
process and software resources. A short overview of existing formal approaches of software metrics and
software measurement helps to motivate our general formal description. This formalization supports the
evaluation of the metrics or measurement level themselves. A short summary describes the benefits and open
tasks in software measurement formalization and application.
1 Introduction
The formal description of software measurement and evaluation combines some existing formal approaches like
•
Descriptive techniques of program construction
•
Software modelling based on the graph theory including semantic networks
•
Software engineering modelling based on the empirical approaches
•
Foundations of descriptive and operative statistics
•
Basics of the set theory and algebra
•
Finally, the measurement theory itself.
In following we will use some of these formal techniques in order to describe the fundamental aspects of the
software measurement. On the other hand, this formalization can be helpful to discuss some kinds of
interpretations, of application and of misinterpretations also – as pitfalls in the field of software evaluation
techniques.
2 The Software Measurement Background
2.1 Software Engineering
Basically, we should formulate the general background on software measurement as the software engineering
itself. The following definition is the classical IEEE description of software engineering in the following form
[IEEE 90]:
„Software engineering is the application of a systematic, disciplined, quantifiable approach to the
development, operation, and maintenance of software; that is, the application of engineering to
software. “
This definition leads us to the simple visualization of the software engineering components in the following
manner ([Dumke 03], [Marciniak 94], [Pfleeger 98]).
standards
(development)
methods
system of
measures
disciplined
systematic
quantifiable
experience
Software Engineering
Definition
engineering
area
engineer
software
engineering
tools
CASE
community
Figure 1: Basic characteristics of software engineering
2
Considering this characterization, we can formulate in the following simple structure of the software engineering
SE area as a system in general:
SE = (MSE, RSE) = ({SE-Methods, CASE, SE-SystemOfMeasures1, SE-Standards,
(2.1)
SE-SoftwareSystems, SE-Experience, SE-Communities}, RSE )
where RSE represents the set of all relations between the elements of the set MSE where the elements of MSE mean
in detail
SE-Methods: “Structured approaches to software development which include system models, notations,
rules, design advice and process guidance.” [Sommerville 04]
CASE: (Computer-Aided Software Engineering) “Software systems which are intended to provide
automated support for software process activities.”[Sommerville 04]
SE-SystemOfMeasures: A set of metrics and measures in order to measure and evaluate all aspects,
components and methodologies of the software engineering areas. [Zuse 98]
SE-Standards: The software engineering standards are a set of rules and principles as a foundation of
control and examination of components achieving special defined characteristics certified by an
consortium like IEEE or ISO. [Dumke 03]
SE-SoftwareSystems: A software system respectively a software product “is a purposeful collection of
interrelated components that work together to achieve some objectives” and requirements. It includes the
computer programs and the associated documentation. [Sommerville 04]
SE-Experience: The experience summarizes the general aspects of laws, principles, criterions,
methodologies and theories in software engineering in the different forms of aggregation, correlation,
interpretation and conclusion based on a context-depended interpretation. (derived from [Davis 95])
SE-Communities: The software engineering community involves the poeples, organisations, events and
initiatives in which interpersonal relationships made an integral part considering aspects or paradigms in
software engineering. [Figallo 98]
Based on (2.1) we can formulate the following examples, components and elements of RSE:
•
The process of producing new or extended experience in software engineering:
r (SESE − Experience ) ∈ RSE: SE-Methods × CASE × SE-SoftwareSystems → SE-Experience
•
The general activities in order to define new standards in the SE:
r (SESE − S tan dards ) ∈ RSE: SE-Methods × SE-SoftwareSystems × SE-Communities → SE-Standards
•
(2.2)
(2.3)
The process of extension the set of measures during the software development, maintenance or
application :
r (SESystemOfMe asures ) ∈RSE: SE-Methods×SE-SoftwareSystems ×systemOfMeasures→systemOfMeasures´(2.4)
•
The characterization of the software quality personnel as following:
r (SESQACommuni ty ) ∈ RSE: SE-Communities × systemOfMeasures → measurementStaff
1
(2.5)
We use this kind of notification adapted from the OO area for more mnemonics.
The kind of the functional reuqirements depends on the kind of the software system which we characterize
later.
3
3
2.2 The Software Product
The main intention of software engineering is to create/produce software products with a high quality for the
customers. A software systems or software product SP was developed by the software process SD and is based
of the supporting resources SR.
processIndicators
systemOfMeasures
standards
experience
applicationDomain
softwareProcess
systemRequirements
for software
softwareProduct
methods
lifecycle
management
programs
documentations
personnelResources
softwareResources
platformResources
developmentStaff
users
customers
COTS
ICASE
systemSoftware
hardwareInfrastructures
Figure 2: The general software development process
At first, we will define the software product as a (software) system as
SP = (MSP, RSP) = ({programs, documentations}, RSP)
(2.6)
where the two sets are divided in the following elements or components (without achieving the completeness)
programs ⊆ {sourceCode, objectCode, template, macro, library, script, plugIn, setup, demo}
(2.7)
documentations = {userManual, referenceManual, developmentDocumentation}
(2.8)
and RSP describes the set of the relations over the SP elements.
The given subsets could be described in following
4
developmentDocumentation = {documentationElements} ={productRequirements,
, productSpecification, productDesign, implementationDescription}
(2.9)
documentationElements ⊆ {model, chart, architecture, diagram, estimation, review,
audit, verificationScript, testCase, testScript, pseudoCode,
extensionDescription, qualityReport }
(2.10)
productRequirements = systemRequirement ⊆ {functionalRequirements, qualityRequirements,
platformRequirements, processRequi rements}
(2.11)
functionalRequirements ⊆ {execution, mapping, information, construction, controlling,
communication, learning, resolution, cooperation, coordination}3
(2.12)
qualityRequirements ⊆ {functionality, reliability, efficiency, usability, maintainability,
portability}4
(2.13)
platformRequirements ⊆ {systemSoftware, hardwareComponent, hardwareInfrastructure,
peripheralDevice, host}
(2.14)
This set of quality characteristics is related to the ISO 9126 product quality standard.
4
processRequirements ⊆ {developmentMethod, resources, cost, timeline, milestone, criticalPath, (2.15)
developmentManagement, lifecycleModel}
A simplified view of the software product shows the following Figure 3 of the product aspects during the
development and application that must be defined through the product requirements.
software architecture
components:
software operation
areas:
software documentation
characteristics:
human interface aspects
user interface
appropriateness
user interface
marketing documents
tutorials
problem domain
configuration
task
management
data
management
product
tasks
user
manual
data
accessing
development
documents
(technology, tests,
tools, supports)
distributed platform
readability
components
tasks behaviour
data basis
completeness
data handling
Figure 3: Simplified visualization of the product characteristics
This visualization could help us for further investigations of the detailed component and aspects of a software
product. The software product is related to the application domain which we explain later. Here, we can define a
software product as a software system as following ([Chung 00], [Dumke 03], [Horn 02], [Maciaszek 01],
[Marciniak 94], [Mikkelsen 97])
SE-SoftwareSystems ⊆ {informationSystem, constructionSystem, embeddedSystem,
communicationSystem, distributedSystem, knowledgeBasedSystem}
(2.16)
Some of the examples of the relations in RSP could we deriving as following
•
The process of the software testing on some software product components:
( test )
r SP
•
∈ RSP: sourceCode × verificationScript × testScript→ testDescription
(2.17)
The elements of the product design considering the necessary components:
( design )
r SP
∈ RSP:
(2.18)
architecture × review × template × library × pseudoCode→ productDesign
•
A special kind of a programming technique could be defined as following:
r (SPprogram min gTechnique ) ∈ RSP: template × macro → sourceCode
•
(2.19)
The process of the software testing on some software product components:
( implementa tion )
r SP
∈ RSP: coding × unitTest × integrationTest → implementation
(2.20)
The following figure summarizes the components and elements of the software product described in the text
above.
5
Figure 4: Components of the software product
2.3 The Software Development Resources
In order to develop a software product we need resources such as developer, CASE tools and variants of
hardware. Therefore, we define the software development resources SR as following
SR = (MSR, RSR) = ({personnelResources, softwareResources, platformResources}, RSR)
(2.21)
where the software resources play a dual role in the software development: as a part of the final system (as COTS
or software components) and as the support for the development (as CASE or integrated CASE as ICASE). The
following Figure 5 shows a possible distribution of the different characteristics addressed to the main parts of the
software development resources.
personnel resources
characteristics:
skills
communication
user
software resources
aspects:
compatibility
customer
paradigm
COTS
ICASE
platform resources
characteristics:
reliability
availability
(mobile)
computers
(hosts)
peripherals
development team
(test team)
system software
architectures
maintenance team
productivity
performance
networks
performance
Figure 5: Simplified visualization of the resources characteristics
We continue our definition as following
6
softwareResources = {COTS} ∪ {ICASE}
(2.22)
ICASE = CASE ∪ CARE ∪ CAME
(2.23)
where CARE stands for computer-aided reengineering and CAME means computer-assisted measurement and
evaluation tools. Considering the WWW aspects and possibilities for software development infrastructures based
on CASE environments, the set of CASE tools could be divided as following
CASEinfrastructure = { ( {UpperCASE} ∪ {LowerCASE} )environment
Further, we can define
}
(2.24)
UpperCASE = {modellingTool, searchTool, documentationTool, diagramTool,
simulationTool, benchmarkingTool, communicationTool}
(2.25)
LowerCASE = {assetLibrary, programmingEnvironment, programGenerator,
compiler, debugger, analysisTool, configurationTool}
(2.26)
Especially, we can describe the following uncompleted list of personnel resources as
personnelResources = {analyst, designer, developer, acquisitor, reviewer, programmer, tester, (2.27)
administrator, qualityEngineer, systemProgrammer, chiefProgrammer, customer}
SE-Communities = {personnelDevelopmentResources, ITadministration, softwareUser,
computerSociety}
Accordingly, some of the examples of the relations in RSR could we deriving in the following manner
•
The process of building an appropriate development environment:
( devEnv )
r SR
•
(2.28)
∈ RSR: ICASE × platformResources → developmentEnvironment
(2.29)
The defining of software developer teams for the agile development:
( agile )
r SR
∈ RSR: programmer × programmer × customer → agileDevelopmentTeam
(2.30)
Now, we summarize the different elements and components of the resources as the basics of the software
development and maintenance in the following figure.
Figure 6: Components of the software development resources
2.4 The Software Development Process
Now, we will define the software development process SD itself (note, that the concrete software process is
known as software project). At the beginning we will show the general process aspects in the following Figure 7.
7
software lifecycle
components:
software management
aspects:
milestones
controlling
problem definition
requirement analysis/
specification
design
... implementation
field test
phase aspects
versioning
project management
quality
configumanageration mament
nagement
maintenance management
evaluation
workflow
software methodology
characteristics:
suitability
support
ap- develop- upper
proach ment me- CASE
thodology
paradigm implemen- lower
tation me- CASE
thodology
efficiency
Figure 7: Simplified visualization of the process characteristics
So, we can define the software process SD as following (including the essential details of every development
component)
SD = (MSD, RSD) = ({developmentMethods, lifecycle, softwareManagement} ∪ MSR, RSD)
(2.31)
developmentMethods ⊆ {formalMethods, informalMethods} = SE-Methods
(2.32)
formalMethods ∈ {CSP, LOTOS, SDL, VDM, Z}
(2.33)
We can see a plenty of “classical” informal development methods as structured methods SAM. Actually, the
informal methods are based on the objects OOSE, the components CBSE, or the agents AOSE. Therefore, we can
define
informalMethods ∈ {SAM, OOSE, CBSE, AOSE}
(2.34)
and especially
SAM ∈ {SA/SD, Jackson, Warnier, HIPO}
(2.35)
OOSE ∈ {UML, OMT, OOD, OOSE, RDD, Fusion, HOOD, OOSA}
(2.36)
CBSE ∈ {DCOM, EJB, CURE, B-COTS, SanFrancisco}
(2.37)
AOSE ∈ {AAII, AUML, DESIRE, IMPACT, MAS, MaSE, MASSIVE, SODA}
(2.38)
The life cycle aspects could be explained by the following descriptions
lifecycle = {lifecyclePhase, lifecycleModel}
(2.39)
lifecyclePhase ∈ {problemDefinition5, requirementAnalysis, specification, design,
implementation, acceptanceTest, delivering}
(2.40)
lifecycleModel ∈ {waterfallModel, Vmodel, evolutionaryDevelopment, prototyping,
incrementalDevelopment, spiralModel, …, winWinModel}
(2.41)
Finally, the software management component of the MSD could be described in the following manner
softwareManagement = developmentManagement ⊆ {projectManagement,
qualityManagement, configurationManagement }
(2.42)
Note that the software development process could be depended or addressed to a special kind of a software
system. Hence, we can make the following characterization
SDinformationSystem ≠ SDembeddedSystem ≠ SDdistributedSystem ≠ SDknowledgeBased System
5
Problem definition is a verbal form of the defined system or product requirements.
8
(2.43)
Furthermore, some of the examples of the relations in RSD could we deriving in the following manner
•
The process of building an appropriate life cycle model:
( lifecycle )
r SD
•
∈ RSD: lifecyclePhase i × … × lifecyclePhase in → lifecycleModel
(2.44)
1
The defining of software development based on the waterfall modell:
( waterfall )
r SD
∈ RSD: problemDefinition × specification × design
(2.45)
× implementation × acceptanceTest → waterfallModel
•
The defining of software development based on the V modell:
( V mod el )
r SD
∈ RSD: (problemDefinition, softwareApplication)
(2.46)
× (specification, acceptanceTest) × (design, integrationTest),
× (coding, unitTest) → Vmodel
•
The characterization of the tool-based software development based on UML:
( UMLdev )
r SD
∈ RSD: UML × developmentEnvironmentUML × systemOfMeasuresUML
(2.47)
× experienceUML × standardUML → developmentInfrastructureUML
These descriptions lead us to the following general model of the software engineering considering the three
dimensions of the software methodology, the software technology and the related application domains or kinds
of systems.
Figure 8: Dimensions of the software engineering
Finally, the components and aspects of the software development process are shown in the following Figure 9.
Figure 9: Components of the software process
9
2.5 The Use of the Software Product
After the software development, the software product goes in two directions: at first (the original sense of a
software product) to the software application SA, second in the software maintenance SM. We define the
different aspects in following
SA = (MSA, RSA) = ({applicationTasks, applicationResources, applicationDomain} ∪ MSP, RSA)
where means
applicationTask ∈ {delivering, operation, migration, conversion, replacement}
(2.48)
(2.49)
applicationResources = {applicationPlatform, applicationPersonnel, applicationDocuments}
(2.50)
applicationPersonnel ⊆ {customer, user, operator, administrator, consultant, trainer}
(2.51)
applicationDomain ⊆ {organisationalDocument, law, contract, directive, rightDocument}
(2.52)
applicationDocument ⊆ {userManual, trainingGuideline, acquisitionPlan, setup,
damageDocument, troubleReport}
(2.53)
Based on these definitions, some of the examples of the relations in RSA could we deriving in the following
manner
•
The process of the first introduction of the software product as deliveration::
( deliver )
r SA
∈ RSA:
(2.54)
SP × trainer × applicationPersonnel × applicationPlatform → deliveration
•
The defining of software migration based on essential requirements:
( migration )
r SA
•
∈ RSA: productExtension × SP × migrationPersonnel→ migration
(2.55)
The characterization of software operation:
( operation )
r SA
∈ RSA: applicationPersonnel × applicationPlatform × SP
(2.56)
× user → operation
•
The defining of the outsourcing of the software operation by extern IT contractors:
( outsourcin g )
r SA
∈ RSA: systemInputs × contractors × systemFeedback → outsourcing
Figure 10: Components of the software product application
10
(2.57)
2.6 The Software Maintenance
The different aspects and characteristics of the software maintenance are summarized by the following formulas
SM = (MSM, RSM) = ({maintenanceTasks, maintenanceResources} ∪ SP)
(2.58)
where means
maintenanceTasks = {extension, adaptation, correction, improvement, prevention}
(2.59)
maintenanceResources = ICASE ∪ {maintenancePersonnel, maintenancePlatform}
(2.60)
maintenancePersonnel = {maintainer, analyst, developer, customer, user}
(2.61)
Accordingly, some of the examples of the relations in RSM could we deriving in the following manner
•
The process of building the extension activity of the maintenance:
∈ RSM: SP × functionalRequirements → SP(extended)
( extension )
r SM
•
The defining of software correction:
( correction )
r SM
•
( adaptation )
( perform )
∈ RSM : SP × platformRequirements → SP(adapted)
(2.64)
∈ RSM : SP × performanceRequirements → SP(improved)
(2.65)
The defining of software prevention:
( prevention )
r SM
•
(2.63)
The defining of software improvement:
r SM
•
∈ RSM : SP × qualityRequirements → SP(corrected)
The defining of software adaptation:
r SM
•
(2.62)
∈ RSM : SP × preventionRequirements → SP(modified)
(2.66)
The characterization of a special kind of software maintenance as remote maintenance:
( remoteMa int)
r SM
∈ RSM : ICASEremote × maintenanceTasks
× maintenancePersonnel → remoteMaintenance
Figure 11: Components of the software maintenance
11
(2.67)
2.7 Software Probabilities and Causalities
At the end of this section we will establish some special experience for motivating the software measurement
intentions ([Basili 01], [Boehm 00a], [Clark 02], [Davis 95], [Dumke 03], [Dumke 95], [Dumke 98], [Dumke
98a], [Endres 03], [Herbsleb 03], [Jones 91], [Kitchenham 02], [Kitchenham 97], [Pfleeger 98], [Putnam 03],
[Zuse 91]).
•
Problems of software management and application:
o
#(development method)IT area >> 1: leads to the problem in managing the software process
including the software personnel and CASE. But, we do not know the thresholds for the
acceptability.
o
More than 99 percent of all executing computer instructions come from COTS products. And, the
average COTS software product undergoes a new release every eight to nine months, with active
vendor support for only its latest three releases.
o
Nondevelopment costs, such as licensing fees, are significant, and projects must plan for and
optimize them.
o
The complexity of IT projects is divided in organizational complexity (structural and dynamic) and
in technological complexity (structural and dynamic) [Xia 04] and must be considered in this
context.
o
Note that the software product depends on the different development aspects such as the kind of
the development method m, the product class p, the variant of the process f, the type of (basic)
platform b, and the kind of the development team e. Hence, we can formulate the software product
elements in the following detailed form
MSP (p,m,b,e,f) = ⊔i {programi(p,m,b,e,f)} ∪
•
⊔j{documentationj(p,m,b,e,f)}
(2.68)
Probabilities in software development:
o
Product specification → formal product implementation: would be achieved with few or more
effort as verification. But the principles of customer satisfied development expressed by the
implication product requirements → product implementation is not kept automatically. That
means that the probability p(validation) << 1 should be the usual situation.
o
Applied test cases << all possible/necessary test cases: this well-known situation implicates an
effect of probability as p(correctness) << 1.
o
Probability of errors in programs: Halstead estiamtes that professional software includes three
errors per 1 KLOC source code (see [Halstead 77]).
o
Schedule probability curve: “The basic idea is to use the probability curves to let us back off from
the planned schedule for enough to provide the risk protection we need. Our work plan must
always be located on the time-effort trade-off line.” [Putnam 03]
o
Limitations of direct model checking: “Model checking and verification are usually performed on a
model of the verified system, rather than being applied directly. One reason for this is that without
introducing some abstraction, the amount of detail we need to consider quickly becomes extremely
large.”[Peled 01]
o
Process modelling by Bayesian networks: The characteristics of probabilistic networks like
Baysian entwroks are helpful in order to modelling software quality aspects and process-related
structures. [Fenton 02]
12
•
Causalities in software engineering (cited from [Davis 95]):
o
General principles: “Productivity and quality are inseparable.” “Communicate with the
customers/users.” “Change during development is inevitable.” “Technique before tools.” “Most
cost estimates tend to be low.” “What applies to small systems does not apply to large ones.” “A
system that is used will be changed.”
o
Software specification:”Prototypes reduce risk in selecting user interfaces.” “Requirement
deficiencies are the prime source of project failures.”
o
Software design: “Design for change, maintenance, and errors.” “Hierarchical structures reduce
complexity.” “Architecture wins over technology.” “Software reuse reduces cycle time and
increases productivity and quality.”
o
Software coding and testing: “Don’t test your own software.” “Don’t integrate before unit testing.”
“Instrument your software.” “Don’t errors personally.” “Online debugging is more efficient than
offline debugging.”
o
Software measurement: “Measurements are always based on actually used models rather than on
desired ones.” “Empirical results are transferable only if abstracted and packaged with context.”
“Measurement requires both goals and models.”
o
The following cause-and-effect diagram shows the classification of some aspects of the software
process [Florac 99].
problem reports not
logged in properly
cannot isolate software
artifact(s) containing
the problem
change control board
meets only once a week
changes decisions not
released in a timely manner
Collection
Evaluation
information missing
from problem reports
Resolution
cannot determine what needs
to be done to fix the problem
cannot replicate
problem
takes time to
make changes
must reconfigure
baselines
delays in approving
changes
Closure
it takes too
long to
process our
software
change
request
delays in shipping
changes and releases
delays enroute
Figure 12: A process classification cause-and-effect diagram
In the following section we will discuss about the current existing formal approaches of software measurement
and a system approach of software measurement itself.
13
3 Some of the Existing Formal Approaches of Software Measurement
Formal approaches for the software measurement frameworks can be divided in algebraic approaches,
axiomatic approaches, functional approaches, rule-based approaches, structure-based approaches and
information-theoretic approaches. Note, that this classification could be involved some effects of covering or
subjective interpretation.
In following we explain some examples of these measurement approaches. The software measurement itself is
directed at the process, product, and resources. These components were characterized by internal and external
attributes, and the measurement includes also the assessment and the prediction. The areas of software
measurement are the cost and effort estimation, the productivity measures, the quality control and assurance, the
data collection, the quality models and measures, the reliability models, the performance evaluation, the
algorithmic/computational complexity, and the structural and complexity measures.
3.1 Algebraic Approaches of Software Measurement
One of the well-known example in the metrics community is given by Shepperd in [Shepperd 95] and includes
the general formal description as
• an algebraic description of the measured model (mod stands for module)
new: → design
add: mod × design → design
(3.1)
• a general description of a metric
metric: design → nat
(3.2)
• a special description of a concrete metric (e. g. module counting)
m: mod
D: design
metric(new) = 0
metric(add(m,D)) = 1 + metric(D)
(3.3)
In [Shepperd 95] is given a full description of a module-based system design metric including the fan-in and fanout characteristics.
Hastings and Sajeev use an algebraic specification language for software design and derive a Vector Size
Measure (VSM) [Hastings 01]. The vector consists of two fundamental size attributes, i. e., functionality and
problem complexity. Functionality could be measured by the Halstead’s intention of the number of operands and
operators (Ops). The problem complexity of an ADT (abstract data type) is defined as the distinct number of
nonredundant semantic properties measured as the sum of the Ops in the semantic section of the ADT. Hastings
represents software size as a two-dimensional vector which has both magnitude and direction. Using plain vector
algebra, magnitude m is defined as
f 2 + c2
m=
(3.4)
where f is the functionality and c the problem complexity. The magnitude is the measure that takes problem
complexity and functionality into account in a balanced and orthogonal manner. The gradient defined as
g=
c
f
(3.5)
is a ration of problem complexity and functionality which tells us about the relative dimensions of systems, i.e.,
the characteristics of software systems.
As a generalized algebraic approach we can consider the measurement approach of Whitmire [Whitmire 97]. He
defines a theory of objects based on categories and derives the possible or appropriate measures and metrics.
This helps to evaluate the object-oriented development process itself. Most of the measures are based on a
distance-based approach using the measurement theory by Krantz (see [Krantz 89]).
14
A special approach by Wang and King uses the process algebra based on the CSP (communicating sequential
processes) description [Wang 00]. This approach considers the different process quality standards: CMM, ISO
9001, Boostrap and SPICE. Examples of the CSP-based model description are
•
CMM: The Capability Maturity Model description includes the different CMM levels (CLi) based on
the typical key process area (KPAj,k) in the following manner:
CL1 ≜ ∅
(3.6)
CL2 ≜ KPA2,1 ∥ KPA2,2 ∥ KPA2,3 ∥ KPA2,4 ∥ KPA2,5 ∥ KPA2,6
CL3 ≜ KPA3,1 ∥ KPA3,2 ∥ KPA3,3 ∥ KPA3,4 ∥ KPA3,5 ∥ KPA3,6 ∥ KPA3,7
CL4 ≜ KPA4,1 ∥ KPA4,2
CL5 ≜ KPA5,1 ∥ KPA5,2 ∥ KPA5,3
These basic process descriptions are used in order to define the algorithms of CMM evaluation which
are estimated in their performance themselves.
•
ISO 9001: This process evaluation is based on different subsystems (SSi) which include the evaluated
main topic areas (MTAj,k)
SS1 ≜ MTA1,1 ∥ MTA1,2 ∥ MTA1,3 ∥ MTA1,4 ∥ MTA1,5 ∥ MTA1,6 ∥ MTA1,7
(3.7)
SS2 ≜ MTA2,1 ∥ MTA2,2 ∥ MTA2,3 ∥ MTA2,4 ∥ MTA2,5
SS3 ≜ MTA3,1 ∥ MTA3,2 ∥ MTA3,3 ∥ MTA3,4 ∥ MTA3,5 ∥ MTA3,6 ∥ MTA3,7 ∥ MTA3,8
In the same manner like in the CMM performance evaluation, the general evaluation algorithm is
defined and is used to compare the ISO 9001 evaluation with the other ones.
•
BOOTSTRAP: This evaluation considers three process areas (PAi) divided in nine process categories
(PCj,k) based on 35 process evaluations (PRl,m,n). The first simple evaluation level can be characterized
as
PA1 ≜ PC1,1 ∥ PC1,2
(3.8)
PA2 ≜ PC2,1 ∥ PC2,2 ∥ PC2,3
PA3 ≜ PC3,1 ∥ PC3,2 ∥ PC3,3 ∥ PC3,4
The PC2,2 for example consists of the process sequence PC2,2,1 ∥ PC2,2,2 ∥ … ∥ PC2,2,10. The algoithmicbased description helps to estimate the evaluation performance effort.
•
SPICE: The SPICE process evaluation considers the process catergories (PCi) divided in customer
supplier criteria (CUSi,j), engineering criteria (ENGi,j), project criteria (PROi,j), support criteria (SUPi,j),
and organization criteria (ORGi,j). The evaluation can be described as
PC1 ≜ CUS1,1 ∥ CUS1,2 ∥ CUS1,3 ∥ CUS1,4 ∥ CUS1,5 ∥ CUS1,6 ∥ CUS1,7 ∥ CUS1,8
PC2 ≜ ENG2,1 ∥ ENG2,2 ∥ ENG2,3 ∥ ENG2,4 ∥ ENG2,5 ∥ ENG2,6 ∥ ENG2,7
15
(3.9)
PC3 ≜ PRO3,1 ∥ PRO3,2 ∥ PRO3,3 ∥ PRO3,4 ∥ PRO3,5 ∥ PRO3,6 ∥ PRO3,7 ∥ PRO3,8
PC4 ≜ SUP4,1 ∥ SUP4,2 ∥ SUP4,3 ∥ SUP4,4 ∥ SUP4,5
PC5 ≜ ORG5,1 ∥ ORG5,2 ∥ ORG5,3 ∥ ORG5,4 ∥ ORG5,5 ∥ ORG5,6 ∥ ORG5,7
In the same manner are defined general algorithms for the application of the different process evaluation
standards which help to compare the efficiency of the different approaches.
3.2 Axiomatic Approaches of Software Measurement
A (classical) axiomatic approach is given by Prather in [Prather 84]. The basic idea is the restricted program
constructs of the structured programming (the sequence, the selection, and the repetition). On this basis was
defined a (complexity) measure as
• measure(sequence) = term1,
(3.10)
• measure(selection) = term2,
• measure(repetition) = term3 .
The description of the measures includes the measure value for a simple statement. The measure value of a
program was executed by use of the three axioms above. This approach can also be used for the classification of
the software measures which consider the characteristics above [Tian 95].
Another axiomatic approach is given by Zuse ([Zuse 89], [Zuse 91] and [Zuse 92]) and was the first application
of the measurement theory consequently. The main idea is the definition of an empirical relational system and a
numerical relational system. Software measurement is a homomorphism as (see also [Poels 99] and [Whitmire
97])
object1 ≥empirical object2 ⇔ measure(object1) ≥numerical measure(object2)
(3.11)
The axioms of the weak order, the weak associativity, the weak commutativity, the weak monotonicity, and the
Archimedean axiom help to determine the scale types of a concrete software measure. The following figure
shows the ZD-MIS-Tool from Zuse that includes more than thousand metrics description and can be used for
teaching of measurement theoretic based software metrics analysis [Zuse 98].
Figure 13: The Zuse-Drabe measurement information system
16
This approach supports the full characterization of a measure including the correct application of statistical
analysis methods ([Dumke 94], [Zuse 03]).
Poels argues in the same manner and prefers the distance-based measures based on the mathematical meaning of
metrics spanning a metric space [Poels 99]. See the following definition: Let X be a set. The function δ: X × X
→ ℝ is a metric if and only if:
1.
2.
3.
4.
∀ x,y ∈ X: δ(x,y) ≥ 0
non-negativity
∀ x,y ∈ X: δ(x,y) = 0 ⇔ x=y
(3.12)
identity
∀ x,y ∈ X: δ(x,y) = δ(y,x)
symmetry
∀ x,y ∈ X: δ(x,y) ≤ δ(x,z) + δ(z,y)
the triangle inequality
If δ: X × X → ℝ is a metric than (X, δ) is a metrics space. Poels especially consider the measurement of objectoriented software systems.
3.3 Functional Approaches of Software Measurement
The notion of functrional measurement approaches intends to the description of the formulas which will be the
conclusion of measurement-based experience and/or laws.
A lexical analysis of source code was intended by Halstead [Halstead 77]. He consider the tokens with a value
are operands denoted by η1 as the number of unique operators and the algorithm operators denoted by η2 as the
number of unique operands in a programming language. The other direct measures are the N1 as the total usage
of all of operators and the N2 as the total usage of all of operands appearing in that implementation. Based on
these characteristics and the measure of vocabulary
η = η1 + η2
(3.13)
Halstead defined the following formulas of software characterization for instance
Program length: N = N1 + N2
(3.14)
Program volume: V = N log 2 η
(3.15)
Program level:
L=
V*
V
(3.16)
with V* as the minimal program volume assuming the minimal set of operands and operators for the
implementation of a given algorithm
Program effort: E =
V
L
(3.17)
Difficulty of implementation: D =
η1 N 2
2η 2
Programming time in seconds:
T=
E
S
(3.18)
(3.19)
with S as the Stroud number (5 ≤ S ≤ 20) which is introduced from the psychological science. Note that the
Halstead metrics leads to some difficulties considering the empirical-based scale characteristics.
17
The COCOMO of Boehm (see also [Boehm 00b]) is an example of a functional measurement approach. The
preprocess of measurement is used for estimation of development effort based on the main formula
effort = α LOCβ
(3.20)
α and β have special values for special project characteristics. The problem is to estimate the lines of code
(LOC). The formula is a summarizing of experience of the software development effort. A special form of the
COCOMO formula is for the personal effort in month (PM) for instance
n
PM = A × SizeE ∏ EMi + PMAuto
(3.21)
i=1
EM and E are scale factors and Auto stands for the part of reused code. Today, some of interesting versions of
COCOMO exist like COPSEMO (constructive phased schedule and effort model), CORADMO (constructive
rapid application development cost model), COCOTS (constructive COTS cost model), and COPROMO
(constructive productivity cost model) [Boehm 00b]. Interesting kinds of cost estimation analysis are the
investigation of the sensitivity(δPM/δEM) by Musilek et al. [Musilek 02] and the use of genetic programming
by Dolado [Dolado 01].
Another functional approach of software measurement is given by Ejiogu in [Ejiogu 91] and is based on the
problem refinement shown in the following Figure 14.
problem definition
subproblem2
subproblem1
subsubproblem1
subsubproblem2
(3.22)
. . .
subproblemn
. . . subsubprobleml
. . .
. . .
module1
module2
. . .
modulek
Figure 14: Hierarchical problem refinement by Ejiogu
The functional approach consists of the definitions of the measures as formulas such as height of the tree, and
characterizes the monadicity, the entropy, the cohesion and coupling of the modules, the degree of refinement,
the modularity, the maintainability, the test coverage, the reliability, and the level depended productivity.
A further functional approach of measurement is the function point method of Albrecht (see also in the
discussion by Jones in [Jones 91]) that was based in the execution of the unadjusted function points (UFP)
UFP = a × inputs + b × outputs + c × requires +
d × internal data + e × external data
(3.23)
with the factors divided in “simple”, “average” and “complex” as a ∈ {3,4,6}, b ∈ {4,5,7}, c ∈ {3,4,6}, d ∈
{7,10,15} and e ∈ {5,7,10} for every (software) component. The final calculations with adjustment factors such
as communication intensity, embedded characteristics, large data base etc. leads to the final function points. This
function points can be mapped to the personal month (effort) of the software product development with the
approximative execution of the personal month PM related to the function points FP by Bundschuh [Bundschuh
00] as
PM ≈ 0.015216 FP1.29
(3.24)
Today, some versions of this approach exist especially for simplification of the counting such as the COSMIC
Full Function Point (FFP) standard [COSMIC 03].
18
Another well-known functional approach considers the whole software life cycle as the software lifecycle
management (SLIM) by Putnam ([Putnam 78], [Putnam 03]) to the following formula for effort E estimation
−t 2
K 2t 2
E(t) = 2 te d
td
(3.25)
with K as effort for the whole life cycle in personnel month (PM), td as the duration of the system development
in years, and t as the time point of estimation during the life cycle.
The measurement intention of monitoring for the specification of real-time systems was described by Peters and
Parnas in a formal manner [Peters 02]. A real-time system can be specified by an environmental state function
based on the quantities Qi
S: Real → Q1 × Q2 × . . . × Qn
(3.26)
defined on all intervals of system operation, the monitored state function as the time t as
mt: Real → Q1 × Q2 × . . . × Qi
(3.27)
is derived from the environmental state function, by including only the monitored quantities, the controlled state
function
ct: Real → Qj × Qj+1 × . . . × Qn
(3.28)
describes the controlled quantities, the input state function
it: Real → I1 × I2 × . . . × In
(3.29)
representing the values of the input quantities during the system operation, and the output state function
ot: Real → O1 × O2 × . . . × Om
(3.30)
representing the values of the output quantities during system operation. Considering that M summarizes the mt ,
I the set of the functions it, C the set of the functions ct, O the set of the functions ot, and the relations IN ⊆ M ×
I and OUT ⊆ O × C it was create a software monitor that directly observes the target system software input and
output variables as INmon as follows
INmon = {((mt , ct), (it , ot)) | IN(mt , it) ∧ OUT(ot , ct)}
(3.31)
Furthermore, a system monitor is a monitor that observes (mt , ct) using its own input devices.
The idea of Munson in his formal description of the program operation [Munson 03] can also be considered as a
functional approach. Based on the user operations O are implemented software functions F described in the
following formula
F ( o ) = {f : F ∀IMPLEMENTS ( o , f )}
(3.32)
We assume a module structure of the software system S and can describe the common modules Mc which are
involved in all of the functionality of S as following
{
Mc = m : M ∀f ∈ F • ASSIGNS( f , m )}
(3.33)
Especially, the set of modules for the operation oi is:
{
M u( oi ) = m : M f ∈ F ( oi ) , c( f , m ) = 1}
19
(3.34)
The relation c shows the set of functions in a module. So we can derive the user operation profile and the system
functional profile which help us to consider the appropriate test strategies.
3.4 Rule-Based Approaches of Software Measurement
One example of this approach is given by Hausen in [Hausen 91] and is based of the definition of rules for the
given (software) product in the form
IF
predicate1 predicate2 . . . predicatel activity estimationexpert predicate
THEN
(3.35)
volumecomponent predicate
where the rules was defined for the component as activities, objects, functions, data, procedures, and variables.
In the same manner was defined the quality rules related to the special software components.
Another rule-based approach consists of the formal language rules by Jacob and Cahill in [Jacob 92] as attribute
grammar approach. The metrics rules are defined in the attributes and the underlying semantic functions. Such
attributes are for example
statement counting: p → statementincrement scount
(3.36)
decision counting:
p → if-statementincrement dcount
p → while-statementincrement dcount
etc.
The semantic function includes the final execution of the defined metric (for example as multi-dimensional form
etc.).
3.5 Structure-Based Approaches of Software Measurement
McCabe investigated the flow graph G in order to estimate the test paths including the test effort [McCabe 76].
He use the cyclomatic number as
V(G) = #edges - #nodes – 2 × #connectedComponents
(3.37)
The connectedComponents consider a module-based software structure and could be simplified as 1 for a first
approximation of determination of the independent text paths in a software (flow) structure. The McCabe
approach is well-known and exists in different variants for call graph, object-oriented structures and general
design structures.
The approach of Fenton and Pfleeger in [Fenton 97] is based on the flow graphs based on the following Dijkstra
structures.
P1
D0
D1
D2
D3
D4
Cn
. . .
if then
if then else
while
repeat
exit
case
Figure 15: The Dijkstra flow graphs for the structured programming
20
Based on this, it was defined an embedded metrics extension in the following manner
• a prime-based definition of program components, e. g. the Dijkstra graphs
• the metric execution for the sequencing of primes
• the metric execution for the nesting of primes
These axioms was put in a language form as a kind of definition for new metrics based on chosen prime
structures as following
metric <name>
define <equations as user definition (optional)>
m1 <prime definitions>
m2 <sequence definitions>
m3 <nesting definitions>
end
(3.38)
Another point of view is the derivation of analysable (graph) structures from the software specification or
implementation themselves. Whitty consider the formal specification language Z [Whitty 90] and suggests such
of the following kinds of graph interpretation for instance:
P = ∃ x.(A(x) ∧ B(x) ∨ C(x))
P = ∀x.(A(x) ∧ B(x) ∨ C(x))
A
A
∧
∧
B
B
∨
∨
C
C
∃
∀
(3.39)
Figure 16: Graph interpretations of logical expressions
The approach of Baudry et al. is directed on class/objects structures [Baudry 02]. At first he define the potential
class interaction from a class A to a class B iff ∃i and j, i ≠ j, such as A ℜi B and A ℜj B. The real object
interaction is explained as the same characteristic over the instantiated objects o1 and o2 from the classes A and
B. Further it was defined the following kinds of complexities:
Complexity of interaction: Let P1, …, Pn be different paths correspond to a class
interaction CI. The complexity of the interaction is linked to the complexity of the
different paths:
n
complexity(CI) = ∑ ( complexity( Pi ) ∑ complexity( P j ))
i =1
j >i
Complexity of a path in a class interaction: Let P be a path involved in a class
interaction. IHi, …, IHk be the inheritance hierarchies crossed by P, the complexity
of P is given as followed:
21
(3.40)
k
complexity(P) = ∏ complexity( IH i , P )
i =1
(3.41)
Complexity of a path going through an inheritance hierarchy: Let IH be an
inheritance hierarchy and P be a path crossing IH. The complexity of IH for P is the
addition of the complexity of dp1, …, dpm, the descendents-path in IH influencing
P’s complexity.
m
complexity(IH,P) = ∑ complexity( dp i )
i =1
(3.42)
Complexity of a descendents-path: Let dp be a descendent-path and h be the height
of dp, the complexity for dp is:
complexity(dp) = h(h-1)
(3.43)
Baudry shows that these complexity metrics are helpful in order to improve the characteristic of design
testability in object-oriented software systems.
3.6 Information-Theoretic Approaches of Software Measurement
Khosghoftaar and Munson are some of the pioneers in the application of information theory in software
measurement ([Khosghoftaar 92], [Khoshgoftaar 02], [Munson 03]). The basic idea is the application of the
entropy definition combined with the appropriate metrics in a dynamic manner.
For a discrete random variable x distributed according to a probability mass function p the entropy is defined as
nx
H(x) = ∑ pl ( − log pl )
(3.44)
l =1
where l is an index over the domain of x and nx is the cardinality of the domain of x. H(x) is the average
information per sample measurement object from the distribution. When all logarithms are base two, the unit of
measure is a bit. Because communications systems are an important application of information theory, many
results are concerned with finding optimal ways to code information for efficient transmission.
On the other hand, the notion of entropy can be a very helpful consideration the dynamic software measurement.
Basically, the program execution could be considered as a stochastic process [Munson 03]. Here, the probability
mass function p is determined by the situation of the executed operations or functions in a software system.
As a kind of measurement the graph abstractions of software Allen use the information theory in order to
evaluate the size, the length, the complexity, the coupling and the cohesion of such a software structure [Allen
02]. The defined formulas are
Size(mk | S) = ∑ ( − log p L( i ) )
(3.45)
⎛
⎞
Length(mk | S) = max ⎜ min (Size( p r ( ij )))⎟
i , j∈mk ⎝ r
⎠
(3.46)
Complexity(mk | S) = ∑ Size( S i ) − Size( m k S # )
(3.47)
Coupling(mk | MS) = Complexity(mk | MS*)
(3.48)
i∈mk
i∈mk
22
Cohesion(mk | MS) =
Complexity(m k | MS ° )
(3.49)
Complexity(m k (nk ) | MS ° )
where S is the software system, MS the modular system, S# the edge-only system graph, , MS* the intermoduleedge graph, MS° the intramodule-edges graph of the software system, and p(…) the special probability
characteristics.
Chapin introduces an entropy-metric in order to evaluate software systems that are based on COTS components
[Chapin 02]. Basically, he defines a L-metric in the following manner
M
L-metric = L(S) = ∑ L( mi ) − H ( S )
(3.50)
i =1
where S stands for the software system, mi means a system component, and H(S) represents the base-level
entropy of the system. L(S) is the entropy loading of the system itself. This approach is very helpful in order to
analyse the maintenance characteristics of a COTS-based software system.
3.7 Methods of Statistical Analysis
A statistical analysis approach can be characterized by the following general scheme of Evanco and Lacovara
for the data collection and analysis ([Evanco 94], see also [Foltin 98]):
Software
Development
Organization
Development Environment Data
(e.g. requirements volatility, reuse capabilities)
Project Data (e.g. faults, maintenance effort)
Software
Project
Utilities
Code, Design, Artefacts
Software
Artefacts
Software
Analyzer
Project
Data
Base
Figure 17: Infrastructure of data collection and analysis
The most used statistical methods are given in the following table (see also [Dao 02], [Dumke 04], [Fenton 02],
[Han 94], [Hanebutte 00], [Juristo 03], [Kitchenham 01], [Lei 03], [Pandian 04], [Singpurwalla 99]).
Type of methodology
Ordinary least squares regression models
Poisson models
Binomial analysis
Ordered response models
Proportional hazards models
Factor analysis
Bayesian networks
Application
Subsystem defects or defect densities
Library unit aggregation defect analysis
Defect injection probabilities
Defect proneness
Failure Analysis incorporating software characteristics
Evaluation of design languages based on code
measurement
Analysis of the relationship between defects detecting
during test and residual defects delivered
Table 1: Examples of methods of statistical analysis
23
In order to use statistical methods it is necessary to know the scale type of the measurement data. For the
correlation methods we must guarantee the following relations
scale type
correlation coefficient
Kendall or Spearman correlation
Pearson or multiple correlation
Pearson, multiple, and variance
ordinal scale type
interval scale type
ratio scale type
Table 2: Appropriate correlation coefficient for the different scale types
Other approaches include the use of classification methods such as pareto classification, factor-based
discriminant analysis, fuzzy classification or neural network approaches ([Ebert 96], [Koshgoftaar 94]).
The main methods of statistical data analysis could be classified as [Pandian 04]
•
data analysis in frequency domain,
•
data analysis in time domain,
•
data analysis in the relationship domain.
(3.51)
The statistical process control is an essential application area of statistical methods. The following Figure 18
shows the main components in this field relating to the Capability Maturity Model CMMI [Dumke 04].
Figure 18: SPC considering the CMMI approach
Note that an essential aspect of the (statistical) measure analysis is given by the metrics validation as a statistical
or application validation and as a predictability validation [Shneidewind 95].
3.8 A Short Summary
We can establish in the formal approaches the situation which is simplified presented in the following table
([Dumke 04], [Henderson 96], [Kitchenham 95] and [Zuse 98]).
24
approach
algebraic
axiomatic
functional
rule-based
structured-based
informationtheoretical
statistics
benefits
a well-defined metrics algebra
weakness
no independence of the development
paradigm
an exact definition of the metrics only few practicable results
characteristics
compact definition of experience
problem in their use for new
development paradigms
a well-defined metrics language
only few empirical evaluation
helpful formalization of interaction and sometimes only considered few
structure-based dependencies
(structural) aspects in the necessary
abstraction process
helpful consideration of system less experience data for system
structures and component interaction
comparison and analysis
essential methods in order to keep the sometimes only few knowledge about
metrics application through the the statistical background (distrimapping to the empirical aspects
butions and scale types)
Table 3: Benefits and weaknesses of formal measurement approaches
Finally, we summarize the different formal approaches including the researchers in a following mind map
overview.
Figure 19: Different kinds of formal measurement approaches
25
4 Software Measurement Systems and Approaches
4.1 Aspects and Trends in Software Measurement
Software measurement is based on different standards and methodologies currently. The following figure shows
the scope of the ISO/IEC 15939 with the feedback and integration aspects of measurement application [ISO
15939].
Requirements for M easurement
Inform ation Needs
Technical and
Management
Process
Measurem ent User Feedback
Information Products
Core Measurem ent Process
Establish &
Sustain
Measurement
Commitment
Com mitm ent
Plan the
Measurement
Process
Planning
Information
Perform the
Measurement
Process
Inform ation
Products &
Perform ance
M easures
Evaluate
M easurement
Measurement Experience Base
Evaluation Results
Im provem ent Actions
Scope of ISO/IEC 15939
Legend
Activity
Data Flow
Data Store
Figure 20: The measurement process standard (ISO 15939)
We use this informal description of the software measurement motivation, planning, performing and evaluation
in order to describe the software measurement processes in a formal manner. Before we start with this
formalization, we describe the general situation in software measurement as following ([Card 00], [Dumke 02],
[Kitchenham 96], [Munson 03], [Zuse 99], [Zuse 02]):
⇒ We don’t have any general system of measures in software engineering like in physics. Hence, we must
consider in the software development the rules of thumb, statements of trends, analogue conclusions,
expertise, estimations and predictions also.
⇒ We also don’t have any standardised measurement system which performs the system of measures.
Therefore, we must use the general techniques of assessment (continues, periodic or certified), general
evaluation, experiences and experimentation. Sometimes, the experimentation is not immediately used
for decision support, improvement or controlling. We also use the experimentation for understanding of
new paradigms or the cognition of new kinds of problems ([Basili 86], [Pandian 04], [Wohlin 00]).
⇒ Software measurement instruments are mostly not based on a physical analogy such the column of
mercury to measure the temperature. In the most cases, software measurement is counting ([Kitchenham
95], [Zuse 91]).
⇒ Software measurement has a context and is not finished with measurement values or the use of
thresholds. Software measurement can be a generic measurement and analysis process ([Card 00],
[Jacquet 97]).
⇒ Software measurement tries to cover all aspects in the software development, operation and maintenance
(such as in the TQM [Talley 91]) and is divided in three historical phases of measurement orientation
[Emam 98]: software product evaluation in the sixties and seventies, process evaluation in the eighties,
and resource evaluation in the nineties.
26
⇒ “In software engineering metrics area, we should place more emphasis on the validity of the
mathematical (and statistical) tools which have been (and are currently being) used in their development
and use. Areas which give cause for concern in the past include the use of dimensionally incorrect
equations, incorrect plotting of equations and consequent incorrect inferences, the sloppy use of
mathematical notation and of calculated values and the lack of underpinning mathematical models.”
[Henderson 96]
⇒ Empirical techniques are divided into informally observing, formal experiments, industrial case studies
and benchmarking exercises or surveys. “There is no objective way of ensuring that one program is
equivalent to another, so that it is impossible to ensure that experimental object (e. g. designs, programs,
modules etc.) are directly comparable. However, the characteristics of a specific experimental object
may influence the experimental results.” [Kitchenham 97]
⇒ “We observed that more than 20 percent of the papers in the journal IEEE Transactions on Software
Engineering have no validation component (either experimental or theoretical). ... the percentage
dropped from 36 percent in 1985 to 29 percent in 1990 and only 19 percent in 1995 ” [Zelkowitz 98]
⇒ Software measurement was sometimes realised in an empirical-less manner. Only the artefact or model
characteristics [Briand 96] as software metrics or as empirical-based measurement and evaluation [Zuse
98] as software measures were considered.
Some of the main investigations and activities in order to obtain an integrated software measurement and
evaluation process in the last years are ([Dimitrov 02], [Dumke 01], [Dumke 01b], [Dumke 02a], [Dumke 03a],
[Ebert 04], [Zuse 99]):
¾
The definition of a measurement maturity model [Comer 93] which divides in two maturity aspects: the
functional and the non-functional. This approach uses the process description by Kellner which was
transformed into the following diagram.
Time Effort
Auditors Cost Management
Controls, Constraints
Inputs
Outputs
Metrics
Methods, Procedures
Measurements
Verification
General/Specific
Automation collection procedures
Figure 21: Evaluation kernel of software measurement by [Comer 93]
¾
In order to define a really measurement process including dynamical characteristics, the static process
aspects (accuracy, fidelity, fitness, precision, redundancy, scalability, maintainability) and the dynamic
aspects (lifeness, robustness, fault tolerance, autonomy, responsiveness) are described by [Feiler 93].
¾
The definition of a measurement information system [Latum 98] that includes all metrics and their
measurement data relating to the different GQM-based software evaluation.
¾
The standard measuring procedure by [Benedicenti 98] considers especially the data analysis which is
described in the Figure 22 (see also [Pfleeger 98] for more details).
27
Measurement data set
Panels
(Normality)
Statistical analysis
(ANOVA, T-test)
(Non-normality)
Nonlinear models
(Neuronale nets)
Statistical data analysis
Figure 22: Measurement data analysis by [Benedicenti 98]
¾
The software measurement was more and more integrated in the efficiency and performance of the IT
process to obtain learning infrastructures [Eickelmann 00].
¾
The definition and application of measurement model life cycles on the area of the evaluation of
software technologies [Cantone 00].
¾
The definition of a new synergy between software product evaluation and the manufacturing views by
[Tervonen 97] visualised as fishbone diagram in the Figure 23.
Quality model for product view
Metrics
Relationships
Structure
Definitions
Hierarchy
Understanding links between views
Goals
Rules
Checklists
Metrics
Model for manufacturing view
Figure 23: Synergy between the product and manufacturing views
¾
The activities to define and modify the measurement approaches more complementary, such as the
GQM to GQM++ or GQM process ([Gray 97], [Solingen 99]).
¾
The integration of software measurement in the education in software engineering as a kernel part and
as motivated by Abran and Bourque in the SWEBOK (software engineering body of knowledge)
initiative [Bourque 99].
¾
In order to consider the essential aspects of the measurement theory for a validation-based measurement
and evaluation Zuse postulated the following rules and experiences for the practitioners [Zuse 03]:
28
1.
Measurement theory gives clear definitions of measures and connects quality
(empirical) properties with numerical ones by a homomorphism.
2. Measurement theory gives criteria for scales and proper statistical operations.
For example: Size measures should assume certain measurement structures, and,
be careful with the Halstead measures when you want to use them as size
measures.
3. The consistency property (among others important for distributed software
development) is one of the most important requirements for software
measurement and assumes certain measurement structures.
4. Additive and strong monotonic transformations measures are useful for
imperative languages.
5. For object-oriented measures additive measures are not useful (only for the
methods). We need for that modified measurement structures (function of belief).
Considering software measures, there is a fundamental difference between the
object-oriented and imperative paradigms.
6. The requirement of wholeness for product measures is a pseudo requirement.
However, for prediction it could be useful.
7. To ignore measurement scales is careless. Scales are everywhere if you want it
or not!
8. Not every measure can be validated.
9. Prediction functions cannot be defined arbitrarily. Very strong conditions have
to be valid.
10. Units and dimensions of measures are based on scales.
¾
The current intentions on the area of the software e-measurement are addressed to the following
initiatives and activities ([Abran 03], [Dumke 01a], [Dumke 02a], [Dumke 03b], [Dumke 04a], [Lother
02]):
•
•
The creating and hosting of e-measurement communities which support the IT companies in the
different cases of information, consultation and education.
This supports are developed in different Web contents like e-experience and/or e-repositories
which are very helpful for small IT companies. The following screenshot shows an example of this
form as a project data repository of the ISBSG [Abran 03].
Figure 24: Interval-directed estimation based on the ISBSG project data repository
29
•
The different measurement phases are only few supported currently as e-measurement services.
Further initiatives are necessary for improving this situation based on the many possibilities in the
World Wide Web.
•
A special kind of this service is the e-quality service which installation is also at the beginning
currently.
•
Another kind of e-measurement support is and could be the e-measurement consulting realized by
special provider or the e-measurement communities.
•
In the Web-based e-measurement support could be included an e-certification which would be
very efficient for the standardization of the quality process evaluations.
•
Finally, the Web promotes the kind of measurement e-learning. In following we show an example
of the SML@b (Software Measurement Lab at the University of Magdeburg) [SMLab].
Figure 25: The GQM tutorial support by the SML@b
In following we will discuss the different methods, models and technologies of software measurement in a
formal manner.
30
4.2 The Formal Description of the Software Measurement System
We define a software measurement system as following ([Card 00], [Dumke 94], [Dumke 01], [Jacquet 97],
[Kitchenham 95], [Potter 00], [Zuse 94], [Zuse 98]):
MS = (MMS, RMS) = ({G, A, M, Q, V, U, E, T, P}, RMS)
(4.1)
where are G the set of the measurement goals, A the set of measured artefacts or measurement objects, M the
set of measurement methods, objects or entities, Q the set of measurement quantities (normally we use ℝ as the
set of real numbers), V the set of measurement values (especially we have the situation Q = V ), U the set of
measurement units, E the set of measurement-based experience, T the set of measurement CASE tools
(respectively CAME tools), and P the set of the measurement personnel. Considering this kind of formulization
we can describe the following elements of the presented sets of the measurement system ([Munson 03], [Offen
97]). This description is based on some basically methods of formalization in the programming and software
context ([Denvir 92], [Ehrig 99], [Neuhold 92], [Nielson 99], [Zuse 98]).
First of all, the following diagram gives an overview about the different characteristics and aspects in software
measurement.
Measurement
Phases
Application
Evaluation
Analysis
Visualization
Measurement
onedimensional
Modelling
multidimensional
causal
concluded
Measurement
Approaches
Assessments
Experimentation
Estimation
Validation
Improvement
Controlling
Measurement
Activities
Figure 26: Software measurement dimensions
4.2.1 The Measurement Goals
The measurement goals G are the main intention of the software measurement. The elements of G could be
summarized as following (see also [Basili 86], [Fenton 97], [Munson 03], [Schmietendorf 02])
G = {understanding, comprehension, learning, proving, validation, improvement,
(4.2)
falsification, investigation, exploration, management, controlling, certification)
Considering the measurement goals in the process of software measurement the following implication for
producing experience holds
G ×M →E
(4.3)
Furthermore, the different measurement goals can be described shortly as following ([Dumke 04], [Fenton 93],
[Munson 03], [Whitmire 97], [Zuse 98]).
31
The measurement goal of understanding considers the essential knowledge about the measured artefacts of
products, processes and resources and is related mainly to the software user [Weinberg 92]:
unders tan ding )
r (MS
∈ RMS: A × M × Puser → E
(4.4)
The goal of comprehension describes the effort to understand some characteristics of the software artefacts by
the developer [Humphrey 00]:
comprehens ion )
∈ RMS: A × M × PITstaff → EIT
r (MS
(4.5)
In our context, learning is the extension of the software engineering knowledge based on different measurement
or experimentation methods related to the personnel resources or communities [Humphrey 97]. We define
learning )
∈ RMS: A × M × P × E → E’
r (MS
(4.6)
The measurement goal of proving considers the confirmation of any assumptions or hypothesis’ about the
measured artefacts [Singpurwalla 99]:
proving )
∈ RMS: A × M × E(hypothesis) → E
r (MS
(4.7)
r (MSfalsificat ion ) ∈ RMS: A × M × E(hypothesis) → False
(4.8)
and especially
The “validation of a software measure is the process of ensuring that the measure is a proper numerical
characterization of the claimed attribute; this means showing that the representation condition is satisfied.”
([Fenton 97], see also [Munson 03] and [Zuse 94])
validation ∈ {criterionOrientedValidation, contentValidation,
constructValidation, empiricalValidation}
(4.9)
criterionOrientedValidation = {internalValidation, externalValidation}
(4.10)
ernalValidation )
r (int
∈ RMS: Q × V × Ecriterion → Boolean
MS
(4.11)
externalVa lidation )
∈ RMS: V × U × Ecriterion → Boolean
r (MS
(4.12)
contentVal idation )
∈ RMS: G × V × U × E → Boolean
r (MS
(4.13)
constuctValidation )
∈ RMS: A × V × U × E → Boolean
r (MS
(4.14)
empiricalValidation )
∈ RMS: A × Mexperiment × E → Boolean
r (MS
(4.15)
The improvement is based on the artefacts evaluation and describes the kind of improved aspects and
characteristics of these artefacts [Kulpa 03]. The improvement could be one-dimensional or multi-dimensional
oriented. Note, that the improving is only valid for the considered aspect or attribute of a measurement object.
improvemen t )
∈ RMS: A × M × E → A(improved)
r (MS
(4.16)
The goal of investigation leads to the analysis of the given artefacts in the direction of assumed laws, rules and
hypothesis’ [Armour 04]:
investigat ion )
∈ RMS: A × M × E → E(hypothesis)
r (MS
32
(4.17)
The exploration considers the artefacts under a special point of view in order to recognize some of meaningful
new measurement goals and interesting artefacts themselves [Pandian 04]:
loration )
r (exp
∈ RMS: G × A × M × E → G’ × A’
MS
(4.18)
The management involves the activities of planning, assigning, coordination, assessment and correction of the
software processes including the appropriate resources [Weinberg 92]:
management )
∈ RMS: A × M × V × U → A(managed)
r (MS
(4.19)
The controlling considers the interactions of components which will provide a desired component or system
response as time progresses [Erdogmus 02]:
controllin g )
∈ RMS: A × M × V × U → V × U
r (MS
(4.20)
The measurement goal of certification considers the evaluation of the artefact by a special institutional
community and standardized assessment rules [Kenett 99]:
certification )
∈ RMS: A × Massessment × Ecriterion → A(certified)
r (MS
(4.21)
The terms of criterion and hypothesis are used in the usually mathematical manner, the terms of assessment and
experiment will be explained later.
Especially, the measurement process MP as one of the instantiations of the software measurement system
could be explained by the following sequence of relations
MP: (G × A × M)T,P → (Q × E)T,P → (V × U)T,P → E’× A’
(4.22)
4.2.2 The Measurement Artefacts
The measured artefacts A can be characterized shortly as ([Dumke 03b], [McGarry 02], [Munson 03],
[Schmietendorf 03], [Zuse 98])
A = {SP, SD, SR}
(4.23)
Therefore, we could specify some of the measurement activities considering the different components and
artefacts in the following manner:
directOOSEMeasurement )
r (MS
∈ RMS: AOOSE × Q → V × U
(4.24)
sourceCodeAssessment )
∈ RMS: G × AsourceCode × Massessment→ V t0 × U t0
r (MS
(4.25)
COTSTrendA nalysis )
∈ RMS: G × ACOTS × E → E’
r (MS
(4.26)
referenceManualMeasu rementAppl ication )
∈ RMS: AreferenceManual × V → A’referenceManual × E
r (MS
managed )
lifecycleManagement )
∈ RMS: Gmanagement × Alifecycle × E → A (lifecycle
r (MS
(4.27)
(4.28)
On the other hand, we have used a special characterization of the artefacts above in the following manner:
•
The abstraction or transformation of software artefacts and components as modelling, such as Amodel in
(4.77)
33
•
The involvement of special characteristics based on evaluation criteria or artefact comparison, such as
A(improved) in (4.16) or A(certified) in (4.21)
•
The embedding of the artefacts in special operation environments or processes, such as A(managed) in
(4.19)
4.2.3 The Measurement Methods
The measurement methods M could be classified as following:
M = {artefactBasedOperation, quantificationBasedOperation,
(4.29)
valueBasedOperation, experienceBasedOperation}
artefactBasedOperation ⊆ {modelling, measurement, experimentation, assessment}
(4.30)
quantificationBasedOperation ⊆ {transformation, regression, factorAnalysis,
calibration}
(4.31)
valueBasedOperation ⊆ {unitTransformation, correlation, visualization, analysis,
adjustment, prediction}
(4.32)
experienceBasedOperation ⊆ {trendAnalysis, expertise, estimation, simulation,
interpretation, evaluation, application}
(4.33)
The following figure shows the distribution of these kinds of measurement operations in the general
measurement and evaluation context.
software development component
artefactBased
Operations
artefact
model
measurement theoretical view
(statistical analysis)
quantificationBased
Operations
flow graph
drawings
call graph
charts
evaluation
criteria
valueBased
Operations
numerical relation
design
documents
evaluation
model
Scale
experienceBased
Operations
empirical relation
ESTIMATION
goal tree
costs
CALIBRATION
factor-criteria
tree
ADJUSTMENT
cause and effect
diagram
text schemata
source code
test tables
etc.
structure tree
code schemata
grade
CORRELATION
GQM paradigm
etc.
abstraction
(tool-based)
etc.
metrication
effort
VALIDATION
metrication
quality
actuality
etc.
abstraction
(expert’s report)
Figure 27: The both sides (numerical and empirical) of the measurement homomorphism
The different measurement phases and activities are described in following and we will start with the
artefactBasedOperation (FSM stands for formal specification method). Note, that we consider the different
operations only in the measurement context.
34
•
“Modelling is the reflection of the properties of a theory and, thereby of some reality of which that
theory is an abstract description.” [Marciniak 94] We consider the process of abstraction from the
(software) artefact to the special kinds of measurement-related models ([Endres 03], [Fenton 93],
[Gomez-Perez 04], [Nielson 99]).
modelling ⊆ {graphicalModelling, statisticalModelling, FSMmodelling}
(4.34)
For the graphical modelling we describe only the different kinds of models which are the result of the
graphical modelling techniques.
•
graphicalModelling ∈ {callGraph, controlflowGraph, dataflowGraph,
componentDiagram, processChart, mindMap, layerDiagram}
(4.35)
statisticalModelling ⊆ {probabilityModelling, reliabilityModelling,
deterministicModelling, strategicalModelling}
(4.36)
FSMmodelling ⊆ {paradigmRelatedModelling, operationalModelling}
(4.37)
paradigmRelatedModelling ⊆ {setBasedModelling, algebraicModelling,
logicalModelling}
(4.38)
operationalModelling ⊆ {sequentialModelling, concurrentModelling
dynamicalModelling}
(4.39)
“Software measurement is a technique or method that applies software measures to a (class of) software
engineering object(s) to achieve a predefined goal.” [Marciniak 94] The phase of measurement is the
kernel element of the software measurement system. We can establish the following general
characteristics ([Fenton 97], [Munson 03], [Potter 00], [Zuse 98]). A first description shows the
involved sets of the measurement system.
( measurement )
r MS
∈ RMS: A × Q → V × U
(4.40)
Furthermore, we can distinguish the following types of measurement
directMeas urement )
∈ RMS: A × Q → V × U
r (MS
(4.41)
indirectMeasurement )
∈ RMS: A × Q × V × U → V’ × U’
r (MS
(4.42)
The software measurement implies a (functional) relation or operation called a software measure or
especially a software metric (shortly measure and metric) ([Zuse 89], [Zuse 94]). We will define these
different kinds of relations. A metric could be defined by Whitmire in the following manner [Whitmire
97]: A metric is only a real-valued function δ: A × A→ ℝ which meets the following conditions for all a,
b, c, d ∈ A: δ(a, b) ≥ 0, δ(a, b) = 0 iff a=b, δ(a, b) = δ (b, a), and δ(a, b) + δ(b, c) ≥ δ(a, c). Note, in
mathematics, and especially in geometry, the term metric is used for describing special characteristics of
spatial dimensions. For some types of measures the term metric is imprecise and confusely. Therefore, we
prefer the term measure for describing the measurement activities.
The measurement process could be considered as a homomorphism between the measured artefacts and
the used metrics or measures µ [Zuse 98]. Let A = (A, ∙≥, ◦) be an empirical relational system, where A
is a non-empty set of objects, ∙≥ is an empirical relation on A and ◦ is a closed binary operation on A.
We can understand the term (A, ∙≥, ◦) as an extensive structure based on the following axioms
35
(A1)
(A2)
(A3)
(A4)
(A5)
(A, ∙≥) is a weak order
(4.43)
a ◦ (b ◦ c) ≈ (a ◦ b) ◦ c, axiom of weak associativity
a ◦ b ≈ b ◦ a, axiom of weak commutativity
a ∙≥ b ⇒ a ◦ c ∙≥ b ◦ c, axiom of weak monotonicity
if c ∙> d then for any a, b exists a natural number n such that a ◦ nc ∙> b ◦ nd,
Archimedean axiom.
Let B = (ℝ, ≥, ⊗) be a numerical (formal) relational system, where ℝ are the real numbers, ≥ a relation
on ℝ, and ⊗ stands for a binary operation on ℝ (such as concatenation). Then we characterize a measure
by Fenton [Fenton 93] as (based on the measurement theory by Krantz and Roberts ([Krantz 89], [Roberts
79]), see also [Zuse 91])
µ: A → ℝ
(4.44)
which means an empirical objective assignment of a number (or symbol) to an entity to characterize a
specific attribute of that entity. It’s hold µ(a ◦ b) = µ(a) ⊗ µ(b) and we can establish the following
ranking homomorphism based on the assumption of additivity as
a ∙≥ b ⇔ µ(a) ≥ µ(b)
(4.45)
The triple (A, B, µ) is called a scale and we can distinguish the following scale types based on special
transformation rules:
•
nominal scale: ((A,≈),(ℝ, =), µ)
(4.46)
where ≈ stands for a equivalence relation and = for a numerical relation (two objects
are equivalent or not). Pandian consider a typological scale where was identified types
or categories [Pandian 04]. The nominal scale and the typological scale are
summarized to the linguistic scales.
•
ordinal scale: ((A, ∙≥),(ℝ, ≥), µ)
(4.47)
interval scale: ((A × A, ∙≥),(ℝ × ℝ, ≥), µ)
(4.48)
ratio scale: : ((A, ∙≥, ◦),(ℝ, ≥, ⊗), µ)
where are valid the describe axioms of an extensive structure above.
(4.49)
where ∙≥ describes ranking properties,
•
where ∙≥ is a preference relation about the measured objects, entities or artefacts,
•
•
Experimentation is one of the empirical strategies for analysis and investigation. “Experiments are
launched when we want control over the situation and want to manipulate behaviour directly, precisely,
and systematically.” [Wohlin 00] As experimentation we can distinguish the following approaches:
survey )
∈ RMS: G × A → E
r (MS
(4.50)
caseStudy )
∈ RMS: G × A → EA
r (MS
(4.51)
controlledExperiment )
∈ RMS: G × A × E → E’
r (MS
(4.52)
The controlled experiment could be explained as following ([Basili 86], [Juristo 01], [Prechelt 01],
[Sheldrake 97], [Wohlin 00]):
controlledExperiment = {definition, planning, operation, interpretation}
36
(4.53)
•
definition = {motivation, object, perspective, domain, scope}
(4.54)
planning = {design, criteria, experimentMethod}
(4.55)
experimentMethod ∈ {simpleMethod, blindMethod,
doubleBlindMethod}
(4.56)
operation = {preparation, execution, analysis}
(4.57)
interpretation = {interpretationContext, extrapolation, impact}
(4.58)
interpretationContext ⊆ {statisticalFramework, studyPurpose,
fieldOfResearch, experimentEffects}
(4.59)
experimentEffects ⊆ {experimentatorEffect, placeboEffect, psiEffect}
(4.60)
In the assessment can be involved all kinds of measurement but the results are defined on a special time
point or moment. The main characteristics of the assessment are ([Kenett 99], [Pfleeger 98], [Putnam
92])
assessment )
r (MS
∈ RMS: G × A × M → V t0 × U t0
(4.61)
where t0 consider a special time point or interval which is the valid moment for the assessment. We
describe the different kinds of the assessment as
assessment ∈ {certifiedAssessment, frequentlyAssessment}
(4.62)
frequentlyAssessment ∈ {periodicAssessment, continuedAssessment}
(4.63)
The artefactBasedOperations are summarized in the following figure.
Figure 28: Different kinds of artefact-based measurement operations
The measurement phases and activities of quantificationBasedOperation are described in following ([Backhaus
96], [Berthold 03], [Fenton 93], [Pandian 04]).
•
The transformation includes all operations about the numbers produced by the measurement achieving
special measurement goals and intentions [Pandian 04]:
37
transforma tion )
r (MS
∈ RMS: G × Q × … × Q → Q’
transformation ⊆ { initialization, modification, tuning}
•
regression ∈ {linearRegression, logisticRegression, non-linearRegression}
(4.66)
(4.67)
The factor analysis considers the reducing of the large number of response variables to a smaller set of
uncorrelated variables as well as to interpret these newly created variables [Hanebutte 03]:
r (MSfactorAnal ysis ) ∈ RMS: Q × … × Q → Q’
factorAnalysis ∈ {exploratorialFactorAnalysis,
confirmatoricalFactorAnalysis, LISREL}
•
(4.65)
The “regression analysis is the process of determining how a variable, y, is related to on, or more, other
variables, x1, x2, … , xn. The y variable is usually called the response by statisticians; the xi’s are usually
called the regressor or simple the explanatory variables.” [Berthold 03]
regression )
∈ RMS: Q × … × Q → Q’
r (MS
•
(4.64)
(4.68)
(4.69)
The calibration of measurement data is the modification of the numerical relational system without
modifying the empirical relational system [Zuse 98]:
calibratio n )
∈ RMS: Q × V → Q’
r (MS
(4.70)
A short summarizing of the different operations based on the set Q is given in the following figure.
Figure 29: Different kinds of quantification-based measurement operations
The measurement phases and activities of valueBasedOperation are described in following ([Berthold 03],
[Feton 93], [Whitmire 97], [Zuse 98]):
•
The unit transformation is the modification of the measured value from one unit to another one [Juristo
03]. It could be based on conversion rules describing the transformation of one measure to another one
[Zuse 98].
unitTransf ormation )
∈ RMS:V × U → V’ ×U’
r (MS
•
(4.71)
The correlation considers the relationship between different variables and is usually based on the
correlation coefficient which characterizes the dependency between these variables [Kan 95].
correlatio n )
∈ RMS: Q × V → E
r (MS
(4.72)
38
•
The visualiziation is a process of data analysis using graphical, optical and other kinds of viewing
techniques [Geroimenko 03]:
visualizat ion )
∈ RMS: V × U → V(visualized)
r (MS
(4.73)
The characteristic visualized means that the measurement values are presented by a special kind of a
visualization object.
visualizationObject ∈ {barDiagram, kiviatDiagram, tieDiagram, cycleDiagram,
scatterPlot, paretoDiagram, histogram, topicMap, checkTable,
clusterMap, tileBar, treeMap, coneTree}
•
Analysis is “the process of integrating measures, decision criteria, and project context information to
evaluate a defined information need.” [McGarry 02]
analysis )
∈ RMS: G ×V × E → V ‘ × E’
r (MS
•
(4.74)
(4.75)
The adjustment is the modification of the empirical relational system without modifying the numerical
relational system based on (new) experience [Dumke 99].
adjustment )
∈ RMS: V × E → V’
r (MS
(4.76)
Typical kinds of analysis are the data analysis in frequency domain, in time domain and in the
relationship domain including general approaches like the Six Sigma model [Pandian 04].
•
“A prediction system consists of a mathematical model together with a set of prediction procedures for
determining unknown variables and parameters. The purpose of prediction is to provide managers with
a forecast of the quality of the operational software early in the development process so that corrective
actions can be taken, if necessary, when the cost is low.” [Zuse 98]
prediction )
∈ RMS: G × Q × V → V’
r (MS
(4.77)
Figure 30: Different kinds of value-based measurement operations
The measurement phases and activities of experienceBasedOperation are described in following ([Boehm 00b],
[Dumke 04], [Kenett 03]).
•
The trend analysis is a special kind of a prediction considering a large future-directed time period and
involving the artefacts [Kenett 99].
trendAnaly sis )
∈ RMS: G × A × E → E’
r (MS
39
(4.78)
•
The expertise or expert review is a kind of evaluation of software artefacts involving (defined) experts
themselves [Boehm 00b].
ertise )
∈ RMS: G × A × Eintuition × Pexpert → V
r (exp
MS
(4.79)
expertise = expertReview ∈ {questionnaires, delphiMethod,
widebandDelphiMethod}
•
(4.80)
“Estimation is a process that uses prediction systems and intuition for cost and resource planning. … It
is concerned about assumptions regarding the future and the relevance of history.” [Pandian 04]
estimation )
∈ RMS: G × A × Q × V → V’ × U
r (MS
(4.81)
The task of estimation could be called as Ρ with
Ρ = f(Μ, Σ )
(4.82)
where Μ = {µ1, µ2, . . . , µn} the set of measured input data and Σ = {σ1, σ2, ..., σm} the set of estimated
data. We can produce a multiple estimation approach as Ρ= {π1, π2, . . . , πk}. The different estimation
levels are shown in the following figure.
M=∅
Intuition
n≈m
n >> m
estimation
strong estimation
n << m
weak estimation
Σ=∅
(exact) execution
Figure 31: Phases of aspect estimation
We can distinguish the following methods of estimation [Boehm 00b]:
estimationMethod ∈ {topDownEstimation, bottomUpEstimation,
parkinsonEstimation, expertOpinionEstimation, analogicalEstimation,
algorithmicEstimation}
(4.83)
Especially, the general intention of the analogical estimation could be described in the following
schema:
given experience
given single estimation
σ1 = x1
σ2 = x2
. . .
π1 = f(x1, yi)
π2 = f(x2, yi)
. . .
πi-1 = f(xi-1-yi)
πi = yi
(4.84)
πi+1 = f(xi+1, yi)
. . .
πn = f(xn, yi) .
σn = xn
•
executed total estimation
The simulation considers a model-based execution of the considered artefacts or systems and leads to
approximated system characteristics and (simulated) system behavior [Erdogmus 02].
simulation )
∈ RMS: Amodel × Qparameters → V
r (MS
simulationMethod ∈ {discreteSimulation, continuousSimulation,
stochasticSimulation, deterministicSimulation}
40
(4.85)
(4.86)
•
The interpretation includes the extrapolation of the results from a particular sample to other
environments and should be visible in professional forums for feedbacks ([Basili 86], [Juristo 03]).
erpretatio n )
∈ RMS: V × U × E → E’
r (int
MS
interpretationMethod ⊆ {contingenceAnalysis, discriminanceAnalysis,
varianceAnalysis, regressionAnalysis, factorAnalysis,
clusterAnalysis}
(4.87)
(4.88)
This activity is one of the sources for the experience building for the different types of the IT
knowledge.
•
The purpose of evaluation is to provide any interested party with quantitative results concerning a
software artefact that is comprehensible, acceptable and trustworthy ([Hausen 93], [Kitchenham 96]).
evaluation )
∈ RMS: Mevaluation × V × E → E’
r (MS
evaluation ∈ {heuristicEvaluation, prospectiveEvaluation, empiricalEvaluation,
roundRobinEvaluation, snowballEvaluation, educationalEvaluation}
•
(4.89)
(4.90)
The measurement application includes the different types of presentation, distribution, operation and
reaction ([Dumke 99], [Kenett 99], [Wille 02]).
applicatio n )
∈ RMS: A × V → A’ × E
r (MS
(4.91)
The presentation and distribution extend the existing experience in the software engineering area.
Especially the kinds of application as operation or reaction represent a form of online measurement
[Pandian 04].
The summarizing of the experience-based operation is shown in the following figure.
Figure 32: Different kinds of experience-based measurement operations
Finally, we will summarize the different kinds of measurement with the descreasing accuracy defined in the
Magdeburg Shell Model (MaSMo).
41
expertise/trendAnalysis
analogicalEstimation
modelBasedEstimation
modelBasedSimulation
measurement at a pendant
measurement at the
origin artifact
Figure 33: The Magdeburg Shell Model MaSMo of measurement methods
4.2.4 The Measurement Quantities
The measurement quantities Q are derived from the measured artefacts and have no empirical meaning
([Fenton 93], [Whitmire 97], [Zuse 98]). Note, that the measurement scale type depends on the characteristics of
Q. We can formulate as the first step in the measurement process MP defined in (4.22)
G ×A×M→Q
(4.92)
The set of Q does not fulfil the scale characteristic. But it determines the essential basis for the potential scale
type [Dumke 99]. Relating to the numerical relative as (ℝ, ≥, ⊗) we can distinguish the following terms (see
(4.40) to (4.43))
•
Potential nominal scaled: (Q, =)
(4.93)
•
Potential ordinal scaled: (Q, ≥)
(4.94)
•
Potential interval scaled: (Q × Q, ≥)
(4.95)
•
Potential ratio scaled: (Q, ≥, ⊗)
(4.96)
Typical kinds of potential ratio scaling are the measured lines of code (LOC), the number of classes or the
number of nodes in a flow graph. Furthermore, we can define different kinds of quantities q ∈ Q ([Munson 03])
•
for the qualified measurement (nominal scale and ordinal scale in the potential manner):
q ∈ {nomination1, . . . , nominationn}
(4.97)
where nominationi consists of the different kinds of string values depending on the kind of the
nomination;
q∈ℕ∨q∈ℤ∨q∈ℚ∨q∈ℝ
(4.98)
where the numbers only held the characteristics of (4.93) and (4.94);
•
for the quantified measurement (interval scale and ratio scale in the potential manner):
42
q∈ℕ∨q∈ℤ∨q∈ℚ∨q∈ℝ
(4.99)
where the numbers imply the characteristics of (4.95) and/or (4.96).
We will call the defined q above an element. Therefore, we can summarize the following elements of Q
([Dumke 04], [Juristo 03], [Pandian 04], [Singpurwalla 99])
Q = [ ({number})structure ]aggregation
(4.100)
quantitySt ructure )
∈ RMS: Q × … × Q → {vector, matrix, tensor}
r (MS
(4.101)
quantityAg gregation )
∈ RMS: Q × … × Q → {pair, Eigenvalue, residual,
r (MS
(4.102)
adaptive, randomized, predictive}
quantityNo rmalizatio n )
∈ RMS: Q × … × Q → {uniform, rated, normativ,
r (MS
(4.103)
harmonic, orthogonal}
4.2.5 The Measurement Values
The measurement values V lead to an (unit-based) measure by mapping the measurement quantities to an
empirical meaning.
Q × E→ V × U
(4.104)
Relating to the empirical relative as (A, ∙≥, ◦) we can derive the following forms of scale types (see (4.46) to
(4.49) and [Denvir 92] and [Zuse 98])
•
Nominal scale: ((A, ≈), (Q, =), µ)
(4.105)
•
Ordinal scale: ((A, ∙≥), (Q, ≥), µ)
(4.106)
•
Interval scale: ((A × A, ∙≥), (Q × Q, ≥), µ)
(4.107)
•
Ratio scale: ((A, ∙≥, ◦), (Q, ≥, ⊗), µ)
(4.108)
The measurement values themselves could be structured as follows ([Bache 94], [Kitchenham 96], [Pfleeger 98],
[Potter 00])
V = [ ({value})valueStructure ]valueRelation
(4.109)
values ∈ {classificator, nominator, naturalNumber, ratioNumber, realNumber}
(4.110)
valueStructures ⊆ {interval, tuple, set, list}
(4.111)
valueRelations ∈ {nominalScale, ordinalScale, intervalScale, ratioScale}
(4.112)
Especially, the tuple could be motivated in the following cases of value aggregation and composition ([Dumke
99], [Foltin 98], [Godart 94]):
•
based on the concept of a multi metric where all metrics are related to the same component [Erni 96],
e.g. with v ∈ V , a ∈ A and µi as different metrics as
43
V(multiMetric) = {v µ1 ( a ) , … , v µ n ( a ) }
•
(4.113)
based on the idea of a measurement timeline for one metric during a special time interval [Pandian 04],
e. g. with v ∈ V , a ∈A and ti as different time points as
V(timeSeries) = {v µ a ( t1 ) , … , v µ a ( t m ) }
•
(4.114)
based on the combination of multi metric and measurement series about different artefact structures or
hierarchies.
In order to evaluate the artefacts, special values – as thresholds – are very helpful. A threshold is “a metric value
below which the probability of a fault in a software component is steady, and above which the probability of a
fault in the component increases.” [Erdogmus 02]
We distinguish between the following characteristics of measurement values by using special notations:
•
in the case of a special value attribute we use the exponential notation, e. g. V(visualized) in (4.73),
•
in the case of a special value characteristic we use the same notation, e. g. V(ratioScaled) showing the
current scale characteristic of the set of values,
•
in the case of the characterization of the measured artefact producing the value we use the index
notation, e. g. Vartefact with the different cases described above.
The large set of measurement values leads to a special characterization of the V as metrics data base (MDB,
see [Bache 94], [Braungarten 05], [Foltin 98] and [Godart 94]). The structure of these metrics data bases could
be defined in the kinds of the tuple in (4.111) and/or (4.112). We formulate in general considering MDB as a
simple relational data base
MDB = V × . . . × V
(4.115)
where the V can assume the value characteristics as V(multiMetric) and/or V(timeSeries).
Furthermore, metrics data bases could be also defined on the basis of sets or list structures, aggregations and
composition.
The different kinds of measurement values are shown in the following figure.
Figure 34: Different kinds of measurement values and measurement value structures
4.2.6 The Measurement Units
The measurement units U is only given in the case of quantified measurement. Therefore, we consider both
sets on the right side in the following relation.
44
V × E → V’ × U
(4.116)
The meaning of these measurement units are the following: “The standardized quantitative amount that will be
counted to derive the value of the base measure, such as an hour or line of code. A unit is a particular quantity,
defined and adopted by convention, with which other quantities of the same kind are compared in order to
express their magnitude relative to that quantity.” [McGarry 02]
In the context of software measurement, we can define the following parts of the measurement units
U = measurementUnits = {physicalMeasureUnits, economicalMeasureUnits,
(4.117)
sociologicalMeasureUnits, softwareMeasureUnits
hardwareMeasureUnits}
For the different sets of measurement units we will only describe some examples without the claim to be
completely.
physicalMeasureUnits ∈ {are, Fahrenheit, gallon, gram, hertz, horsepower,
(4.118)
hour, joule, karat, lux, metre, mile, milli second, parsec,
pound, watt, yard}
economicalMeasureUnits ∈ {Euro, GNP, hurdleRate, MBI, ROI, $, shareIndex,
uninflatedCost, VAT}
(4.119)
where means GNP (gross national product), MBI (manpower buildup index), ROI
(return on investment), VAT (value added tax);
sociologicalMeasureUnits ∈ {IQ, Mfactor, StroudNumber, Ufactor}
(4.120)
where means IQ (intelligence quotient), Mfactor (ordinal scaled part of motivation),
Ufactor (part of unbroken working hours per day);
softwareMeasureUnits ∈ {FFP(COSMIC), FP, KDSI, L(Halstead), N(Halstead), PM, RSI, SLOC, (4.121)
toperation, tprojectDuration,tresponse, V(McCabe)}
where means FFP (full function points by the COSMIC community), FP (function
points), KDSI (kilo delivered source instruction), L(Halstead) (program level by
Halstead), N(Halstead) (program length by Halstead), PM (personnel (effort in) month),
RSI (reused source instruction), SLOC (source lines of code), V(Halstead) (program
volume by Halstead), V(McCabe) (cyclomatic number by McCabe);
hardwareMeasureUnits ∈ {MTFD, MTDD, MTDR, MTFR, MTBF, MTTD, MTTF}
(4.122)
where means MTFD (mean time between failure and disclosure); MTDD (mean time
between disclosure and diagnosis), MTDR (mean time between diagnosis and repair),
MTFR (mean time between failure and repair), MTBF (mean time between failures),
MTTD (mean time to defect), and MTTF (mean time to failure).
The measurement unit plays an important role in the operations of calibration, adjustment and unit
transformation or conversion.
Figure 35: Different kinds of measurement units in the context of software measurement
45
4.2.7 The Measurement Experiences
The measurement experiences E summarizes the general aspects of the concrete measurement results in
different forms of aggregation, correlation, interpretation and conclusion based on a context-depended
interpretation. We can define
G × V × U × E → E’
(4.123)
Note that the measurement experience is divided in the experiences of the measurement results and the
(evaluated-based) experience of the measurement itself. In following we only consider the first aspect. Some
kinds of measurement experience are ([Armour 04], [Davis 95], [Endres 03], [Kenett 99])
E ⊆ {analogies, axioms, correlations, criterions, intuitions, laws, lemmas, formulas,
(4.124)
methodologies, principles, relations, ruleOfThumbs, theories}
Some examples of these kinds of experience are (see also [Basili 01], [Boehm 89], [Dumke 03], [Halstead 77]
and [Putnam 03])
analogies ∈ {analogicalEstimation, systemAnalogy, hardwareSoftwareAnalogy}
(4.125)
criteria ∈ {fulfilCondition, qualityAspect, minimality, maximality}
(4.126)
laws ∈ {BrooksLaw, DijkstraMillsWirthLaw, FagansLaw, GlassLaw,
GraySerlinLaw, McIlroysLaw, MooresLaw, SimonsLaw}
(4.127)
The following figure shows the varity of intentions of such laws. The detailed content of these laws is described
in [Endres 03].
Bayes’ law
processIndicators
Dijkstra’s law
systemOfMeasures
standards
experience
Simon’s law
Glass’ law
Parnas’ law
Curtis’ law
Humphrey’s law
softwareProcess
systemRequirements
for software
Miller’s law
methods
lifecycle
management
applicationDomain
softwareProduct
programs
documentations
Davis’ law
DeRemer’s law
personnelResources
softwareResources
platformResources
developmentStaff
users
customers
COTS
ICASE
systemSoftware
hardwareInfrastructures
Corbato’s law
Weinberg’s law
Lehman’s law
McIlroy’s law
Basili’s law
Moore’s law
Cooper’s law
Figure 36: Intentions of choosen software engineering laws
lemmas ∈ {‘any system can be tuned’, ‘installability must be designed in’,
‘human-based methods can only be studied empirically’}
(4.128)
methodologies ∈ {agileMethodology, cleanroomMethodology,
empiricalBasedMethodology}
(4.129)
46
principles ∈ {‘don’t set unrealistic deadlines’, ‘evalvate alternatives’,
‘manage by variance’, ‘regression test after every change’}
(4.130)
rulesOfThumb ∈ {‘one dollar in development leads to two dollars maintenance’,
‘1 KLOC professional developed programs implies 3 errors’,
‘more than 99 percent of all executing computer instructions come from
COTS’, ‘more than the half of the COTS features go unsused’}
(4.131)
We distinguish between the following charaterizations of the measurement experience:
•
in the case of unproven experience we use the exponential notation, e. g. E(conjecture) or E(hypothesis) in
(4.7) and (4.8)
•
in the case of a special experience characteristic we use the index notation, e. g. Eintuition in (4.79) or
Ecriterion in (4.9), (4.10) and (4.21).
Like the metrics data base we can build some structures about the measurement experience as experience factory
(EF) which could be characterized in general as ([Basili 93], [Ciolkowski 02])
EF = E × . . . × E
(4.132)
In the following figure we summarize the essential aspects of the measurement experience.
Figure 37: Different kinds of measurement experience
4.2.8 The Measurement Tools
The measurement tools T are also be named as Computer-Assisted software Measurement and Evaluation
(CAME) tools [Dumke 96]. Some typical kinds of these CASE tools are summarized as following
T = {modellingTool, measurementTool, (statistical) analysisTool,
(4.133)
visualizationTool, metricsRepositoryTool}
We distinguish between the following characteristics of measurement tools by using special notations:
•
in the case of the special measured artefact by CAME tools we use the index notation, e. g. TsourceCode or
Tprocess.,
•
in the case of the application of a special measurement method by the measurement tool we prefer the
exponential notation, e. g. T(modelling) or T(estimation) ,
•
in the case of different characteristics of a metrics tool we use the same notation, e. g. T(open) or
T(flexible).
47
An example of such combined notation could be the following term
measurement ,open )
T (OOdesign
(4.134)
The tool operations as composition, combination and integration are essential aspects for the effectiveness of
CAME tool application.
The following figure shows an example of the CAME tool combination in order to cover the complete software
measurement process [Dumke 96].
measurement
definition &
scheduling
Ami tool
SPQR/20
modeling
measurement
data
analysis
evaluation
(presentation)
Cosmos
SPQR/20
Logiscope
Figure 38: Examples of CAME tool combination
Examples of CAME tool cobination in the software development life cycle is described in the following fiugre
[Dumke 96].
problem
definition
SOFT-ORG
PDM
analysis/
specification.
design
SOFT-CALC
implementation
SOFT-AUDOTIR
OOM
maintenance
SOFT-MESS
PMTool
Figure 39: Examples of CAME tool combination in the software life cycle
Another aspect of CAME tools or metrics tools id their openness, flexibility, stability and validity. Openness is
required to use any components of the CAME tool for the applied measurement framework such as the
modelling part, the metrics execution part or the statistical analysis and the presentation part. The flexibility is
necessary to manage the software development complexity in the company and allows to define new metrics in
the tool or to change the empirical evaluation criteria. The stability correspond to the reliability of the tool at runtime.The validity is the characterization of the correctness of the metrics values and the reconstructability of the
results. Some examples of tool evaluation based on these criteria is given in [Dumke 97].
Furthermore, CAME tool could be integrated considering the modern aspects of the WWW possibilities and
intentions. The following figure shows some of the essential aspects for costructing measurement
infrastructures.
48
Web-based
technology
online consultation and communication in the Web
agent-based information gathering in the Web
Web links as information resources
given application
domain
chosable application
domain
ad hoc
searched/
defined
application
domain
Software
development
focus
product
product
and process
product,
process and
resources
typical Web-based
support currently
Application domain
Figure 40: General characterization of Web-based measurement infrastructures
Future directions consider the different kinds of e-measurement involving the possibilities of measurement
dashboards and portals.
4.2.9 The Measurement Personnel
The measurement personnel P includes all the personnel resources in a creative, motivated and active manner
in order to
•
define metrics, measurement processes, measurement strategies, validation techniques, and
measurement methodologies,
•
carry out the measurement process installation and realization in a practical environment for software
(process) improvement and controlling.
The personnel resources themselves could be divided in two kinds of measurement application
•
the origin measurement staff like ([Dumke 03], [Pandian 04], [Pfleeger 98]) as
P ⊆ {measurementAnalyst, certificator, librarian, metricsCreator
(4.135)
user, validator}
•
the IT staff which use the software measurement aditionally as
P ⊆ {administrator, analyst, auditor, designer, developer, expert, maintainer,
(4.136)
programmer, reviewer, tester, customer}
The measurement personnel of the second class is one of the developers which use such technologies like the
personal software process (PSP) and/or the team software process (TSP) ([Humphrey 97], [Humphrey 00]).
We distinguish between the following characteristics of measurement personnel by using special notations:
•
in the case of the general characterization of the origin measurement personnel or measurement adapter
we use the exponential notatation, e. g. P(origin) or P(adaptor).
•
in the case of the concrete different measurement role we use the inde notation, e.g. Pexpert or
PmetricsCreator .
49
4.3 Software Measurement Frameworks
The measurement frameworks are directed on the efficient and valid installation and application of a software
measurement process. In (4.21) we have defined
MP: (G × A × M)T,P → (Q × E)T,P → (V × U)T,P → E’× A’
The following approaches or frameworks are defined in order to support this measurement process in a
successful manner.
4.3.1 A Formal Characterization of the ISO 15939
In following we will formulate a short description of the essential tasks and objects of an ISO 15939 concurring
measurement infrastructure ([ISO 15939], see also figure 20). The basic tasks and objects are:
(G × A × M)T,P → (Q × E)T,P:
establishingTasks = {acceptanceOfMeasurement, assignTheResources}
(4.137)
establishingArtefacts = {managementCommitment, measurementRequirements,
planForMeasurementResources, descriptionOfTheOrganisationalUnit}
(4.138)
where means
•
•
•
•
•
•
acceptanceOfMeasurment: in order to accept the requirements for measurement
considering the identification and communication of/in the organisational unit;
assignTheResources: for assigning the competent resources including the responsibilities of
the roles of the measurement user, the measurement analyst, and the measurement librarian;
managementCommitment: in the form of a measurement policy for the organisational unit
or in the form of a contract with a customer stipulating that certain measures be used
measurementRequirements: includes the commitment of resources to the measurement
process and the willingness to maintain this commitment
planForMeasurementResources: resources shall be made available for implementing the
planned measurement tasks
descriptionOfTheOrganisationalUnit: the part of the organisation that is the subject of the
measurement
planningTasks = {organisationCharacterization, needsIdentification, measuresSelection,
(4.139)
evaluationProcedures, evaluationCriteria, measurementResourcing,
measurementSupporting}
planningArtefacts = {informationNeeds, descriptionOfTheOrganisationalUnit,
(4.140)
candidateMeasures, selectedMeasures, measurementTasks, informationProducts,
toolsDescriptions, trainingCourseMaterials}
where means
•
•
•
•
organisationCharacterization: in order to characterise the organisational unit explicitly;
needsIdentification: for identifying the information needs which should be prioritized,
selected, documented, and communicated;
measuresSelection: in order to select measures that satisfy the selected information needs
including their documentation (name, unit, formal definition, method of data collection,
link to the information needs); selecting criteria: relevance, feasibility, availability of
resources, ease of data collection, protection of privacy, number of users, evidence
evaluationProcedures: for the definition of the data collection, analysis and reporting
procedures;
50
•
•
•
•
•
•
•
•
•
•
•
evaluationCriteria: for the definition of the criteria for evaluating the information products
and measurement process;
measurementResourcing: in order to review, approve, and staff the measurement tasks;
measurementSupporting: for the aquisition and deployment of the supporting technologies;
informationNeeds: structured as indicator(model(derivedMeasure(function(baseMeasure
(measurementAttribute(attribut), . . . , baseMeasure(measurementAttribut(attribut))), . . . ,
derivedMeasure(function(baseMeasure(measurementAttribute(attribut), . . . , baseMeasure
(measurementAttribut(attribut)))))
descriptionOfTheOrganisationalUnit: see above
candidateMeasures: the set of measures in the considered measurement context
selectedMeasures: selection of the candidate measures reflect to the information needs
measurementTasks: the set of measurement activities based on the measurement
requirements
informationProducts: an indicator an its associated interpretation
(Statistical) toolsDescriptions: including the method that would be needed to perform the
data analysis
trainingCourseMaterials: lectures for training of the measurement staff
(Q × E)T,P → (V × U)T,P:
performingTasks = {processIntegration, dataCollection, dataAnalysis,
userCommunication}
(4.141)
performingArtefacts = {measuredArtefacts, collectedData, storedData,
informationProducts, measurementExperienceBase}
(4.142)
where means
•
•
•
•
•
•
•
•
•
processIntegration: for integration of the data generation and collection into the relevant
processes based on the measurement of the measured artefacts;
dataCollection: the collected data shall be verified and stored including any context
information necessary to verify, understand, and evaluate the data;
dataAnalysis: for collecting and interpreting the analysed data according to the planning
information;
userCommunication: for the documentation of the informations products and their
communication to the data providers and measurement users;
measuredArtefacts: the product, processes or resources which will be measured reflecting
to the information needs
collectedData: the structure of the measures
storedData: the measurement data base contents
informationProducts: see above
measurementExperienceBase: a data store that contains the evaluation of the information
products and the measurement process as well as any lesson learned during measurement
and analysis
(V × U)T,P → E’× A’:
evaluationTasks = {measurementEvaluation, measurementImprovement}
(4.143)
evaluationArtefacts = {evaluationCriteria, informationProducts,
lessonLearned, measurementExperienceBase}
(4.144)
where means
•
•
measurementEvaluation: for the evaluation of the measures and the measurement process
themselves;
measurementImprovement: for the identification of the potential improvements to the
(performance of the) measurement process (as input for the planning agents);
51
•
•
•
•
evaluationCriteria: use of information products, confidence in an information product,
evidence and fitness for purpose of an information product, understandability of
information products, satisfaction of the assumptions of an indicator model, accuracy of a
measurement process, reliability of a measurement method, performance of the
measurement process (timeliness, efficiency, defect containment, customer satisfaction,
process conformance)
informationProducts: see above
lessonLearned: empirical rules and experiences
measurementExperienceBase: see above
explorationTasks = {measurementReporting, measurementExploration,
measurementConclusion}
(4.145)
explorationArtefact = measurementExperienceBase
(4.146)
where means
•
•
•
measurementReporting: in order to generate measurement reports for different users and
contents;
measurementExploration: for the generation of some statistical data analysis;
measurementConclusion: for generating some general conclusions on the base of the
experience data;
•
measurementExperienceBase: see above
controllingTasks = {measurementFeedback, informationGeneration}
(4.147)
controllingArtefacts = {informationProducts, informationNeeds}
(4.148)
•
•
measurementFeedback: for presentation and information of the measured basis processes;
informationGeneration: for the creation and/or generation of the information needs;
•
•
informationProducts: see above
informationNeeds: see above
Further details could be described in order to define the algorithms for a support system for IOS 15939 related
measurement automation.
4.3.2 The Zuse Measurement Framework
In following we describe the main measurement structural intentions based on the largest and profoundest
considerations of the existing metrics approaches and investigations by Zuse (see [Zuse 98] for the details). The
Zuse approach is based on the measurement theory by Roberts and Krantz ([Krantz 89], [Roberts 79]), see also
[Zuse 91]) and uses the extensive structure described above (see (4.39), [Zuse 89] and [Zuse 92]). In following
we characterize some of the framework intentions of this approach:
(G × A × M)T,P → (Q × E)T,P:
•
The following kinds of requirements for software measures are defined by Zuse ([Zuse 98, p. 399 ff):
requirementsForMeasures = {reqgeneral, reqrankingProperties, requnits, reqsubstitutionOperation,
reqadditiveRatioScale, reqnon-additive, reqdensityMeasures, reqwholeness,
reqpredictionModels, reqsizeMeasures, reqsimpleMeasures,
reqcomponentIndependence}
52
(4.149)
•
The types of measures based on the measurement theory are the following ([Zuse 98], p. 242 ff)
typesOfMeasures = {TM-CNT, TM-ADD, TM-HYA, TM-HYAS, TM-DEN,
TM-DENM, TM-NADDR, TM-BSA, TM-HYM, TM-ORD,
TM-PC, TM-FB, TM-MIMA, TM-IND, TM-NGEXT}
(4.150)
where means TM-CNT (counting on attribute), TM-ADD (additive measure), TM-HYA (hybrid measure
by additivity), TM-HYAS (hybrid measure additive plus a constant), TM-DEN (density measures), TMDENM (density measure with an additional condition), TM-NADDR (non-additive ratio scale), TM-BSA
(two concatenation operations), TM-HYM (hybrid measure by multiplication), TM-ORD (purely ordinal
measure), TM-PC (percentage measure), TM-FB (modified function of belief), TM-MIMA (minimum –
maximum), TM-IND (independence conditions), TM-NGEXT (negative extension structure and nonadditive ratio scale);
•
The following artefacts are considered in detail in the Zuse framework ([Zuse 98, p. 487 ff):
Aframework = { AISO9000-3, AfunctionPoints, ACOCOMO, Amaintainability, AstructureCharts,
AinformationFlow, AmeasuresOfBowles, Acoding, Atesting, Acohesion,
Amaintenance, AdocumentQuality, AOOmeasures, AlifeCycle}
(4.151)
(Q × E)T,P → (V × U)T,P:
•
In the scale analysis (see (4.105) – (4.112)) the following operations about the components of the
measured artefact are helpful ([Zuse 98], p. 115 ff):
Mmodelling ∈ {BALT, BSEQ, CINT, CUNI, DSEQ, HAGG,
(4.152)
HGEN, LSEQ, RALT, RSEQ, SBSEQ}
where means BALT (parallel concatenation), BSEQ (sequential concatenation), CINT (object-oriented
concatenation), CUNI (object-oriented unification), DSEQ (charts sequential concatenation), HAGG
(object-oriented hierachical concatenation), HGEN (object-oriented generalizational concatenation),
LSEQ (boards sequential concatenation), RALT (resistors parallel concatenation), RSEQ (resistors
sequential concatenation), SBSEQ (source code sequential concatenation);
•
The application of the viewpoint principle for using metrics. The viewpoint and the empirical system
are identical. The term viewpoint is more intuitive for the user and well known from the daily life.
(V × U)T,P → E’× A’:
•
The principles of metrics application are defined by Zuse as following ([Zuse 98, p. 479 ff):
1.
2.
3.
4.
5.
There is no best measure.
One number cannot express software quality.
The properties of the used measure should be well known.
One measure is not sufficient because software systems concists of many properties.
A so-called software quality or maintainability index, combined by many measures, has to be
considered carfully.
6. It is a widely spread misunderstanding, that validated measures only are useful.
7. It is important to know whether a measure assumes an extensive structure or not.
8. A measure that is validated in one environment is not autonatically a validated measure.
9. A measure that has a good correlation to a validated measure is not automatically validated,
either.
10. The environment in every company for measurement is differently.
11. There does not exist an initial set of measures or even a final set of measures.
The application of the software measurement framework by Zuse keeps the consideration of the measurement
theoretical aspects for achieving really measures and their correct involvement in the software evaluation.
53
4.3.3 The CAME Approach
We use the acronym CAME in three meanings to meet the requirements of software measurement. The CAME
aspects are defined in a layer model which is described in the following figure [Dumke 99].
CAME strategy
CAME framework
CAME tools
Figure 41: The CAME approach
The CAME strategy stands for
• Community: the necessity of a group or a team that is motivated and has the knowledge of software
measurement to install software metrics. In general, the members of these groups are organized in metrics
communities such as our German Interest Group on Software Metrics (http://ivs.cs.uni-magdeburg.de/sweng/us/giak/).
• Acceptance: the agreement of the (top) management to install a metrics program in the (IT) business area.
This aspect is strong connected with the knowledge about required budgets and personnel resources.
• Motivation: the production of measurement and evaluation results in a first metrics application which
demonstrates the convincing benefits of the metrics application. This very important aspect can be
achieved by the application of essential results in the (world-wide) practice which are easy to understand
and should motivate the management. One of the problem of this aspect is the fact that the management
wants to obtain one single (quality) number as a summary of all measured characteristics.
• Engagement: the acceptance of spending effort to implement the software measurement as a permanent
metrics system (with continued measurement, different statistical analysis, metrics set updates etc.). This
aspect includes also the requirement to dedicate personnel resources such as measurement teams etc.
The CAME framework itself consists of the following four phases:
• Choice: the selection of metrics based on a special or general measurement view on the kind of
measurement and the related measurement goals -- as measurement objects,
• Adjustment: the measurement characteristics of the metrics for the specific application field – as
attributes of the measurement objects,
• Migration: the semantic relations between the metrics along the whole life cycle and along the system
architecture – as services of the measurement objects,
• Efficiency: the automation level as construction of a tool-based measurement – as implementation of the
measurement objects by CAME tools (computer assisted measurement and evaluation tools).
The CAME approach could be used in order to evaluate the measurement level itself in the following manner:
Eval(G × A × M ):
choiceLevel = | AmeasuredKindsOfProducts | + | AmeasuredKindsOfProcesses | + | AmeasuredKindsOfResources |
We establish
choiceLevel = {1, 2, …, n},
where n depends on the metrication structure of the different measurement areas (product, process, resources). In
[Dumke 99] we defined a triple structure of these areas obtaining n = 81.
54
The following adjustment level considers the part of quantified measurement of the used measures for the
different measurement areas characterized above. Hence, only the ratio scale and interval scale are involved in
this characterization.
adjustmentLevel = (| V(ratioScale) | + | V(intervalScale) |)/choiceLevel
(4.153)
Note that the considered values are mapped to the metrication structure described above. Therefore, we establish
adjustmentLevel = {m | m ∈ ℚ ∧ 0 ≤ m ≤ 1}.
(4.154)
On the other hand there can defined further level structures considering the different kinds of scale types in
different assoziation and aggregations.
Eval(Q × E→ V × U→ E’× A’):
The migrationLevel considers the measured areas and their implementation as a full measurement, evaluation
and application process. Therefore we define
migrationLevel = | M(product) | + | M(process) | + | M(resources) |
(4.155)
migrationLevel = {1, 2, …, k}
(4.156)
We establish
where k includes the number of the elements of the metrication structure adding by the number of the necessary
measurement methods themselves. We have described 21 measurement methods above (4.28 – 4.91) as a
maximum number. Using the value example of our metrication structure as 81 we can establish
migrationLevel ≤ 21 · 81 = 1701.
Finally, the toolLevel considers the part of tool support for the different measurement methods
Eval((Q × E)T → (V × U)T → (E’× A’)T):
toolLevel = (| M(toolBased) |)/migrationLevel
(4.157)
toolLevel = {l | l ∈ ℚ ∧ 0 ≤ l ≤ 1}.
(4.158)
We establish
The CAME approach is very helpful in order to analyze the general measurement situation in different IT areas,
development paradigms and kinds of software systems and can establish necessary research activities and
missing measurement field applications.
5 Conclusions
This preprint shows a first general formalization of the whole software measurement system including the
aspects of the measurement processes, their intentions and applications. The intention was to define a declarative
environment on the measurement areas, methods and variations that could be a base for detailed description of
the software measurement for different paradigms, technologies and systems.
55
6 References
[Abran 03] Abran, A.; Braungarten, R.; Dumke, R. R.: The second generation of the ISBSG Effort Estimation
Prototype. In: Dumke/Abran: Investigations in Software Measurement, Shaker Publ., 2003, pp. 218-231
[Allen 02] Allen, E. B.: Measuring Graph Abstractions of Software: An Information-Theory Approach. Proc. of
the Eight IEEE Symposium on Software Metrics (METRICS 2002), June 4-7, 2002, Ottawa, pp. 182-193
[Armour 04] Armour, P. G.: The Laws of Software Process – A New Model for the Production and Management
of Software. CRC Press, 2004
[Bache 94] Bache, R.; Bazzana, G.: Software Metrics for Product Assessment. McGraw Hill Publ., 1994
[Backhaus 96] Backhaus et al.: Multivariate Analysemethoden. Springer Publ., 1996
[Balcàzar 95] Balcàzar, J., L.; Diaz, J.; Gabarrò, J.: Structural Comlexity I. Springer Publ., 1995
[Basili 93] Basili, V. R.: Applying the Goal Question Metric Paradigm in the Experience Factory. Proc. of the
Tenth Annual Conderence of Software Metrics and Quality Assurance in Industry, Amsterdam September
29 – October 1, 2002, Section 1
[Basili 01] Basili, V. R.; Boehm, B. W.: COTS-Based Systems Top 10 List. IEEE Computer, May 2001, pp. 9195
[Basili 86] Basili, V. R.; Selby, R. W.; Hutchens, D. H.: Experimentation in Software Engineering. IEEE
Transactions on Software Engineering, 12(1986)7, pp. 733-734
[Baudry 02] Baudry, B.; Traon, Y, L.; Sunye, G.: Testability Analysis of a UML Class Diagram. Proc. of the
Eight IEEE Symposium on Software Metrics (METRICS 2002), June 4-7, 2002, Ottawa, pp. 54-63
[Benedicenti 98] Benedicenti, et al.: A Standard Measuring Procedure for Software Engineering. Proc. of the
FESMA, Brussels, May 1998, pp. 129-136
[Berthold 03] Berthold, M.; Hand, D. J.: Intelligent Data Analysis. Springer Publ., 2003
[Boehm 89] Boehm , B.W.: Software Risk Management. IEEE Computer Society Press, 1989
[Boehm 00a] Boehm, B. W.; Basili, V. R.: Gaining Intellectual Control of Software Development. IEEE
Computer, May 2000, pp. 27-33
[Boehm 00b] Boehm, B.W. et al.: Software Cost Etsimation with COCOMO II. Prentice Hall Inc., 2000
[Bourque 00] Bourque, P.; Dupuis, R.; Abran, A.; Moore, J. W.; Tripp, L.: The Guide to the Software
Engineering Body of Knowledge. IEEE Software, November/December 1999, pp. 35-44
[Braungarten 05] Braungarten, R.; Kunz, M.; Dumke, R.: An Approach to Classify Software Measurement
Storage Facilities. Preprint No. 2, University of Magdeburg, Faculty of Informatics, 2005
[Briand 96] Briand, L.; Morasca, S.; Basili, V. R.: Property Based Software Engineering Measurement. IEEE
Transactions on Software Engineering, 22(1996)1, pp. 68-85
[Bundschuh 00] Bundschuh, M.; Fabry, A.: Auswandschätzung von IT-Projekten. MITP Publ., Bonn, 2000
[Cantone 00] Cantone, G.; Cantone, L.; Donzelli, P.: Modelling and measuring software technologies. Proc. of
the ESCOM 2000, Munich, May 2000, pp. 83-93
[Card 00] Card, D. N.: Making Measurement Understandable. IEEE Software, January/February 2000, pp. 95-96
[Chapin 02] Chapin, N.: Entropy-Metric For Systems With COTS Software. . Proc. of the Eight IEEE
Symposium on Software Metrics (METRICS 2002), June 4-7, 2002, Ottawa, pp. 173-181
[Chung 00] Chung, L.; Nixon, B. A.; Yu, E.; Mylopoulos, J.: Non-Functional Requirements in Software
Engineering. Kluwer Academic Publ., 2000
[Ciolkowski 02] Ciolkowski, M.; Hartkopf, S.; Laitenberger, O.; Rombach, D.: Das ViSEK-Projekt: Aufbau
einer nationalen empirisch-basierten Erfahrungsdatenbank für Software-Engineering: In:
Dumke/Rombach: Software-Messung und –Bewertung, Deutscher Universitätsverlag, 2002, pp. 1-12
[Clark 02] Clark, B.: Eight Secrets of Software Measurement. IEEE Software, September/October 2002, pp. 1214
[Cole 93] Cole, R. J.; Woods, D.: Measurement Through the Software Lifecycle: A Comperative Case Study. .
Proc. of the 10th Annual Conference on Application of Software Metrics and Quality Assurance in
Industry, Amsterdam, Netherlands, September 1993, Section 19
[Comer 93] Comer, P.; Chard, J.: A measurement maturity model. Software Quality Journal, 40(1997)12, pp. 95103
[COSMIC 03] COSMIC FFP 2.2: Measurement Manual. ISO 19761, January 2003
[Dao 02] Dao, M.; Huchard, M.; Libourel, T.; Leblanc, H.: A New Approach to Factorization – Introducing
Metrics. Proc. of the Eight IEEE Symposium on Software Metrics (METRICS 2002), June 4-7, 2002,
Ottawa, pp. 227-236
[Davis 95] Davis, A. M.: 201 Principles of Software Development. McGraw Hill Publ., 1995
[Denvir 92] Denvir, T.; Herman, R.; Whitty, R. W.: Formal Aspects of Measurement. Springer Publ., 1992
[Dimitrov 02] Dimitrov, E.; Schmietendorf, A.; Dumke, R.: UML-Based Performance Engineering Possibilities
and Techniques. IEEE Software, January/February 2002, pp. 74-83
[Dolado 01] Dolado, J. J.: On the problem of the software cost function. Information and Software Technology,
43(2001), pp. 61-72
56
[Dumke 03] Dumke, R.: Software Engineering – Eine Einführung für Informatiker und Ingenieure. (4th edn)
Vieweg Publ., 2003
[Dumke 03a] Dumke, R.; Abran, A. (Eds): Investigations in Software Measurement. Shaker Publ., 2003
[Dumke 01] Dumke, R.; Abran, A. (Eds): New Approaches in Software Measurement. LNCS 2006, Springer
Publ., 2001
[Dumke 04] Dumke, R.; Cotè, I.; Andruschak, O.: Statistical Process Control (SPC) – A Metrics-Based Point of
View of Software Processes Achieving the CMMI Level Four. Preprint No. 7, University of Magdeburg,
Fakultät für Informatik, 2004
[Dumke 99] Dumke, R.; Foltin, E.: An Object-Oriented Software Measurement and Evaluation Framework.
Proc. of the FESMA, October 4-8, 1999, Amsterdam, pp. 59-68
[Dumke 98] Dumke, R.; Foltin, E.: Metrics-Based Evaluation of Object-Oriented Development Methods. Proc.
of the Euromicro Conference on Software Maintenance and Reengineering, Florence, Italy, March 8-11,
1998, pp. 193-196
[Dumke 98a] Dumke, R.; Foltin, E.; Schmietendorf, A.: Kausalitätsprobleme bei der Aufwandsschätzung in der
Softwareentwicklung und –wartung. Preprint Nr. 13, Otto-von-Guericke-Universität Magdeburg, 1998
[Dumke 96] Dumke, R.; Foltin, E.; Koeppe, R.; Winkler, A.: Softwarequalität durch Meßtools – Assessment,
Messung und instrumentierte ISO 9000. Vieweg Publ., Braunschweig, Germany, 1996
[Dumke 97] Dumke, R.; Grigoleit, H.: Usefulness of CAME tools in Software Quality Asurance. In: Hawkins et
al.: The Quality Challenge, MEP Publ., London, 1997, pp. 291-303
[Dumke 01a] Dumke, R.; Koeppe, R.: Conception of a Web-Based SPE Development Infrastructure. In: Dumke
et al.: Performance Engineering, LNCS 2047, Springer Publ., 2001, pp. 1-19
[Dumke 95] Dumke, R.; Koeppe, R.: Komplexität bei der Software-Entwicklung und Softwarezuverlässigkeit.
Tagungsband zum Workshop Software hoher Zuverlässigkeit, Verfügbarkeit und Sicherheit, München,
17.5.1995, S. 6.1 – 6.22
[Dumke 02] Dumke, R.; Lother, M.; Wille, C.: Situation and Trends in Software Measurement – A Statistical
Analysis of the SML@b Metrics Bibliography. In: Dumke/Abran: Software Measurement and Estimation,
Shaker Publ., 2002, pp. 298-314
[Dumke 03b] Dumke, R.; Lother, M.; Wille, C.; Braungarten, R.: eMeasurement – Gegenwärtiger Stand und
Perspektiven. In: Büren et al.: Software-Messung in der Praxis, Shaker Publ., 2003, pp. 135-148
[Dumke 01b] Dumke, R.; Rautenstrauch, C.; Schmietendorf, A.; Scholz, A. (Eds): Performance Engineering –
State of the Art and Current Trends. LNCS 2047, Springer Publ., 2001
[Dumke 02a] Dumke, R.; Rombach, D. (Eds): Software-Messung und –Bewertung. Deutscher Universitätsverlag,
Wiesbaden, 2002
[Dumke 04a] Dumke, R.; Schäfer, U.; Wille, C.; Zbrog, F.: Agentenbasierte Web-Technologiebewertung für das
Performance Engineering. In: Schmietendorf et al.: 5. Workshop Performance Engineering in der
Softwareentwicklung (PE2004), Siemens München, Mai 2004, pp. 37-66
[Dumke 94] Dumke, R.; Zuse, H.: Theorie und Praxis der Softwaremessung. Deutscher Universitätsverlag,
Wiesbaden, 1994
[Ebert 97] Ebert, C.: Complexity Traces – an Instrument for Software Project Management. Proc. of the 10th
Annual Conference on Application of Software Metrics and Quality Assurance in Industry, Amsterdam,
Netherlands, September 1993, Section 17
[Ebert 96] Ebert, C.: Evaluation and Application of Complexity-Based Criticality Models. Proc. of the Third
International Software Metrics Symposium, March 25-26, Berlin, 1996, pp. 174-185
[Ebert 04] Ebert, C.; Dumke, R.; Bundschuh, M.; Schmietendorf, A.: Best Practices in Software Measuerement.
Springer Publ., 2004
[Ehrig 99] Ehrig, H. et al.: Mathematisch-strukturelle Grundlagen der Informatik. Springer Publ., 1999
[Eickelmann 00] Eickelmann, N.: Integrating the Balanced Scorecard and Software Measurement Frameworks.
Proc. of the IRMA 2000, Anchorage, Alaska, May 2000, pp. 980-983
[Ejiogu 91] Ejiogu, L.O.: Software Engineering with Formal Metrics. QED Technical Publ., 1991
[Emam 98] Emam, K. E.; Drouin, J. N.; Melo, W.: SPICE – The Theory and Practice of Software Process
Improvement and Capability Determination. IEEE Computer Society Press, 1998
[Endres 03] Endres, A.; Rombach, D.: A Handbook of Software and Systems Engineering. Addison Wesley
Publ., 2003
[Erdogmus 02] Erdogmus, H.; Tanir, O.: Advances in Software Engineering – Comprehension, Evaluation, and
Evolution. Springer Publ., 2002
[Erni 96] Erni, K.; Lewerentz, C.: Applying Design-Metrics to Object-Oriented Frameworks. Proc. of the Third
International Software Metrics Symposium, March 25-26, Berlin, 1996, pp. 64-74
[Evanco 94] Evanco, W. M.: Lacovara, R.: A Model-Based Framework for the Integration of Software Metrics.
The Journal of Systems and Software, 26(1994), pp. 77-86
[Feiler 93] Feiler, P. H.; Humphrey, W. S.: Software Process Development and Enactment: Concepts and
Definitions. Proc. of the 2nd Int. Conference on Software Process, Los Altimos, 1993, pp. 28-40
57
[Fenton 93] Fenton, N.E.; Hill, G.: Systems Construction and Analysis – A Mathematical and Logical
Framework. McGraw-Hill, 1993
[Fenton 02] Fenton, N.; Krause, P.; Neil, M.: Software Measurement: Uncertainty and Causal Modeling. IEEE
Software, July/August 2002, pp. 116-122
[Fenton 97] Fenton, N. E.; Pfleeger, S. L.: Software Metrics - a rigorous and practical approach. Thompson
Publ., 1997
[Figallo 98] Figallo, C.: Hosting Web Communities. John Wiley & Sons, Inc. 1998
[Florac 99] Florac, W. A.; Carleton, A. D.: Measuring the Software Process – Statistical Process Control for
Software Process Improvement. Pearson Education, 1999
[Foltin 98] Foltin, E.; Dumke, R.: Aspects of Software Metrics Database Design. Journal of Software Process –
Improvement and Practice, 4(1998)1, pp. 33-42
[Geroimenko 03] Geroimenko, V.; Chen, C.: Visualizing the Semantic Web – XML-based Internet and
Information Visualization. Springer Publ. 2003
[Godart 94] Godart, C.; Charoy, F.: Databases for Software Engineering. Prentice-Hall Publ., 1994
[Gomez-Perez 04] Gomez-Perez, A.; Fernandez-Lopez, M.; Corcho, O.: Ontological Engineering. Springer
Publ., 2004
[Gray 97] Gray, A.; MacDonell, S. G.: GQM++ A Full Life Cycle Framework for the Development and
Implementation of Software Metric Programs. Australian Software Measurement Conference, Canberra,
November 1997
[Halstead 77] Halstead, M. H.: Elements of Software Science. Prentice Hall, New York, 1977
[Han 94] Han, K.J.; Yoon, J.; Kim, J.; Lee, K.: Quality Assessment Criteria in C++ Classes. Microelectronics
Reliability Journal, 34(1994)2, pp. 361-368
[Hanebutte 00] Hanebutte, N.; Dumke, R. R.: Analyzing Software Design using a Measurable Program Design
Language. Metrics News, 5(2000)2, pp. 23-33
[Hanebutte 03] Hanebutte, N.; Taylor, C.; Dumke, R. R.: Techniques of successful application of factor analysis
in software measurement. Empirical Software Engineering, 8(2003)1, pp. 43-57
[Hastings 01] Hastings, T. E.; Sajeev, A. S. M.: A Vector-Based Approach to Software Size Measurement and
Effort Estimation. IEEE Transactions on Software Engineering, 27(2001)4, pp. 337-350
[Hausen 91] Hausen, H.: A Rule-Based Approach to Software Quality Engineering. in: Fenton/Littlewood:
Software Reliability and Metrics, Elsevier Publ., 1991, pp. 48-68
[Hausen 93] Hausen, H.; Welzel, D.: Guides to Software Evaluation. Arbeitspapiere der GMD 746, Bonn, 1993
[Henderson 96] Henderson-Seller, B.: The Mathematical Validity of Software Metrics. Software Engineering
Notes, 21(1996)5, pp. 89-94
[Herbsleb 03] Herbsleb, J. D.; Mockus, A.: Formulation and Preliminary Test of an Empirical Theory of
Coordination in Software Engineering. Software Engineering Notes, , pp. 138-147
[Horn 02] Horn, E.; Reinke, T.: Softwarearchitektur und Softwarebauelemente. Hanser Publ., 2002
[Humphrey 97] Humphrey, W. S.: Introduction to the Personal Software Process. Addison-Wesley Publ., 1997
[Humphrey 00] Humphrey, W. S.: Introduction to the Team Software Process. Addison-Wesley Publ., 2000
[IEEE 90] IEEE Standard Glossary, IEEE Computer Society Press, 1990
[ISO 15939] ISO/IEC 15939: Information Technology – Software Measurement Process. Metrics News
6(2001)1, pp. 11-46
[Jacob 92] Jacob, P.; Cahill, T.: Software Product Metrics as Attributes in an Attribute Grammar. Proc. of the
2ICSQ, October 1992, Research Triangle Park, USA, pp. 40-49
[Jacquet 97] Jacquet, J.; Abran, A.: From Software Metrics to Software Measurement Methods: A Process
Model. Proc. of the ISESS, 1997
[Jones 91] Jones, C.: Applied Software Measurement. McGraw-Hill, 1991
[Juristo 03] Juristo, N.; Moreno, A. M.: Basics of Software Engineering Experimentation. Kluwer Academic
Publishers, Boston, 2003
[Kan 95] Kan, S. H.: Metrics and Models in Software Quality Engineering. Addison-Wesley Publ., 1995
[Kenett 99] Kenett, R. S.; Baker, E. R.: Software Process Quality – Management and Control. Marcel Dekker
Inc., 1999
[Khoshgoftaar 92] Khoshgoftaar, T. M.; Munson, J. C.: Predictive Modeling Techniques of Software Quality
from Software Measures. IEEE Transactions on Software Engineering, 18(1992)11, pp. 979-987
[Khoshgoftaar 02] Khoshgoftaar, T.M.; Seliya, N.: Tree-based Software Qualitzy Estimation Models for Fualt
Prediction. Proc. of the Eight IEEE Symposium on Software Metrics (METRICS 2002), June 4-7, 2002,
Ottawa, pp. 203-215
[Khoshgoftaar 94] Khoshgoftaar, T.M.; Szabo, R.M.: ARIMA models of software system quality. Proc. of the
Annual Oregon Workshop on Software Metrics, April 10-12, 1994, Oregon
[Kitchenham 96] Kitchenham, B. A.: Software Metrics – Measurement for Software Process Improvement. NCC
Blackwell Ltd, 1996
[Kitchenham 02] Kitchenham, B. A.: The question of scale economies in software – why cannot researchers
agree? Information and Software Technology, 44(2002)1, pp. 13-24
58
[Kitchenham 95] Kitchenham, B.; Pfleeger, S.L.; Fenton, N.E.: Towards a Framework for Software
Measurement Validation. IEEE Transactions on Software Engineering, 21(1995)12, pp. 929-944
[Kitchenham 97] Kitchenham et al.: Evaluation and assessment in software engineering. Information and
Software Technology, 39(1997), pp. 731-734
[Kitchenham 01] Kitchenham, B. A.; Pickard, L. M.; MacDonell, S., G.; Shepperd, M. J.: What accuracy
statistics really measure. IEE Proceedings Software, 148(2001)3, pp. 81-85
[Krantz 89] Krantz, D. H. et al.: Foundations of Measurement. Academic Press, Volume I, 1989
[Kulpa 03] Kulpa, M. K.; Johnson, K. A.: Interpreting the CMMI – A Process Improvement Approach. CRC
Press Company, 2003
[Latum 98] Latum et al.: Adopting GQM-based Measurement in an Industrial Environment. IEEE Software,
January/February 1998, pp. 78-86
[Lei 03] Lei, S.; Smith, M. R.: Evaluation of Several Nonparametric Bootstrap Methods to Estimate Confidence
Intervals for Software Metrics. IEEE Transactions on Software Engineering, 29(2003)11, pp. 996-1004
[Lother 02] Lother, M.; Dumke, R.: Application of eMeasurement in Software Development. Proc. of the IFPUG
Annual Conference, San Antonio, Texas, 2002, Chapter 5
[Maciaszek 01] Maciaszek, L. A.: Requirements Analysis and System Design – Development Informatik Systems
with UML. Addison Wesley Publ., 2001
[Marciniak 94] Marciniak, J. J.: Encyclopedia of Software Engineering. Vol. I and II, John Wiley & Sons Inc.,
1994
[McCabe 78] McCabe, T. J.: A Complexity Measure. IEEE Transactions on Software Engineering, 2(1976)4, pp.
308-320
[McGarry 02] McGarry, J.; Card, D.; Joines, C.; Layman, B.; Clark, E.; Dean, J.; Hall, F.: Practical Softwarer
Measurement – Objective Information for Decision Makers. Addison-Wesley Publ., 2002
[McGregor 95] McGregor, J. D.: Managing metrics in an iterative environment. Object Magazine, October
1995, pp. 65-71
[Mikkelsen 97] Mikkelsen, T.; Phirego, S.: Practical Software Configuration Management. Prentice Hall Publ.
1997
[Munson 03] Munson, J., C.: Software Engineering Measurement. CRC Press Company, Boca Raton London
New York, 2003
[Musilek 02] Musilek, P.; Pedrycz, W.; Sun, N.; Succi, G.: On the Sensitivity of COCOMO II Software Cost
Estimation Model. Proc. of the Eight IEEE Symposium on Software Metrics (METRICS 2002), June 4-7,
2002, Ottawa, pp. 13-20
[Neuhold 92] Neuhold, E. J.; Paul, M.: Formal Description of Programming Concepts. Springer Publ., 1991
[Nielson 99] Nielson, F.; Nielson, H. R.; Hankin, C.: Principles of Program Analysis. Springer Publ., 1999
[Offen 97] Offen, R. J.; Jefferey, R.: Establishing software measurement programs. IEEE Software, March/April
1997, pp. 45-53
[Pandian 04] Pandian, C. R.: Software Metrics – A Guide to Planning, Analysis, and Application. CRC Press
Company, 2004
[Peled 01] Peled, D. A.: Software Reliability Methods. Springer Publ., 2001
[Peters 02] Peters, D. K.; Parnas, D. L.: Requirements-Based Monitors for Real-Time Systems. IEEE
Transactions on Software Engineering, 28(2002)2, pp. 146-158
[Pfleeger 98] Pfleeger, S. L.: Software Engineering – Theory and Practice. Prentice-Hall Publ., 1998
[Poels 99] Poels, G.: On the Formal Aspects of the Measurement of Object-Oriented Software Specifications.
Dissertation, Katholieke Universiteit Leuven, Belgium, No. 125, 1999
[Potter 00] Potter, R. W.: The Art of Measurement. Prentice Hall Publ., 2000
[Prather 84] Prather, R.E.: An Axiomatic Theory of Software Complexity Measure. The Computer Journal,
27(1984)4, pp. 340-347
[Prechelt 01] Prechelt, L.: Kontrollierte Experimente in der Softwaretechnik – Potenzial und Methodik. Springer
Publ., 2001
[Putnam 78] Putnam, L. H.: General Empirical Solution to the Macro Software Sizing and Estimation Problem.
IEEE Transaction on Software Engineering, 4(1978)4, pp. 345-361
[Putnam 03] Putnam, L. H.; Myers, W.: Five Core Metrics – The Intelligence Behind Successful Software
Management. Dorset House Publishing, New York, 2003
[Putnam 92] Putnam, L. H.; Myers, W.: Measures for Excellence – Reliable Software on Time, within Budget.
Yourdon Press Comp., 1992
[Roberts 79] Roberts, F. S.: Measurement Theary with Applications to Decisionmaking, Utility, and the Social
Sciences. Addison Wesley Publ., 1979
[Schmietendorf 02] Schmietendorf, A.; Dimitrov, E.; Dumke, R.: Enterprise JavaBeans. MITP, 2002
[Schmietendorf 03] Schmietendorf, A.; Lezius, J.; Dimitrov, E.; Reitz, D.; Dumke, R.: Aktuelle Ansätze für Web
Service basierte Integrationslösungen. Preprint Nr. 10, Otto-von-Guericke-Universität Magdeburg, 2003
[Sheldrake 97] Sheldrake, R.: Seven Experiments That Could Change The World. (German) Goldman Publ. 1997
[Shepperd 95] Shepperd, M.: Foundations of Software Measurement. Prentice Hall Publ., 1995
59
[Shneidewind 95] Shneidewind, N.F.: Controlling and predicting the quality of space shuttle software using
metrics. Software Quality Journal, 4(1995), pp. 49-68
[Singpurwalla 99] Singpurwalla, N. D.; Wilson, S. P.: Statistical Methods in Software Engineering. Springer
Publ., 1999
[SMLab] SML@b Web Site: On-line unter http://ivs.cs.uni-magdeburg.de/sw-eng/us/
[Solingen 99] Solingen, v. R.; Berghout, E.: The Goal/Question/Metric Method. McGraw Hill Publ., 1999
[Sommerville 04] Sommerville, I.: Software Engineering. Seventh Edition, Addison Wesley, 2004
[Talley 91] Talley, D. J.; Total Quality Management – Performance and Cost Measures. ASQC Publ.,
Milwaukee, 1991
[Tervonen 97] Tervonen, I.; Kerola, P.: Towards deeper co-understanding of software quality. Information and
Software Technology, 39(1997), pp. 995-1003
[Tian 95] Tian, J.; Zelkowitz, V.: Complexity Measure Evaluation and Selection. IEEE Transactions on Software
Engineering, 21(1995)8, pp. 641-650
[Wang 00] Wang, Y.; King, G.: Software Engineering Processes – Principles and Applications. CRC Press,
2000
[Weinberg 92] Weinberg, G. M.: Quality Software Management. Vol. I and II, Dorset House Publ., 1992
[Whitmire 97] Whitmire, S.A.: Object Oriented Design Measurement. John Wiley & Sons, 1997
[Whitty 90] Whitty, R. W.: Structural Metrics for Z Specifications. Proc. of the Z User Workshop, Oxford 1989,
Springer Verlag, 1990, S. 186-191
[Wille 02] Wille, C.; Dumke, R., Stojanov, S.: New Measurement Intentions in Agent-Based System
Development. In: Dumke et al.: Software Measurement and Estimation, Shaker Publ., 2002, pp. 203-227
[Wohlin 00] Wohlin, C. et al.: Experimentation in Software Engineering: An Introduction. Kluwer Academic
Publ., 2000
[Xia 04] Xia, W.; Lee, G.: Grasping the Complexity of IS Development Projects. Comm. of the ACM,
47(2004)5, pp. 69-74
[Zelkowitz 98] Zelkowitz, M. V.; Wallace, D. R.: Experimental Models for Validating Technology. IEEE
Computer, May 1998, pp. 23-31
[Zuse 98] Zuse, H.: A Framework of Software Measurement. de Gruyter Publ., 1998
[Zuse 02] Zuse, H.: Problems and Pitfalls in Software Metrics Application. In: Dumke/Abran/Bundschuh/
Symons: Software Measurement and Estimation, Shaker Publ., 2002, pp. 1-8
[Zuse 91] Zuse, H.: Software Complexity - Measures and Methods. de Gruyter Publ., 1991
[Zuse 94] Zuse, H.: Software Complexity Metrics/Analysis. In: Marciniak: Encyclopedia of Software
Engineering, Volume I, John Wiley & Sons Inc., 1994, pp. 131-166
[Zuse 99] Zuse, H.: Thirty Years of Software Measurement. In: Dumke/Abran: Software Measurement – Current
Trends in Research and Practice, DUV, 1999, pp. 3-37
[Zuse 03] Zuse, H.: What can Practioneers learn from Measurement Theory. In: Dumke et al.: Investigations in
Software Measurement, Proc. of the IWSM 2003, Montreal, September 2003, pp. 175-176
[Zuse 92] Zuse, H.; Bollmann, P.: Measurement Theory and Software Measures. Proc. of the BCS-FACS
Workshop, London, May, 1992
[Zuse 89] Zuse, H.; Bollmann, P.: Using Measurement Theory to Describe the Properties and Scales of Static
Software Complexity Metrics. SIGPLAN Notices 24(1989)8, pp. 23-33
60