Maintenance Effort and Cost Estimation
Using Software Functional Sizes
De Tran-Cao 1
Ghislain Levesque 2
[email protected]
[email protected]
Software Engineering Management Research Laboratory (LRGL)
University of Quebec in Montreal (UQAM)
Abstract
This paper presents two models for estimating
software maintenance effort and cost using software size
measured by COSMIC Full Function Points (FFP) as the
independent variable. The proposed models are
established experimentally using ten real software
maintenance
projects
belonging
to
a
large
telecommunications company.
To establish the estimation models, the linear
regression technique is used to analyze the historical data.
This technique is also applied in combination with
logarithm transformation of the collected data. Our results
show that the correlation coefficients (or R2) between size
(FFP) and real effort and size (FFP) and cost are 89% and
88% respectively. With logarithm transformation, the
correlation coefficients are 85% and 86% for effort and
cost respectively. We recommend using the established
models in combination with logarithm transformation
because the prediction quality of these models (PRED) is
better than that of the models established without it.
In addition, we verified the quality of our models by
analyzing the Mean Magnitude of Relative Error
(MMRE) on ten other maintenance projects which were
carried out by the same company. The MMREs of the
effort and cost estimation models obtained are 48% and
45% respectively.
We also tried to estimate maintenance effort using
IFPUG 4.1 Function Points (FPA). Unfortunately, none of
the three effort models we used, including the Albrecht
and Gaffney model, the Kemerer model, and the Matson,
Barnett and Mellichamp model, provides good results
with our historical data.
Keywords: maintenance effort, maintenance cost, effort
estimation, cost estimation, Function Point Analysis
(FPA), COSMIC Full Function Points (FFP).
1
2
1. Introduction
Estimating software cost and effort for software
development and software maintenance has been a
research issue for over 25 years now. Many of the
estimation models that have been proposed are regressionbased, that is, the estimation model is derived using
regression analysis on data collected from past software
projects. Examples are the Walston Felix, Bailey-Basili,
COCOMO Basic and Doty models (see Pressman 1997,
p.121). Software size (in Lines of Code) is an independent
variable of these models.
In the literature, there are some well-known methods
for measuring software size, such as Lines of Code
[CON86] and Function Point Analysis (FPA) [ALB79,
83]. Recently, a number of new methods have been
proposed to adapt Albrecht's Function Point Analysis to
new software characteristics such as real-time software,
embedded software and scientific software [ABR01,
WHI95, JON91, REI90, SYM88].
Some models have been proposed for estimating
software development effort and cost using FPA [PRE97
p.121], but there are very few models that can be used to
estimate maintenance effort and cost from the software
size measured by these new sizing methods.
The main goal of this paper is to propose some
estimation models for software maintenance using
COSMIC Full Function Points (FFP) [ABR99]. Our
models have been established experimentally on the
historical data of ten software maintenance projects which
were carried out by a large telecommunications company
between 1999 and 2001. We use twelve other projects
used by this company (hereinafter referred to as Company
C) to verify our models by analyzing the Mean Magnitude
of Relative Error (MMRE). All projects mentioned in this
paper were maintenance projects of MIS software.
PhD student, Computer Science Department, UQAM.
Professor, Computer Science Department, UQAM.
1
The software measurement literature shows that every
estimation model to date has been established on a single
data set, which is limited in the size of its population and
originates from a specific contextual environment.
Therefore, it may fail when it is used to estimate the effort
or cost of a project in other environments. In fact, three
effort models using function points (the Albrecht and
Gaffney, Kemerer, and Matson, Barnett and Mellichamp
models) are not relevant for application to our collected
data. We accept that changing the environment in which
an experimental model was established might have a
significant effect on the accuracy of the model, and stress
that an experimental model should therefore be used with
care. It should also be remembered that some implicit
variables are inherent to each model.
In the next section, a survey of some of the work
related to this research is provided. Then, we establish
initial effort and cost models with linear regression
analysis in section 3. Section 4 is devoted to establishing
effort and cost models by applying linear regression
analysis to our collected data after transforming them into
logarithms. A comparison of these models and the initial
models will also be performed in this section. In section 5,
the quality of the two models obtained in section 4 will be
validated by analyzing their MMREs on ten other
projects. Section 6 presents some results obtained using
other models on our historical data. Finally, some
discussion and a conclusion make up the last section.
2. Related work
The size of software is often used as an independent
variable in many well-known models for estimating
software effort and cost. Software size can be measured in
terms of Lines of Code (LOC or KLOC, e.g. the Putnam
model [PUT78], the Basic COCOMO model [BOE95])
and function points (Albrecht's Function Points or
variations, e.g. the Albrecht and Gaffney model).
Estimation methods using function points are gaining
more and more attention because software size expressed
in terms of function points can be estimated much earlier
in the software process than LOC. Moreover, function
points are independent of technical and language
considerations [ALB83]. The wide acceptance of function
points in estimation models has motivated a great deal of
research, on the one hand to estimate software size more
accurately, and, on the other, to establish effort or cost
estimation models based on size in function points.
For software sizing, a number of new methods have
been proposed to adapt Albrecht's Function Points to new
software characteristics such as real-time, embedded and
scientific software. Examples are COSMIC Full Function
Points (FFP) [ABR01], 3D Function Points [WHI95],
Feature Points [JON91], Assert-R [REI90] and Mark II
[SYM88]. However, there are very few effort or cost
models using software size in terms of function points
with these new methods.
For COSMIC Full Function Points in particular,
although there are some reports on experiments involving
their use in predicting software development effort
[BOO99, BUR99, DUM99], no explicit model has been
proposed. Moreover, very few models can be used to
estimate software maintenance effort and cost from FFP.
The earliest models for this purpose might be those
reported by Levesque [LEV01], as follows:
Effort (man-months) = 3.39992* FFP + 36.531, with the
correlation coefficient R2 = 62%.
Cost (K$) = 0.2599* FFP + 4.3402, with the correlation
coefficient R2 = 63%.
These models were established by performing linear
regression analysis on fifteen software maintenance
projects, but, as seen above, the resulting correlation
coefficient is quite weak. Moreover, there is no report on
the quality of these models.
Some other models have been developed by Abran
[ABR02], but his models were established based on small
maintenance projects to maintain a single existing
application. Therefore, they might not be relevant to the
estimation of the maintenance effort associated with a big
project (many man-months).
In our research, we analyze ten maintenance projects
with efforts varying from 3 man-months to 121 manmonths. The average effort of these projects is 26 manmonths. All ten projects were carried out according to a
standard process which is similar to a development
process in that it includes specification, design,
implementation and test phases.
3. Effort and cost estimation model:
correlation analysis
In this section, we establish an effort estimation model
and a model for estimating cost by analyzing the
correlation between software size (FFP) and effort and
cost. For this purpose, we use the data collected on the
fifteen projects (Table 1) used by Levesque [LEV01] to
establish his cost and effort estimation model. In Table 1,
the cost and effort columns represent two independent
variables, and contain the real data collected by the
project managers of Company C. In the FFP column are
listed software sizes measured by the COSMIC FFP
method (version 2.0, 1999 [ABR99]). These measures
were obtained by a PhD student during his research
internship at Company C and were revised by an expert
who has worked for more than five years with Function
2
Point methods (Function Point Analysis, Full Function
Points, COSMIC-FFP).
Effort
Man-months Cost/ffp Effort/ffp
#
Project
FFP
COST $K
1
A
30
49
3.80
1.63
0.13
2
B-1
7
51
5.00
7.29
0.71
3
C
170
254
21.70
1.49
0.13
4
B-2
17
41
3.40
2.41
0.20
5
B-3
61
59
7.80
0.97
0.13
6
D
167
137
12.60
0.82
0.08
7
E
8
40
3.20
5.00
0.40
8
F
13
81
6.50
6.23
0.50
9
G
158
574
51.30
3.63
0.32
10
H
349
1617
121.20
4.63
0.35
11
I
18
713
55.70
39.61
3.09
12
J
60
259
19.40
4.32
0.32
13
K
15
91
7.40
6.07
0.49
14
L
8
97.5
9.30
12.19
1.16
15
M
35
278
25.10
FFP
Effort
COST $K Man-months Cost/ffp
#
Project
1
B-1
7
51
5.00
7.29
0.71
2
C
170
254
21.70
1.49
0.13
3
B-2
17
41
3.40
2.41
0.20
4
E
8
40
3.20
5.00
0.40
5
F
13
81
6.50
6.23
0.50
6
G
158
574
51.30
3.63
0.32
7
H
349
1617
121.20
4.63
0.35
8
J
60
259
19.40
4.32
0.32
9
K
15
91
7.40
6.07
0.49
10
M
35
278
25.10
7.94
0.72
Mean
4.82
0.37
Average deviation
4.90
0.41
Standard deviation
2.05
0.20
Table 2: Ten projects used to establish cost and effort models.
Software size in our project collection varies from 7 to
349 FFP and software cost from $40,000 to $1,617,000
Canadian dollars. The average cost for these ten projects
is $4,900 Canadian dollars per function point (FFP) and
the average effort is 0.41 man-months per function point.
0.72
4.63
0.35
Average deviation
6.95
0.58
Effort vs FFP
Standard deviation
9.55
0.75
effort=0,31*FFP+0,83
correlation determination = 89%
The two models proposed by Levesque (see section 2)
were derived from a regression analysis performed on
these fifteen projects, however the correlation
determination is quite weak, so these models might be not
relevant in practice to the estimation of effort and cost.
Our aim, therefore, was to build new models, and these
fifteen projects were used for this purpose. Some of them
were eliminated, however, as we were looking for more
robust models in terms of their correlation coefficients.
First, we eliminated project A because it is not of the
same type as other projects in our collection. In fact, the
objective of this project was to completely remove an
application from a system. No new functionality was
added to the system, unlike what was occurred in the
other projects. Then, a further four projects were
eliminated because of their cost/FFP ratio. In our
experiments, two projects with too large a cost/FFP ratio
(projects I and L) and two others with too small a ratio
(projects B3 and D) were eliminated. The ten remaining
projects (Table 2) were then used to establish effort and
cost models. The population used for the experiments was
quite small, but its homogeneity may be beneficial for the
generalization of an estimation model.
Effort (manmonths)
7.94
Mean
Table 1: Historical data of fifteen maintenance projects (copied
from [LEV01])
Effort/ffp
150
100
50
0
0
100
200
300
400
FFP
Figure 1: Relation between effort and size (FFP).
By linear regression analysis, we obtain the relation
between effort and size (FFP), as shown in Figure 1,
where the correlation coefficient (R2) is 89%. The relation
between cost and size (FFP) is shown in Figure 2, with
the correlation coefficient being 88%.
The effort estimation model derived from the linear
regression (Figure 1) is:
Effort = 0.31*FFP + 0.83
(1)
This model is quite good in terms of the correlation
coefficient (R2 = 89%). Moreover, the free constant of this
model, 0.83, conforms well to our intuitive understanding
if we interpret this constant as the fixed effort required to
start a project.
3
V
n∑
= 1
Cost vs FFP
Cost = 4,1*FFP-8,4
Correlation determination = 88%
Cost (K$)
2000
i =1
(Pi)−V real (Pi)
, where v is
V real(Pi)
estimted
cost or effort; and Pi (i=1..n) is the ith project considered.
1500
Some conclusions can be drawn from Table 3:
1000
-
500
0
0
100
200
300
400
FFP
Figure 2: Relation between cost and size (FFP).
The cost estimation model derived from the linear
regression (Figure 2) is:
(2)
Cost = 4.1*FFP – 8.4
MRE<=25%
25%<MRE<=50
%
MRE>50%
Effort
MRE MRE
Cost Man- Estimated Estimated of
of
Project FFP $K months effort
cost
effort cost
1
B-1
7
51
5.00
3.00
20.30
40% 60%
2
C
170
254
21.70
53.53
688.60
147% 171%
3
B-2
17
41
3.40
6.10
61.30
79% 50%
4
E
8
40
3.20
3.31
24.40
3%
5
F
13
81
6.50
4.86
44.90
25% 45%
6
G
158
574
51.30
49.81
639.40
3%
7
H
349 1617 121.20
109.02
1422.50
10% 12%
8
J
60
259
19.40
19.43
237.60
0%
9
K
15
91
7.40
5.48
53.10
26% 42%
10
M
35
278
25.10
11.68
135.10
53% 51%
MMRE
39% 49%
Max
147% 171%
For the effort estimation model (formula (1)): 50%
of projects are estimated with an MRE lower than
25% or Pred(0.25) = 0.5. Twenty percent of projects
are estimated with an error ranging from 26% to
50% or Pred(0.50) = 0.70. Thirty percent of projects
have an MRE of more than 50% (Figure 3). The
MMRE is 39%. In practice, the recommended
prediction quality of a model is Pred(0.25) ≥ 0.75
[CON86]. Clearly, this model is still far from the
recommended threshold.
MRE of effort model (1)
This is quite good in terms of the correlation
coefficient (R2=88%), but the free constant of this model,
which is normally interpreted as the fixed cost to start a
project, is –8.4. Therefore, the model (2) does not seem to
conform to our intuitive understanding.
#
n
Figure 3: Analyzing MRE of the effort model.
MRE of cost model (2)
39%
11%
MRE<=25%
25%<MRE<=50%
8%
MRE>50%
Figure 4: Analyzing MRE of the cost model.
Min
0%
8%
Table 3: MRE and MMRE of models (1) and (2).
Now, these models are evaluated by analyzing their
accuracy in terms of error range and the PRED. The
Magnitude of Relative Error (MRE) of each project and
the MMRE of all ten projects are presented in Table 3.
The MRE and MMRE are computed as follows:
V
MREv(P) =
(P)−V real(P) , where v is cost or
V real(P)
estimated
effort; and P represents the project considered.
n
MRE v(Pi)
n∑
i =1
MMREv = 1
-
For the cost estimation model (formula (2)): 30% of
projects fall into an error range lower than 25% or
Pred(0.25) = 0.30. Forty percent of projects fall into
an error range of between 26% and 50% or
Pred(0.50) = 0.70. Thirty percent of projects have an
error rate of over 50% (Figure 4). The MMRE is
49%. These results show that the cost model in
formula (2) is quite weak, and the residual error is
large, which suggests that we need to find another
cost model by supposing that the relation between
cost and size takes power form.
4. Effort and cost estimation model:
logarithm transformation analysis
In an attempt to find better models for effort and cost
estimation, we suppose that the relationship between
4
effort (or cost) and software size (FFP) is in power form
[MAT94]: V = b*FFPa, where V represents cost or effort.
MRE of effort model with log.
transform
By logarithm transformation, we have:
log(V) = a*log(FFP)+B, where B = log(b).
MRE<=25%
This formula takes the form of the linear regression
between the logarithm of cost (or effort) and the logarithm
of size (FFP). The base of the logarithm is not important,
so in our analysis we chose 10 for this base. The linear
regression between effort and size after logarithm
transformation is shown in Figure 5. A similar analysis for
cost is shown in Figure 6. The correlation coefficients are
85% and 86% respectively. These correlation coefficients
are almost the same as those of models (1) and (2) above.
Presenting the data in the form of log-log graphs of
Figures 5 and 6 has the merit of forcing the data to be
spread evenly along both axes, so the fitted curve is less
dependent on individual project data points, which is a big
advantage for statistical analysis.
25<MRE<=50%
MRE>51 %
Figure 7: MRE of the effort model established with LOG
transformation.
MRE of cost model with log.
transform
MRE<=25%
25<MRE<=50%
MRE>51 %
The models derived from the regression analysis in
Figure 5 and 6 are as follows:
Effort = 0.72* FFP0.81.
0.82
Cost = 8.32* FFP
(3)
.
(4)
Figure 8: MRE of the cost model established with LOG
transformation.
Once again, we used MRE and PRED to analyze these
models.
-
Effort vs FFP with log transform
log(Effort)
log(effort)=0,81*log(FFP)-013
correlation deternimation = 85%.
2.5
2.0
1.5
1.0
0.5
0.0
0.00
1.00
2.00
These results are almost the same as those of the
effort model in formula (1). However, the Pred(0.50)
of model (3) is a little higher than that of model (1).
We therefore suggest using model (3) rather than
model (1).
3.00
log(FFP)
Figure 5: Relation between effort and size (FFP) after logarithm
transformation.
Cost vs FFP with Log transform
log(cost)
4.0
log(cost)=0,82*log(FFP)+0,92
correlation determination = 86%.
3.0
2.0
1.0
0.0
0.00
1.00
2.00
3.00
log(FFP)
Figure 6: Relation between cost and size (FFP) after logarithm
transformation.
For the effort model (formula (3)): 50% of projects
are estimated with an MRE lower than 25%. Thirty
percent of projects are estimated with an error
ranging from 26% to 50%. Twenty percent of
projects have an MRE of more than 50% (Figure 7).
In other words, Pred(0.25) = 0.50, Pred(0.50) = 0.80.
The MMRE of this model is 40%.
-
For the cost model (formula (4)): 60% of projects are
estimated with an MRE lower than 25%. Twenty
percent of projects are estimated with an error
ranging from 26% to 50%. Twenty percent of
projects have an MRE of more than 50% (Figure 8).
In terms of PRED, we have Pred(0.25) = 0.60 and
Pred(0.50) = 0.80. The MMRE of this model is 39%.
These error margins are much better than those of
the model in formula (2).
A summary of the comparison of effort models (1) and
(3) is shown in Table 4. Cost models (2) and (4) are
compared in Table 5.
Effort Model
Correlation
2
R
Pred(
0.25)
Pred(
0.50)
MMRE
5
Effort = 0.31*FFP + 0.83
0.81
Effort = 0.72* FFP
0.89
0.50
0.70
39%
11
7.8
59
61
0.85
0.50
0.80
40%
12
12.6
137
167
Cost
$K
Effort
Cost
MRE of
MRE of
estimated estimated estimated estimated
by (3)
by (4)
effort
cost
Correlation
2
R
Pred(
0.25)
Pred(
0.50)
MMRE
%
#
1
6.3
98
23
9.13
108.83
45%
11%
0.88
0.30
0.70
49%
2
19.4
259
93
28.30
342.20
46%
32%
0.86
0.60
0.80
39%
3
59.29
769
64
20.91
251.88
65%
67%
4
7.84
98
24
9.45
112.69
21%
15%
Cost = 4.1*FFP – 8.4
0.82
Cost = 8.32* FFP
Table 5: Comparison of two cost models.
Comparison of the PRED of these pairs of models
leads us to propose the use of the two models that were
established with the logarithm transformation. The effort
model is as in formula (3), and the cost estimation model
is as in formula (4).
5. Model validation
In this section, we present our experimental validation
of the proposed models. The historical data of twelve
software projects are grouped for this purpose as in Table
6.
The real effort and real cost columns represent the
real effort and the real cost of the software projects
respectively. The FFP and FPA columns represent
software size in terms of COSMIC-FFP and IFPUG 4.1
Function Points [IFP99] respectively. For the first eight
projects (#1-#8), software sizes were counted by a
Master's student who had undertaken a research internship
for six months at the Software Engineering Management
Research Laboratory, University of Quebec in Montreal
in 2001. These measurements were revised by an expert
who has worked with Function Point methods (Function
Point Analysis, Full Function Points, COSMIC FFP) for
more than five years. The last four projects in Table 6
(B3, D, I, L) are those that were eliminated when we
established the estimation models, but which are now
being reused for validation purposes.
Real
Effort
Real
Cost $K
FFP
FPA
1
6.3
98
23
87
2
19.4
259
93
204
3 59.29
769
64
118
#
D
Table 6: Twelve projects used to validate the proposed models.
Table 4: Comparison of two effort models.
Cost Model
B3
244
Remark
4
7.84
98
24
24
5
58
794
76
270
6
40.7
505
32
32
7 53.65
747
49
62
8
28.2
408
110
165
9
9.3
98
34
42
L
10
55.7
713
68
105
I
Effort
FFP
5
58
794
76
24.03
289.99
59%
63%
6
40.7
505
32
11.93
142.67
71%
72%
7
53.65
747
49
16.84
202.34
69%
73%
8
28.2
408
110
32.42
392.70
15%
4%
9
9.3
98
34
12.53
149.95
35%
53%
10
55.7
713
68
21.96
264.72
61%
63%
11
7.8
59
61
20.11
242.16
158%
310%
12
12.6
137
167
45.47
553.03
261%
304%
MMRE of the first 10 projects
48%
45%
MMRE of 12 projects
75%
89%
Table 7: Validation results of effort and cost models.
The results of analyzing the MRE and MMRE of these
twelve projects are shown in Table 7. The last two
projects have very large MREs, which makes the MMREs
of these models very high. The interpretation might be
that the proposed models are not adapted to these kinds of
projects. There might be factors other than size which
significantly influence the efforts and costs of these two
projects. But, for the moment, we do not have enough
data to analyze what makes the errors of the estimated
efforts and costs of these projects so high. We therefore
continue the analysis of the MMRE with the first ten
projects only.
For the first ten projects, the MMRE of effort
estimated by formula (3) is 48% and the MMRE of cost
estimated by formula (4) is 45%. These results are higher
than the MMRE encountered for the ten projects that were
used to establish the models (40% and 39% for effort
model and cost model respectively). But the difference is
quite small. Therefore, we can say that, if projects #11
and #12 in Table 6 are excluded, the proposed models are
stable and applicable.
6. Comparisons with other models
In the research, we observed that, with software size
measurement in IFPUG 4.1 Function Points (FPA), there
is no correlation between effort and size, nor is there a
correlation between cost and size. In fact, the correlation
6
determination is 11% and 14% for effort and cost
respectively. The scatter-plots of data are shown in
Figures 9 and 10.
log(cost)
Cost v. FPA with log. transform
Effort
Effort v. FPA
100
50
0.00
0.00
1.00
0
100
200
2.00
3.00
300
Figure 12: No correlation between the logarithm of cost and the
logarithm of FPA.
FPA
Figure 9: No correlation between effort and FPA.
Cost v. FPA
1000
Cost
2.00
log(FPA)
0
500
0
0
100
200
300
FPA
A similar conclusion can be drawn about the
relationship between effort or cost and FPA after
transforming them into logarithmic form. There is no
correlation between the logarithm of effort and that of
FPA (R2 = 17%), nor is there a correlation between the
logarithm of cost and that of FPA (R2 = 22%). The
scatter-plots of data are shown in Figures 11 and 12.
Therefore, no effort or cost model using FPA can be
established adequately by linear regression analysis on
the first ten projects in Table 6.
Effort v. FPA with log. transform
2.00
1.00
0.00
0.00
1.00
We also observed that some well-known effort
estimation models based on function points, such as the
Albrecht and Gaffney, Kemerer, and Matson, Barnett and
Mellichamp models (see Pressman, 1997, p.121) are not
capable of giving good estimations for these projects.
Table 8 presents the results of applying the following
three models to eleven projects in Table 6 (we do not have
the size of project #11 in FPA).
-
Albrecht and Gaffney model:
E(man-months) = 0.0545*FPA – 13.39
Figure 10: No correlation between cost and FPA.
log(Effort)
4.00
2.00
3.00
log(FPA)
Figure 11: No correlation between the logarithm of effort and the
logarithm of FPA.
-
Kemerer model:
E(man-months) = 60.62*7.728*10-8*FPA3
-
(5)
(6)
Matson et al. model:
E(work hours) = 585.7 + 15.12*FPA.
(7)
The Albrecht and Gaffney model (5) provides
estimated results which are too far from real data, so it is
not necessary to analyze it further.
The Kemerer model (6) provides some significant
results when compared with the real effort. But the MRE
is large, varying from 25% to 105% (analyzed on the first
ten projects). The MMRE is 81%, which is significantly
higher than the MMRE of our effort model (48%).
The Matson et al. model (7) gives better results than
those of the Kemerer model, but the MMRE (54%) is still
higher than that of our model (48%). Note that the Matson
et al. model measures effort in hours of work. Therefore,
to compare with real effort (man-months) in our data
collection, the effort estimated by this model is divided by
140 (1 man-month = 140 hours of work).
We do not mean to imply here that our model is better
than those models that use FPA, since the context and the
environment in which these three models were established
may be very different from the environment of our
projects, hence productivity. Moreover, these models
were proposed for estimating software development
effort, and so they might not be relevant for estimating
maintenance effort. Another potential reason for the
7
inaccuracy of these models may be the version of the
document used to count FPA. In our historical data,
software size is counted using IFPUG 4.1 [IFP99], while
these three models are based on Albrecht's Function
Points [ALB83].
Effort (man-months)
MRE %
of
MRE of
by
by
by
Real
Albrecht Kemerer Matson Kemerer Matson
Effort FPA model
model
model
model
model
#
1
6.3
87
-8.65
3.08
13.58
51%
116%
2 19.4
204
-2.27
39.77
26.22
105%
35%
3 59.29 118
-6.95
7.74
16.95
87%
71%
24
-12.06
0.07
6.82
99%
13%
270
1.33
92.21
33.34
59%
43%
6 40.7
32
-11.65
0.15
7.64
100%
81%
7 53.65
62
-10.03
1.10
10.84
98%
80%
8 28.2
165
-4.40
21.04
22.00
25%
22%
4 7.84
5
9
58
9.3
42
-11.12
0.34
8.68
96%
7%
10 55.7
105
-7.67
5.42
15.52
90%
72%
12 12.6
244
-0.09
68.05
30.54
440%
142%
81%
54%
114%
62%
MMRE of the first 10 projects
MMRE of all 11 projects
sample is quite small, but we hope the homogeneity of the
environment allows us to generalize the effort and cost
estimation models for application in other, similar
environments.
The validation of our models on ten projects shows
that these models are readily applicable. The MMRE of
the proposed effort model is 48% and that of the cost
model is 45%. These results are really significant because,
on the one hand, we do not have any well-known model
which permits the estimation of effort/cost from FFP, and,
on the other, there were three effort models using FPA
which could not be applied satisfactorily on our collected
data.
Actually, COSMIC FFP measures software size
regardless of problem complexity [TRA01, 02], which
means that estimation models using FFP do not take it
into account. Further research will investigate this issue
with the aim of measuring problem complexity and then
of establishing an estimation model in which effort and
cost are a function of both size and complexity.
Table 8: Results of applying 3 models using FPA.
However, we can conclude that, in our context, FFP
is, in fact, more helpful than FPA in predicting software
maintenance effort and cost.
7. Conclusion
Estimating effort and cost for software development
and software maintenance remains a research issue in both
software engineering and software management.
Managers and programmers need the accuracy and
robustness of estimation methods. However, they need
simple estimation methods. These two criteria are
contradictory, because by their nature software effort and
cost depend on many factors, from the production process
to staff skills, from programming methodologies to used
tools.
Our work aims to establish two simple models, one for
effort and one for cost estimation. These models are
established by analyzing the linear regression of the
logarithm transformation of historical data. All twentytwo projects collected (ten used for establishing the
models and twelve used for validating the models) were
carried out by the same company. The population of the
8
References
[ABR02] Abran A., Silva I., Primera L., Field Studies
Using Functional Size Measurement in Building
Estimation Models for Software Maintenance, Journal of
Software Maintenance and Evolution: Research and
Practice, 2002, Vol.14, pp. 31-64.
[ABR01] Abran, A.; Desharnais, J.-M.; Oligny, S.; StPierre, D.; Symons, C., COSMIC FFP – Measurement
Manual – Field Trials Version, Montreal, May, 2001.
[ABR99] Abran, A.; Desharnais, J.-M.; Oligny, S.; StPierre, D.; Symons, C., COSMIC FFP – Measurement
Manual – Version 2.0, edited by Oligny S., Software
Engineering
Management
Research
Laboratory,
University of Quebec in Montreal, 1999.
[ALB83] Albrecht A., Gaffney J., Software Function,
Source Lines of Code, and Development Effort Prediction:
A Software Science Validation, IEEE Transactions on
Software Engineering, Vol. SE-9, No. 6, November 1983,
pp. 639-648.
[ALB79]
Albrecht
A.,
Measuring
Application
Development Productivity, presented at the IBM
Applications Development Symposium, 1979.
[BOE95] Boehm, B.W., Clark, B., Horowitz et al., Cost
Model for Future Life Cycle Processes: COCOMO 2.0,
Annals of Software Engineering 1 (1), 1-24, Nov. 1995.
[BOE81] Boehm, B., Software Engineering Economics,
Englewood Cliffs, NJ Prentice-Hall, 1981.
[BOO99] Bootsma F., Applying Full Function Points to
Drive Strategic Business Improvement within Real-Time
Software Environment. Proceedings of the 1999 Annual
IFPUG Conference. IFPUG, Westerville OH, 1999, pp.
159-218.
[BUR99] Büren G, Koll I., First experiences with the
introduction of an effort estimation process. Proceedings
of the 12th International Conference on Software and
System Engineering and their Application (ICSSEA).
CNAM: Paris, France, 1999, pp. 14-30.
[CON86] Conte ,S.D., Dunsmore, H.E., Shen, V.Y.,
Software Engineering Metrics and Models,
The
Benjamin/Cummings Publishing Company, Inc., 1986.
[JON91] Jones C., Applied Software Measurement –
Assuring Productivity and Quality, New York, McGrawHill Inc, 1991 (2nd edition, August 1996).
[LEV01] Lévesque G., Bevo V., Measuring Size for the
Development of a Cost Model: A Comparison of Results
Based on COSMIC FFP and Slim Back-Firing Function
Points, Proceedings of the 11th International Workshop
on Software Measurement, Montreal, Canada 2001,
IWSM 2001.
[MAT94] Matson, J., Barrett B., Mellichamp J., Software
Development Cost Estimation Using Function Points,
IEEE Transactions on Software Engineering, vol. 20,
no.4, April 1994, pp. 275-287.
[PRE97] Pressman R.S., Software Engineering: A
Practitioner’s Approach, fourth edition, The McGrawHill Companies, Inc., 1997.
[PUT78] Putnam, L.H., A General Empirical Solution to
the Macro Software Sizing and Estimating Problem, IEEE
Transactions on Software Engineering, Vol. 4, No. 4, July
1978, pp. .345-361.
[REI90] Reifer D.J., Assert-R: A Function Point sizing
tool for scientific and real-time systems, Journal of
Systems Software, vol. 11, pp. 159-171, 1990.
[SYM88] Symons C., Function Point Analysis:
Difficulties and Improvements, IEEE Transactions on
Software Engineering, Vol. 14, No. 1, January, 1988.
[TRA01] Tran-Cao D., Abran A., Levesque G.,
Functional Complexity Measurement, in Proceedings of
the 11th International Workshop on Software
Measurement, Montreal, Canada 2001 (IWSM’01), pp.
173-181.
[TRA02] Tran-Cao D., Levesque G., Abran A.,
Measuring Software Functional Size: Towards an
Effective Measurement of Complexity, IEEE International
Conference on Software Maintenance, Montreal, Canada,
Oct. 2002 (ICSM’02), pp. 370-376.
[WHI95] Whitmire S.A., An Introduction to 3D Function
Points, Software Development, pp. 43-53, 1995.
[DUM99] Dumke R., Foltin E., Applicability of Full
Function Points for Siemens AT. Proceedings of the
International Workshop on Software Measurement
(IWSM’99), 1999.
[IFP99] IFPUG, Function Point Counting Practices
Manual, Release 4.1. International Function Point Users
Group, January 1999.
9