Analyzing the Feature Models
Maintainability over their
Evolution Process: An
Exploratory Study
Carla I. M. Bezerra, Ph.D. Student
Prof. Rossana M. C. Andrade, Ph.D.
Prof. José Maria Monteiro, Ph.D.
Prof. Lincoln Rocha, Ph.D
10th International Workshop on Variability Modelling of
Software-intensive Systems
January 27 – 29
2016
Summary
• Introduction
• Feature Model Evolution
• Goal
• Exploratory Study
• Feature Model Dataset
• Results
• Threats to Validity and Related Work
• Concluding Remarks and Future Work
2
Introduction
• One of the most important assets of an SPL is the feature model [1]
• The feature model and the SPL itself evolve and grow together [2]
• It is important to build feature models that can be easy to
understand and change (i.e., maintainable feature models)
[1] K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-oriented domain analysis (fFODS) feasibility study. Technical Report, Pittsburgh, PA, 1990.
[2] R. Lotufo, S. She, T. Berger, K. Czarnecki, and A. Wasowski. Evolution of the linux kernel variability model. In Software Product Lines: Going Beyond, pages 136{150. Springer, 2010.
3
Feature Model Evolution
• The feature model evolution occurs when some changes are
performed in the model structure [3]
• An example of a feature model evolution
[3] N. Gamez and L. Fuentes. Software product line evolution with cardinality-based feature models. In Top Productivity through Software Reuse, pages 102{118. Springer, 2011.
4
Feature Model Evolution: Change Types (1/4)
• Change type: To modify features
Modifying
Features
5
Feature Model Evolution: Changes Types (2/4)
• Change type: To add new features
Adding features
6
Feature Model Evolution: Changes Types (3/4)
• Change type: To add relationships
Adding ORrelationship
7
Feature Model Evolution: Changes Types (4/4)
• Change type: To add new constraints
Adding constraint
8
Motivation
• Quality evaluation of a feature model is essential to ensure that
errors in the early stages do not spread in the whole SPL [4]
• To evaluate the quality of an artifact, a popular strategy is the use
of measures
• It is important to evaluate the feature models to guarantee their
consistency and maintainability during their evolution
[4] Etxeberria, L. and Sagardui, G.: Evaluation of quality attribute variability in software product families. In: 15th Annual IEEE International Conference and Workshop on
the Engineering of Computer Based Systems, pp. 255-264 (2008)
9
Goal
• To analyze the impact of feature models maintainability in
their evolution using an exploratory study
10
About the Exploratory Study
• To support this study:
• We built a dataset containing a compiled set of 21
maintainability structural measures extracted from 16 feature
models and respective versions
Feature Models
Dataset
FM16
FM V16.1
FM V16.2
Measures
FM1
.
.
.
FM V1.1
NF
3
FM V1.2
4
FM V1.3
6
FM V1.3
FM V1.2
Measures
FM V1.1
NF
FM V16.1 3
...
...
FM V16.2 5
11
Feature Model Dataset
• Maintainability Measures [5]
Characteristics
Maintainability
Quality Attributes
Measures
Analyzability
Number of leaf Features (NLeaf)
Complexity Cyclomatic
Cyclomatic complexity (CyC)
Compound Complexity (ComC)
Cross-tree constraints (CTC)
Coefficient of connectivity-density (CoC)
Depth of tree (DT)
Number of features (NF)
Number of top features (NTop)
Extensibililty
Feature EXtendibility (FEX)
Flexibility
Flexibility of configuration (FoC)
Variability
Multiple Hotspot Features (MHoF)
Number of variable features (NVF)
Number of variation points (NVP)
Ratio of variability (RoV)
Number of valid configurations (NVC)
Rigid Nohotspot Features (RNoF)
Single Hotspot Features (SHoF)
[5] C. I. M. Bezerra, R. M. C. Andrade, and J. M. Monteiro. Measures for quality evaluation of feature models. In ICSR 2015, Miami, FL, USA. Proceedings, pages 282-297, 2015.
12
Feature Model Dataset
• The dataset generation methodology is described as
following:
• Step 1 - Feature Models Selection and Extraction
• Step 2 - Feature Models Loading
• Step 3 - Measure Values Exporting
continue
13
Feature Model Dataset
• The DyMMer tool
14
Feature Model Subdataset
• Step 4 - Identification of Feature Model Evolutions
• Step 4.1 - We identified features models with the same, or similar
name
• Step 4.2 - We verified its structure to ensure that they correspond
to different version of the same feature model evolution
• Step 4.3 - The feature models versions were sorted
Feature Model Versions
Sorting
Feature Model Versions
Selection
Feature Models
Dataset
Feature Model Structure
Verification
Feature Models
SubSet
Feature Models
SubSet
15
Exploratory Study
• Evolution types and the number of times that each change
type occurred in the dataset analysis of different versions of
16 feature models
16
Exploratory Study
• Evolution Types
17
Exploratory Study
Research Questions
Measure Name
RQ1 - Does the number of features
tend to increase as the feature
model evolves?
Number of Features
RQ2 - Does the number of child
features involved in alternative
(XOR) or OR relationships tend to
increase as the feature model
evolves?
Measurement Function
NF = number of features
Number of features in
OR relationships
Number of child features involved in
OR relationships
Number of features in
alternative
relationships
Number of child features involved in
alternative relationships
RQ3 - Does the depth of the feature
model tend to increase as
Depth of Tree
the SPL evolves?
Number of features of the longest
path from the root of the feature
model
18
Exploratory Study
Research Questions
Measure Name
Measurement Function
RQ4 - Does the width of the
feature model tend to increase as
the SPL evolves?
Number of Leaf Features
Number of features without
children
Number of Variable
Features
NVC = NA + NO
NA = Number of alternative
features
NO = Number of optional features
Number of Valid
Configurations
Number of possible and valid
configurations of the feature model
RQ5 - Does the variability tend to
increase as the feature
model evolves?
19
Exploratory Study
Research Questions
RQ6 - Does the complexity grow
as the feature model evolves?
Measure Name
Measurement Function
Cyclomatic Complexity
Number of integrity constraints
Compound Complexity
NF2 + (Rand 2 + 2*Ror2 + 3*Rcase2 +
3*Rgr2 + 3*R2)/9
Rand = Number of Mandatory
relationship
Ror = Number of OR relationship
Rcase = Number of XOR relationship
Rgr = Number of XOR or OR
relationship
R = Number of relationships among
nodes including constraints
20
Exploratory Study
• Results
Research Questions
Measure Name
RQ1 - Does the number of features tend to
Number of Features
increase as the feature model evolves?
RQ2 - Does the number of child features
involved in alternative (XOR) or OR
relationships tend to increase as the
feature model evolves?
Values
High
Number of features in OR
relationships
High
Number of features in
alternative relationships
High
RQ3 - Does the depth of the feature model
Depth of Tree
tend to increase as the SPL evolves?
Constant
21
Exploratory Study
• Results
Research Questions
Measure Name
RQ4 - Does the width of the feature model
Number of Leaf Features
tend to increase as the SPL evolves?
Values
High
High
RQ5 - Does the variability tend to increase Number of Variable Features
as the feature model evolves?
Number of Valid Configurations High
RQ6 - Does the complexity grow as the
feature model evolves?
Cyclomatic Complexity
High
Compound Complexity
High
22
Exploratory Study
• Results
• We can conclude:
• The SPL evolution decreases the feature model maintainability
23
Threats to Validity
• Internal Validity
• The identification of the evolution types in each version of each
feature model was conducted manually
• External Validity
• Our study considers only 16 feature models (and their respective
versions)
• Construct Validity
• The generation of the dataset was conducted in a manual manner
• Many feature models of the S.P.L.O.T. repository are "toy examples"
24
Related work
• Bagheri and Gasevic [6] proposed a number of structural
measures to assess the quality of feature models in SPLs
• Furthermore, the authors did not evaluate the evolution of feature
models, evaluating only one version of each feature model
• In Dintzner et al. [7] was conducted a study about the
extraction of feature models changes from Linux Kernel
• However, they did not evaluate the feature models using
maintainability measures
[6] E. Bagheri and D. Gasevic. Assessing the maintainability of software product line feature models using structural metrics. Software Quality Control, 19(3):579{612, Sept. 2011.
[7] ‘N. Dintzner, A. van Deursen, and M. Pinzger. Analysing the linux kernel feature model changes using fmdi . Software & Systems Modeling, pages 1{22, 2015.
25
Concluding Remarks
• We built a dataset containing a compiled set of 21
maintainability measures extracted from 16 feature models
and their different versions to support our analysis
• We analyzed the different versions of the feature models
through 6 research questions
• We also identified the most common changes performed in a
feature model during its evolution
• Our findings indicate that the feature models maintainability
tends to decrease as the feature models evolve
26
Future Work
• Future Work
• To expand the study with more maintainability measures and a
realistic sample of feature model evolutions (e.g., the Linux kernel)
• To investigate how we can use the measures catalog proposed by
Bezerra et al. [5], to improve the feature model maintainability
[5] C. I. M. Bezerra, R. M. C. Andrade, and J. M. Monteiro. Measures for quality evaluation of feature models. In ICSR 2015, Miami, FL, USA. Proceedings, pages 282-297, 2015.
27
Thank you!
Carla I. M. Bezerra
[email protected]
28