FZI FORSCHUNGSZENTRUM
INFORMATIK
From Product Copies to Product Lines
Guest-Lecture – University of Stuttgart
Benjamin Klatt, 16.01.2014
Customized
Copy 1
Consolidation
Original
Product
Customized
Copy 2
Software
Product Line
Short Profile – Benjamin Klatt
 Research Associate & Projectmanager
@ FZI – Forschungszentrum Informatik (Karlsruhe)
Education
Industry
• Diploma in Computer Science
• Certified Engineer in
Computer Science
7/28/2014
• ~10 Years IT Consulting &
Product Development
© FZI Forschungszentrum Informatik
2
Example of a typical customization scenario
E-COMMERCE INTEGRATION
7/28/2014
© FZI Forschungszentrum Informatik
3
E-Commerce Integration
Online Shop
Customer
Product
Database
Load Data
Search
Enterprise Resource
Planing (ERP)
-
7/28/2014
Stock Information
Prices
…
© FZI Forschungszentrum Informatik
4
E-Commerce Integration
Customized Solutions
Customer 2
PD2 & ERP2
Customer
PD1 & ERP1
Time
Pressure
Customer 3
PD3 & ERP3
7/28/2014
© FZI Forschungszentrum Informatik
5
E-Commerce Integration
Consolidate to Extendable Connectors
Online Shop
PD - Connector
Product
Database
Product
Product
Database
Load Data
Database
Product
Database
Search
Enterprise
Resource
ERP Connector
Planing (ERP)
ERP -
7/28/2014
Stock Information
Prices
ERP
…
ERP
© FZI Forschungszentrum Informatik
6
E-Commerce Integration
Slightly different Setups for the same ERP Product
Online Shop
Product
Database
Product
Database
PD - Connector
Product
Database
Search
ERP Connector
ERP 1
Customer-specific
ERP Connections
7/28/2014
ERP 2
ERP 3
ERP 2a
ERP 2b
ERP 2c
ERP 2d
ERP 2e
…
© FZI Forschungszentrum Informatik
Customer
Support Teams
7
„Copy & Customize“ / „Clown & Own“
 We cannot prevent copy-based customization
 Reasons








7/28/2014
Time pressure
Instable requirements
Intellectual properties
Developer knowledge
Culture & society
Organization
Flexibility is intended
many more …
© FZI Forschungszentrum Informatik
8
HANDLING CUSTOMIZED
PRODUCT COPIES
7/28/2014
© FZI Forschungszentrum Informatik
9
Approaches to Handle Customized Copies
2 General
Strategies
Difference
Management
„Single-Copy“
Consolidation
Advantages of the „Single-Copy“ Approach
•
•
•
•
Benefit from common SPL advantages
Clear code base to start next project
Prevent trace evolution problem
Enable new feature combinations to sell
[rubin2013a]
7/28/2014
© FZI Forschungszentrum Informatik
10
How does a
consolidation look like?
Customized
Copy 1
Original
Product
Software
Product Line
Customized
Copy 2
7/28/2014
© FZI Forschungszentrum Informatik
11
How does a
consolidation look like?
Programcomprehension
7/28/2014
Design
Refactoring
© FZI Forschungszentrum Informatik
12
Identify program variants in customized copies
PROGRAM COMPREHENSION
7/28/2014
© FZI Forschungszentrum Informatik
13
Step 0: Prepare Consolidation Candidates
 Identify Consolidation Candidates

Requires expert knowledge
 Developer or product manager identifies potential candidates
 Candidate Scoping

Complete copies or just specific modules?
 Try to reduce complexity
 Choose Leading Variant


7/28/2014
Select a copy to integrate modifications to
Must not be the original product
© FZI Forschungszentrum Informatik
14
package org.splevo.examples.calculator;
package org.splevo.examples.calculator;
/**
/**
* The command line tool to run the calculator. * The command line tool to run the calculator.
*/
*/
public class CalculatorTool {
public class CalculatorTool {
package org.splevo.examples.calculator;
/**
* The command line tool to run the calculator.
*/
public class CalculatorTool {
/**
/**
/**
* Executing the calculator tool.
* Executing the calculator tool.
* Executing the calculator tool.
*
*
*
* @param args The command line args provided to the
* @param
tooling.
args The command line args provided to the tooling.
* @param args The command line args provided to the tooling.
*/
*/
*/
public static void main(String[] args) {
public static void main(String[] args) {
public static void main(String[] args) {
if(args.length > 0 && args[0] == "sqrt"){
String value = "13.25";
if(args.length > 0 && args[0] == "sqrt"){
String value = "13.25";
if(args.length > 0 && args[0] == "sqrt"){
String value = "13.25";
CalculatorSqrt calculator = new CalculatorSqrt();CalculatorSqrt calculator = new CalculatorSqrt();
String sqrt = calculator.sqrt(value);
String sqrt = calculator.sqrt(value);
CalculatorSqrt calculator = new CalculatorSqrt();
String sqrt = calculator.sqrt(value);
System.out.println(sqrt);
System.out.println(sqrt);
System.out.println(sqrt);
} else {
} else {
} else {
String value1 = "9876543210987654321098765432109876543210";
String value1 = "9876543210987654321098765432109876543210";
String value1 = "9876543210987654321098765432109876543210";
String value2 = "1234567891234567891234567891234567891234";
String value2 = "1234567891234567891234567891234567891234";
String value2 = "1234567891234567891234567891234567891234";
CalculatorGCD calculator = new CalculatorGCD(); CalculatorGCD calculator = new CalculatorGCD();
String gcd = calculator.gcd(value1, value2);
String gcd = calculator.gcd(value1, value2);
System.out.println(gcd);
CalculatorGCD calculator = new CalculatorGCD();
String gcd = calculator.gcd(value1, value2);
System.out.println(gcd);
}
System.out.println(gcd);
}
}
}
}
}
}
}
}
package org.splevo.examples.calculator;
package org.splevo.examples.calculator;
package org.splevo.examples.calculator;
import org.jscience.mathematics.number.LargeInteger;
import org.jscience.mathematics.number.LargeInteger;
java.math.BigInteger;
import java.math.BigInteger;
/**
/**
* The calculator to perform numeric calculations.* The calculator to perform numeric calculations.
* This makes use of the LargeInteger class of the* This makes use of the LargeInteger class of the
* JScience mathematics library (www.jscience.org)* JScience mathematics library (www.jscience.org)
*/
*/
public class CalculatorGCD {
public class CalculatorGCD {
/**
* The calculator to perform numeric calculations.
*
*
*/
public class CalculatorGCD {
/**
/**
/**
* Calculate the greatest common denominator (divider)
* Calculate
(GCD) of
theangreatest
integer common denominator (divider) (GCD)
* Calculate
of an integer
the greatest common denominator (divider) (GCD) of an integer
* @param value1 The first number to get the gcd. * @param value1 The first number to get the gcd.
* @param value1 The first number to get the gcd.
* @param value2 The second number to get the gcd.* @param value2 The second number to get the gcd.
* @param value2 The second number to get the gcd.
* @return
* @return
* @return
*/
*/
*/
public String gcd(String value1, String value2){ public String gcd(String value1, String value2){
public String gcd(String value1, String value2){
LargeInteger integerValue1 = LargeInteger.valueOf(value1);
LargeInteger
BigInteger integerValue1
integerValue1==new
LargeInteger.valueOf(value1);
BigInteger(value1);
BigInteger integerValue1 = new BigInteger(value1);
LargeInteger integerValue2 = LargeInteger.valueOf(value2);
LargeInteger
BigInteger integerValue2
integerValue2==new
LargeInteger.valueOf(value2);
BigInteger(value2);
BigInteger integerValue2 = new BigInteger(value2);
LargeInteger gcd = integerValue1.gcd(integerValue2);
LargeInteger
BigInteger gcd
gcd==integerValue1.gcd(integerValue2);
integerValue1.gcd(integerValue2);
BigInteger gcd = integerValue1.gcd(integerValue2);
return gcd.toString();
return
returngcd.toString();
gcd.toString();
return gcd.toString();
}
}
}
}
}
}
package org.splevo.examples.calculator;
package org.splevo.examples.calculator;
package org.splevo.examples.calculator;
import org.jscience.mathematics.number.FloatingPoint;
import org.jscience.mathematics.number.FloatingPoint;
/**
/**
* Calculator to get the positive square root of * Calculator to get the positive square root of
* a floating point number.
* a floating point number.
* This makes use of the FloatingPoint class of the
* This makes use of the FloatingPoint class of the
* JScience mathematics library (www.jscience.org)* JScience mathematics library (www.jscience.org)
*/
*/
public class CalculatorSqrt {
public class CalculatorSqrt {
/**
* Calculator to get the positive square root of
* a floating point number.
*
*
*/
public class CalculatorSqrt {
/**
/**
/**
* Calculate the square root (sqrt) of a floating *point
Calculate
number.
the square root (sqrt) of a floating point number.
* Calculate the square root (sqrt) of a floating point number.
* @param value1 The number to get the square root* for.
@param value1 The number to get the square root for.
* @param value1 The number to get the square root for.
* @return The calculated square root.
* @return The calculated square root.
* @return The calculated square root.
*/
*/
*/
public String sqrt(String value1){
public String sqrt(String value1){
public String sqrt(String value1){
FloatingPoint floatingPointValue = FloatingPoint.valueOf(value1);
FloatingPoint
Double floatingPointValue
floatingPointValue
= Double.valueOf(value1);
= FloatingPoint.valueOf(value1);
Double floatingPointValue = Double.valueOf(value1);
FloatingPoint sqrt = floatingPointValue.sqrt(); FloatingPoint
Double sqrt = sqrt
Math.sqrt(floatingPointValue);
= floatingPointValue.sqrt();
Double sqrt = Math.sqrt(floatingPointValue);
return sqrt.toString();
return sqrt.toString();
}
}
7/28/2014
return sqrt.toString();
}
}
}
}
© FZI Forschungszentrum Informatik
15
package org.splevo.examples.calculator;
/**
* The command line tool to run the calculator.
*/
public class CalculatorTool {
/**
* Executing the calculator tool.
*
* @param args The command line args provided to the tooling.
*/
public static void main(String[] args) {
if(args.length > 0 && args[0] == "sqrt"){
String value = "13.25";
CalculatorSqrt calculator = new CalculatorSqrt();
String sqrt = calculator.sqrt(value);
“import org.jscience.mathematics.number.LargeInteger;”
versus
“import java.math.BigInteger;”
System.out.println(sqrt);
} else {
String value1 = "9876543210987654321098765432109876543210";
String value2 = "1234567891234567891234567891234567891234";
CalculatorGCD calculator = new CalculatorGCD();
String gcd = calculator.gcd(value1, value2);
System.out.println(gcd);
}
}
}
package org.splevo.examples.calculator;
import org.jscience.mathematics.number.LargeInteger;
java.math.BigInteger;
/**
* The calculator to perform numeric calculations.
* This makes use of the LargeInteger class of the
* JScience mathematics library (www.jscience.org)
*/
public class CalculatorGCD {
/**
* Calculate the greatest common denominator (divider) (GCD) of an integer
* @param value1 The first number to get the gcd.
* @param value2 The second number to get the gcd.
* @return
*/
public String gcd(String value1, String value2){
/**
* The calculator to perform numeric calculations.
* This makes use of the LargeInteger class of the
* JScience mathematics library (www.jscience.org)
*/
LargeInteger
BigInteger integerValue1
integerValue1 ==new
LargeInteger.valueOf(value1);
BigInteger(value1);
LargeInteger
BigInteger integerValue2
integerValue2 ==new
LargeInteger.valueOf(value2);
BigInteger(value2);
LargeInteger
BigInteger gcd
gcd == integerValue1.gcd(integerValue2);
integerValue1.gcd(integerValue2);
return
returngcd.toString();
gcd.toString();
}
}
package org.splevo.examples.calculator;
import org.jscience.mathematics.number.FloatingPoint;
/**
* Calculator to get the positive square root of
* a floating point number.
* This makes use of the FloatingPoint class of the
* JScience mathematics library (www.jscience.org)
*/
public class CalculatorSqrt {
/**
* Calculate the square root (sqrt) of a floating point number.
* @param value1 The number to get the square root for.
* @return The calculated square root.
*/
public String sqrt(String value1){
FloatingPoint
Double floatingPointValue
floatingPointValue
= Double.valueOf(value1);
= FloatingPoint.valueOf(value1);
FloatingPoint
Double sqrt = sqrt
Math.sqrt(floatingPointValue);
= floatingPointValue.sqrt();
return sqrt.toString();
}
}
7/28/2014
© FZI Forschungszentrum Informatik
16
= Changed Code
(multiple modified lines per block)
7/28/2014
© FZI Forschungszentrum Informatik
17
= Related changes
= Changed Code
(multiple modified lines per block)
7/28/2014
© FZI Forschungszentrum Informatik
18
Program Comprehension
Difference Analysis
• Detect differences / modifications
between the copies
Kopie 1
1
Differenzanalyse
Differenzen
Kopie 2
Variability Analysis
• Identify related Changes
• Reduce information load
• Derive Variability Design
2
Variabilitätsanalyse
Differenzen
Beziehungen
7/28/2014
© FZI Forschungszentrum Informatik
19
Remarks about Clone Detection
 Inverse Goal: Finding similar code
not finding differing code
 Project / Product internal analysis
not between multiple projects
 Based on Heuristics
no clear changed or not changed
 For removing redundant code
not for combining co-existent alternative code
 Might serve as vague preparation step.
Don’t expect too much.
7/28/2014
© FZI Forschungszentrum Informatik
20
Product
Copy 1
Difference
Analysis
Product
Copy 2
Differences
Find program changes
PROGRAM COMPREHENSION:
DIFFERENCE ANALYSIS
7/28/2014
© FZI Forschungszentrum Informatik
21
Difference Analysis
 Goal

Find differences between consolidation candidates
 Stakeholders

Developer: Capture the degree of customization
 Challenges

Possibly lot of changes
 Consider software structures (not only text)
 Approach

Software specific difference analysis
 Filter irrelevant changes
7/28/2014
© FZI Forschungszentrum Informatik
22
Model- vs. Text-based Difference Analysis
Compare
Model-based
Model
CodeDifferences
Extraction
CodeCopies
Interpretation
Compare
TextText-based
Differences
Preferable Approach: Model-based
• Extraction better supported than interpretation
• Compare can be optimized with semantic of the model elements
• Downstream process needs extracted information anyway
7/28/2014
© FZI Forschungszentrum Informatik
23
2 (½)-Phase Model-Difference Analysis
Matching
Diffing
Post
Processing
[xing2006] / [kehrer2012]
 Matching
Find model elements representing the same object
 Diffing
Identify differences between matches
 Post Processing
„Semantic Lift“ and model optimization
7/28/2014
© FZI Forschungszentrum Informatik
24
Software-Difference Analysis: Matching-Phase
 „False Positives“ better than „False Negatives“

Prefer manual investigation over lost differences
 Depth-First Search

Based on model containment relationships (Top-Down)
 Scoping: Filter irrelevant elements
 Similarity Check based on

7/28/2014
Types, attributes, references (e.g. import)
© FZI Forschungszentrum Informatik
25
Software-Difference Analysis: Diffing-Phase
 Specific Diff-Types
ClassChange, MethodChange, StatementChange,…

Typed for downstream process
 Changes refer to changed software elements
 Minimum granularity depending on intended variability mechanism
e.g. Configuration-based: Statements and above
 Modification Kinds
Add, Delete, Change
 Diff-Detection

Single-side matches
 Minimum granularity and above: Add / Delete
 More fine granular: Change of the next minimum element
(e.g. Expressions)
7/28/2014
© FZI Forschungszentrum Informatik
26
Result of the Difference Analysis
 Changes between the copies to consolidate

Typed changes (classes, methods, statements, …)
 Example: Eclipse JDT Compare
 Don’t forget other artifacts!
7/28/2014
© FZI Forschungszentrum Informatik
27
Remaining Challenges
 Finding related changes to derive variability
 Variation Point Design
7/28/2014
© FZI Forschungszentrum Informatik
28
Variability
Analysis
Differences
Relationships
Productline
Design
Find technical related differences
0
VARIABILITY ANALYSIS
7/28/2014
© FZI Forschungszentrum Informatik
29
Variability Analysis
= Changed Code
(multiple modified lines per block)
7/28/2014
© FZI Forschungszentrum Informatik
30
Variability Analysis
Identify related Changes
7/28/2014
© FZI Forschungszentrum Informatik
31
Variability Analysis
 Goal

Find related changes to identify the required variability
 Stakeholders

Developer: Technical software design perspective
 Challenges

High information load
 Many clustering opportunities
 Approach

Identify technical required dependencies
 Design reasonable variability
7/28/2014
© FZI Forschungszentrum Informatik
32
Variation Points and Features
Product
Management
SoftwareDesign
SoftwareImplementation
7/28/2014
© FZI Forschungszentrum Informatik
33
Variation Point Relationships
Code Location
• VPs at the same code location
Program Structure
• VPs with program references
Dependency
Data
public void doSth(){
... // VP1
...
Dependency
VPs with data dependencies
... // •VP2
Import
org.package.ClassA; // VP1
}
public void doSth(){
Program
• VPs on the same execution path
...
Execution Trace
ClassA obj = new ClassA() // VP2
public ...
void doSth(){
... }
String
VP1 a = getInput()
…
• … // VP1
...
VP2
VP3
processInput(a);
//
VP2
}
7/28/2014
© FZI Forschungszentrum Informatik
34
Variability Analysis Strategies
…
•…
• VPs with the same custom change pattern
Change Pattern
Cloned Change
Semantic
Relationship
• VPsparameter
with cloned
change
Method
with
name
“session” added which is checked
for null as the first statement
Added
the with
samesame
4 statements
at for change
• VPs
terms used
different code locations
Common
ModificationsNew
…
7/28/2014
• VPs with changes in same context
classes and methods all
containing the term “UseCase” in
their names
• others possible
Changes committed at the same time
or with a reference to the same
issue.
© FZI Forschungszentrum Informatik
35
Variability
Analysis
Differences
Relationships
Productline
Design
0
Define final variation point structure and implementation
VARIATION POINT DESIGN
7/28/2014
© FZI Forschungszentrum Informatik
36
Variation Point Design
 Goal

Design variation points of the desired Software Product Line
including their variability implementation technique
 Stakeholders

Developer: Technical software design perspective
 Product Manager: „Customer compatible“ features
 Challenges

Balance technical constraints and product management
 Decide for implementation techniques
 Approach

Technical guidelines
 Stakeholder interaction
7/28/2014
© FZI Forschungszentrum Informatik
37
Individuality of SPL Design Decisions
Stakeholder Interests
• Technical Demand (Software management)
• Marketing Demand (Client satisfaction)
Product Line Quality Dimensions
• Redundant code vs. complex code
• Flexibility vs. manageability
[rubin2013a]
Variability Handling
• Binding Time
• Extendibility
• Extension Management
7/28/2014
© FZI Forschungszentrum Informatik
[svahnberg2005]
38
Characteristics of Implementation Techniques
Types of variability
Technology
• Additive
• Subtractive
• Language-Based
• Tool-Based
Cardinality of variability
Representation
•
•
•
•
• Annotation-based
• Composition-based
Optional
Alternative
Mandatory
(External)
Binding Time
•
•
•
•
Design-time
Compile-time
Load-time
Runtime-time
7/28/2014
© FZI Forschungszentrum Informatik
39
Implementation Techniques and Characteristics
[apel2013a]
7/28/2014
© FZI Forschungszentrum Informatik
40
Refactoring
0
Software
Product Line
Migrate implementation
REFACTORING
7/28/2014
© FZI Forschungszentrum Informatik
42
Refactoring
 Goal

Merge the copies according to the variation point design
 Stakeholders

Developer: Modify code efficiently
 Challenges

General refactoring tools are not sufficient
 Produce input for SPL management tools
 Approach

Define refactoring steps
 Automate as much as possible
7/28/2014
© FZI Forschungszentrum Informatik
43
Common Refactoring Tools
 Typically integrated into IDE
 Example: Eclipse JDT Refactorings
7/28/2014
© FZI Forschungszentrum Informatik
44
Refactoring Patterns
 Structured description why, when and how to perform a refactoring
 Described by





Name – identify refactorings
Summary – brief description of the transformation
Motivation – when to apply and when not to apply
Mechanics – step-by-step instructions on how to apply
Examples
 Goal

Repeatable refactoring
 Quality assurance
 Automation
7/28/2014
© FZI Forschungszentrum Informatik
45
Refactoring Pattern Example
 Name:
 Summary:


Extract method
Turn a code fragment into a method whose name
explains the purpose of the fragment
Motivation:
 Improve readability, reduce potential duplication
 Use only when code fragment is cohesive
(easy to find a suitable name for it)
 Do not use when fragment has many
dependencies to variables defined outside
Mechanics:
1. Create a new method
2. Name after purpose of extracted fragment
3. Copy extracted fragment to new method
4. Replace old fragment with call to new method
5. Resolve data dependencies
6. Compile and test code
7/28/2014
© FZI Forschungszentrum Informatik
void drawWindow(Graphics g) {
// draw frame
…
// draw panels
…
}
void drawWindow(Graphics g) {
drawFrame(g);
drawPanels(g);
}
void drawFrame(Graphics g) {
…
}
Void drawPanels(Graphics g) {
…
}
46
Refactoring for Consolidation
Differences to common refactorings
• Introduce variability mechanism
• Reuse existing code of product copies
Non-code artifacts involved
• Configuration files
• Component descriptors,
• etc.
Tool Support
• Not available out-of-the-box
• Reuse and combine standard refactorings
• Infrastructures to build on
• Eclipse Language Toolkit
• EMF Refactor
7/28/2014
© FZI Forschungszentrum Informatik
47
Plan and Perform Refactorings
For each variability implementation technique
Identify
involved
artifact types
Specify
refactoring
steps
Identify
available
basic
refactorings
Add missing
steps
For each variation point (group)
Select elements
to refactor
7/28/2014
© FZI Forschungszentrum Informatik
Run refactoring engine
and perform manual
steps
48
SPL Management Tools
 Purpose

Continuously manage SPL features,
variability, and implementation
 Derive product instances
 Example Tools

Commercial
 BigLever: Gears
 Pure System: Pure Variants
 Open Source
 FeatureIDE
 FeatureMapper
7/28/2014
© FZI Forschungszentrum Informatik
49
Tool support to reduce the manual effort
CURRENT RESEARCH
7/28/2014
© FZI Forschungszentrum Informatik
50
Toolsupported consolidation of customized
produktCopies to Software Product Lines
www.kopl-projekt.de
Project Partners
7/28/2014
Supported by
Grant No.:
01IS13023 A-C
© FZI Forschungszentrum Informatik
51
What did we talk about
RECAP
7/28/2014
© FZI Forschungszentrum Informatik
52
Lessons Learned
Copy & Customize will always happen
Use a structured approach for consolidation
7/28/2014
© FZI Forschungszentrum Informatik
53
Thanks for your attention
TIME FOR QUESTIONS
Benjamin Klatt
FZI Forschungszentrum Informatik
Software Engineering
Haid-und-Neu-Str. 10-14 | 76131 Karlsruhe
[email protected]
+49 721 9654 690
7/28/2014
© FZI Forschungszentrum Informatik
54
APPENDIX
7/28/2014
© FZI Forschungszentrum Informatik
55
References
 [apel2013] – Apel et al.: “Feature-Oriented Software Product Lines”,
Book Springer 2013
 [bosch2002] – Jan Bosch,: “Maturity and Evolution in Software Product
Lines”, Chapter in the book “Software Product Lines” 2002
 [dubinsky2013] – Dubinsky et al.: “An exploratory study of cloning in
industrial software product lines”, CSMR 2013
 [kehrer2012] – Kehrer et al.: “Integrating the Specification and
Recognition of Changes in Models” WSR 2012
 [rubin2013a] - Rubin, J. & Chechik, M.: “A Framework for Managing
Cloned Product Variants”, ICSE 2013
 [rubin2013b] - Rubin, J. & Chechik, M.: “Quality of merge-refactorings
for product lines”, FASE 2013
 [svahnberg2005] – Svahnberg et al.: “A taxonomy of variability
realization techniques”, Software: Practice and Experience 2005
 [xing2006] – Xing et al.: “UMLDiff: an algorithm for object-oriented
design differencing” ASE 2006
7/28/2014
© FZI Forschungszentrum Informatik
56