OORPT
Object-Oriented Reengineering Patterns
and Techniques
9. Architecture Recovery
Prof. O. Nierstrasz
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
> Formal Concept Analysis
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.2
OORPT — Architecture Recovery
References
>
>
>
>
>
>
>
D. D'Souza, A. Wills, Objects, Components and Frameworks with
UML, Addison-Wesley, 1999
F. Buschmann, et al., Pattern-Oriented Software Architecture — A
System of Patterns, John Wiley, 1996
M. Shaw, D. Garlan, Software Architecture: Perspectives on an
Emerging Discipline, Prentice-Hall, 1996
C. Riva, View-based Software Architecture Reconstruction, Ph.D.
thesis, Technical University of Vienna, 2004.
H. Gall, Architecture Recovery and Reference Architecture, U
Zürich, presentation, 2004
M. Pinzger, et al., “Architecture Recovery for Product Families,”
PFE-5, LNCS 3014, Springer-Verlag, 2004.
R. Kazman and S. J. Carriere, “Playing detective: Reconstructing
software architecture from available evidence.,” Automated
Software Engineering, April 1999.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.3
OORPT — Architecture Recovery
Roadmap
>
What is Architecture?
— Viewpoints
— Styles
— ADLs
Architecture Extraction
> Formal Concept Analysis
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.4
OORPT — Architecture Recovery
What is Software Architecture?
The architecture of a system consists of:
> the structure(s) of its parts
— including design-time, test-time, and run-time hardware and
software parts
>
the externally visible properties of those parts
— modules with interfaces, hardware units, objects
>
the relationships and constraints between them
in other words:
— The set of design decisions about any system (or subsystem)
that keeps its implementors and maintainers from exercising
“needless creativity”
— D’Souza, Wills
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.5
OORPT — Architecture Recovery
Roadmap
>
What is Architecture?
— Viewpoints
— Styles
— ADLs
Architecture Extraction
> Formal Concept Analysis
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.6
OORPT — Architecture Recovery
IEEE 1471-2000
>
“A software architecture is the fundamental organization of a system embodied in its
components, their relationships to each other, and to the environment, and the
principles guiding its design and evolution.”
>
“A system stakeholder is an individual, team, or organization with interests in, or
concerns relative to, a system.”
>
“A concern is an interest which pertains to the system’s development, its operation or
any other aspects that are critical or otherwise important to one or more
stakeholders. Concerns include system considerations as performance, reliability,
security, distribution, and evolvability.”
>
“A view is a representation of a whole system from the perspective of a related set of
concerns.”
>
“A viewpoint is a specification of the conventions for constructing and using a views.
A pattern or a template from which to develop individual views by establishing the
purposes and audience for a view and the techniques for its creation and analysis.”
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.7
OORPT — Architecture Recovery
Architectural Viewpoints
Run-time
How are responsibilities distributed amongst run-time
entities?
Process
How many concurrent threads/processes exist; how do
they they communicate and synchronize?
Dataflow
How do data and tasks flow through the system?
Deployment
How are components physically distributed?
Module
How is the software partitioned into modules?
Build
What dependencies exist between modules?
File
How is the software physically distributed in the file
system?
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.9
OORPT — Architecture Recovery
Roadmap
>
What is Architecture?
— Viewpoints
— Styles
— ADLs
Architecture Extraction
> Formal Concept Analysis
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.10
OORPT — Architecture Recovery
Architectural Styles
An architectural style defines a family of systems in
terms of a pattern of structural organization. More
specifically, an architectural style defines a vocabulary of
components and connector types, and a set of
constraints on how they can be combined.
— Shaw and Garlan
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.11
OORPT — Architecture Recovery
Some Classical Styles
Layered
Elements in a given layer can only see the layer below.
Callbacks used to communicate upwards
Client-Server
Separate application logic from interaction logic. Clients
may be “fat” or “thin”
4-Tier
Server is further divided into generic part, business logic
and legacy adaptor.
Dataflow
Data or tasks strictly flow “downstream”.
Blackboard
Tools or applications coordinate through shared repository.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.12
OORPT — Architecture Recovery
Product Line Architectures
Reference family
architecture
Global style for all the products. Fixes the
architecture requirements, rules, patterns,
component types, infrastructure etc.
Family architecture
Families with similar UI styles, features, standards,
hardware generations etc.
Lead product
architecture
A reference product implementation, most typical of
the family.
Copy product
Products “copied” and adapted from the lead
product.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.13
OORPT — Architecture Recovery
Roadmap
>
What is Architecture?
— Viewpoints
— Styles
— ADLs
Architecture Extraction
> Formal Concept Analysis
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.14
OORPT — Architecture Recovery
Architectural Description Languages
>
ADLs
— Formal languages for representing and reasoning about
software architecture.
— Provide a conceptual framework and a concrete syntax for
characterizing architectures.
— Some are executable, or implemented in a general-purpose
programming language.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.15
OORPT — Architecture Recovery
Common ADL Concepts
System
connector
component
component
role
port
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.16
OORPT — Architecture Recovery
Common ADL Concepts
>
Component: unit of computation or data store. Typically contains
interface (ports) and formal behavioral description.
>
Connector: architectural building block used to model interactions
among components. Typically contains interface (roles) and formal
behavioral description.
>
Configuration: connected graphs of components and connectors
that describe architectural structure.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.17
OORPT — Architecture Recovery
Some ADLs
>
Darwin: focuses on supporting distributed applications. Components are
single-threaded active objects.
>
Wright: underlying model is CSP, focuses on connectivity of concurrent
components.
>
C2: component- and message-based architectural style with concurrent
components linked together by connectors in accordance with a set of style
rules.
>
Rapide: focuses on developing a new technology for building large-scale,
distributed multi-language systems.
http://en.wikipedia.org/wiki/Architecture_Description_Language
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.18
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
>
—
—
—
—
>
Extract-abstract-present
Rigi, Dali, PBS, …
Reflexion models
Querying and Soul
Formal Concept Analysis
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.19
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
>
—
—
—
—
>
Extract-abstract-present
Rigi, Dali, PBS, …
Reflexion models
Querying and Soul
Formal Concept Analysis
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.20
OORPT — Architecture Recovery
Extract-abstract-present
Tilley et al., 1996; Krikhaar 1999
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.21
Architecture Reconstruction
Domain
Knowledge
Sources of information:
- Documentation
- Experts
- Source Code
Architectural
Model
1. Model Refinement
2. Composition rules
3. View Selection
Hierarchical graphs
UML logical diagrams
Message sequence charts
HTML reports
4. Visualisation
3. Abstraction
Develop
Low level
model
Architecturally
Signifcant
Concepts
Extract
Source Code
1. Definition of Architectural Concepts
2. Data gathering
Analyze & record
rationales
Rationales for
design decisions
5. Re-document
© Harald Gall, UniZH
OORPT — Architecture Recovery
Extracting Architecture
>
Various techniques
—
—
—
—
—
Human experts
Recognizing known patterns
Static and dynamic analysis
Clustering and data mining
Comparing products within a family
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.23
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
>
—
—
—
—
>
Extract-abstract-present
Rigi, Dali, PBS, …
Reflexion models
Querying and Soul
Formal Concept Analysis
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.24
OORPT — Architecture Recovery
Rigi
Programmable reverse engineering
environment
— C parser; relational data import
— Visualization of hierarchical typed
graphs
— Graph manipulation, filtering, layout
— Tcl-programmable
— www.rigi.csc.uvic.ca/
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
10.25
OORPT — Architecture Recovery
Dali Workbench
>
>
Workbench built on Rigi, PostgreSQL, perl scripts …
Three techniques used:
— Architectural extraction from source artifacts
— User-defined patterns
— Visualization
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.26
OORPT — Architecture Recovery
Software Bookshelf
>
A web-based paradigm for presentation and navigation
of large software systems
— PBS (Portable Bookshelf) is one implementation
>
Extract-abstract-present
— C fact extractor generates Rigi Standard Format (RSF) files
— Reconstructors define tree-structure by assigning files to
subsystems
— GROK relational calculator determines high-level relations and
visualizes as graphs
>
Applications to Linux, Apache and Mozilla
— swag.uwaterloo.ca/pbs/
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.27
OORPT — Architecture Recovery
SWAGkit
>
Toolkit to extract, abstract and present Software
Architectures
— Pipeline architecture based on PBS and GROK
— Mainly used to reconstruct module views
— swag.uwaterloo.ca/swagkit/
facts
Source Code
Extraction
facts
Manipulation
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
facts
Analysis
pictures, text, etc
10.28
OORPT — Architecture Recovery
SWAGkit tools
Three categories of tools:
> Fact extractors
— cppx - C++ fact extractor based on GNU g++ compiler.
–
>
One output file (TA) generated for each source file.
Fact manipulators
— Several grok scripts and C++ programs used to manipulate the
facts into a form that can be meaningfully analyzed
–
>
linking, abstracting, layout, containment
Fact analyzers
— lsedit - visualize system as navigable “box and arrow” diagrams.
— Sgrep - query facts via a command line interface
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.29
OORPT — Architecture Recovery
GUPRO
>
Integrated workbench of program understanding tools
— Based on extract-abstract-present
— Graph-based source model
–
Graph querying, manipulation and browsing
— www.uni-koblenz.de/~ist/gupro.en.html
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.30
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
>
—
—
—
—
>
Extract-abstract-present
Rigi, Dali, PBS, …
Reflexion models
Querying and Soul
Formal Concept Analysis
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.31
OORPT — Architecture Recovery
Reflexion models
> Semi-automated approach
> Repeat
— Define/Update high-level model of interest
— Extract a source model
— Define/Update declarative mapping between high-level model
and source model
— System computes a software reflexion model
— Interpret the software reflexion model.
Until “happy”
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.32
OORPT — Architecture Recovery
Reflexion Approach
Reflexion Model
Mapping
High-Level Model
Reflexion
Model
Tools
Extraction
Specification
Source
Code
Extraction
Tool
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
Source
Model
10.33
OORPT — Architecture Recovery
High-Level Model
>
Describes boxes and arrows model of system.
Memory
HardwareTrans
VirtAddressMaint
User
VMPolicy
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
KernelFaultHandler
Pager
FileSystem
10.34
OORPT — Architecture Recovery
Mapping
>
Relates source model entities to high-level model
entities.
>
Example:
[
[
[
[
[
[
[
file= .*pager.*
file= vm_map.*
file=vm_fault\.c
dir=[un]fs
dir=sparc/mem.*]
file=pmap.*
file=vm_pageout\.c
mapTo=Pager ]
mapTo=VirtAddressMaint ]
mapTo=KernelFaultHandler ]
mapTo=FileSystem ]
mapTo=Memory ]
mapTo=HardwareTrans ]
mapTo=VMPolicy ]
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.35
OORPT — Architecture Recovery
Source Model
Particular information extracted from source code
> Calculated with Lightweight Source extraction:
>
— Specifications easy to write
— Flexible: few constraints on source
— Tolerant: source code can be incomplete, not compilable, …
Lexical Approach
> Intrinsically Approximate
> For every Source Model, a new scanner is generated
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.36
OORPT — Architecture Recovery
Source Model Specification Example
>
Writing out C calls
[ <type> ] <functionName> \( [ { <formalArg> }+ ] \)
[ { <type> <argDecl> ; }+ ] \{
<calledFunction>
@ write ( functionName, “ calls “, calledFunction ) @
\( [ { <parm> }+ ] \) ( \) | ; )
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.37
OORPT — Architecture Recovery
Software Reflexion Model
>
Indicates where the source model and high-level model
differ:
— Convergences
— Divergences
— Absences
>
Has to be interpreted by developer.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.38
OORPT — Architecture Recovery
Software Reflexion Model Example
convergence
Memory
divergence
absence
HardwareTrans
VirtAddressMaint
User
VMPolicy
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
KernelFaultHandler
Pager
FileSystem
10.39
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
>
—
—
—
—
>
Extract-abstract-present
Rigi, Dali, PBS, …
Reflexion models
Querying and Soul
Formal Concept Analysis
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.40
OORPT — Architecture Recovery
Soul
>
Express source-code views and relations as logic
programming queries
>
A source-code view
— Is a set of source-code entities that address a same concern
— One view can contain many entities
— Views may crosscut dominant implementation decomposition
>
A source-code entity
— Can be any tangible language construct: method, class,
variable, …
— One source-code entity can reside in multiple source-code
views
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.41
OORPT — Architecture Recovery
Defining Source-code views
>
Structure
—
—
—
—
>
Name
Comments
Definition(s)
Default definition
Definition can be defined
— Extensionally = by explicit enumeration of their elements
— Intentionally = by declaratively describing their elements
>
All definitions
— Should yield the same extension
— This can be checked automatically
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.42
OORPT — Architecture Recovery
Source-code view example
view(soulPredicates,<byCategory,byHierarchy>).
viewComment(soulPredicates,
['This intentional view contains ALL classes that implement SOUL
predicates (i.e., Prolog-like predicates that may use Smalltalk code
due to language symbiosis).']).
default(soulPredicates,byCategory).
intention(soulPredicates,byCategory,?class) if
category(?category),
name(?category,?name),
startsWith(?name,['Soul-Logic']),
classInCategory(?class,?category).
include(soulPredicates,byCategory,[Soul.TestClassifications]).
intention(soulPredicates,byHierarchy,?class) if
…
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.43
OORPT — Architecture Recovery
Software Tomography
>
Repository-based toolset
— Detects and checks architectural layers
— Uses metrics and graph-based queries to detect “bad smells”
and architectural violations
— www.software-tomography.com
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.44
Software Tomography
© Software Tomography GmbH, 2003, 2004
OORPT — Architecture Recovery
Feature-oriented analysis
Select a feature and a use case
> Step through scenarios and collect traces
> Combine the traces with the high-level static view to
generate the feature view
> Abstract the feature view by detecting interaction
patterns
>
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.46
The Dynamic Analysis Process
Extract/Filter
Abstract
Present
Static Model
Execute a
scenario
Instrumented
system
Traces
Original MSC
A
B
Visualisation
Vertical abstraction
Horizontal abstraction
C
AB
C
A
C
B
abstractions
AB
C
Horizontal and Veritcal
M1
M2
M2
M1-M4
M3
M4
M3
M7
M5
M6
M7
M8
M9
M6
M7
M10
M11
M8
M10
M11
M1-M4
M5
M7
M9-M12
M9-M12
M12
© Harald Gall, UniZH
An Example of Feature View
MSC_COMMANDS
CANVAS
FILE
DATA_STRUCTURES
DIALOGS
GUI
EXECUTION
msc_env _get
DIALOGS
exec_initializeMsc
FILE
TREE_CREATE
HASH_CREATE
GUI
guisel_initBindings
exec_Command
CANVAS
f ile_loadUseCase
EXECUTION
ADD_PARTICIPANTS
ADD_MESSAGES
exec_initialDraw
DATA_STRUCTURES
gui_initialDraw
COMMANDS
exec_getVisibleParticipants
tree_children
DRAW_PARTICIPANTS
exec_getVisibleMessages
tree_children
exec_f ilterMessagesToDisplay
FILTERING_MESSAGES
DRAW_MESSAGES
gui_scaleCanv asToFit
exec_getVisibleMessages
tree_children
exec_f ilterMessagesToDisplay
FILTERING_MESSAGES
exec_getVisibleParticipants
tree_children
msc_env _set
© Harald Gall, UniZH
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
> Formal Concept Analysis
>
— Detecting implicit patterns
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.49
OORPT — Architecture Recovery
Identifying Design Patterns
Template Method
Look for protected and abstract methods being
overridden in subclasses (hook methods). The
caller is often a template method.
Proxy, Decorator
Look for interfaces implemented by classes in
different hierarchies. Look for delegation to
instance variables of the same type.
Composite
Look for instance variables whose type is the
superclass.
Observer
Look for methods “update” and “notify”.
Visitor
Look for an “accept*” method.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.50
OORPT — Architecture Recovery
ConAn: Detecting hidden contracts
>
Concept analysis
— Identify groups of entities (software artifacts) with common
properties (contracts)
>
X-Ray Views
— Understanding the internals of a class
>
Inheritance Patterns
— Analysing behavioral dependencies in a class hierarchy
>
Collaboration Patterns
— Detecting structural relationships in an object oriented systems
Gabriela Arévalo, “High Level Views in Object-Oriented Systems using
Formal Concept Analysis,” Ph.D. thesis, University of Berne, 2005.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.51
OORPT — Architecture Recovery
Concept Analysis: Definitions
Rock
Jazz
Houari
True
True
Petko
True
True
Yann-Gael
Pierre
Pop
Rap
{Houari, Petko,
Pierre}, {Jazz}
Top
True
True
Blues
True
True
True
Mohamed
True
True
Gaby
True
True
{Houari, Petko},
{Rock, Jazz}
 Specification of Elements and Properties
 Concept: Maximal groups of elements based on
their common properties
Bottom
{Pierre}, {Jazz,
Pop, Rap}
 Lattice: partial order over the concepts
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.52
OORPT — Architecture Recovery
ConAn: Tool Architecture
X-Ray Views
Software Patterns
Inheritance Patterns
Abstraction of
the lattice
Identify groups
Concept
Analysis
CA Pre-Filters
CA Mapping
Keep one access to an attribute in a method
Element: <add:>
Property: access: #firstIndex
Code
Representation
Class: Ordered
Collection
attribute
defines
firstIndex
add:
Source Code
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.53
OORPT — Architecture Recovery
Roadmap
What is Architecture?
> Architecture Extraction
> Formal Concept Analysis
>
— Detecting implicit patterns
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.54
OORPT — Architecture Recovery
Collaboration Patterns
Abstract X
P
Y
Abstract A
Z
T
B
C
D
Collaboration Patterns: Common
structural aspects between classes
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.55
OORPT — Architecture Recovery
Using CA to detect Patterns
Elements: tuples of classes
Properties: relationships over the tuples. (i,j)relation and (i)feature
Tuples of classes
2
(1,2)Sub
(3,2)Sub
(C,A,P)
True
True
(C,A,B)
True
True
(Z,X,Y)
(Z,X,P)
(1,2)Acc
(3,2)Acc
(2)Abs
True
True
True
True
True
True
True
True
True
True
A
C
P
1
3
True
A B C P X Y Z: all classes in a system
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.56
OORPT — Architecture Recovery
Found Collaboration Patterns
Subclass Star
A
Attribute Star
A
A
Order = 3
Order = 4
A
Order = 3
Bridge
Composite
A
Order = 3
A
Order = 4
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
Order = 4
A
Order = 3
A
Order = 4
10.57
OORPT — Architecture Recovery
License
>
http://creativecommons.org/licenses/by-sa/2.5/
Attribution-ShareAlike 2.5
You are free:
• to copy, distribute, display, and perform the work
• to make derivative works
• to make commercial use of the work
Under the following conditions:
Attribution. You must attribute the work in the manner specified by the author or licensor.
Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting
work only under a license identical to this one.
• For any reuse or distribution, you must make clear to others the license terms of this work.
• Any of these conditions can be waived if you get permission from the copyright holder.
Your fair use and other rights are in no way affected by the above.
© Stéphane Ducasse, Serge Demeyer, Oscar Nierstrasz
10.58