A model-driven framework for domain specific languages
demonstrated on a test automation language
Thesis by
Martin Karlsch, 711330
In Partial Fulfillment of the Requirements
for the Degree of
Master of Science in Software Systems Engineering
Supervisors
Prof. Dr. Andreas Polze (HPI)
Dr. Martin von Löwis (HPI)
Dr. Klaus Ries (BMW Group)
Dipl.-Inf. Markus Hillebrand (BMW Group)
Hasso-Plattner-Institute of Software Systems Engineering
Potsdam, Germany
January 12, 2007
(Submitted March 7, 2007)
ii
c January 12, 2007
Martin Karlsch, 711330
All Rights Reserved
iii
Abstract
Innovation in today’s automotive industry is largely based on electronics and software. To
get a competitive edge a company has to deliver new features steadily. This trend is boosted
by a decreasing technology life cycle. The result is an exponential growth of software inside a car. As quality plays a critical role software and hardware have to be tested properly,
manual testing is not practicable anymore. At the BMW Group a framework for test automation has been developed including a Microsoft Excel based test specification language.
The language and its implementation approach offer various places for improvement, however due to the chosen technology the emendation possibilities are limited. Therefore the
development of a new domain specific language (DSL) was decided.
However, the development of a new DSL is a difficult task at the moment. The developer needs expertise in the field of language development which involves design, lexical
analysis, parser and compiler construction, implementation of a suited runtime and most
importantly knowledge of the concepts within the specific domain. A various number of
tools supporting domain specific language development is available, yet most lack in categories like reusability, maintainability, interoperability or ease of use.
For that reason this thesis proposes a new framework for the development of domain
specific languages which is based on the vision of Model-Driven Engineering (MDE). In
MDE models are the central artifact of software engineering and enable reuse. A domain
and its language (syntax and semantic) are described by models. The framework allows
the reuse of existing textual grammars for compiler generator tools, translating them into
models. Thereby it becomes possible to apply a large number of MDE transformation and
generation tools to them. In turn a language parser and generator can be produced from
these grammar models. An additional goal is to automatically generate a suitable language
editor and debugger to improve developer experience.
Using the proposed framework, the test automation language is reimplemented. This
is realized by reusing an existing language and extending it. Also different problems like
error reporting or tool support for the DSL are solved.
iv
Zusammenfassung
Die Grundlage für Innovation in der heutigen Automobilindustrie findet sich zu 90% im
Bereich der Elektronik- und Softwareentwicklung. Der Lebenszyklus neuer Produkte wird
immer kürzer, und um konkurrenzfähig zu bleiben, müssen Hersteller immer schneller neue
Funktionen integrieren. Das hat zur Folge, dass die Menge an Software im Auto exponentiell wächst. Die verwendete Software wie auch die Hardware müssen sorgfältig getestet
werden, um eine hohe Qualität garantieren zu können. Aufgrund der Menge und Komplexität ist dies nicht mehr manuell möglich und muss automatisiert werden. Bei der BMW
Group ist hierfür ein spezielles Testautomatisierungsframework einschließlich einer Microsoft Excel basierten Testspezifikationssprache entwickelt worden. Die Sprache und ihre
Implementierung lassen Raum für Verbesserungen. Dies erweißt sich allerdings aufgrund
der gewählten Technologie als schwierig. Deshalb entschied man sich für die Entwicklung
einer neuen domänenspezifischen Sprache (DSL) für die Testautomatisierung.
Die Entwicklung einer neuen Sprache ist jedoch nicht trivial. Die Entwickler brauchen
ein breitgefächertes Repertoire an Wissen vor allem in den Bereichen Sprachdesign und
-implementierung. Nicht zu unterschätzen ist die genaue Kenntnis der Domäne, für die
die neue Sprache bestimmt ist, um Syntax und Semantik möglichst domänenspezifisch
festlegen zu können. Dieser Prozess wird durch eine Vielzahl an verfügbaren Werkzeugen unterstützt. Häufig sind diese allerdings in einem oder sogar mehreren Aspekten wie
Wiederverwendbarkeit, Wartbarkeit oder Interoperabilität unzureichend.
In dieser Arbeit wird ein neues Framework für die Entwicklung von domänenspezifischen Sprachen vorgeschlagen und implementiert. Es baut auf den Prinzipien des ModelDriven Engineering (MDE) auf, welches Modelle in das Zentrum der Softwareentwicklung rückt und mit ihrer Hilfe eine einfachere Wiederverwendbarkeit ermöglicht. Eine
Domäne und ihre Sprache (Syntax und Semantik) werden mit Modellen beschrieben. Das
Framework ermöglicht die Wiederverwendung von exestierenden textuellen Grammatiken,
indem es sie in Grammatikmodelle überführt. Diese können mit Werkzeugen, welche ebenfalls den MDE Ansatz verfolgen, modifiziert, analisiert, überprüft und umgewandelt werden. Außerdem ist es möglich, aus diesen Grammatikmodellen wiederum einen Parser zu
erzeugen, sowie zusätzlich einen Editor und einen Debugger für die domänenspezifische
Sprache, was der Verwendbarkeit durch Entwickler zu Gute kommt. Zusätzlich ist die
Entwicklung einer komplett neuen Sprache möglich.
Die bestehende Testautomatisierungsprache wird mit Hilfe des neuen Frameworks reimplementiert – unter Wiederverwendung bestehender Artefakt und einer Erweiterung dieser.
Hiermit werden Problem im Bereich Fehlerbehandlung und Werkzeugunterstützung gelöst.
v
Contents
Abstract
iii
Zusammenfassung
iv
Acknowledgments
x
I
Preliminaries
2
1
Introduction
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Description of the remaining chapters . . . . . . . . . . . . . . . . . . . .
3
3
5
2
Technological context and related work
2.1 Domain specific languages . . . . . . . . . . . . . .
2.1.1 What is a domain specific language? . . . . .
2.1.2 Advantages . . . . . . . . . . . . . . . . . .
2.1.3 Disadvantages . . . . . . . . . . . . . . . .
2.1.4 Development phases and patterns . . . . . .
2.1.5 Decision phase . . . . . . . . . . . . . . . .
2.1.6 Analysis phase . . . . . . . . . . . . . . . .
2.1.7 Design phase . . . . . . . . . . . . . . . . .
2.1.8 Implementation phase . . . . . . . . . . . .
2.1.9 Approaches supporting DSL development . .
2.2 Model-Driven Engineering . . . . . . . . . . . . . .
2.2.1 Models - the foundation of MDE . . . . . .
2.2.2 Domain specific modeling languages . . . .
2.2.3 Generators and transformations . . . . . . .
2.2.4 Different MDE technologies in detail . . . .
2.3 Model-driven DSL development . . . . . . . . . . .
2.3.1 Model-driven DSLs by example . . . . . . .
2.3.2 Model-based domain specific language tools
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . .
3
Domains in the automotive industry
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6
6
6
8
8
9
10
11
13
15
18
22
22
26
28
31
36
36
37
42
43
vi
II
4
5
III
6
7
Frodo
45
One framework to bind them all?
4.1 Requirements and solutions . . . . . . . . . . . .
4.2 User roles . . . . . . . . . . . . . . . . . . . . .
4.3 Framework Architecture . . . . . . . . . . . . .
4.3.1 Components of the framework . . . . . .
4.3.2 Language specification artifacts . . . . .
4.3.3 Back-end . . . . . . . . . . . . . . . . .
4.4 Implementation . . . . . . . . . . . . . . . . . .
4.4.1 Overview . . . . . . . . . . . . . . . . .
4.4.2 ECore - Metametamodel Implementation
4.4.3 Grammar / Syntax View Handling . . . .
4.4.4 Transformation Engine . . . . . . . . . .
4.4.5 Plugin Generation . . . . . . . . . . . .
4.4.6 Implementation problems and issues . . .
Examples
5.1 A simple state machine language . . . . .
5.1.1 Design . . . . . . . . . . . . . .
5.1.2 Implementation . . . . . . . . . .
5.2 Test automation language . . . . . . . .
5.2.1 Introduction . . . . . . . . . . . .
5.2.2 The current system . . . . . . . .
5.2.3 A better test automation language
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
46
46
47
48
49
53
55
56
56
57
60
63
63
64
.
.
.
.
.
.
.
66
66
66
68
71
71
72
76
Conclusions
Future work
6.1 DSL Framework . . . . . . . . .
6.1.1 Theoretical outlook . . .
6.1.2 Practical outlook . . . . .
6.2 TestDSL . . . . . . . . . . . . . .
86
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
87
87
87
88
88
Summary
90
7.1 Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
A Fundamental Modeling Concepts
93
B Statement of Academic Honesty
96
vii
List of Figures
2.1
2.2
2.3
2.4
2.5
2.6
2.7
Simplified cost prediction for DSL-based methodologies [59]
Domain analysis, taken from [147] . . . . . . . . . . . . . . .
Different technological space and the metamodeling stacks . .
Example of MOF layers . . . . . . . . . . . . . . . . . . . .
MDA PIM to PSM (FMC) . . . . . . . . . . . . . . . . . . .
ECore metametamodel . . . . . . . . . . . . . . . . . . . . .
XMI used for exchange. . . . . . . . . . . . . . . . . . . . .
4.1
4.2
4.3
Proposed architecture for an model-driven DSL framework (FMC) . . . . . . 49
Frodo’s architecture (FMC) . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Excerpt from Frodo grammar metamodel . . . . . . . . . . . . . . . . . . . 61
5.1
5.2
5.3
5.4
5.5
Finite state machine domain model
Example of a finite state machine .
Screenshot of the FSM editor . . . .
Current PyTAF architecture (FMC)
Excel TestDSL example . . . . . .
A.1
A.2
FMC block diagram example . . . . . . . . . . . . . . . . . . . . . . . . . . 94
FMC block diagram reference sheet . . . . . . . . . . . . . . . . . . . . . . 95
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
9
11
25
32
33
34
35
67
67
70
74
76
viii
List of Tables
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
Well known DSLs [92] . . . . . . . . . . . . . . . . . . . .
Lines of Code for different approaches to implement FDL .
Different MDE implementations, different metametamodels
Different tools for creating DSML . . . . . . . . . . . . . .
Different model-to-text approaches . . . . . . . . . . . . .
Different generator tools . . . . . . . . . . . . . . . . . . .
Transformation engines from different technology spaces . .
Some model-to-model transformers . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
8
18
24
27
29
29
30
31
ix
Listings
2.1
2.2
2.3
4.1
4.2
4.3
4.4
4.5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
5.10
5.11
SPL metamodel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SPL TCS template specification . . . . . . . . . . . . . . . . . . . . . .
A TCSSL grammar example . . . . . . . . . . . . . . . . . . . . . . . .
Creating an ECore metamodel for a carpool in Python . . . . . . . . . . .
Carpool metamodel as XMI . . . . . . . . . . . . . . . . . . . . . . . . .
Auto importing the carpool metamodel and creating a model . . . . . . .
XMI representation of a carpool model . . . . . . . . . . . . . . . . . . .
Excerpt from the generative grammar for ANTLR grammars . . . . . . .
Example of FSM DSL (demo.fsm). . . . . . . . . . . . . . . . . . . . . .
Grammar of the FSM DSL as EBNF . . . . . . . . . . . . . . . . . . . .
The FSM parser grammar . . . . . . . . . . . . . . . . . . . . . . . . . .
The FSM lexer grammar . . . . . . . . . . . . . . . . . . . . . . . . . .
A simple FSM interpreter in Python, interpreting demo.fsm . . . . . . . .
The XMI representation of demo.fsm . . . . . . . . . . . . . . . . . . .
A generative grammar, producing a C back-end. . . . . . . . . . . . . . .
Example of generated code for TestDSL . . . . . . . . . . . . . . . . . .
Example of the embedded Python test language . . . . . . . . . . . . . .
Example of the new wait statment . . . . . . . . . . . . . . . . . . . . .
Exerpt from the Python grammar metamodel to Frodo metamodel transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.12 Example of unmatchable rule with finite lookahead and a solution. . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
38
38
41
59
59
59
60
62
67
68
68
69
70
70
71
80
81
81
. 82
. 84
x
Acknowledgments
I would like to thank Klaus Ries and Markus Hillebrand at the BMW Group, for many
interessting discussion even for the philosophical ones. Inspiring and wild ideas were discussed. But after all their guidance helped me to keep the right direction. Also Martin von
Löwis, my tutor at the HPI helped with additional ideas and many corrections. Furthermore
I would like to thank Hannah Ulferts for motivation, correcting this thesis and eating my
chocolate. Additional corrections were done by Thomas Hille, Friederike Karlsch and Jens
Ulferts, many thanks go to them, too, especially to Thomas because he shared my thesis
writing fate.
Last but not least I could not have done all of this without the never ending support of
my family. Thank you.
1
The limits of my language determine the limits of my world.
Ludwig Wittgenstein
2
Part I
Preliminaries
3
Chapter 1
Introduction
1.1
Motivation
Today the German automotive industry holds a high market share and has a good reputation
all over the world [57]. However, competition with low cost manufacturers forces companies like BMW Group, Audi or DaimlerCrysler to concentrate on innovative features [57],
for example sophisticated control over processes like engine, brake or acceleration control,
furthermore infotainment functionality like GPS navigation, head-up displays or advanced
audio/video functionality. Almost every new feature contains a combination of electronics
and software [35]. To reduce costs, devices are built out of standard hardware components
while the software is well customized, not least to avoid imitation.
This has led to an exponential growth of software inside a car [35]. However, the
software engineering capabilities within the automotive industry, suppliers and OEMs alike
have not grown as fast as the complexity. The cause for the increased complexity is not only
the amount of software but more and more distributed, interconnected and interdependent
functionality leading to a higher error rate.
A number of measures have been taken to manage it. One is to automate testing, especially integration testing. Many different test automation solutions exist nowadays but
not all fulfill the requirements of automotive integration testing. Therefore PyTAF[62], a
practical Python1 test automation framework, has been developed at the BMW Group. It is
programmed in a domain specific test automation language, not in Python directly. From
this language Python code is generated.
Domain specific languages (DSLs) are computer languages tailored to a special field of
application. Compared to general purpose languages they offer an advantage in expressiveness and ease of use in their domain. This is possible due to higher abstraction, a clear and
concise program specification and improved program checking possibilities.
For the current test automation language, test cases have to be written inside Microsoft
Excel. This is not satisfying because of the implied and inappropriate spreadsheet structure and limited integration into the development environment. Furthermore the language
itself suffers from design problems such as an incomplete data model. As a case in point
this can lead to ineffective test development or even worse defective tests and increased
1
The programming language. http://www.python.org
4
maintenance costs. Later in this thesis the current issues will be elaborated in more detail.
Most of the problems can be solved by a new textual domain specific language. This
language needs to be equally powerful but has to improve upon the current shortcoming.
Nevertheless, the development of a new language is a difficult task to undertake, even if the
language is domain specific. The developer needs to be qualified for language design and
implementation and to possess a good understanding of the target domain. For tasks such
as compiler development several tools are available. Likewise tools explicitly supporting
the development of domain specific languages have been created. To utilize most of them
great expertise is still required. Additionally reuse is difficult in most cases.
A vision which focuses on reuse is Model-Driven Engineering (MDE). Models move
into the center of the development process. They allow a more domain specific description
of problems than modern third generation programming languages are able to offer, exploiting advanced modeling techniques, code generation and model transformations. This
approach is very promising as first results show [17].
Recently first attempts have been made to combine technologies for DSL development
with the MDE paradigm, motivated by improved reuse properties and the need to specify
domain specific languages more easily [64, 94, 96, 146, 151].
In this thesis, beyond the improved test automation language, a model-driven framework is presented allowing to implement the former in a more easy and reusable way.
Inside [106] Klint demands an “engineering discipline for grammarware” stating that the
engineering aspect of grammar-depended software is insufficiently understood. Most DSL
development tools are grammar based. Still questions like grammar transformation, maintenance, verification and reuse are hard to answer. While the available DSL tools answer
some of the question in a versatile way, they remain unconnected island solutions. For
example grammar refactoring is available in nearly none of the current tools. Providing
answers will reduce needed development efforts and decrease the number of errors.
For that reason the proposed framework tries to combine DSL development and MDE
backed by the idea of explicitly enabling reuse. A metamodel for grammars is specified
which is able to represent a large number of grammars as grammar models. These models
can then be modified utilizing a large number of available MDE-enabled tools. For example it is possible to refactor through transformation of the grammar model or to apply
aspects. Nevertheless, refactoring or aspect introduction is not the focus of this thesis. To
enable MDE in the first place, this thesis also discusses the implementation of a popular
metametamodel on a new platform.
Furthermore, not only formal aspects are important in DSL development but also the
surrounding toolset and its integration. Developers are used to capable editors with syntax
highlighting, folding and code completing and moreover excellent support of debugging
in modern IDEs. These feature can greatly improve productivity though are seldom available for domain specific languages as a high development effort is necessary to implement
editors and debuggers. The proposed framework shows a possible way to automatically
deliver such an environment.
While implementing the new test automation language, for example the formal Python
grammar is reused and extended. This allowed to save a large amount of development time,
base the new language on an already existing formal specification and create a reusable
5
model of Python and the DSL.
Additionally problems like error reporting, exception mapping from back-end to the
DSL and some debugging capabilities are enabled by using the presented framework. These
features were not included in the original test automation language.
1.2
Description of the remaining chapters
This thesis is divided into three parts.
Part one gives an extensive overview of concepts and current technologies. It consist
of three chapters. First, domain specific languages are described. What is a DSL and what
advantages and disadvantages are associated with it? Then development phases and design
patterns for each phase are discussed including references to different technologies. Finally
a review of some more concrete tools and frameworks allowing the implementation of DSL
is given.
In the chapter Model-Driven Engineering the former term is introduced as a new software engineering paradigm. Definition for model, metamodel, metametamodel and technology space are provided followed by a formal specification attempt. Moreover the concepts of domain specific modeling languages, generator and transformations are elucidated,
in each case underpinned by a description of current technology. The last section explains
concrete technologies in more detail such as the Meta Object Facility, Model-Driven Architecture, Eclipse Modeling Framework, and XML Metadata Interchange.
That last chapter of part one outlines research which is more closely related to this
thesis, namely examples of DSLs implemented using model-driven techniques and MDEenabled DSL development frameworks.
Part two contains the main contributions of this thesis, split up into a framework proposal, the realization of that proposal and two examples. While the proposal is generic and
does not discuss concrete technologies, the implementation shows what technologies are
able and which of them were used to realize the framework. Furthermore different problems encountered are depicted. At the end two examples are implemented with the help
of the framework: a simple state machine language and a more complex test automation
language. The later one is elaborated in more detail because it is major contribution of
this thesis. At first the currently used system is described, followed by an enumeration of
its problems in design and implementation. Then, a new language and its development is
shown.
Part three, the last part of this work consists of two chapters. The first chapter shows
possibilities for future work in general for the proposed and implemented framework and
in special for the test automation language. The second chapter draws a summary of this
thesis and names important contributions. Finally the appendix contains an overview about
the FMC2 notation often used in this work.
2
Fundamental Modeling Concepts.
6
Chapter 2
Technological context and related work
Initially this chapter introduces domain specific languages, explains their advantages and
disadvantages, gives a summary about common patterns and categorizes the domain specific language development process. Furthermore different approaches and tools, supporting language development, are discussed. Secondly the approach of Model-Driven Engineering is presented, including an explanation of terms such as metamodel or technical
space. Afterwards different Model-Driven Engineering incarnations are depicted together
with the meaning of generators and transformations in this context and what kinds of approaches are available. A closer look on MOF/MDA, ECore/EMF and XMI follows. Finally an insight into various frameworks which directly support model-driven language
development is given.
2.1
Domain specific languages
“Works of imagination should be written in very plain language; the more
purely imaginative they are the more necessary it is to be plain.” 1
This section describes what a domain specific languages is, what kind of advantages and disadvantages a DSL has and also what common DSL analysis,
design and implementation patterns exist.
2.1.1
What is a domain specific language?
To understand the meaning Of the term domain specific language or more precisely domain
specific programming language the term programming language is defined. One possibility
is given by [72]:
“A programming language or computer language is a standardized communication technique for expressing instructions to a computer. It is a set
of syntactic and semantic rules used to define computer programs. A language
enables a programmer to precisely specify what data a computer will act upon,
1
Samuel Taylor Coleridge
7
how these data will be stored/transmitted, and precisely what actions to take
under various circumstances.”
However there exists no definition which all authors agree upon. Watts therefore proposes
[181] some criteria which have to be fulfilled by a programming language:
• Must be universal (every problem must have a solution that can be programmed in
the language, if that problem can be solved at all by computer).
• Must be implementable on a computer.
• Should also be reasonably natural for solving problems, at least problems within its
intended application area.
Programming languages in general can be grouped or classified by different criteria. Possible criteria are the purpose (for example Fortran for scientific programming versus C [102]
for system programming), the paradigm (LISP as a functional language or Smalltalk as a
object oriented language), the generation (1GL up to 5GL) , whether it is imperative or
declarative and domain specific or general purpose. General purpose languages (GPLs) are
less specialized and are suited for a wide area of applications from business processing up
to scientific computing. Java2 is a prominent representative.
The term domain specific means that the language is explicitly tailored to a target domain. Complex constructs and abstraction of the domain are offered within the language
increasing its expressiveness in comparison to GPLs. It is possible to express solutions
for domain problems with a lesser effort. The higher abstraction and the compactness and
therefore better readability and writability enables a larger group of people with less programming knowledge to be productive using the DSL. This leads to productivity gains in
general and also to decreased maintenance costs.
Often a DSL does not fulfill all criteria given by Watts. Nevertheless, many DSLs are
regarded as special programming languages.
Today there are many well known DSLs like HTML, SQL, VHDL, make (software
build process), Latex (document preparation), BNF (context free grammars) or even Excel.
The use of DSLs is not new. These languages had been named special-purpose languages,
end-user languages or as Bentley [23] called them “little languages” before the term domain specific language was coined. Already in 1957 APT [34], a language for numeric
controlled machines was developed at the MIT, which can be considered as one of the first
available DSLs. The boarder between a DSL and a GPL is fuzzy, for example COBOL
was considered a GPL but also a DSL for business applications. Another example is Prolog which can be understood as a DSL for applications specified by the predicate calculus.
One attempt to classify a language has been done by Jones [92]. A higher level stands for
more domain specific whereas a lower level means more generality (table 2.1). As stated
by Mernik [126] the domain-specificity of a language is a matter of degree. In this thesis a
definition by the former will serve as a guidance:
2
http://java.sun.com/
8
“DSLs are languages tailored to a specific application domain. They offer
substantial gains in expressiveness and ease of use compared with GPLs in
their domain of application”
However many similar definition can be found in the literature [99].
DSL
Java
VHDL
HTML
SQL
Excel
domain
GPL
hardware design
web pages markup
database queries
spreadsheets
level
5
17
22
25
57
Table 2.1: Well known DSLs [92]
2.1.2
Advantages
As stated in 2.1.1 a DSL offers different advantages. Productivity and maintainability [172]
are increased due to an appropriated domain specific notation. DSLs are more suitable
for end-user programming. Domain experts are able to understand , validate, modify and
develop within the language (better readability, writability and high abstraction). The gains
can be measured quantitatively and qualitatively. Most qualitative reasoning is backed up
by practical observations. According to [126] the quantitative validation of DSL advantages
is an ongoing field of research, yet supporting results are reported. Figure 2.1 shows the
advantage of DSLs regarding to long term cost.
Because of the concise nature and the domain fitting notation DSLs are up to a certain
degree self-documenting. This also facilitates the embodying of domain knowledge which
eases reuse [63, 166] and conservation.
Another advantage is the possibility to validate at domain level [49]. While normal GPL
compilers do not know about any domain concept beyond the general language constructs,
a DSL can be checked for any domain specific constraint. An example may be real time
properties: as long as for every language construct a certain execution time is ensured, it is
possible to automatically proof the whole program. Just as verification, optimization can
be done more effectively at the domain level [18].
2.1.3
Disadvantages
A DSL has not only advantages, but also potential shortcomings. One drawback is the high
development effort which is needed for a new language. The language developer needs
at least experience in language design and knowledge about the target domain. He has
to find fitting abstractions, the right scope and balance between GPL and DSL constructs.
Furthermore the language must be implemented and maintained.
9
Figure 2.1: Simplified cost prediction for DSL-based methodologies [59]
Other problems are tooling, user training costs and performance. While general purpose languages such as Java or C#3 have a strong tool support, corresponding tools for a
new DSL have to be created. IDEs like Eclipse or Visual Studio offer deep integration
with these languages like powerful editors with syntax highlighting and checking, integrated compilers and advanced debuggers. Creating a tool ecosystem for a DSL is a time
consuming process which adds to the total costs caused by language design and implementation. Without a development methodology and suitable tools the risk is high that the DSL
development costs surpass the estimated saving by using a DSL.
The mentioned training costs originate from the fact that possible DSL users have by
definition never used the language before, However this is mitigated as in most cases the
new language should match the domain expert’s expectations.
Often a DSL will suffer from a lower performance than a hand written software. As
long as performance is not critical the other DSL benefits will make this a minor problem.
Nevertheless is some cases performance can be equal or faster because optimization is
possible on a high abstraction level but in most cases the potential is limited.
2.1.4
Development phases and patterns
The development of a DSL can be divided into different phases. The design and the implementation phase were already mentioned. A more fine grained phase subdivision is
possible. Five stages can be distinguished: decision, analysis, design, implementation and
deployment. The development process of a DSL has not to follow these phases sequentially.
Different authors [161, 166] have identified numerous patterns which are reoccurring in
3
http://www.ecma-international.org/publications/standards/Ecma-334.htm
10
DSL development and can serve as guidance for a developer without prior expertise in this
field. Each pattern can be assigned to one of the five phases. The patterns are divided into
decision patterns, analysis patterns, design patterns and implementation patterns each capturing common approaches. In the following section phases and patterns will be described
according to the extensive analysis by Menrik et al. [126].
2.1.5
Decision phase
Before the development of a new DSL can begin, a decision has to be made. Is it feasible
or not? Economic considerations have to be taken into account. Do the accumulated development, deployment and maintenance costs justify a new DSL in comparison with other
conventional approaches? Is there already a suited existing DSL? If so are documentation
and maintenance good enough? If not, is the risk developing a new DSL acceptable?
The following decision patterns have been identified. Most of them based on the same
general concerns such as allowing domain experts with less programming experience [130,
163] to develop software or improving software economics.
• Notation An improved new or existing domain specific notation can be a definitive
factor. Two common subpatterns are the transformation of a visual to a textual notion
and the creation of a user friendly notation for an existing API. The first pattern for
example enables easier composition for large artifacts.
• AVOPT Domain-specific analysis, verification, optimization, parallelization and transformation for applications developed in a GPL are in general time consuming and
hard to automate due to for example source code complexity. With a well defined
DSL AVOPT is more feasible.
• Task automation In some cases GPL programming suffers from repetitive programming tasks. Automatic code generation driven by an appropriated DSL can ease this
[160].
• Product line Some software products do not exist as a single standalone application
but are part of a product line or software family, sharing common parts. A DSL can
facilitate the specification and support automated assembly [182].
• Data structure representation Representing structured data in an easy to read, write
and maintainable form assists in making complex structures accessible. An appropriated DSL can help achieving this goals. YAML [22] and JSON [52] are examples.
• Data structure traversal Like representation, traversal of data structures can often be
expressed more effective with a fitting DSL (for example SQL [84]).
• System front-end DSL based configuration and adaption for system front-ends
• Interaction Text, menu, dialog or voice based applications which interact with the
user can benefit from a DSL which specifies input and reaction in a high level representation.
11
• GUI construction Often GUI design is done by using a DSL. For example XUL and
XAML are XML based DSL for GUI description [25].
2.1.6
Analysis phase
After the decision in favor for a (new) DSL is made, the specific domain has to be analyzed
with the goal of gathering as much domain knowledge as possible. It is important to ensure
a high quality of the gathered material and to have access to domain experts. The term domain analysis was introduced by Neighbors [132] and defined as identifying similar objects
and operations in a particular domain. Different sources of information can be examined
for example already existing technical documents, APIs and GPL code or knowledge from
domain experts.
After gathering the knowledge must be clustered to find meaningful abstractions and
must be consolidated. In most cases the results of the analysis are a domain definition,
the domain specific terminology and concepts, a domain model, the domain scope and
a description of the (operational) semantics. Figure 2.2 summarizes different sources and
possible results. Yet there is no widely adopted notation to capture the results of the analysis
phase.
Three different domain analysis patterns can be identified: informal, formal and extraction from code.
Figure 2.2: Domain analysis, taken from [147]
12
Informal pattern
The informal pattern means that the domain analysis is done informally and therefore no
formal process is used. Most DSLs are developed without an analysis methodology [126].
This often leads to incomplete requirements and can complicate the development process.
While it is possible to get first results earlier the quality is not as high as with formal
patterns. For simple domains an informal process is often enough.
Formal pattern
Domain analysis can also be done using a defined process/methodology. Those which use a
methodology can be counted to those that follow the formal pattern. Using a formal pattern
helps to avoid missing important parts of the domain and can lead to more appropriate
requirements. A large number of methodologies used, come from another field of research:
domain engineering. Domain engineering is derived from the area of software reuse and
refers to the systematic modeling of a target domain. This is strongly related to the notation
of program families [171] and software product lines [113, 162].
While domain engineering and analysis techniques focus mainly on commonalities,
family and product line analysis examine the variations inside a domain. Several methodologies exist today: FAST (Family-Oriented Abstractions, Specification and Translation)
[182] , Sherlock [168], DSSA (Domain-Specific Software Architectures) [165], DARE
(Domain Analysis and Reuse Environment) [75], FODA (Feature-Oriented Domain Analysis) [100], PROTEUS [89], ODE (Ontology-based Domain Engineering) [54] or ODM
(Organization Domain Modeling) [158]. This list consists of the most well know methods
but is by no means complete.
An example where FODA and FAST are applied can be found in [126]. While most
methodologies have a graphical feature diagram or domain model as result, Deursen and
Klint propose a formalized textual4 representation [58] which can be used to generate UML
[133] diagrams or other types of documentation even code.
Semi formal
A specific semi formal approach (domain driven design) covering analysis is proposed in
[66]. The creation of a fitting domain model is most important in domain driven design. At
first domain experts and software architects try to find a domain model which serves as a
base for a common communication language (Ubiquitous Language). This language will
be used later on in all aspects of the development process. It is advised that the notation for
the domain model is UML. Not only one large diagram, but several small diagrams each
describing a certain aspect or part should be used. The reason behind this is avoiding cluttering and reducing complexity. The UML artifacts should be accompanied by documents
that contain information not captured by UML like the meaning of concepts or what certain
objects are supposed to do. In comparison to other methodologies Evans gives extensive
information how to continue after the domain model is established or the feature analysis
is done.
4
The Feature Description Language (FDL), which is a separate DSL again.
13
Extraction from code pattern
The last identified pattern extraction from code derives a DSL directly from an existing
implementation. In most cases this implementation is done in a GPL though it is also
possible to derive from another DSL.
2.1.7
Design phase
The design of a DSL and therefore the development of the language itself is based on the
results of the earlier phases. Two questions have to be answered approaching the design:
How is the DSL related to existing languages and what kind of formal description for the
language is chosen? With each question different possible design patterns are associated
helping to find an appropriated answer.
Creating a language based on an existing one can have different advantages. Some
users may be familiar with the base language resulting in reduced training cost. Common
operations such as arithmetic’s for the family of C languages are well known to many
developers. Furthermore an existing implementation and/or eco system can be leveraged.
Three different approaches reusing existing language can be distinguished. The fourth
approach is the entirely new development of a language.
Piggyback
The new language can piggyback domain specifics feature on part of the existing language.
Examples are Hancock [51], lava [159] or Facile [154]. Hancock is a DSL for high performance signature processing and it piggybacks on C by modifying language parts and
adding processing related constructs. From this DSL, C code is generated again. Similar
to that lava, a production grammar DSL to describe and generate test cases for a JVM, piggybacks on the textual Java byte code representation. The byte code is generated from the
DSL. The Facile language helps developing high performance processor simulation, also
by augmenting C.
Extension
A related pattern is extension. The base language is extended by features corresponding
to domain concept. In comparison to piggybacking the base language is not modified or
replaced. A problem of this approach is the seamlessly integration of new features with
existing ones. A DSL which follows the extension pattern is SWUL [32]. SWUL supports
the development of Java SWING GUIs and is embedded into Java.
Specialization
Developing a new DSL does not always mean to create something new. A more uncommon
pattern is specialization (not to confuse with specialization in UML). An existing language
is reduced to fit the needs of a special domain. Examples are RPython [149] or OWL-Lite
[174]. RPython is a subset of the Python language used inside the PyPy project [149]. The
complexity of Python is reduced in order to make C code generation from RPython easier.
14
OWL-Lite is a subset of the OWL Web Ontology Language with the goal to reason easier
about it than OWL.
Entirely new language
While building on another language can have several advantages as discussed above, creating a completely new language offers great flexibility. However with great flexibility
comes an increased difficulty designing the language. The design of most GPLs is directed
by a number of desirable characteristics [72], namely simplicity, abstraction, expressiveness, uniformity, efficiency, safety, modularity, clarity and orthogonality. More essential or
desirable characteristics and also language design advices are stated by Horowitz [90] in
his evaluation of programming languages. Also software economics may influence the design, too. Costs can be generated by programmer training, software creating, compilation,
execution and maintenance time which depends on the DSL.
Following the guidelines for GPL design while designing a new DSL is considered a
good idea, many DSL researcher believe that the same rules apply for general and domain
specific languages. This needs caution though as real life DSL development has shown
[186]. Some domain models or established domain notations can be in conflict with the
basic rules. The DSL users can be the end-users and do not need to be programmers. Wile
formulates some learned lessons from real life DSL development:
“Lesson 3: You are almost never designing a programming language.
Lesson 3 Corollary: Design only what is necessary. Learn to recognize your
tendency to over-design.
Most DSL designers come from language design backgrounds. There the admirable principles of orthogonality and economy of form are not necessarily
well-applied to DSL design. Especially in catering to the pre-existing jargon
and notations of the domain, one must be careful not to embellish or overgeneralize the language.” [186]
Informal or formal design
After the question is answered if and how the DSL might be related to other languages, the
designer has to specify the language itself. This can be done formally or informally.
Using the informal design pattern is often easier. The specification is usually written in
some sort of natural language. Sample DSL code snippets or short programs are sometimes
used exemplary to show language constructs and what their meaning is. This approach suffers from certain restriction: it is not clear if the descriptions are precise enough - language
syntax and semantic problems can only be detected in the implementation phase and often
informal artifacts are not directly useful for the implementation.
A formal language design can solve some of these problems. Syntax and semantic
problems can be checked automatically if an appropriated formalism is used. With a formal specification it is possible to derive parts of the implementation and therefore reduce
the needed implementation efforts. Different formal methods may be applied including
15
grammars for syntax and abstract state machines for semantic definition (like BNF) , attributed grammars or rewrite systems.
2.1.8
Implementation phase
After the hurdles of analysis and design are taken, the DSL has to be implemented (in case
it is executable, editable or should be analyzed). A wide range of possible implementation
strategies exists and the best appropriated one should be chosen. This decision can strongly
influence the needed development efforts and should be considered carefully. As with
decision, analysis and design different possible implementation patterns can be identified.
Fowler [74] distinguished between external and internal DSL. Spinellis [161] and Mernik
[126] found similar patterns but did a deeper analysis. Some of them will be discussed in
the following sections.
Compiler5 / Interpreter pattern
The most obvious strategy is implementing a DSL in the way a large number of GPLs are
implemented: by using an interpreter or compiler/application generator approach.
An interpreter interprets DSL code in a recognize-fetch-decode-run cycle. A great number of dynamic languages (such as Ruby6 or Python) is implemented following this pattern.
An interpreter is much easier to implement and later to extend, compared to a compiler.
Furthermore the runtime environment is often easier to control.
A compiler approach offers different advantages. Because the complete code has to
be analyzed by the compiler extensive static analysis including error checking and appropriated error reporting can be done. The source is then transformed into library calls and
lower level language constructs (For example when compiling C#, it is transformed into
MSIL and calls to .Net Libraries). This allows fine grained optimizations and often offers
a performance advantaged due to native execution of the compilation results.
Preprocessors
Another classical approach to DSL implementation involves preprocessors. The preprocessing step happens before the code is interpreted or compiled. Different flavors can be
distinguished.
The macro preprocessor is one of the most used forms of preprocessing. Macros can
be simple substitutions on the lexical level before any parsing is done. One or more tokens
can be replaced by a sequence of characters or other tokens. These substitutions can often
be parametrized. The C language with the C preprocessor is an example of simple macros.
However working only on the lexical level has many shortcomings. The preprocessor is
independent of the underlying language and therefore syntax or logical errors can be easily
introduced while expanding the macros. These errors can only be detected at interpret or
compile time.
5
6
In the context of DSL development, compilers are sometimes called application generators.
http://www.ruby-lang.org/
16
This problem is solved by syntactical macros that work on the syntactical level hence
on the parse tree. Scheme or <bigwig> for example offer hygienic macros [109]. These
macros are checked and guarantee to prevent the production of collisions with existing
symbol definitions during expansion. The language C++ also offers syntactical macros,
called templates or template metaprogramming7 . An example DSL constructed by C++
template metaprogramming techniques and operator overloading is Spirit (a C++ Parser
Framework [55]).
Another form of preprocessing is the source-to-source transformation. Unlike macros
where the scope is restricted to some tokens, the whole source is translated into another
language. An example is the in 2.1.7 mentioned language SWUL. Here the complete DSL
which is embedded inside a Java source file is replace by Java code.
A more uncommon preprocessing pattern is the pipeline. The source is not change in
one but in multiple steps whereas each processor is responsible for a subset of the DSL.
The language CHEM is an example using a pipeline architecture.
Embedding
While preprocessors replace the DSL code the embedding approach8 directly embeds DSL
constructs9 inside a host language. The basic embedding, is the creation of a simple application library. To allow a more domain specific notation, new types, operators and other
constructs depending on the base language are created. The combination of the new constructs is then used to describe a domain specific problem. In C++ an often used feature to
allow domain specific notation is operator overloading10 .
Not all languages are equally well suited as a host for a DSEL. Hudak states that functional languages like Haskell11 or MetaOCaml12 are far more convenient than classical
static typed compiled languages such as Java (Lava is a Haskell DSEL example [27]).
Mostly this is due to features like partial and lazy evaluation, high order functions or
generic and polymorphic strong types. However also dynamic languages like Smalltalk
[81] or lately Ruby [167] have proven to be equally suitable for embedding a DSL because
of their very flexible syntax and advanced introspection, reflection and dynamic runtime
modification features. In contradiction to Hudak, Freeman and Pryce [77] show that it is
possible to implement a acceptable DSEL in Java. By separating syntax from interpretation
through object call chains which create an object graph, a good compromise is reached. In
the end they conclude, now in conformance with Hudak, that even though they reached
their goal other languages are better suited.
Some future languages are especially built with extensibility in mind like Fortress [9]
or Scala[138].
7
The C++ template system is a Turing-complete almost functional compile time programming
language[176].
8
Often called domain specific embedded language (DSEL) or embedded domain specific languages
(ESDL)
9
DSL constructs have to be valid constructs in the context of the host language.
10
For example Liboctave a C++ math library allows matrix addition and multiplication using the normal
addition or multiplication operator.
11
http://www.haskell.org/
12
http://www.metaocaml.org/
17
Extension
A related approach to embedding is the extension of the language itself. The compiler
or interpreter is modified to work with domain specific constructs. This is more difficult
than other approaches since most compiler or interpreters are not designed for extension.
Modular and safe techniques are still a matter of research. One possible solution are Meta
Object Protocols (MOP) like the runtime MOP [103] for CLOS or the compiletime MOP
for C++ [48].
Metaobjects are objects modeling the object system itself and they allow introspection
and customization. A Metaobject protocol guarantees that the Object System will follow a
documented protocol for various parts of its operation. therefore a MOP defines the meaning and behavior of a program. Through the MOP an existing language can be modified
and extended.
COTS
With the rise of XML another form of DSL implementation has evolved. DSL implementation can also be approached by using commercial of the shelf (COTS) tools and library
and restricting them to fit a special domain [101]. The grammar for a XML DSL can be
customized by using a DTD or a XML Schema. As a broad range of XML library and
tools is available for parsing (DOM,SAX and STAX parsers), for analyzing (XQuery), and
for transforming (XSLT) the effort for a standalone parser and compiler can be reduced
[139]. In addition to that tools for writing and displaying XML in a structured manner are
available. However XML is not the only possibility as Wile showed [186].
Hybrid
The different strategies have on the one hand advantages but disadvantages on the other..
While it is possible to have a syntax very close to the domain, domain specific verification
and optimization (AVOPT in general) and good error reporting using the compiler/interpreter pattern often causes problems due to the complexity of the tools development and
the need to design a language from zero compared to the solution of leveraging a existing
solutions. In 2.1.9 some frameworks and tools will be discussed which try to solve some
of these problems. Additionally the framework presented in this paper tries to solve some
problems.
The embedding approach has similar advantages and problems but the sides are flipped.
The advantages: often existing languages and their tool chains can be reused and the implementation effort is reduced. On the other side it is hard to create a domain fitting syntax
being bound to a host language. Although the existing infrastructure is leveraged, the DSL
concepts are not known to the base language resulting in the problem that error reporting
or debugging happens in terms of the hosting language not the DSL.
Therefore it is also possible that a hybrid approach is used. Examples are combining macros and an extensible compiler [164], combining embedding and compiling [65],
combining macros and embedding [71] and many others.
18
It has to be decided per case which kind of approach is appropriated. A non general
applicable and simplified comparison of needed effort is presented in table 2.2. Mernik
implemented the Feature Description Language (FDL) [58] using several approaches and
compared the results.
Approach
Compiler
Macro-processing
Embedding (Prolog)
XML-based
Lines of Code
1100
50
40
310
Table 2.2: Lines of Code for different approaches to implement FDL
2.1.9
Approaches supporting DSL development
Many different tools and frameworks are available to support the development of domain
specific languages. They differ in their approaches and their maturity. Some support
the whole development life cycle and offer a solution for editor and debugger generation
whereas others are simple compiler generators. Most of them are based on various forms
of language descriptions.
Compiler Compiler
The lowest common denominator are compiler compiler tools13 , that can be used to develop
new domain specific languages. They are able to construct a lexer/parser out of a given
grammar. This can be the base for an interpreter or compiler. The most prominent ones
are Lex/Yacc14 , Flex/Bison15 , JavaCC16 , SableCC17 , CoCo/R18 and ANTLR [141] (a more
comprehensive list is given by [2, 185]). They often differ only slightly in the used grammar
or the power of the generated lexers/parsers (GLR, LALR, SLR, LR or LL). A large number
of different DSLs has been constructed using one of these tools. However they only solve
the problem of syntax recognition, so a parser/compiler has not to be developed manually.
It should not remain unmentioned that they often require a solid background in language
development theory, like to understand the difference in power between a GLR and LL
generator.
JTS, Maya, JSE
In section 2.1.8 different approaches to DSL implementations are described. As stated
normally it is hard to extend existing languages if no special arrangements were taken by
13
Often also named lexer/parser generators.
http://dinosaur.compilertools.net/
15
http://www.gnu.org/software/flex/
16
https://javacc.dev.java.net/
17
http://sablecc.org/
18
http://www.ssw.uni-linz.ac.at/Research/Projects/Coco/
14
19
the original language implementers . Java is such a case. Nevertheless the Jakarta Tool
Suite [19] supports the extension of the Java language with domain specific constructs.
JTS consists of two components Bali and Jak. Bali is a Java Precompiler which takes
an annotated EBNF grammar as input and Jak is a JTS generated superset of the Java
Language with meta programming capabilities . Together they allow an easier extension of
Java corresponding to a macro approach. Similar approaches are Maya [16] and the Java
Syntax Extender (JSE) [15].
JLE
While JTS provides facilities for error checking there is no support for more general static
analysis. The Java Language Extension Framework (JLE) [188] is an alternative to the
JTS. The primary idea behind JTS was macro processing, the JLE framework builds a new
customized Java compiler out of the different selectable language extension. The language
extensions are formulated in an attributed grammar and are strictly modular. They can be
used for the simple extension of the host language grammar productions or for the definition
of static analysis and optimizations. Finally the customized compiler generates pure Java
from the input code.
EasyExtend
Another approach to language extension with domain specific constructs is EasyExtend.
EasyExtend allows to extend the Python programming language. This is done by modular
modifications to the original Python grammar. The new grammar is used in combination
with a mapping to valid Python constructs creating a new domain specific parser. One that
can parse and run the DSL on top of Python. The original CPython Parser is not modified
and has not to be recompiled19 .
metafront
A framework which goes beyond the shortly before introduced tools is metafront [30].
Metafront is language agnostic. It consists of two parts: grammar and syntactic transformation specifications. An input stream according to the grammar is parsed by a new parsing algorithm (specificity parsing). The input grammar can be created in a modular way
combining multiple grammar input files (one grammar can extend another grammar). The
resulting parse tree can then be transformed by given declarative transformation rules. For
every transformation an input and an output language (grammar) has to be defined. This
allows metafront to statically check the transformations for type safety and to guarantee
the termination of the rules. The metafront approach can be used as a (macro) preprocessor
for domain specific language extension or for the development a completely new language.
The language is then transformed into a base language like C.
19
A modified parser from the PyPy project is used (written in Python).
20
LISA
The previous approaches could be used for the extension of an existing language or for a
complete new DSL, they do not solve the DSL ecosystem problems. Nevertheless different
frameworks try to improve this situation.
The Language Implementation System on Attribute Grammars (LISA) [88] uses attribute grammars not only to generate a compiler or interpreter. A large number of different tools can be generated by the Java based LISA tool including a syntax aware editor or
finite state automate visualization editors. One reason for that is the possibility to separate
semantic rules from grammar production rules. This in addition to advanced templates and
inheritance formalism support for grammars allows better modularization. Another part of
LISA is debugger generation, although this was not a target for LISA and is therefore only
rudimentary.
MetaEnvironment - ASF+SDF, TIDE, AspectAsf
While LISA is based on attribute grammars the MetaEnvironment [169] is based on algebraic specifications. It has been in development for several years and had several rewrites,
but can be considered stable today. The most important subpart is ASF+SDF (algebraic
specification formalism and syntax definition formalism) allowing a integrated but still
modularized definition of language syntax and semantics. ASF+SDF delivers executable
specifications based on conditional equations and term rewriting20 . Other important parts
are a SGLR parser, the ToolBus for coordinating different tools and ATerms, a format to
exchange annotated trees between the several tools. Based on this foundation the MetaEnvironment can generate interpreters, editors or pretty printers for DSLs.
With the ToolBus Integrated Debugging Environment (TIDE) [170] it is also possible to
get debugging support for a DSL, but it still needs several manual steps. Another solution
to get debug support is AspectAsf [143]. AspectAsf annotates the grammar with all possible debug actions at every potential place, This increased the complexity of the resulting
grammar.
The MetaEnvironment aims to reduce development effort for new languages, but the
developer has to learn the complex ASF+SDF formalism. Additionally the MetaEnvironment has grown over the years to a large collection of components making the framework
flexible and generic but decreasing the ease of maintenance. Due the flexibility learning
and mastering the environment is difficult. The MetaEnvironment is only available for
Linux based systems.
Strongly related to the MetaEnvironment is Stratego/XT21 [177], which is based on
SDF, too. It uses a different and more powerful language for term rewriting. With it different tools can be generated. Stratego is even more complex due to highly advanced concepts.
The MetaEnvironment and LISA are both suited to implement DSLs and moreover GPLs
or can be used for program transformations.
20
21
An extended explanation can be found in [107]
Stratego is the language and XT is a tool collection.
21
Smarttools
As LISA SmartTools [13] is a development environment generator based on Java and it
additionally makes heavy use of XML. A DSL can be define as an AST22 . This AST is
in turn manipulated. From it SmartTools is capable of generating a structured editor, a
pretty-printer, a UML model and a parser specification. A debugger can not be generated,
though.
Horn logic approach
An approach able to generate a compiler and even a profiler is presented in [180]. It is based
denotational semantic and uses Horn logic where Horn logic code syntax and semantic are
executable and yield an interpreter. This interpreter can be partially evaluated together
with a DSL program and resulting in compiled code. This approach is very powerful,
though mathematical, and requires many theoretical skills from the DSL developer. Also
no surrounding toolset is generated.
DSL Debugging Framework
As manually building debugging tools for each new DSL is time consuming, Hu [187]
proposes a DSL Debugging Framework (DDF) based on the Eclipse Debugger Platform
and on DMS. A DSL grammar is defined with ANTLR. From this grammar a DSL parser
and additional source mappings are generated. The mappings relate each DSL construct
to the back-end code. A mapping component communicating with the Eclipse Debugging
perspective and the back-end debugger is derived of the mapping . The mapping instructions are applied automatically to the grammar by aspects, by using DMS (a transformation
system). They contain instructions which are inserted in the grammar actions. The aspects
are specified in AspectG, a language for modifying the ANTLR grammar.
DDF supports the generation of DSL debugger for imperative, declarative and hybrid
languages. Currently only Java as back-end and ANTLR as front-end are supported. Moreover the actions are specified in Java. For this reasons the approach can be seen as rather
inflexible at the moment.
Other frameworks
Most of the presented frameworks and tools are based on previous research results. For
the completeness a few should be mentioned: Draco [131]which allowed the creation of
a parser, transformer, optimizer, analyzer and pretty printer for a DSL in an integrated
way, KHEPERA [67] which worked as a source-to-source translator, powered by a simple
AST constructing parser, a tree transformer and a pretty printer for the resulting trees and
furthermore Sprint [55] based on denotational semantics and partial evaluation of a DSL
interpreter. Other approaches will not be discussed but they are similar: APPLAB [26],
DMS [20],smgn [104], Eli [97], SPARK [121] and TXL [50]. This list is by no means
complete and several other solutions exist.
22
abstract syntax tree
22
Also some commercial solutions promise DSL development support by generating editors,debuggers and other tool such as JetBrains MetaProgrammingSystem23 or IntentionalProgramming24 . However non of these has reached production state or is available to the
public.
2.2
Model-Driven Engineering
While section 2.1 focused on DSLs this section discusses Model-Driven Engineering. In the following the terms model, metamodel, metametamodel technology space will be defined and several examples given. Afterwards a possible formalization of the terms is presented. Then some key techniques are
described like domain specific modeling languages, transformations and generators. Finally the Model-Driven Architecture will be described shortly including MOF, EMF/ECore and XMI.
Model-Driven Engineering25 (MDE) is a vision that moves the model into the center of the
software development process. Moreover any software artifact is considered a model or
model element. While previous approaches used models for documentation or communication of ideas, they are first class entities throughout the whole model-driven engineering
life cycle [40]. MDE is an open and integrative approach not tied to a special standard,
therefore different implementations exist.
The problem that should be solved is the growth of platform complexity beyond the
ability to mask it with current GPLs. MDE tries to capture what is often expressed in an
informal way (prose,diagrams) as formal model based specifications.
Key concepts to mitigate current software engineering problem are models, metamodels, technology spaces, domain specific modeling languages and different kinds of model
transformation like model to model or model-to-text26 .
Model-Driven Engineering is currently only described informally but different formalization approaches are researched at the moment [68, 150].
2.2.1
Models - the foundation of MDE
Model, Metamodel and Metametamodel
Models are the key artifact of MDE. For that reason the term model is defined first. The
word model has its etymological root in the Latin word modullus that is the diminutive of
modus. Then again modus meant a special architectural constraint ratio between parts of a
building. A general definition merely based on the etymological meaning is not easy. Today
the interpretation of the word model strongly depends on the view point of the observer and
his domain.
23
http://www.jetbrains.com/mps/
http://intentsoft.com/
25
The often used term Model-Driven Development (MDD) is equivalent to MDE, but is a registered
trademark of OMG. Also used: model driven software development (MDSD) .
26
Often also called generators
24
23
As this thesis is a computer science thesis a definition from the computer scientists
Gerbé and Bézivin [41] is used (a more general discussion of the model term can be found
in [24, 114, 156]).
“A model is a simplification of a system built with an intended goal in mind.
The model should be able to answer questions in place of the actual system.”
While this serves as a starting point, Kleppe et al. [105] gives a definition even more
directed to MDE.
“A model is a description of a (part of) systems written in a well-defined
language. A well-defined language is a language with well-defined form (syntax), and meaning (semantics), which is suitable for automated interpretation
by a computer.”
Kleppe speaks about a “well-defined language” which can be used to create a model. In
MDE metamodels define how a model can look alike.
The word “meta” is Greek and means “above”, therefore the term metamodel can be
interpreted as a model describing another model27 . To understand the term metamodel a
simple analogy to languages is drawn. A language consists of words whose combination
is constraint by a grammar. If a sentence in a language is seen as one possible model, the
definition of its structure, the grammar, can be seen as its metamodel. Earlier it was said
that in MDE a metamodel defines how a model can lookalike, this can be more precisely
formulated as: a metamodel defines the constructs and rules usable to create a class of
models. This is consistent with the following definition :
“A meta-model is a model of a set of models.” [69]
“A meta-model is a model that defines the language for expressing a model.”
[135]
Notice that a metamodel is itself a model. If this is true the metamodel has also to be
defined in a “language” (needs a metamodel). For that reason the metametamodel is introduced, allowing to specify metamodels. The question which can be raised, is how the
metametamodel is defined. To avoid an infinite stacking of meta levels, metametamodels
are often specified self reflexive and therefore the metamodel of the metametamodel is the
metametamodel itself.
Most approaches implementing MDE define a three level meta stack - model, metamodel, metametamodel. A metamodel can be used to clearly define a class of models and
the metametamodel should allow the specification of all possible metamodels including
itself. Therefore one metametamodel should be enough.
The one metametamodel needs still to be defined. Currently different models compete: the MetaObjectFacility (MOF [135]), the Kernel Metametamodel (KM3 [93]), EMF
27
It is important to differentiate between a model about a model and a model of a model. While a painting
of a painting showing a house can be seen as a model of a model it does not describe for example of what
colors can be used in a painting.
24
MDE
implementation
Model-Driven
Architecture(MDA)
Software
Factories
Model Integrated
Computing
(MIC)
Model Centric
Software
engineering
Main contributor
Metametamodels
Tools
OMG
MOF, ECore,
KM3, SMD,
Kermeta
unnamed
Eclipse EMF
Microsoft
University of
Vanderbilt
MetaGME
Lockheed
?
Microsoft DSL
Tools
GME
Lockheed
internal
Table 2.3: Different MDE implementations, different metametamodels
ECore [37], MetaGME [119], the Coral Simple Metamodel Description (SMD [8]), Kermeta [146], the Microsoft DSL Tools Metametamodel [127] and many others. All serve
a certain MDE implementation as metametamodel but differ in expressiveness. Unifying
these or enabling translation between them will mitigate fragmentation in the long term.
Different approaches have already been made [42, 43, 79] to bridge metametamodels for
instance MDA to MIC or MDA to Microsoft DSL Tools and vice versa. Yet as Bézivin
states [45] simple translation does not solve all fragmentation problems and is an area of
ongoing research.
On the end of this section it should be mentioned that the described terms are still
subjects of philosophical debates. For slightly different views [24, 39, 69, 70, 148] are
good starting points.
Technological spaces
Different notations like BNF, XML Schema or Ontology’s description can be seen as
metametamodel, too (for grammars respective XML documents or ontologies). This depends on the technological space (TS) under consideration. Technological spaces were
introduced in [116]. The idea is having an abstract base to reason about similarities, differences and integrations possibilities for different technologies. The following definition is
given by Kurtev et al. [117]:
“A technical space is a model management framework with a set of tools
that operate on the models definable within the framework.”
Kurtev continues with the observation that each technology space is based on a three-level
metamodeling stack (model, metamodel and metametamodel) as can be seen in figure 2.3.
This has already been described for the model technological space (MDE or ModelWare in
25
general) but is also true for technological spaces such as DataWare, GrammarWare [106]
or XMLWare.
Technology space
OMG/MDA
Layer 3
Grammar
conformsTo
XML
conformsTo
conformsTo
MOF
EBNF
conformsTo
XML Schema
conformsTo
Layer 2
UML
metamodel
C Grammar
conformsTo
XML Schema
Definition
conformsTo
Layer 1
UML model
a C program
a XML
document
Figure 2.3: Different technological space and the metamodeling stacks
As different metametamodels exist, various frameworks, languages and tools are available for generation and transformation.
Formalization
Currently the terms model, metamodel and metametamodel have only been defined informal. Kurtev et al. [117] propose a formal definition of these terms. This is done out of
an organizational point of view (as a set of graphs constrained by other graphs). If in the
remainder of this thesis is referred to the mentioned terms the beneath definitions should
be used.
Definition 2.1: A directed multigraph G = (NG , EG , ΓG ) consists of a set of nodes NG , a
set of edges EG , and a mapping function ΓG : EG → NG × NG .
Definition 2.2: A model M = (G, ω, µ) is a triple where: G = (NG , EG , ΓG ) is a directed
multigraph, ω is itself a model (called the reference model of M ) associated to a graph
26
Gω = (Nω , Eω , Γv ) and µ : NG ∪ EG → Nω is a function associating elements (nodes and
edges) of G to nodes of Gω . The relation between a model and its reference model is called
conformance. We denote it as conformsTo28 .
Definition 2.3: A metametamodel is a model that is its own reference model (i.e. it conforms to itself).
Definition 2.4: A metamodel is a model such that its reference model is a metametamodel.
Definition 2.5: A terminal model is a model such that its reference model is a metamodel.
2.2.2
Domain specific modeling languages
The ideas behind domain specific modeling languages (DSML) and domain specific modeling are similar to the DSL concepts presented in 2.1 but applied to the world of models.
Whereas in DSLs the language, its textual representation, is domain specific, in DSM29 the
models are domain specific. In comparison with DSLs the representation is almost in every
case graphical. The gained benefits are similar, too.
UML according to its name a unified modeling languages tries to offer a set of different model types able to describe every imaginable domain. Yet often the UML or other
traditional modeling languages lack the facilities to precisely capture domain knowledge
leading to abuse and a large number of proposed extensions [124, 137]. The recent version
of UML allows the modification by profiles, enabling specialization. Of greater importance
is the fact that UML diagram types are now based on a single metamodel30 (MOF).
Using a metamodel for the definition of a DSML offers a number of advantages:
• Decreased development effort for visualization and editors because of a common
metamodel
• Model verification and checking according to a metamodel with available tools [144]
• The DSML stays extensible (for example SysML based on the UML2 metamodel
[137])
• Uniform exchange formats exist (like XMI) and a next generation model exchange
is proposed [111]
• Approaches for models comparison based on a metamodel exist [110, 123]
• The semantics of a DSML can be described by a metamodel [60]
28
Alternatively instanceOf is also correct.
Domain Specific Modeling
30
In this case MOF is the metamodel for the definition of the definition of what elements can exist in a
diagram or simply the metamodel of the DSML. If the definition of what kind of elements can exist is treated
as a metamodel, MOF is its metametamodel. It depends on the point of view when MOF is treated as model,
metamodel or metametamodel.
29
27
Name
Eclipse EMF/GMF
Metamodel
ECore/XML Schema
MetaEdit+
GOPRR
Generic Modeling
Environment
Generic Eclipse
Modeling System
MetaGME
XMF-Mosaic
MOF/XCore
Tau G2
MOF/UML
Rational Software
Architect
MOF/UML
MagicDraw 12.0
MOF/UML
Microsoft DSL Tools
unnamed
ECore
information
XMI support, complex but most
complete and powerful compared with
others [11]
customizable diagram editor including
element constraints
OCL and XMI support, complete
graphical customization possible
based on Eclipse GMF, a large number of
DSML development steps is automated
in comparison to raw GMF.
based on Eclipse, editors customizable,
OCL support
DSML are created by defining UML 2.0
profiles through extending the UML
metamodel. Modification of graphical
representation is restricted to model
elements (for example link could not be
customized).
DSML are created by defining UML 2.0
profiles but only stereotypes are possible.
Graphical representation is even more
restricted then Tau G2.
extension by UML 2.0 profiles, API for
defining custom diagram elements
plugin for Visual Studio.Net,
representation completely customizable
Table 2.4: Different tools for creating DSML
Current tools
A large number of tools usable to specify and then use a DSML are available. They allow
to define allowed elements and how their graphical representation look like. Most of them
are based either on the ECore or the MOF. A detailed view goes beyond this thesis but a
few are listed in table 2.4.
Earlier approaches
The described approach to modeling a specific domain is not new. Already CASE tools
implemented many of the ideas. In contradiction to the described DSML tools however they
were based on proprietary modeling languages, metamodels and ineffective code generator
and failed for this reasons [82].
“Most of these tools did not take advantage of platform-specific features,
28
and produced naive, inefficient, least common denominator code. The upfront
costs of adopting the modeling techniques they required were prohibitively
large. Added to this was the risk inherent in spending a substantial portion
of the projects budget building models with the promise of code appearing
only in later stages. This required enormous confidence in the tools and in the
longevity of their vendors. Also, concepts like round-trip engineering, where a
model could be synchronized with changes made independently to the code it
had generated from them, were overwhelmingly complex. Another criticism
was that many CASE tools imposed a methodical top-down process. This
was the ultimate kiss of death, since rapid iteration of partial solutions has
proven to be critically important to the success of application development, as
so clearly demonstrated by agile development methods.” [21]
2.2.3
Generators and transformations
Transformations and generators allow to synthesize different artifacts such as source code
or alternative model representations and also transform traceable between different models with the possibility to ensure consistency. Furthermore it opens the possibility to do
model refactoring, specialization, annotation or model aspect weaving. Transformations
and generators are enabled by metamodels.
Generators
In general most available generators can be categorized as either template or visitor based.
Often generators are referred to as model-to-text transformations. Template based generators replace certain parts inside a template document with information from the model. One
instance of a template based generator is JET which is part of the EMF. Additional to JET
there are a large number of different template based generators like MOFScript, XPand or
Acceleo which are focused on model-to-text or more general ones such as Smarty, Contemplate, Cheetah, Jinja, Savant, or Liquid to name only a few.
The visitor approach works as described by the visitor pattern [78], though the visitee
is the model. Each model element is visited and an according action is triggered. In the
context of the current visited model element specific text is generated.
Another approach, falling under the category of template generators but less frequently
used are generative grammars. The main idea is to use grammars as in compiler generator
tools, but reverse the direction. While in compiler tools a lexer and a parser are generated
being able to consume arbitrary text conforming to the grammar, reverse grammar tools
generate text conforming to the grammar, according to given model (input data). Such
tools are ST [140] or Anti-Yacc [87].
29
Approach
Template
Generative
Tools
AndroMDA[128], FUUT-je[173],
OptimalJ31 , Xpand32 , ArcStyler33 ,
MOFScript34 , Acceleo35
ST [140], Anti-Yacc [87]
Table 2.5: Different model-to-text approaches
Tool
AndroMDA, FUUT-je, OptimalJ
MOFScript, XPand, Acceleo
Smarty36 , Contemplate37 ,
Cheetah38 , Jinja39 , Savant40 , or
Liquid41
ST, Anti-Yacc
Description
template engines based on UML
template engines based on MOF,
more flexible then UML based
engines
general applicable template
engines, no special MDE support
generative grammar engines, no
special MDE support
Table 2.6: Different generator tools
Transformations
Transformations and the executing systems can be grouped by different criteria, for example whether they are imperative or declarative, how many input and output models are
involve, are they horizontal42 or vertical43 , whether they directly manipulate44 a model, are
31
http://www.compuware.com/products/optimalj/
http://www.openarchitectureware.org/
33
http://www.interactive-objects.com/
34
http://www.eclipse.org/gmt/mofscript/
35
http://www.acceleo.org/
36
http://smarty.php.net/
37
http://www.typea.net/software/contemplate/assembled/home.html
38
http://www.cheetahtemplate.org/
39
http://wsgiarea.pocoo.org/jinja/http://smarty.php.net/
40
http://phpsavant.com/yawiki/http://smarty.php.net/
41
http://home.leetsoft.com/liquid
42
The source and the target model share the abstraction level, for example when a refactoring is performed.
43
Source and target model exist on a different abstraction level, for example a refinement.
44
The user has to implement transformation rules and sequence trough API calls in a GPL like Java, which
directly modify the model.
32
30
Technology space
MDA/EMF
Metametamodel
MOF/ECore
XML
Grammar
Schema
Grammar in (E)BNF
Data
Graph
different
Standard / Tools
QVT [136], ATL [95], Tefkat [118],
YATL [142], MTF45 , UMT[83], BOTL
[31], MOLA [98], SmartQVT46 , Borland
TogetherQVT47 , MiA48 , MTRANS[145]
XSLT49
ANTLR AST Transform50 , TXL [50],
ASF+SDF [169]
SQL
51
AGG , GReaT [6], Viatra, PROGRES
[155], Groove, FUJABA [179]
Table 2.7: Transformation engines from different technology spaces
described via algebraic relations, are graph based or using a hybrid approach. Which of
the techniques is most appropriated? That has to be decided depending on the requirements. Mens and Gorp proposed a taxonomy [125] that can help with this decision. Again
the technology space has to be considered. Is the transformation intra or inter technology space? An intra-TS transformation for example would be a transformation XML to
XML performed with XSLT and XQuery. An intra-TS transformation could be UML to
XML[178].
A large number of model-to-model transformation frameworks have been developed
lately, caused by a growing interest in MDA/MOF/EMF and MDE in general. The OMG
proposed the Query View and Transform (QVT) standard based on MOF. There are several
implementations claiming to be compliant with QVT up to a certain level. Nonetheless,
there exists no complete implementation of the standard version 1.0 today. The most advanced freely available implementation is the Atlas Transformation Language(ATL).
Note that transformations themselves can be expressed as models [44]. These models are called transformations models and behave like regular models. They conform to
a transformation metamodel and can be transformed themselves (metatransformations or
high order transformations).
While there are several attempts to classify [53] or benchmark [175] different model
transformation approaches, an exhaustive comparison of the various transformation technologies is still pending but would be of great use.
45
http://www.alphaworks.ibm.com/tech/mtf
http://sourceforge.net/projects/smartqvt
47
http://www.borland.com/de/products/together/index.html
48
http://www.mia-software.com/miaStudio/indexOfMiaStudio.php?lang=
en&theme=prod-miagen
49
http://www.w3.org/TR/xslt
50
http://www.antlr.org/doc/sor.html
51
http://tfs.cs.tu-berlin.de/agg
46
31
Tool
YATL
MOLA
BOTL
Tefkat
ATL
MTRANS
UMLX
BOTL
Viatra2
VMT
MISTRAL [115]
Description
part of the Kent Modeling Framework, textual, MOF
based, declarative/imperative hybrid
visual language, simple and loop patterns
bijective, visual language, declarative, object oriented
mathematical foundation
textual, declarative/imperative hybrid, based on theory of
stratified logic programs
ATLAS Transformation Language, based on ECore EMF
metametamodel, very actively developed,
declarative/imperative hybrid
textual, MOF based, translated into XSLT which modifies
XMI
graph bases, visual language, only for UML models,
translated into XSLT which modifies XMI
graph based, bidirectional, only for UML models
graph based and abstract state machine based hybrid,
pre/post conditions for a transformation, recursive and
negative pattern support
graph based, visual language, declarative
MOF based, textual, declarative/imperative hybrid
Table 2.8: Some model-to-model transformers
2.2.4
Different MDE technologies in detail
MOF
As described in Section 2.2.2 different metametamodels exist. A well-known one is described in the OMG Meta Object Facility (MOF) standard. The idea behind MOF is a
four-layered architecture as depicted in figure 2.4. Sometimes it is referred to as a closed
metamodeling architecture because the defined metametamodel conforms to itself. Currently with version 2.0 different flavors of the MOF exist: the Complete MOF (CMOF),
the Essential MOF (EMOF) and the Semantic MOF. CMOF is the whole MOF whereas
EMOF is only a subset of the most important elements.
The widespread use of the Universal Modeling Language (UML) is one of the reasons
for the recent popularity of MOF. The current version is the base for UML2. All different
diagram types are defined conforming to the metamodel. The MOF is the foundation for
OMG Model Driven Architecture52 (MDA), too.
52
The term Model Driven Architecture is misleading. While the part “Model Driven” has its explicit
meaning in the systematic use of models, the part “Architecture” was chosen by accident [70]. In contrast to
what could be assumed to be “Architecture” it does not mean software architecture in this case. The OMG
has been criticized more than once for the misleading naming. In fact the etymology of MDA derives from
the renaming of the earlier Object Management Architecture (OMA) to MDA. However the OMG sometimes
refers by architecture to their “four layer architecture”.
32
conformsTo
instanceOf
describes
M3: metametamodel
(MOF)
describes
conformsTo
instanceOf
M2: metamodel
(UML metamodel)
describes
conformsTo
instanceOf
M1: model
(UML Class)
describes
conformsTo
instanceOf
M0: object
Figure 2.4: Example of MOF layers
MDA
The original idea behind MDA is to separate design from architecture with the help of
models on different abstraction levels. MDA proposes Platform Independent (PIM) and
Platform Specific Models (PSM). Where PIMs express a higher abstraction level, leaving
out implementation details and PSMs describe lower levels including platform specifics.
The PSM should be almost generated automatically by transforming a PIM.
In practice this approach causes problems. The standards that MDA is based upon
(UML, MOF, OCL53 , QVT, ...) are complex and incomplete at the moment. For example
the transformation standard QVT has no fully compliant implementation right now and
there exists no standard for action semantics. Another problem is that interoperability
between MDA tools is often problematic. Not every vendor implements all OMG standards
correctly, moreover sometimes the specifications are not precise enough. Furthermore no
practical tool environment is available for the MOF leading to the abuse of UML tools (to
define class diagrams (terminal models) which would be then translated or interpreted as
metamodels).
53
Object Constraint Language
33
PIM to PSM Transformation
Engine
PIM
PSM to Code Transformation
Engine
PSM
PIM
Code
Code
Figure 2.5: MDA PIM to PSM (FMC)
Eclipse / EMF
There are several tools and framework following the MDA approach and leveraging standards from the OMG. A prominent example is the Eclipse Modeling Project (EMF), a
framework for modeling and code generation. EMF is one of the most successful approaches to MDE. Since 2002 it has been developed [37] and integrated into the Eclipse
Platform (now version 2.2). In contrast to the MDA proposal the metametamodel used
is ECore. EMF is split into three large components: core, edit and codegen. While core
contains the ECore metametamodel, persistence, serialization, a model tracer (change notification and recording) and a validation framework, edit includes a default model viewer
and helps in building rich user interfaces (view, editor) for models and codegen supplies a
code generator for ECore based models and several importers (for example from Rational
Rose).
On top of EMF a number of other frameworks are build, like GEF (Graphical Editor
Framework) allowing to create graphical editor and GMF (Graphical Modeling Framework) for creating a domain specific modeling language editor.
ECore - EMF metametamodel
ECore is the metametamodel of EMF. In the initial development of EMF, MOF should have
been used as the metametamodel but was dismissed in favor for ECore. It was developed as
MOF 1.x was to complex and did not fit to the needs of EMF developers. Therefore ECore
is much simpler than MOF 1.4. MOF evolved, tough. And with version 2.0 different
metametamodel flavors were developed. Interestingly the EMOF was developed in parallel
to the ECore/EMF development with a steady exchange of ideas. So EMOF is strongly
aligned to ECore, and has only minor differences. EMF can read, use, manipulate and
write EMOF based models equally well as native ECore Models.
ECore is described in ECore itself. Figure 2.6 shows the complete model (version 2.2).
MOF / ECore Model exchange formats - XMI, HUTN, emfantic and KM3
For the exchange of metadata information like models or metamodel the OMG has defined
the XML Metadata Interchange Standard (XMI). The current version is 2.1. This standard
describes how any model (MOF compliant) can be mapped to XML. Additionally it also
shows how a schema can be automatically generated of the MOF model to represent any
MOF compliant model. That allows the validation of model data which is stored as XML.
Figure 2.6: ECore metametamodel
34
35
At first the primary goal of XMI was UML model exchange. Today it is used to exchange all kinds of models. In figure 2.7 different applications in need to exchange models
and the advantages of a standard for exchange (in this case XMI) are shown.
Application
Application
Application
Application
Application
Application
Application
Application
Application
Application
Application
XMI
Application
Application
Application
Application
Application
Figure 2.7: XMI used for exchange.
XMI suffers from some practical problems. Again there are incompatibilities between
different XMI implementations of different vendors that need to be resolved. Another problem is that a great number of models being exchanged (most UML drawings for example)
are concrete models. Most of the times these models have a visual representation. In this
representation layout information, colors and shapes are often important to the user. But
these information are not part of the model (itself). For example UML2 defines class diagrams but the visual representation or a way how to exchange it is nowhere specified.
To solve this problem the OMG defined the Diagram Interchange (DI, XMI-DI) standard,
which captures visual information. At the moment DI is not widely implemented by tools
vendors. Nevertheless there are several efforts to optimize the support. In the meantime
model exchange including visual representation is not working seamlessly between different vendor tools.
XMI suffers not only of slow implementation. It is difficulty to read and write manually
without tools support. The OMG therefore proposed the Human-Usable Textual Notation
(HUTN) [134]. This language allows description and exchange of MOF models in a way
which is more comfortable to read and to write than XMI. It was primarily targeted for
writing textual UML. It failed because of UMLs complex metamodel. The author of this
thesis only knows two implementations of HUTN: TokTok (out-dated54 ) [4] and HUTN
Kermeta (prototype) [129].
With Emfatic55 a similar human friendly notation for ECore exists. The current version
is 0.0.2 and no development occurred since 2004. It can still be considered as usable (with
Eclipse 3.0).
As stated before ECore is closely aligned to EMOF. The EMF offers XMI support for
ECore as its main exchange format. This benefits the interchangeability of ECore Models.
54
55
It is based on an old dMOF specification.
http://www.alphaworks.ibm.com/tech/emfatic
36
Yet the ECore XMI implementation has some minor anomalies, as described in section
4.4.6.
Another possibility to exchange metamodels is KM3. The Kernel MetaMetaModel is
a textual language for specifying metamodels. KM3 as metametamodel resembles ECore,
only stripping out Java related entities. It can be converted to and from ECore/Emfatic/XMI
automatically and serve as a persistence format for metamodels. In contraction to XMI it
is not possible to store a instance of a metamodel with KM3.
2.3
Model-driven DSL development
This section will show two DSLs implemented with an Model-Driven Engineering approach. After the concrete DSLs various model-driven DSL frameworks will be presented, which allow the definition of a textual language.
2.3.1
Model-driven DSLs by example
Two DSL implemented using an MDE approach are WebML [46] and SPL [151].
WebML
WebML is a language for expressing the structure of web applications with a high-level
description. It offers different models, together forming a website: structure, derivation,
composition, navigation and presentation model. Earlier versions of WebML were based
on implicit metamodels contained inside the WebRation tool in form of a DTD. This approach suffers from the of verbosity and limitation of DTD. Moreover because no explicit
metamodel exist, the profits gained by MDE, like a common exchange format or powerful
model transformations are not available. Recently a ECore based metamodel was proposed and implemented [152] to overcome these constraint. The metamodel was derived
semi-automatically of the DTD. Based on that metamodel WebML was further evolved for
example the addition of aspect oriented concepts.
SPL
One step further goes Ruscio et al. [151] with the Session Processing Language (SPL) [38].
This language is designed for implementing telephony services over the Session Initiation
Protocol (SIP). The Atlas Model Management Architecture (AMMA) is used to implement
SPL. AMMA is based on Eclipse and has a strong connection to EMF. The earlier described
KM3 and the ATL are part of AMMA. While KM3 is needed to define the metamodels and
ATL for different transformation, two more parts play an important role. The concrete
syntax to domain model mapping is done by TCS which will be described in 2.3.2.
Most other MDE implemented DSLs end here. For SPL also the (dynamic) semantic
is described by model-based abstract state machines (defined by a metamodel based on
XASM [12]). This is a novel strategy because even for languages such as UML or MOF
37
the semantic is only defined by prose56 . Only a small number of other approaches are
known to the author: Kermeta which extends MOF with operations to define operational
semantic and the denotational metamodeling approaches by Hausmann [86] and Alvarez
[10].
2.3.2
Model-based domain specific language tools
MDE based approaches to DSL development are still a field of ongoing research. Frameworks or toolset already being usable are XText/oAW, Kermeta TCS/AMMA and TCSSL.
Atlas Model Management Architecture
AMMA was discussed as framework used for the implementation of SPL. To rephrase, it
consists of the parts KM3, ATL, AM3 (model management) and TCS (Textual Concrete
Syntax) [94] and is based on the Eclipse Modeling Framework. As KM3 and ATL were
discussed before and AM3 is not of importance, TCS will be discussed more in detail.
TCS allows definition of textual syntax for models and was introduced [96] in October
2006. It is a submission for the Eclipse Textual Modeling Framework (TMF) call for
proposals. At the time of writing of this thesis TCS was not publicly available. It works
by associating syntactical elements with elements of the domain metamodel. From this
information an ANTLR grammar is generated. The parser derived from this grammar is
able to consume a DSL program and deliver its representation as a model conforming to
the metamodel. This component is also called injector. The second component of TCS
is the extractor. It implements model-to-text generation. No additional specification is
needed. A model, its metamodel and the TCS syntax mapping are used to generate a
textual representation. This is novel as most other approaches separate this into two distinct
steps: a model conforming to a meta model is create manually by using grammarware
like yacc. Then the grammar is annotated with special actions creating model elements.
Finally another tool like JET is used for model-to-text transformation. With TCS only
the metamodel and the syntactic mapping are required for text-to-model and additionally
a valid model for mode-to-text transformation. The syntax mapping consists of a template
file defining the mapping for each metamodel element. Different mappings are available:
• PrimitiveTemplates describe lexer tokens for a KM3 DataType inside the metamodel
• ClassTemplates describes syntactical elements that are used for KM3 Classes inside
the metamodel. This could be keywords, special symbols and properties. Where
properties correspond to structural feature inside the KM3 Class.
• Abstract ClassTemplates enabling navigation inside the inheritance hierarchy
• Conditionals constraining the existence of a sequence by a condition
• Operators with priority, associativity and a symbol
56
One exception is the semantic model verification with constraints formulated within OCL.
38
Metamodel elements are identified by name inside the TCS. For a better understanding the
listings 2.1 and 2.2 show an excerpt from the metamodel in KM3 and the mapping in TCS
for the SPL language, taken from [96].
1 datatype String ;
3 c l a s s Program e x t e n d s L o c a t e d E l e m e n t {
reference service container : Service ;
5 }
7 c l a s s Service extends LocatedElement {
a t t r i b u t e name : S t r i n g ;
9
r e f e r e n c e d e c l a r a t i o n s [∗] ordered c o n t a i n e r : D e c l a r a t i o n ;
r e f e r e n c e s e s s i o n s [∗] ordered c o n t a i n e r : Session ;
11 }
Listing 2.1: SPL metamodel
1 p r i m i t i v e T e m p l a t e i d e n t i f i e r f o r S t r i n g d e f a l u t u s i n g NAME;
3 t e m p l a t e Program main
: service
5
;
7 template Service
: " s e r v i c e " name " { "
9
" processing " "{"
declarations
11
sessions
"}"
13
"}"
;
Listing 2.2: SPL TCS template specification
TCS also has some limitations. The possible languages are restricted by the parser
generator and therefore the supported grammar class. The used ANTLR version has the
power of LL(k). Furthermore error reporting is currently insufficient. It is possible to create
ambiguous grammars. ANTLR recognizes this problem, but the error is not propagated up
to the corresponding rule inside TCS or the KM3 metamodel. If the metamodel differs
strongly from the desired syntax, the mapping gets difficult and is sometimes not possible
due to limitations of TCS. For example it is currently impossible to create blank delimited57
or case insensitive languages.
The AMMA platform itself does not offer a way to describe the dynamic semantics
and no debugger can be generated. However an editor TGE (Textual Generic Editor)
parametrizable by a TCS file, offers outlining, syntax highlighting and hyperlinking for
any in TCS defined DSL is available.
Notably the approach chosen in TCS is similar to the framework described in this thesis.
As both were developed independently this supports some of the later described design
decisions.
open ArchitectureWare
Open ArchitectureWare (oAW) is another Eclipse based platform suited to develop DSL
corresponding to the MDE approach. The current version is 4.1, it is based on the ECore
57
An example for a blank delimited GPL is Python. A blank delimited DSL is YAML.
39
metamodel58 and consists of a set of tools which can interact with each other:
• Xpand a statically typed template language for model-to-text transformation with
support for template polymorphism and template aspects. It is comparable to any
modern template engine for instance Velocity or Smarty.
• Xtend a DSL for extending metamodels, for example with custom properties for a
metaclass. It is use for template modularization and also enables aspect-oriented
templates59 . As of version 4.1 Xtend can also be used for functional style model-tomodel transformations. However only simple and limited transformations are possible in comparison to ATL.
• Check for defining declarative constraints over a model and checking them. (Similar
to OCL).
• XText enables text-to-model transformations and was also submitted as a proposal
for TMF. In contrast to TCS the metamodel is derived from a XText grammar file.
The grammar describes the syntax of the DSL and is then transformed to a ANTLR
grammar and a ECore metamodel. The parser generated of the ANTLR grammar
creates model elements conforming to the metamodel. The generated metamodel
corresponds to an AST specification for the DSL. This approach suffers of the inability to create a custom metamodel. The authors propose [64] to transform from
the generated metamodel into a “real” metamodel. This helps separating concerns
(parsing, linking60 ) but additional effort is needed compared to TCS. Furthermore
the XText language itself is limited, so only simple DSLs are possible. The XText
grammar which is similar to BNF consists of a set of rules. Each rule can be composed out of a combination of three token types(keyword, identifier or string) and
references to other rules. Modifiers such as multiplicity, optionality or alternation
are available. How a string token or a identifier token is build, is not customizable,
nevertheless it is possible to define a simple custom token type for strings. XText
generates an Eclipse based editor including an outline view, syntax highlighting and
checking for a DSL.
• A workflow engine which can be used to coordinate the different parts.
The oAW framework offers integration with Eclipse GMF. Xpand and Xtend are already
used successfully in MDE environments, however not many DSL based on XText expect of
a abstract state machine example and an overly simple example of describing an ADT61 are
known to the author. Moreover the use of XText is limited because only simple grammars
can be formulated and the according metamodel is build automatically and can not be
customized.
58
oAW comes also with its own metametamodel.
Extensions can be applied to the metamodel in an aspect like fashion.
60
The proposed term for AST metamodel to domain metamodel transformation.
61
Abstract Data Type
59
40
Kermeta
Kermeta [76] is also based on the Eclipse platform and allows the definition of a domain
specific language. However only a graphical representation is possible. The metametamodel used is derived from EMOF and extended by a behavior package. Therefore in
Kermeta it is possible to specify the semantic behind a model, using the Kermeta language,
a DSL which directly maps to the behavior model. Moreover it is possible to transform a
Kermeta model which contains semantic information into an ECore model. The semantic
is preserved within ECore annotation. This approach is interesting because it contains a
model-based representation of semantic information similar to the SPL example. Yet it
is not possible to create a customizable textual representation for the model itself and the
reason why Kermeta is included in this discussion is because in comparison to the other
frameworks it is possible to describe the language semantics in a model-driven way.
TCSSL
The Textual Concrete Syntax Specification Language (TCSSL) [73] is a metamodel-aware
specification language for grammars. It promises a bidirectional mapping text-to-model
mapping. A new grammar language consisting of three different types of rules is proposed:
• Simple rules are EBNF rules instantiating a model element.
• Seek rules are used to resolve references by looking for existing model elements
matching a given condition.
• Singleton rules work like simple rules but do not instantiate a new model element if
an instance already exists.
In addition several elements a rule can contain like alternatives, actions with guards, model
expression (inside a language such as Kermeta or MTF), model queries (side effect free
expressions), sub rule calls or multiplicities are defined.
At the moment only a prototype implementation of TCSSL exists in Java, based on
ANTLR and EMF Jet. ANTLR rules are mapped to TCSSL rules one to one and their actions contain TCSSL specific code. Complex queries and expressions are not implemented.
The model-to-text transformation is handled by simple EMF templates, where each template must be associated with a rule, this makes indention handling tricky because JET is
not completely controllable.
Moreover the authors make no statements about the used lexer and use a standard implementation in all cases. This problem and the JET generator will cause the same problems
as already seen with TCS such as no support of blank delimited languages or no standard
token types. The implementation is not available to the public.
Other approaches
Currently a number of different frameworks for model-driven DSL implementation are
developed. While documentation for XText, TCS and TCSSL is available to some extend
41
1 syntax UMLStatecharts
w i t h d e f a u l t L a n g u a g e KerMeta ;
3
entry rule for StateMachine : : =
5 " S t a t e M a c h i n e " s e l f . name
7
s e l f . top ;
9
r u l e f o r S i m p l e S t a t e : : = < i n i t a l > " S t a t e " s e l f . name
;
11
r u l e for CompositState : : =
13 < i n i t i a l > " C o m p o s i t e S t a t e " s e l f . name " { "
mixed ( s e l f . s t a t e | | s e l f . t r a n s i t i o n )
15 " } "
;
17
macrorule i n i t a l ::=
19 ( < < { s e l f } . i s I n i t a l : : = { t r u e } >> " i n i t i a l "
|_
21 ) ;
r u l e for T r a n s i t i o n : : =
" T r a n s i t i o n " s e l f . name " from " s e l f . s o u r c e <name>
25
" t o " s e l f . t a r g e t <name> ( s e l f . e v e n t o p e n e r " on " )
;
27
s e e k r u l e name f o r S t a t e w i t h c r i t e r i u m { s e l f . name } : : =
29 [ 1 ] ( s e l f . name ) ;
23
31
s i n g l e t o n r u l e f o r E v e n t w i t h c r i t e r i u m { s e l f . name } : : =
[ 1 ] ( s e l f . name ) ;
Listing 2.3: A TCSSL grammar example
this is not the case for some newer frameworks like IBM SAFARI[47] or TEF62 .
The initial presentation of SAFARI was at the EclispeCon 200663 / GPCE’0664 but no
documentation and no public binary/source code access is available. SAFARI allows the
generation of Eclipse based DSL environments offering a rich user experience such as syntax highlighting, source-text folding, hyperlink detection, content outlining, content assist,
hover annotations, hover help, parsing, and project building. The DSL has to be specified
using the LPG65 parser generator. Furthermore parsing support includes incremental scanning and parsing, and grammar-directed error recovery. SAFARI also provides support for
debugging breakpoints (via JSR 04566 ) and a framework of components for multi-language
static analysis. If all of the named feature work as promised, SAFARI is a powerful alternative to the other presented frameworks, however the author of this thesis could not prove
any of these statements as only a short introduction paper is accessible.
The Textual Editing Framework (TEF) is announced by the Humbolt-University of
Berlin and offers similar feature as SAFARI. Yet only a website and the source code of
a unusable alpha version is available at the moment. TEF is also based on Eclipse and
allows the generation of convenient DSL editor.
62
http://www2.informatik.hu-berlin.de/sam/meta-tools/tef/tool.html
http://www.eclipsecon.org/2006/
64
http://www.program-transformation.org/GPCE06/WebHome
65
http://sourceforge.net/projects/lpg
66
http://jcp.org/aboutJava/communityprocess/review/jsr045/index.html
63
42
2.4
Summary
A large number of solutions for DSL development are available. Most of them do not
support a model-driven approach. Those of them which do lack some important features.
Non of the model-driven frameworks is able to generate a debugger. In XText the domain
model is generated automatically and it is not possible to use a custom domain model. TCS
allows the usage of a custom model and includes implicit mapping from text-to-model and
vice versa. Yet it is restricted to languages not differing so much from the model and has
some other arbitrary limitations such as no support for blank delimited languages. The
approach TCSSL has taken is also interesting but premature Java based implementation
does not make it an ideal solution. Moreover JET is an insufficient template engine and
the proposed language needs improvements (like a way to define influence lexer). The
frameworks SAFARI and TEF are interesting alternatives if they reach a matured status
and are made available to the public.
None of the discussed model-driven tools allows the reuse of existing grammars. In the
remainder of this thesis some solutions for these problems are proposed.
43
Chapter 3
Domains in the automotive industry
After the concepts of domain specific languages and Model-Driven Engineering have been discussed above, this section introduces specific domains inside
the automotive industry. This serves as a general background for a later described and implemented automotive domain specific language,
The key innovation areas in today’s automotive industry are software and electronics. A
modern car contains a multiplicity of connected controllers running under different kinds of
operating systems. While the first software solutions inside a car were local and unrelated,
today up to 70 bus interconnected ECUs1 can be found. Many different development tools
and frameworks are orchestrated. However the principles and methods of modern/efficient
software engineering are not sufficiently known and applied in this industry. Proprietary
solutions are used [29] and software reuse from one car generation to the next lies often
below 10% [85].
The controllers used range from very limited low level sensors up to high level processors empowering multimedia functionality. They are interconnected by different bus
systems such as CAN, MOST or FlexRay and have to be coordinated. Most of the controllers and their software are also constrained by real time , safety and always by cost
requirements.
Additionally this complexity has not only to be managed for one car, but for various
models, product lines and equipment combinations. Furthermore the different stages of
development have to be coordinated from requirement analysis, architectural design to implementation and testing. A wide range of software development and management tools is
therefore in use, each fitting a special domain. These tools are not always well integrated.
Currently in a number of different domains the supporting software is not sufficient with
regard to not allowing the domain expert to express his intentions optimally. As depicted
in 2.1.2 domain specific languages are often able to solve this problem. Despite that a large
part of the software development is still utilizes in C. Requirements engineering is still done
in prose with tools like DOORs without any formal foundation2 and testing is realized by a
wide variety of tools, not inherently developed for testing. At least in some domains DSLs
have emerged, for example ASCET-SD suited for the development of embedded controller
1
2
Electronic Control Unit
For the MMI specification modeling guidelines exist, resulting in a semi formal description.
44
software. Compared to C it offers hierarchical state machines, block diagrams and pseudo
code similar to Java to define control logic. Another example is Matlab/Simulink. Matlab
is a DSL for numerical computing especially vector and matrix numerics. Simulink is an
extension of Matlab. It allows modeling, simulation and analysis of multidomain dynamic
systems which are described by block diagrams. Message State Charts3 (MSC) an ITU
standard for modeling the communication behavior between distributed instances similar
to UML sequence diagrams are used too. test cases are generated from these specifications.
One domain which can be found at the BMW Group Information and Communication
Systems division is automation of multimedia oriented system testing. Currently a DSL
exists but could be improved. This is explained in detail in 5.2.
3
A textual and graphical representation is available.
45
Part II
Frodo
46
Chapter 4
One framework to bind them all?
“To achieve great things, two things are needed: a plan, and not quite
enough time.”1
One framework to bind them all? The answer to this question is the same as to the question after the silver bullet in software development. Since no silver bullet exists [33], no
framework can satisfy all needs. Therefore the framework presented in this section tries to
answer a set of special questions. A large number of the them have aroused during analyzing available tools for DSL development and the current implementation of the TestDSL.
In general the framework should allow the implementation of domain specific languages with a reduced effort and better reuse properties compared to other solutions.
4.1
Requirements and solutions
Reusability As discussed in 2.1.8 the classic compiler/interpreter approaches to DSL development often only utilizes a compiler generator. This raises the question of reusability. Almost all grammar definitions are tool specific. They depend on the chosen
syntax and capabilities of the generator such as the the parser class. In some cases
the specification of syntax and semantic has to be done inside one grammar file (for
example Yacc/Lex). Possibilities for reuse are heavily limited. A modular approach
to grammar definition improves this and is proposed and implemented by different
frameworks (see 2.1.9). These modularized artifacts are not interchangeable between
tools due to the already listed reasons. So the proposed framework therefore uses a
Model-Driven Engineering solution to mitigate this problem. A grammar metamodel
is suggested. If a model conforms to this metamodel a tool specific grammar can be
generated and the grammar can be modularized by separating concerns in different
models. As meta- and model are easily exchangeable in MDE they come with reuse
built in. Other MDE tools can handle the models, offering visualization and edit capabilities, applying transformations or deriving other artifacts. Many of the available
tools for DSL development tie the developer to a platform like LISA or ASF+SDF.
Only a small number of them are MDE-enabled. Even the MDE-enabled tools have
1
Leonard Bernstein
47
some problems for example XText suffers from the inability to define own domain
meta models. Only TCS which was published after the development of the proposed
framework has equal properties.
Decrease development effort Another critical point is the needed skills of the DSL developer. Developing a new DSL from scratch needs serious effort and knowledge
in language design. On the other side, embedding a DSL inside another language
like Ruby, needs less effort but has some disadvantages such as the syntax constraint
of the host language and the missing deep of integration of domain concepts. One
possible answer given by the framework which is also enabled through MDE is reuse
again. It allows the conversion of an existing grammar into a model of that grammar.
A less experienced developer can extend or modify this model for example by adding
new constructs to the Java language. It is possible to develop a complete new DSL.
Ideally this should work for syntax and semantics.
Usability Not only reuse, modularization and interchangeability with other tools can be
questioned. Today the users of modern IDEs are accustomed to syntax highlighting,
code completion, refactoring and deeply integrated debugging capabilities. These
are standards in IDEs like Microsoft Visual Studio.net or Eclipse. Just a small number of DSL tools have a solution for this demand. Most compiler compiler only
allow specification of the language itself and include nothing else. Exceptions are
described in 2.1.9.Some offer the generation of an editor and to a less extend a debugger. These are important feature for increasing the efficiency of a developer.
Therefore the framework described afterwards is able to generate an editor and a
debugger of a given metamodel. Often the used models do not provide enough information for this task, so additional work has to be done be the developer in specifying
needed in formations.
Language evolution A DSL may evolve over time. If changes were made to the language,
rendering artifacts written in older versions incompatible, all artifacts will need to be
updated manually. This is a time consuming and error prone task as non of the
current tools support can automate this easily. The presented framework will support
automatic conversion between language versions if a transformation on the metalevel
is defined.
4.2
User roles
Several user roles can be distinguished.
• A domain expert who is simply reading the DSL to understand what is expressed.
Only a simple editor is needed.
• A domain expert developing in a new DSL. He only needs to know how to use the
DSL editor and how to execute or debug the DSL. Often he is a source for language
improvement proposals.
48
• The DSL developer: He has to understand the basic principles of the proposed framework and the target domain to define a new language. This is important because his
task is the definition of the syntax and of the semantic of the DSL. Furthermore he
has to test and debug the newly developed language. Finally to accomplish his task
language design and domain modeling expertise is needed.
• The DSL back-end developer: His role is to implement the back-end for the new
language. A large number of different possibilities such as an interpreter or virtual machine approach exists. A good understanding of the domain and the needed
functionality is a prerequisite but expertise in language development is not required.
Often the role of the DSL and the back-end developer coincide in one person.
• The last role is the framework developer. No explicit domain knowledge is require
expect for a deep understanding of the framework: How the different parts are related
and how they work together. He has to extend, maintain and port the framework.
4.3
Framework Architecture
The proposed architecture consists of three parts:
• The framework itself, which contains modules for model handling, transformation
and exchange and also different parsers/generators,
• Language specification artifacts like grammars or transformations,
• An interchangeable back-end.
In 4.4 a subset of the features of the in the following described architecutre are implemented.
49
Edit, Debug
...
Backend
R
IDE
Debugger
Debugger
(Plugin)
R
Framework
Developer
Backend Plattform (Runtime,
Language & Tools)
Editor
Editor
(Plugin)
DSL Artifacts
Model Bus
R
DSL Specification
R
Metamodel / domain model
Backend APIs / Frameworks
DSL Framework
Grammar model
Model Manager
Syntax
SyntaxView
View
Metametamodel
Semantic Mappings
Grammar metamodel
Plugin Generator
Tool Templates /
Tool Template
Debugging
Aspects
R
R
Configuration
Get syntax model
R
Model consumable by backend
(source,bytecode,dbg symbols,...)
Transformations
Concrete DSL
Model Transformation
Engine
(Model2Text,Model2Model)
Model
Parser/Generator
Generator
Textual representation
DSL Parser /
Generator
Misc Frameworks, APIs, Runtimes
R
Figure 4.1: Proposed architecture for an model-driven DSL framework (FMC)
4.3.1
Components of the framework
The framework consists of a number of interacting components.
Metametamodel
Because the framework follows a Model-Driven Engineering approach, one unique metametamodel is defined. Ideally this is one of the widely used metametamodels As already stated
such a model enables reuse with other tools and frameworks.
Model Manager
The Model Manager (MM) plays a central role inside the framework. It manages the available models and metamodels, allows to create new models conforming to a metamodel,
loads and stores models and keeps track of inter-model referencing. The metamodels have
to conform to the unique metametamodel, which serves as the base for the Model Manager to understand every other conforming model. The ModelManager offers access to this
50
metametamodel. In most cases the models will be handled in memory by the MM. Through
the model import and export capabilities other tools and frameworks which understand the
same metametamodel are able to reuse any model created or loaded and modified and then
exported by the MM.
The MM can also be seen as a model repository like the Netbeans Metadata Repository2
or the repository implementation of EMF.
Model Bus
The Model Bus is not part of the framework but is used by the MM. The MM itself only
supports exporting models which at some point conform to the unique metametamodel
in a special persistence format whereas the Model Bus enables a broader reuse. Some
tools may not support the MMs export format but are connected to the Model Bus and
can therefore work with the models managed by the MM. Sometimes a transformation
into different metamodels can be automatically handled by the Model Bus by metamodel
similarity matching. Moreover the Model Bus allows connecting model operations from
different tools. This is an advantage as no single tool offers all operations like edition, storage, manipulation, transformation, validation, checking or code generation and a manual
connection needs additional developer efforts.
A concrete Model Bus is implemented by [28], still the framework is completely functional without a Model Bus connection.
Model Transformation Engine
To allow complex programmatic model modification a Model Transformation Engine (MTE)
is included into the framework. The MTE offers to different high level operations: modelto-model and model-to-text transformations.
Model-to-model transformations are needed at several places. For instance existing
grammar models can be modified, or a concrete DSL model can be transformed into a
back-end model.
Model-to-text transformations are used to create textual representations of grammar
models that could after that be process by the parser generator or the generator generator.
Furthermore not all back-ends understand models, so a textual representation which they
are able to understand (like C) can be created from an concrete DSL model.
It is possible to use an external transformation engine, too.
Grammar metamodel, parser- and generator
Not only the domain information should be described by a model but also the grammar
of their textual representation; therefore a grammar metamodel is proposed within this
framework. Accompanying a grammar parser and a grammar generator are included in the
framework. The parser creates a grammar model conforming to the grammar metamodel
from a textual grammar and the generator transforms the grammar model into a textual
grammar. The textual grammars are used to generate the DSL parser. Different parser
2
http://mdr.netbeans.org/
51
generators can be utilized. For each generator a separate grammar parser and grammar
generator is needed because their textual grammar representation is dissimilar, however the
grammar metamodel is tool independent.
Parser Generator
A textual representation for a domain is of no value if it can not be processed. A parser
is needed making the DSL understandable to a machine. This parser is constructed from a
grammar by a parser generator. As described in 4.3.1 several parser generators can be used,
as long as a grammar parser/generator is available for the format inside the framework
used by the parser generator. As parser generators differ in the employed paradigm and
in power, it is not possible to utilize any generator. The mapping to the generic grammar
metamodel can be difficult up to impossible. Nevertheless creating a parser/generator for
parser generators like CoCo/R, JavaCC, ANTLR, Sprint (Parser Framework), or Parsec and
cross targeting them through the common metamodel is possible due to their equal LL(k)
power.
Notice that one domain can have multiple textual representations and different grammars. In the context of this framework these grammars are syntax views of the domain.
The created parser is not only able to recognize a DSL. It creates a model from the
textual representation that conforms to the domain metamodel. This model can then be
used for any further operations. To be able to instantiate the right metamodel elements on
recognition of a certain DSL construct, a mapping from the grammar rules to the domain
elements has to be done. Two ways are possible: implicit and explicit mapping. If the
mapping is explicit the developer has to specify the actions instantiating model elements
on its own for every grammar rule. An implicit approach is able to create model elements
automatically by convention. For example grammar rule names are mapped to metamodel
element names.
While explicit mapping offers greater flexibility a larger effort is needed. For this framework an implicit approach is favored. This can reduce development work by clever mapping conventions.
The usable parser generator are restricted by the target programming languages3 . Not
for every language exists a metametamodel implementation. This is essential as the parser
has to understand the domain metamodel in order to instantiate a model. One possibility
if that is not the case is to directly write XMI XML. File output is supported by a vast
majority of available languages.
Generator Generator
Sometimes it is of value to have the possibility to generate the textual representation for
a domain metamodel instance. A model-to-text transformation engine can create such a
representation. However a developer has to write the transformation specification manually. The idea of the Generator Generator or more precisely of the generative grammar4
3
The parser generator creates code in a specific programming language. The code represents the parser.
Sometimes also called unparser. The first theoretical foundation goes back to Axel Thue in 1914 [184].
Later Chomsky based his work on the by Thue defined Thue system.
4
52
generator is the creation of a generative grammar from the existing grammar model. This
is feasible because the grammar model contains the complete syntax representation. The
idea to derive an unparser grammar from a parser grammar is long known [91] and was
recently demonstrated in the context of model driven DSL development [94]. This reduces
the needed effort a as no model-to-text transformation like a model visiting generator has
to be developed.
Plugin Generator
As previously discussed besides the domain specific language itself the surrounding eco
system is important. Users of modern GPL environments are accustomed to features as
syntax highlighting and code completion. Furthermore a debugger is essential for most
GPLs. To further enhance the advantages of a DSL the framework proposes the generation
of the according tools. The goal is not to generate a complete environment, but to extend
existing environments and reuse already built-in functionality. Some IDEs like Eclipse5 or
Microsoft Visual Studio.Net6 offer interfaces that explicitly support the extension of the
environment with new components.
Two components are proposed an editor plugin and a debugger plugin. Both should
be produced by the Plugin Generator module of the framework. For every IDE a set of
particular templates is needed. In combination with the grammar model it is possible to
generate the components.
The way to derive a standalone editor from a grammar is shown by [88, 169] and the
generation of an Eclipse editor plugin is explained by [64, 96]. A similar approach is used
by the presented framework. Basically the templates provide a tool specific frame. They
contain grammar independent code and placeholders for grammar specific parts. The parts
are filled with information from the grammar model. The derived editor can also serve as a
base for custom modifications.
A debugger plugin which allows DSL debugging requires greater effort. Only a very
small number of DSLs, independently of the way they are developed come with debugging
tools. While an editor can be generated only from a grammar model this is not the case
for a debugger. For executable DSL the back-end has to support debugging first of all.
Secondly the mapping from DSL to back-end has to be traceable. If it is not possible to
relate back-end instructions back to the DSL parts they are generated from, debugging is
impossible. Letting the DSL user debug directly inside the back-end is no option because
he has to understand the back-end technology and therefore often deals with non domain
specific concepts. Finally it is not feasible to implement the debugger manually as this is a
time consuming and error prone task and would increase DSL tool development duration.
However as discussed in 2.1.9 some frameworks allow the generation of a debugger. An
approach which concentrates on imperative and declarative DSL debugging serves as guidance for idea how a debugger can be generated. Wu et al. [187] propose the weaving of
debugging control instructions into the grammar. In the context of this DSL framework the
weaving of debugging aspects by simple model transformations on the grammar model is
5
6
http://www.eclipse.org
http://msdn2.microsoft.com/en-us/vstudio/default.aspx
53
proposed. The available debugging aspects are back-end dependent and have to exist for
every back-end a debugger should be generated for.
4.3.2
Language specification artifacts
Several artifacts are needed to define a new DSL. They are described in the now.
Domain Metamodel
The domain metamodel is a central part in the definition of a new DSL. It describes what
kind of elements can exist inside the domain. Every domain metamodel instance7 has to
conform to the metamodel, which itself has to conform to the metametamodel on that the
framework is built. Designing a new DSL the domain metamodel should be established
in the first place. In some cases a domain metamodel already exists for instance as a base
for a domain specific modeling language and only needs to be translate into the correct
metametamodel. Without the metamodel a domain model can not be precisely defined.
How the domain metamodel looks alike depends merely on the developer. It can be
close to the textual representation or far away from it, however the former makes and
implicit mapping from the textual representation to a domain model and vice versa difficult.
Special caution8 should be taken, because the domain metamodel is an important artifact.
A model which does not fit the domain, can increase the needed development effort.
A similar approach is taken by TCS. XText in comparison derives the domain metamodel from a grammar.
Syntax View
A Syntax View describes the textual representation for a domain metamodel and is the
grammar for the domain under consideration. This view can be a grammar model itself
conforming to the grammar metamodel included in the framework or a textual grammar
supported by the framework. A textual grammar can then automatically be converted into a
model. Again using a grammar model makes manipulation through transformation easier,
especially global modifications. As transformations are applied manually, they are less error prone than a manually modified textual grammar. Allowing the import of any grammar
written for the supported parser generators enables the reuse of existing grammars (as for
prominent GPLs or previously developed DSLs).
The syntax view consists of a set of parsing rules and additionally of the construction
actions for model elements. As mentioned in 4.3.1 an implicit and an explicit mapping
is possible. To facilitate an implicit mapping, different conventions are proposed for instance naming production rules equal to model elements. The grammar model can then
automatically be augmented with matching actions derived from rule naming and type,
7
The terms domain metamodel instance and domain model are synonymous in this context. A domain
models is in this case a terminal model.
8
The development of a precise world map, which can be seen as a model took hundreds of years. It was
refined multiple times. The domain modeler should not be satisfied with his first design but try to refine it.
However this should take considerably less time than the development of the world map.
54
cardinality and connection of domain metamodel elements. In some cases more information is needed, like such as how to map primitive types for example to lexer rules or to
define special symbols like delimiters or operators. So a special grammar DSL which is
model and convention aware is proposed. Furthermore it contains additional constructs for
the description of special cases and mappings.
Additionally for debugging and error handling purpose the line and column numbers
inside the textual representation are stored for every created model entity .
Semantic Mapping
Describing the syntax and its mapping to a model can be done with Syntax Views. The
question what a model exactly means and its (operative) semantic is defined by Semantic
Mappings. They are the connection to the back-end. DSLs which are not executable do not
need this mapping.
The goal is to map the domain model to an representation understandable by the backend. Different possibilities are available.
• The back-end understands the domain metamodel. It can directly execute any given
model by either interpreting it or compiling it to another executable representation.
In this case no mapping at all is required.
• The back-end uses a special back-end model. If the back-end’s metamodel uses the
same metametamodel as the framework the semantic mapping is a transformation
from domain metamodel to back-end metamodel. The meaning of a domain model
element is then directly related to one or more back-end model elements.
• The back-end is a general purpose language such as Java or Python eventually combined with a domain specific library. A number of approaches are possible. A textual
representation can be created from the domain model using a model-to-text transformation. After that the generated code can be compiled/interpreted. Furthermore
the domain model can be transformed into an abstract syntax representation of the
back-end/hosting language. Some platforms allow the direct execution of ASTs. Depending on the domain it is even imaginable to target lower level abstraction like Java
Byte code or MSIL.
• The back-end is unclear. One option is to use an established format for the specification of semantics such as abstract state machines. Ruscio et al. [151] uses XASM to
describe the semantics of a DSL (see 2.3.1). This could be integrated into the model
flow if the representation supports the MDE approach. The chosen representation can
afterwards be transformed into an executable format, an example is Kermeta which
specifies a new language for semantics deeply integrating semantics into the model
flow.
This is not a complete list of potential semantic mappings, as the mapping strongly depends
on the back-end.
55
To allow error propagation from the back-end up to the DSL this mapping is of importance. As a mapping from the domain model to line and column information inside the
textual representation is preserved, a mapping from a model element to back-end permits
to derive a mapping from back-end to textual DSL representation. If an error occurs in the
back-end it can be propagated to the related DSL fragment.
Currently no help or instructions for semantic mapping beyond the model management
and transformation tools are proposed.
Transformations
Additional transformations can be included in the description of a DSL. This can be modelto-model or model-to-text transformations. Some example could be the generation of a documentation skeleton for the grammar model in HTML or custom modifications to any of the
involved models. A use case with more importance is the transformation between different
DSL versions. Ideally it is sufficient to define a transformation between distinct versions
of the domain metamodel. All domain models can then be converted automatically. This
leads to the importance of keeping the old domain metamodels between different revisions.
Admittedly no special arrangements for versions are included in this framework.
Another use case is the transformation of multi variant models into simple models by
resolving constraints or replacing generic with concrete elements.
Configuration
The configuration controls the parts of the framework. It can be used to parametrize tools
and to define workflows steering the different components like parser generator or transformation engine. This approach is similar to the oAW workflow engine.
4.3.3
Back-end
The back-end allows the execution of the DSL. As shown in 4.3.2 several back-end types
are imaginable as long as a semantic mapping exists. In general the framework enforces no
restrictions on the back-end, but to allow certain functionality the back-end needs to offer
several interfaces.
Control interface
To integrate the back-end with the rest of the framework and generated editors, the backend needs to be externally controllable. It should be possible to trigger the execution of
back-end consumable artifacts and to query state information. If an error has happened
inside the back-end it should be possible to get an error trace to allow the mapping back to
the DSL.
Debugging interface
If debugging a DSL should be possible control over the executing back-end is needed.
The back-end has to offer interfaces which allow different operations. What operations
56
are required depends on the DSL. For an imperative DSL the back-end should support
breakpoints, and step by step execution. Furthermore to inspect DSL datatypes the backend has to offer introspection capabilities. This makes the retrievement of variables/data
information from the back-end and map them to the DSL possible.
4.4
Implementation
“Python code is easy to throw away”9 .
Frodo - a Framwork for domain specific languages is an implementation of the in 4.3
proposed architecture. It is developed in Python for practical reasons. The use case which
motivated the development of the framework is a large Python application. However as
will be shown, that does not have an influence on the generality of the framework.
In the next sections the implementation is presented. Frodo implements a large number
of the described components. It will be mentioned if an implemented feature differs from
the proposal.
4.4.1
Overview
Frodo consists of several components. The core is built around the EMF ECore metametamodel, enabling MDE. The transformation engine is a mixture of ATL and Python code
for model-to-model transformations and a modified ST10 engine for model-to-text. For
parser generation ANTLR is chosen as it comes with Python support. The target of the plugin generators is the Eclipse development environment, because of its good extensibility
features.
Most of the framework is implemented directly in Python version 2.4, Python 2.5 which
was released after the initial development can run Frodo equally well.
9
10
Tim Peters, long time Python contributor
The orginal name was StringTemplate but ST is a name one its own by now, see table 2.6.
57
Edit, Debug
...
Backend
R
Eclipse IDE
Debugger
Debugger
(Plugin)
R
Framework
Developer
Backend Plattform (Runtime,
Language & Tools)
Editor
Editor
(Plugin)
DSL Artifacts
ECore/XMI
R
DSL Specification
R
ECore domain model
Backend APIs / Frameworks
DSL Framework
Grammar model
Model Manager
Syntax
SyntaxView
View
ECore
Semantic Mappings
Grammar metamodel
Plugin Generator
Tool Templates /
Tool Template
Debugging
Aspects
R
R
Get syntax model
R
Configuration
Model consumable by backend
(source,bytecode,dbg symbols,...)
ATL/Python Transformations
Concrete DSL
ATL
Python
ST
Model
ANTLR Parser
Generator /
ST Generator
Textual representation
DSL Parser /
Generator
Misc Frameworks, APIs, Runtimes
R
Figure 4.2: Frodo’s architecture (FMC)
4.4.2
ECore - Metametamodel Implementation
The core of Frodo is the functionality built around the ECore metametamodel. ECore is
described in 2.2.4 and is chosen because it has proven its practical usability, as a wide
range of tools and libraries is based on or able to work with ECore (Eclipse GEF/GMF,
Omondo11 , Topcased12 , MagicDraw13 ). Major MDE implementers like GME announced to
support EMF/ECore[3]. Yet because of its strong connection with the Eclipse/EMF project,
only a Java implementation is available. Frodo implements the ECore metametamodel in
Python. In comparison with Java a cleaner integration into Python is possible due to its
dynamic nature. The implementation consists of various components.
11
http://www.omondo.de/
http://www.topcased.org/
13
http://www.magicdraw.com/
12
58
Metametamodel
ECore is implemented as a set of Python classes. In the first version of the framework
a skeleton of these classes was developed manually. In the current version the metametamodel classes are bootstrapped by generating the Python classes from the ECore metametamodel description defined in ECore itself14 . As there is no specification for the semantic of
the operations defined in the metametamodel, the Java implementation is consulted on.
A metamodel which conforms to the ECore metametamodel is expressed as an inmemory-object-tree of Python ECore class instances. Other models use ECore as metametamodel. They can link to it as metamodel and use the defined classes, too.
Model Management
The Model Management keeps track of all in-memory models (models, metamodels and
the ECore metametamodel). It manages the mapping of URIs to metamodel files and allows
to query for a specific metamodel by URI or filename. Moreover different paths are configured by the Manager such as the default model storage location. Currently only filesystem
locations are supported. Finally it offers a wrapper-function to allow the loading of ECore
metamodels, creating instances of metametamodel classes. Often these classes contain
methods defined in the metamodel but without an implementation. Through mixins the
ModelManager can automatically inject implementations defined by the by the developer.
XMI Handling
To support the exchange of models Frodo implements the XMI 2.0 standard. This allows
storing and loading of ECore models in the XMI format, that is also used by EMF. As XMI
is a XML format it is implemented inside Frodo on top of the lxml library for Python. The
library implements ElementTree a container for accessing hierarchical data structures like
XML from Python and supports XPath and XML Schema validation.
When a model is loaded the corresponding Python classes are instantiated according
to their metamodel and they are filled with the stored information. In most cases an XML
element is mapped to a Python class and its attributes to class attributes. Stored intra-model
references are resolved and replaced by real object reference in memory. Frodo supports
both referencing types used by ECore XMI: by ID and by location references. Furthermore
the framework is able to resolve external references and load models and their metamodel
on demand.
Storing an in-memory model as XMI/XML is equally implemented as loading. Inside
the XMI document object references are replaced by positional or ID references. The
resulting object graph is then written to an output stream.
Python integration
Making the use of models as easy as possible, Frodo is well integrated with Python. Therefore model elements are represented by in memory objects. The framework generates ap14
org.eclipse.emf.ecore_2.2.0.jar/model/ecore.ecore
59
from e c o r e . modelmanagment . c o r e import ∗
2 from e c o r e . s t o r a g e . dumper import dump
4 c o l o r = E A t t r i b u t e ( name= " c o l o r " , e A t t r i b u t e T y p e = " E S t r i n g " )
c a r = E C l a s s ( name= " Car " , e S t r u c t u r a l F e a t u r e s = [ c o l o r ] )
6 r r = E C l a s s ( name= " R o l l s R o y c e " , e S u p e r T y p e s = [ c a r ] )
m i n i = E C l a s s ( name= " Mini " , e S u p e r T y p e s = [ c a r ] )
8
p a c k a g e = EPackage ( e C l a s s i f i e r s = [ r r , m i n i ] )
10 dump ( p a c k a g e , " c a r s . e c o r e " )
Listing 4.1: Creating an ECore metamodel for a carpool in Python
1 < e c o r e : E P a c k a g e x m l n s : e c o r e = " h t t p : / / www. e c l i p s e . o r g / emf / 2 0 0 2 / E c o r e "
x m l n s : x s i = " h t t p : / / www. w3 . o r g / 2 0 0 1 / XMLSchema−i n s t a n c e "
3
x m l n s : x m i = " h t t p : / / www. omg . o r g / XMI" x m i : v e r s i o n = " 2 . 0 "
name= " C a r s " >
5
< e C l a s s i f i e r s x s i : t y p e = " e c o r e : E C l a s s " name= " R o l l s R o y c e " e S u p e r T y p e s = " / / Car " / >
< e C l a s s i f i e r s x s i : t y p e = " e c o r e : E C l a s s " name= " Mini " e S u p e r T y p e s = " / / Car " / >
7
< e C l a s s i f i e r s x s i : t y p e = " e c o r e : E C l a s s " name= " Car " >
< e S t r u c t u r a l F e a t u r e s x s i : t y p e = " e c o r e : E A t t r i b u t e " name= " c o l o r " upperBound = " 1 "
9
eType = " e c o r e : E D a t a T y p e h t t p : / / www. e c l i p s e . o r g / emf / 2 0 0 2 / E c o r e # / / E S t r i n g " / >
</ eClassifiers>
11
< e C l a s s i f i e r s x s i : t y p e = " e c o r e : E C l a s s " name= " C a r P o o l " >
< e S t r u c t u r a l F e a t u r e s x s i : t y p e = " e c o r e : E R e f e r e n c e " name= " c a r s " upperBound = "−1" eType = " / / Car " / >
13
</ eClassifiers>
</ ecore:EPackage>
Listing 4.2: Carpool metamodel as XMI
propriated Python classes for each metamodel. These classes can then be instantiated to
create a model, can be loaded from an ECore XMI file or saved to it. A modified ST
template engine is used for generation of the Python classes.
To ameliorate the integration, Frodo hooks on the standard Python module import facilities and allows the direct import of ECore metamodels. During the process of importing
the corresponding classes are generated and loaded. Its use is optional.
Example
The following paragraph illustrates the Frodo ECore components using a simple example.
At first an ECore metamodel for a car pool that can contain cars and some car types is
created (listing 4.1). This is done in Python but can be produced with any other tools
supporting ECore, too. The metamodel is serialized into XMI shown in listing 4.2. The
way the autoimport works and creates a model conforming the metamodel is demonstrated
in listing 4.3.
In this oversimplified example only a subset of the features is shown. For example
loading or inter-metamodel referencing are possible, but not demonstrated. Finally listing
4.4 show the XMI representation of the model created before.
import e c o r e . a u t o i m p o r t
2 from c a r s import ∗
4 myCarPool = C a r P o o l ( name= " M a r t i n s C a r s ! " )
myCarPool . c a r s = [ R o l l s R o y c e ( c o l o r = " b l a c k " ) , Mini ( c o l o r = " g r e e n " ) ]
6
from e c o r e . s t o r a g e . dumper import dump
8 dump ( " myCars . c a r p o o l " , myCarPool )
Listing 4.3: Auto importing the carpool metamodel and creating a model
60
1 <? xml v e r s i o n = " 1 . 0 " e n c o d i n g = "UTF−8" ? >
< c a r p o o l : C a r P o o l x m l n s : c a r p o o l = " h t t p : / / k a r l s c h . com / C a r P o o l "
3
x m l n s : x m i = " h t t p : / / www. omg . o r g / XMI"
x m l n s : x s i = " h t t p : / / www. w3 . o r g / 2 0 0 1 / XMLSchema−i n s t a n c e "
5
x m i : v e r s i o n =" 2.0 "
name= " M a r t i n s C a r s ! " >
7
< c a r s x s i : t y p e =" Cars:RollsRoyce " c o l o r =" black " / >
< c a r s x s i : t y p e =" Cars:Mini " c o l o r =" green " / >
9 </ carpool:CarPool>
Listing 4.4: XMI representation of a carpool model
4.4.3
Grammar / Syntax View Handling
Frodo is employs the ANTLR v2 parser generator (LL(k)) which is able to generate parsers
in Python. The grammars used by the generator can be transformed into a grammar model
and a grammar model into an ANTLR grammar.
Grammar Metamodel
The grammar metamodel can be used to describe grammars. It has been developed manually in an ECore aware modeling tool. Contrary to the proposed one the grammar metamodel is not completely tool independent but contains some ANTLR specific elements at
the moment. The reason is the incremental development of the framework.. Nevertheless
some model refactoring can improved that.
Figure 4.3 shows an excerpt of the metamodel. Important is the definition of rules and
of what elements a rule can consist of. The picture does not contain the set of operations
each element owns because they are currently mixed in at runtime . A number of methods
for manipulating a grammar are available: finding and modifying rules, annotating rule
elements with actions or labels and replacing/deleting elements.
Grammar Parser
Creating a model from an existing parser grammar enables reuse. So Frodo contains a
parser built with ANTLR which can convert an ANTLR grammar into a grammar model.
The attributed parser grammar contains actions to instantiate elements from the grammar
metamodel. These actions are written in Python and use the Frodo ECore implementation.
After reading the grammar ANTLR generates a parser in Python. This parser is used for
every ANTLR compatible grammar.
Currently a various grammars are available for reuse, ranging from C, Java, C# over
SDL, SQL and Eiffel up to Verilog and HTML [1]. The mentioned grammars have been
tested with Frodo and all of them can be successfully transferred into a grammar model.
Grammar Generator
To create an ANTLR grammar from a grammar model, a modified ST engine is used.
Currently the generative template is written manually and consists of a set of rules. Each
rule is a production for a certain model element. The rules are invoked top-down and the
Figure 4.3: Excerpt from Frodo grammar metamodel
61
62
top rule is parametrized by the top-level grammar model element and generates a complete
grammar. Listing 4.5 gives an example of the grammar generation template used .
A generative rule has a name and one or multiple parameters. The template associated
with the rule is delimited by double angle brackets. In this case, the rule parameter corresponds directly to a model element. Inside the template the placeholders inside single angle
brackets are replaced by the values they refer to for simple placeholders, or by the results
of the calculation for complex placeholders such as conditionals or subrule calls. Subrule
calls can take parameters or can be appended to a collection and are then called for every
element in the collection (this happens in block.alternatives handling).
1 B l o c k ( b l o c k ) : : = <<
<block . a l t e r n a t i v e s : A l t e r n a t i v ( ) ; s e p a r a t o r =" | ">
3 >>
5
R u l e ( r u l e ) : : = <<
7 <rule . access>
<rule . id> < i f ( rule . returnAction )> returns [<rule . returnAction >]<endif>
9
< i f ( r u l e . o p t i o n s ) >< O p t i o n s ( r u l e ) >< e n d i f >
< i f ( r u l e . i n i t A c t i o n ) >{< r u l e . i n i t A c t i o n >}< e n d i f >
11
:
<Block ( r u l e . block ) >
13
;
>>
Listing 4.5: Excerpt from the generative grammar for ANTLR grammars
Parser and Generator generation
A DSL parser is generated with ANTLR from an ANTLR grammar file. The grammar
file is produced with the grammar generator for a grammar model. Currently ANTLR is
the only supported parser generator. After the generation process the parser can be used
immediately to recognize the DSL. A grammar model without actions does not construct
the model elements described in the domain metamodel.
Two possibilities described earlier in this thesis solve this problem. The current version
of Frodo supports the explicit definition of creational actions inside the grammar metamodel. Similar to the grammar parser these actions are by default written in Python, as the
generated Parser is Python, leveraging the ECore implementation. It is possible to retarget
ANTLR to generate a Java, C++ or C# parser. Afterwards the actions have to be defined in
the matching target language. The only precondition is the ability to import ECore models.
This explicit approach allows a very flexible mapping of the language elements to model
elements, but comes along with some drawbacks. Firstly, if the parser is retargetted, the
actions have to be converted manually to the new target language. Secondly, an additional
development effort and a good understanding of the grammar model is need. Moreover
the grammar model is not target independent anymore. Finally, the goal described in 4.3.2
of generating a textual syntax generator or a fitting generative grammar is impossible, as
Frodo would have to analyze the custom action and interpret their meaning.
To solve this problem, the implicit approach is needed. Frodo has only experimental
support for an implicit mapping by conventions and some additional information have to
be defined manually. The actions to create model elements and populate them are then
generated automatically. At the moment only automatic generation of Python actions is
supported. A description of the conventions can be found inside the framework documentation.
63
Automatic generation of the textual representation from a domain metamodel instance
is currently not implemented. As already mentioned it is impossible for grammar models
with explicit actions. Nevertheless a generative grammar can be defined in ST. To enable
the automatic generative grammar generation for implicit grammar models a preliminary
metamodel to describe ST generative grammars was developed accompanied by a transformation for the grammar metamodel to the ST metamodel. This step corresponds to the
proposed inversion of a grammar.
To simplify the whole process a separate DSL which is able to serve as a foundation for
the generation of a parser and a generative grammar was suggested in 4.3.1. Initial work
was done, but deferred for later research due to reasons of complexity.
4.4.4
Transformation Engine
ATL is elected as transformation engine because transformation is an important functionality in Frodo. With ATL it gets possible to perform model-to-model transformation of
ECore models. ATL is similar to the QVT proposal, is opensource and compared to other
transformation tools superior in functionality. It is written in Java, so a JVM is needed to
execute the ATL transformations included in Frodo.
Additionally it is possible to transform models with Python by directly manipulating
objects in memory. At the moment no special API is included in Frodo for this task.
Practical experience showed, that it is sometimes easier to modify models in Python than
with ATL. Nevertheless for complex transformations from one metamodel to another, ATL
is recommended.
An example how transformations can look alike is found in 5.1.2 and 5.2.3. A large
number of reusable transformations is available for the Atlantic transformation zoo [5] .
4.4.5
Plugin Generation
The plugin generator creates an editor and a debugging plugin for the Eclipse platform
based on a ST template that is filled with information from the grammar metamodel. While
the editor is usable, the debugger is unfinished.
The editor derives from the AbstractUI class provided by Eclipse and implements a
simple keyword highlighting. Therefore a listener listens to changes made inside the document and reparses it on every change in a separate thread. After parsing the resulting tokens
are colored according to the grammar model and additional color information specified in
another template.
The debugger takes ideas from the Eclipse DDF but also from general available Eclipse
documentation and is suited for imperative languages. A plugin is generated which communicates with the debug target via TCP. Out of this reason the back-end has to implement
a debug server understanding at least the following commands: set and clear breakpoints,
start, stop, resume and step. On the side of the plugin several Eclipse Debugging Platform
interfaces are implemented: ILauch, IDebugTarget, IProcess, IDebugElement, IVariable,
IValue, IBreakpoint and several others. The IDebugTarget implementation is the central
64
place where communication with the back-end takes place. Currently only breakpoints by
line can be set. The work on the debugger generation was not finished.
4.4.6
Implementation problems and issues
ModelBus
A ModelBus connection is not implemented, and all exchange with external tools is done
via XMI. This is due to the fact that no matured bus implementation is available by now.
The Model Bus Integrated Environment for Eclipse [28] is only available as a prototype.
The same is true for ModelCVS [112] which applies similarity matching to automatically
derive transformation between metamodels.
ECore
The current ECore implementation inside Frodo suffers shows some problems. Some are
directly related to the implementation, others to the ECore XMI format.
• When loading a model from an ECore XMI file, the top-level model element has to
be defined manually. This is only a minor issue and can be resolved by implementing
a function which checks the root XMI element, loads the metamodel automatically
and instantiates the top level model element.
• Not all ECore operations are implemented. Particularly methods allowing to query
information redundantly stored in queryable attributes are missing, for example the
getFeatureCount method of EClass that returns the number of features. Another way
for retrieving this information is getting the number of items inside the collection
eAllStructuralFeatures, an attribute of EClass.
• The ECore XMI file does not conform to the XMI standard. That requires the usage
of xmi:type for type definition. Instead xsi:type is utilized. This can be followed back
to the usage of XML Schema inside ECore XMI serialized files. Frodo solves this
problem by using xsi:type.
• Inter-model referencing of elements inside ECore XMI uses a proprietary format.
The Python reimplementation was done by studying the Java source code. Some
sources suggest [7] that part of the link is valid XPath, however this is wrong and can
easily be seen examining some links carefully. This was stated by Ed Merk15 . If the
EMF implementation changes, Frodo has to aligned with it.
15
Ed Merk is one of the EMF ECore API maintainers and states that these links are not XPath, and that
ECore will not support XPath in the near future.
“XPath is no doubt far more complex than ECore itself, so it seems like
kind of a big boat anchor for our little speedboat.”
- It will not be used.
(http://dev.eclipse.org/newslists/news.eclipse.tools.emf/msg10055.html)
65
• Using the EMF ECore classes and tools sometime an invalid ECore metametamodel
is referenced (../../org.eclipse.emf.ecore/model/Ecore.ecore). Usually the model is
identified by http://www.eclipse.org/emf/2002/Ecore. This problem ia solved by allowing to map arbitrary URI to the correct model inside the ModelManager.
ANTLR
ANTLR proved to by very stable during the development of Frodo, nevertheless some flaws
like possible deadlocks were discovered in the ANTLR Python runtime, which is used by
generated Python parsers. In consequence Frodo includes a fixed ANTLR runtime.
ATL
It is of great impact that model transformations are done correctly. Some problems were
encountered while using ATL . The latest available binary release targets Eclipse 3.1. This
allows the correct execution of transformations used inside Frodo. Using this version of the
ATL plugin with Eclipse 3.2 (current stable version) leads to wrong transformation results
due to changes in EMF used by ATL. As ATL can currently not be used without Eclipse
the usage of Eclipse 3.2 is not advised in combination with the official ATL release. A
solution for this problem is the utilization of ATL 2006, to day only available from the
Eclipse GMT CVS16 source repository. It has to be compiled and installed manually, but
executes all transformations correctly.
16
http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gmt/ATL/?root=
Technology_Project
66
Chapter 5
Examples
“Example is not the main thing in influencing others. It is the only thing.”1
To illustrate the practical usage of Frodo two examples are given. A simple state machine language and a complex test automation language based on
previous work inside the automotive industry.
To give a guideline how Frodo can be used some examples are helpful. From a merely
theoretical viewpoint, it may not be possible to derive the steps needed to create working solutions with the framework. One simple example demonstrates the basic usages and
implements a state machine language with two back-ends. To show that more sophisticated solutions are realizable with Frodo, a test automation language based on Python is
developed in multiple steps.
5.1
A simple state machine language
The design and implementation of a new DSL using Frodo, should be done in several steps
. At first the domain metamodel should be defined, followed by one or more syntax views.
Finally the back-end can be plugged in. A minimal finite state machine (FSM) DSL will
be developed in the next paragraphs.
5.1.1
Design
At first it is important to define a suitable domain model. It consists of states and transitions
between these states. Each state has a name and a number of possible transitions. A
transition can fire if a certain input value is given. If it fires, an output associated with the
transition is displayed. As the state machine should be finite, it needs a start and a stop
state. Finally the machine itself is named. Figure 5.1 shows one possible domain model for
these requirements, modeled in ECore.
1
Albert Schweizer
67
Figure 5.1: Finite state machine domain model
After the domain model is finished, a adequate syntax is developed. The language
should work with blocks, known from C or Java. Each state is represented by an own
block, delimited by curly brackets. The state block contains the transitions associated with
a target state, an input condition and some output. Furthermore the start and the stop state
should be marked. Listing 5.1 shows an example of a possible language, defining three
state including transitions. This example represents the state machine depicted in figure
5.2.
2
4
6
8
10
12
s t a t e m a c h i n e Demo {
start state A {
g o t o B on " b " p r i n t " u " ;
g o t o C on " c " p r i n t " v " ;
}
state B {
g o t o C on " c " p r i n t " x " ;
g o t o A on " a " p r i n t " y " ;
}
stop s t a t e C {
}
}
Listing 5.1: Example of FSM DSL (demo.fsm).
a/y
b/u
A
c/x
B
C
c/v
Figure 5.2: Example of a finite state machine
68
From this listing and the previous considerations a grammar can be derived as proposed
in listing 5.2.
To complete the design it is decided to target two back-ends: first an interpreter for the
model developed in Python and second a C code generator. The C should be compilable
afterwards.
Now after these steps realizing the implementation with the help of Frodo is presented.
It should be mentioned that a FSM can be more effectively represented visually but this
example serves merely for the demonstration of Frodo.
StateMachine
State
Transition
4 Identifier
String
2
:=
:=
:=
:=
:=
’ statemachine ’ I d e n t i f i e r ’{ ’ State∗ ’} ’ ;
( ’ s t a r t ’ | ’ stop ’ )?
’ s t a t e ’ I d e n t i f i e r ’{ ’ Transition∗ ’} ’ ;
’ g o t o ’ I d e n t i f i e r ’ on ’ S t r i n g ’ p r i n t ’ S t r i n g ’ ; ’ ;
( ’ a ’ . . ’ z ’ | ’A ’ . . ’Z ’ | ’ _ ’ | ’ 0 ’ . . ’ 9 ’ )∗
’ " ’ (~( ’ " ’ | ’ \ r ’ | ’ \ n ’ ))∗
’" ’ ;
Listing 5.2: Grammar of the FSM DSL as EBNF
5.1.2
Implementation
For the implementation of the proposed DSL different steps are necessary. The first is already done, as the FSM metamodel has already been created in the design phase. Also a
possible grammar for the DSL has been proposed and serves as a base for the following
Frodo syntax view specification. As Frodo derives a parser from a grammar model, currently two ways to create the FSM parser are feasible. At first the grammar model can be
created directly in any modeling tool supporting ECore. The second approach is defining
an ANTLR grammar which is in turn converted to a grammar model. In listing 5.3 the
result of using the second possibility is shown. For the resulting grammar model, the mapping to the FSM domain model is done implicitly. It is sufficient to specify the domain
model, a starting rule inside the grammar and attribute mapping hints.
1 s t a t e M a c h i n e : " s t a t e m a c h i n e " name:IDENTIFIER LCURLY s t a t e s RCURLY ;
s t a t e s : ( s t a t e )∗ ;
3 s t a t e : ( s t a r t S t a t e : " s t a r t " | s t o p S t a t e : " s t o p " ) ? " s t a t e " name:IDENTIFIER LCURLY t r a n s i t i o n s RCURLY;
t r a n s i t i o n s : ( t r a n s i t i o n )∗;
5 t r a n s i t i o n : " g o t o " to:IDENTIFIER " on " input:STRING " p r i n t " output:STRING COLON;
Listing 5.3: The FSM parser grammar
The tokens IDENTIFIER, STRING, LCURLY, RCURLY and COLON are provided by
a standard lexer grammar model. Though the lexer can easily be exchanged or defined
within the FSM grammar model. Additional rules for skipping whitespaces and comments
are found there. For the FSM DSL the lexer rules are slightly modified as shown in listing
5.4.
The mapping rules are always looked up inside the context of the current rule and model
element. For example the model element StateMachine owns an attribute name and name
is looked up in the stateMachine rule. If no mapping could be found this way (for example
as with startState and stopState of the StateMachine) the rule associated with the model
element type under consideration is examined (in this case the state rule).
To allow referencing of other model elements (such as the to property of Transition)
from within the grammar model an additional hint, currently in the form of a model annotation, is given. For the FSM example the name of a State owns an annotation declaring
69
the name as unique to identify a certain State. In figure 5.1 the collapsed Frodo annotation
contains this hint (TCSSL solves this by extra seek rules inside the grammar).
The implicit mapping happens after the grammar (in listing 5.3) is transformed into a
grammar model. A transformation that has knowledge about the domain model and the
grammar model creates the actions inside the grammar model needed to create the correct
domain model elements. From this final grammar model the parser and a simple editor are
generated (figure 5.3). Invoking the parser returns a FSM model for further use.
As required in the design a Python interpreter should be able to interpret the given
model. Listing 5.5 shows a very rudimentary interpreter (error handling is left out for
simplicity). The C back-end is more complicated because in C no facilities for runtime
ECore model access are available. The machine should be directly compiled. A generative
grammar is therefore defined (listing 5.7), creating a C language compatible representation
of the model.
Beyond the developed back-ends the model created from a DSL code fragment can
be used in any other tools supporting XMI (listing 5.6 shows the XMI representation of
demo.fsm).
It should be mentioned that in a real life DSL more aspects have to be considered.
Model checking is unavoidable. In this example it is possible to create a FSM without a
start state or generate C code containing deadlocks. However this is beyond the scope of
this example. Also no debugging support is generated using a simple generative grammar.
The back-end needs to explicitly offer a debugging interface.
1 LCURLY : ’ { ’ ;
RCURLY : ’ } ’ ;
3 COLON : " ; " ;
# $ s k i p p r e v e n t s t h e l e x e r from g e n e r a t i n g a t o k e n , $ n e w l i n e i n c r e a s e s t h e l i n e number
5 COMMENT : ’ # ’ ( ~ ’ \ n ’ )∗ {$ s k i p } ;
WHITESPACE : ( ( ’ \ 0 0 3 ’ . . ’ \ 0 1 0 ’ | ’ \ t ’ | ’ \ 0 1 3 ’ | ’ \ f ’ | ’ \ 0 1 6 ’ . . ’ \ 0 3 7 ’ | ’ \ 1 7 7 ’ . . ’ \ 3 7 7 ’ | ’ ’ )
7
| " \ r \ n " {$ n e w l i n e }
| ( ’ \ n ’ | ’ \ r ’ ) {$ n e w l i n e }
9
) {$ s k i p } ;
# t h e e x c l a m a t i o n mark p r e v e n t s t h e i n c l u s i o n o f a c h a r a c t e r i n t o t h e t o k e n t e x t
11 STRING : ’ " ’ ! ( ~ ( ’ " ’ | ’ \ r ’ | ’ \ n ’ ) ) ∗
’" ’! ;
IDENTIFIER : ( ’ a ’ . . ’ z ’ | ’A ’ . . ’Z ’ | ’ _ ’ | ’ 0 ’ . . ’ 9 ’ ) ∗ ;
Listing 5.4: The FSM lexer grammar
70
i m p o r t fsm . grammar . p a r s e r # i m p o r t t h e g e n e r a t e d p a r s e r
2 from u t i l i m p o r t f i n d
4 def run ( s t a t e m a c h i n e ) :
p r i n t " Running s t a t e m a c h i n e : " , s t a t e m a c h i n e . name
6
currentState = statemachine . s t a r t S t a t e
print statemachine . stopState
8
w h i l e n o t c u r r e n t S t a t e == s t a t e m a c h i n e . s t o p S t a t e :
p r i n t " I n s t a t e " , c u r r e n t S t a t e . name
10
i n p u t = r a w _ i n p u t ( ) . s t r i p ( ) # r e a d i n p u t and c l e a n i t
t r a n s i t i o n = f i n d ( lambda t : t . i n p u t == i n p u t , c u r r e n t S t a t e . t r a n s i t i o n s )
12
if transition:
# found a t r a n s i t i o n
14
print t r a n s i t i o n . output
curre ntState = t r a n s i t i o n . to
16
else::
p r i n t " I n p u t was i n v a l i d . "
18
r u n ( fsm . p a r s e r . p a r s e ( " demo . fsm " ) )
Listing 5.5: A simple FSM interpreter in Python, interpreting demo.fsm
1 <? xml v e r s i o n = " 1 . 0 " e n c o d i n g = "UTF−8" ? >
< f s m : S t a t e M a c h i n e x m l n s : x s i = " h t t p : / / www. w3 . o r g / 2 0 0 1 / XMLSchema−i n s t a n c e "
3
x m l n s : f s m = " h t t p : / / k a r l s c h . com / fsm " x m l n s : x m i = " h t t p : / / www. omg . o r g / XMI"
x m i : v e r s i o n =" 2.0 "
5
name= "Demo"
s t a r t S t a t e =" / / @states . 0 "
7
s t o p S t a t e =" / / @states . 2 ">
< s t a t e s name= "A" >
9
< t r a n s i t i o n s i n p u t ="b" o u t p u t ="u" to =" / / @states . 1 " / >
< t r a n s i t i o n s i n p u t =" c " o u t p u t ="v" to =" / / @states . 2 " / >
11
</ states>
< s t a t e s name= "B" >
13
< t r a n s i t i o n s i n p u t =" c " o u t p u t ="x" to =" / / @states . 2 " / >
< t r a n s i t i o n s i n p u t =" a " o u t p u t ="y" to =" / / @states . 0 " / >
15
</ states>
< s t a t e s name= "C" / >
17 < / f s m : S t a t e M a c h i n e >
Listing 5.6: The XMI representation of demo.fsm
Figure 5.3: Screenshot of the FSM editor
71
1 g r o u p fsm ;
3
5
7
9
11
13
15
17
19
21
T r a n s i t i o n ( t r a n s i t i o n ) : : = <<
i f ( ! strcmp ( input , "< t r a n s i t i o n . input >" ) )
{
p r i n t f ( " Going t o s t a t e < t r a n s i t i o n . t o . name > , o u t p u t : < t r a n s i t i o n . o u t p u t > \ \ n " ) ;
r e t u r n &S t a t e _ < t r a n s i t i o n . t o . name> ;
}
>>
S t a t e ( s t a t e ) : : = <<
p S t a t e S t a t e _ < s t a t e . name> ( ) {
v o i d ∗ s e l f = &S t a t e _ < s t a t e . name> ;
p r i n t f ( " I n s t a t e < s t a t e . name > \ \ n I n p u t : " ) ;
s c a n f ( "%20s " , i n p u t , 2 1 ) ;
< s t a t e . t r a n s i t i o n s : T r a n s i t i o n ( ) ; s e p a r a t o r =" \ n">
p r i n t f ( " Invalid input ! \ \ n" ) ;
r e t u r n &S t a t e _ < s t a t e . name> ;
}
>>
S t a t e F o r w a r d ( s t a t e ) : : = <<
p S t a t e S t a t e _ < s t a t e . name> ( ) ;
>>
23
Main ( s t a t e m a c h i n e ) : : = <<
25 # i n c l u d e \ < s t d i o . h \ >
27 t y p e d e f v o i d ∗ (∗ p S t a t e ) ( ) ;
char input [ 2 1 ] ;
29
< s t a t e m a c h i n e . s t a t e s : S t a t e F o r w a r d ( ) ; s e p a r a t o r =" \ n">
31
< s t a t e m a c h i n e . s t a t e s : S t a t e ( ) ; s e p a r a t o r =" \ n">
33
v o i d main ( ) {
35
p S t a t e n e x t S t a t e = &S t a t e _ < s t a t e m a c h i n e . s t a r t S t a t e . name> ;
p S t a t e s t o p S t a t e = &S t a t e _ < s t a t e m a c h i n e . s t o p S t a t e . name> ;
37
p r i n t f ( " Running S t a t e m a c h i n e Demo \ \ n " ) ;
while (1) {
39
i f ( n e x t S t a t e && ( n e x t S t a t e ! = s t o p S t a t e ) )
nextState = ( pState ) nextState ( ) ;
41
else
break ;
43
}
}
Listing 5.7: A generative grammar, producing a C back-end.
5.2
Test automation language
“Testing can only reveal the presence of errors, never their absence.”2
This section starts with presenting the existing test automation framework
and its Microsoft Excel language. Then the problems caused by the current
approach are highlighted. After that an improved design based on language
embedding and extension is depicted. This design is afterwards implemented
with the help of Frodo.
5.2.1
Introduction
With growing complexity, the testing needed is going to increase exponentially. Different
system parts have to be tested on their own but also while integrated into one complete
2
E.W. Dijkstra [61]
72
system. In reality this is complicated. Product lines often share a large number of common parts but differ subtly. Furthermore the soft and the hardware version of components
changes steadily, sometimes causing new incompatibilities. All these components have to
be tested to ensure their quality. For example infotainment and telematic systems inside a
modern car require the cooperation and interchange among systems like CD charger, mobile phone, navigation system, video assistants, speech recognition and combi-instruments3
by different buses.
The manual testing of such systems is not feasible anymore. Single component4 tests
and even integration tests have to be automated. The automotive industry does not fulfill
this requirement in all of the cases [36]. Still, a large number of automation solutions exists
today (like AutomationDesk5 or ECU-Test6 ), even at the BMW Group multiple solutions
are in use depending on the concrete domain. To unify some of them, at least inside the
BMW Group Hardware/Software Platforms, Methods and Tools division, the Python Test
Automation Framework (PyTAF) was developed. It contains a DSL reimplemented with
the help of Frodo.
5.2.2
The current system
A test automation framework
For PyTAF an extensive domain analysis was performed and all test automation requirements for automotive infotainment and telematic related systems were gathered. This analysis led to a domain model serving as foundation for the framework.
The framework itself was developed in Python. This had multiple reasons. First, the
language is easy to learn and to read compared to C for example. That eases maintenance.
In addition the effort needed to change parts of a system is lower than for an equal system developed in C. Second, Python offers are large and stable standard library and can
access existing C libraries with minimal effort. This is important because a large number of used libraries for accessing hardware components are written in C. Finally several
other approaches [80, 122, 157] have shown that Python is suited to develop automation
solutions.
The framework allows the execution of a test sequence, which is specified in a TestDSL.
At first the TestDSL is translated into a Python program that uses PyTAF. The resulting
sequence is run on a test host (a Python interpreter on a computer with a Windows operating
system). In most cases special hardware is attached to the computer. This is the target of
the test7 or contains functionality needed by the test.
PyTAF is built upon three parts: the Central Controlling Unit (CCU), Services and
TestDSL infrastructure.
3
Displays speed, gear, revolution ...
In the automotive domain often the term ECU (embedded control unit) is used for such hardware components.
5
http://www.dspace.de/ww/de/gmb/home/products/sw/expsoft/automdesk.cfm
6
http://www.tracetronic.de/produkte/ecutest.html
7
Sometimes called SUT, System-Under-Test
4
73
Central Controlling Unit The CCU is the core of the framework. It contains the runtime
facilities to execute a test sequence. This includes logging and reporting, an error
manager and a service manager which manages the orchestration of the available
services. The service manager starts each service in a separate thread and provides
a proxy to all the service users (the test sequence or another service). This proxy
asychronizes every service call. Decoupling the service execution from the sequence
and other services allows a robust event processing and solves the problem of long
running service tasks stalling the complete system. Synchronization is done by synchronized queues used for communication in general and event notification. This
approach follows roughly the active object pattern[153].
Services A service offers functionality to a test sequence. This functionality could be
low level library calls for controlling an attached device. An example is sending
a message over the CAN bus, being attached to the host executing the sequence.
The second option is a software-only service, for example an OCR module. Finally
a meta service implementing functionality on top of one or more other services is
imaginable, like the MMI8 service leveraging the ZBE9 that uses the CAN service
for communication10 . At the moment a growing number of services are available
(approximately 50) from CAN or MOST bus control up to image and text recognition. The service can consist of a driver wrapping low level functionality (for example C libraries), basic functions, main service functions and exposed events. Basic
functions and service functions differ in, that basic functions are always executed
synchronously.
DSL infrastructure The DSL infrastructure consists of three parts. A domain model11 ,
a front- and a back-end. The domain model serves as an input language agnostic
representation of a test sequence. The front-end is used to create a model instance
from an input language. Finally the back-end generates a test sequence (Python
code) orchestrating the framework and its services. Only one front-end is currently
available. It parses a test specification done in Microsoft Excel and creates a domain
model instance.
Excel test automation language
The Excel language offers different language constructs and options. It is possible to start
the test out of Excel via a custom macro that invokes Python/PyTAF. Furthermore a test
8
Multi Media Interface
“Zentrale Bedien-Einheit” - central control unit,
10
The MMI service for example offers a function for detecting weather the MMI is currently inside the
navigation menu. This is not done by relying on a MMI internal data structure but by capturing an image
of the screen. This image is analyzed, with the help of the OCR service. If the current screen is not the
navigation menu, the MMI service can navigate to it automatically by instructing the ZBE service with a
navigation sequence. The ZBE then uses the CAN service to turn this sequence into actual messages send
over the bus.
11
sometimes also information or data model.
9
74
R
Excel
execute test
Testsequence in Excel Sheet
R
PyTAF
Frontend
Central Control Unit
Excel Parser
Data
model
R
Backend
Services
Testcase
Generated
Python
(generated Python)
Datamodel to
Python Transformer
R
Libraries
Figure 5.4: Current PyTAF architecture (FMC)
can span multiple sheets and workbooks and can be parametrized by internal or external
parameters (an INI or YAML file). The most important constructs are:
• a comment cell row, B 12
• a comments command, commenting out an entire row if set inside the cell row, A
• an Errorhandler inside cell row, S which is executed if a command like ASSERTTRUE fails. The type is given in row, S. A subsequence can be called by specifying
its type inside row, U (SHEET, FILE, FILESHEET) and parametrized by row, V .
Also the number of repetition for the handler can be set in row, T . Different Errorhandlers are available:
◦ CONTINUE - continues execution of the test
◦ LOG_AND_IGNORE - logs the error but continues the execution
◦ HALT_AND_WAIT - stops the execution and put all services on hold
◦ SHUTDOWN_AND_RESTART - restarts the test
◦ SHUTDOWN - terminates the test
12
row is the current row inside Excel, A − Z stands for the column and value(row, column) the value
inside the cell at row and column
75
• An explicit log entry specified in row, Y if the row is executed Commands inside the
column C
◦ SETCONFIG - sets global configuration variables to certain values. The nam-
ing follows a hierarchical schema like toplevel.sublevel.variable : value where
the separating dot stands for the next column starting from column G and the
actual variable and its value are represented in the last column, separated by a
colon.
◦ FILE - includes another workbook at the current row
◦ SHEET - calls another sheet, the call can be parametrized (like a method call)
◦ FILESHEET - calls another sheet of another workbook
◦ SET - setting a variable with its name inside cell row, G to a value inside cell
row, H
◦ QUEUEVARIABLE creates a new queue with the name taken from cell row, G
◦ FOR - a loop from i to j using a step size of s with a loop variable name of
n where n = value(row, G), i = value(row, H), j = value(row, I), s =
value(row, J) and executing the loop body. The loop body starts in the row
following the FOR command beginning with a open bracket and is terminated
by the next matching closing bracket at the same bracket depth.
◦ WHILE - similar to FOR, but executes its body until a condition inside cell
row, G is false
◦ BREAK - terminates a loop
◦ CONTINUE - continues a loop from the start, not executing anything after that
statement
◦ WAIT - waits until a event has happened inside the “wait-body”. This can be a
timeout event created with the TIMEOUT keyword which cancels the wait or
an event from a service enqueued in a queue variable13 . In the later case the
wait is canceled by an IF statement whose condition evaluates to true. A wait
can not be nested.
◦ TIMEOUT - creates a timeout event which is raised after row, G seconds, only
valid inside a WAIT block.
◦ IF/ELSE - creates a conditional block, whose execution depends on the evalua-
tion of the cell row, G
◦ ASSERTTRUE - tests the condition in cell row, G and asserts if the condition
is violated. Control is passed to a Errorhandler if defined.
◦ LOGGER - makes a log entry, type is given inside row, G and the content inside
row, H. The type can be either info, debug, error, warning or critical.
◦ END - the test case has to end with this command
13
Also called a trigger.
76
◦ ServiceName.ServiceFunction - calling a function from a service, where Servi-
ceName is the name of the service and ServiceFunction the name of the function. The parameters are read from cell row, G to cell row, R. This can also be
the enabling or disabling of a trigger/event.
An important fact about the DSL beyond the discussed commands is, that there is no formal
definition how a condition for an IF or ASSERTTRUE command has to look alike. The
same is true for the name/value for SET and in other places. These parts taken as uninterpreted strings from Excel are stored inside the data model uninterpreted, too. The current
implementation simply puts these strings inside the generated Python code.
Figure 5.5: Excel TestDSL example
5.2.3
A better test automation language
In this section a new test automation language will be presented. At first problems of
the earlier described language are identified. Afterwards a solution is discussed. A new
domain specific language will be designed and implemented with the help of Frodo. This
is described in the remaining section.
Problems of the Excel language
The current implementation of the test automation language and its framework suffers from
some drawbacks:
Test representation Excel is not ideally suited for the formulation of a structured test
case as it forces the spreading of semantically coherent information across multiple
77
cells. (For example the definition of FOR, WHILE, and WAIT blocks, or method
parameters). It is easy to introduce misunderstandings or, even worse, bugs due to
formatting or layout changing.
Test specification format Test cases are currently developed within Microsoft Excel. Excel was primarily developed as a spreadsheet application but is used in many other
different domains. However Excel and its storage format are proprietary, therefore to
be able to modify a test case Excel is needed. Format and application are a subjects
of change. Moreover the format is binary which makes versioning and comparison
complicated.
Data model The current data model is incomplete and tightly coupled to Python. Each
data model element class directly contains the Python source code that is used for
generation. Furthermore as already mentioned inside the description of the DSL the
data model stores statements like condition, value assignments or parameter as simple string. No validation is performed and they are simply inserted in the generated
Python code. Therefore they must be valid Python. That may not be obvious to the
test developer who can easily produce errors.
Error reporting and validation If the test case contains a syntactical error, the user has
to interpret the error message of the Python interpreter which interprets the generated
Python code. For a user without experience in Python it may be difficult to backtrack
the error to a specific statement inside the Excel test case. This is due to the fact
that there is a change in the abstraction level from Excel to Python but this is not
reversed for error reporting. Furthermore by now no satisfying validation of the test
specification is performed. Part of the reason lies in the next elaborated point.
Generated code From the Excel specification Python code is generated. As there is no
special debugging facility sometimes this code has to be debugged manually. However it is not as easy readable as a normal Python code because each service call
is wrapped inside the step function which ensures correct error handling and large
repetitive blocks for WAIT instruction processing.
Framework It is not possible to stop or restart a service. Moreover only one service
instance can exist. Starting the same service with different parameters is impossible.
Services are tightly coupled to the service manager.
Tools As Excel is the main language editor no supporting features like code completion,
syntax highlighting or error checking before the execution is available. Also no debugger is available. So a tester has no support for debugging his tests.
Language constructs Different language constructs have drawbacks. The QUEUEVARIABLE command creates a special variable. To query this variable (a queue) implementation and Python knowledge is needed. Another example is the WAIT command.
It is not possible to explicitly wait for more then one event (trigger). Additionally if
an event occurs, the wait is always terminated.
78
Flexibility Deployment and practical use of the DSL and the framework showed that often a complex Python code is embedded inside commands like setting a variable.
Some service calls take or return complex structures like lists or hash tables. This
code is not captured by the data model and moreover is not checked. Nonetheless
there seems to be a need for more complex statements and functionality like lists or
mathematical functions.
This can be summarized in the following statements:
• The TestDSL language is practically just powerful enough with embedded Python.
• The current specification of the data model is lacking of important properties.
• Error checking and reporting capabilities are improvable.
• There is no good reason to use Excel as test specification environment despite an
easier parser implementation. A better editor and a debugger would be useful.
Solution approach
A number of solutions are possible to solve the discussed problem. It was decided to
develop a new domain specific language which orchestrates the existing framework but
does not suffer from the described problems. Not all facts are known to the author but
other possibilities like using TTCN3 were discarded for reasons such as its only commercial
availability, vendor lock-in, invested effort into the existing framework which is difficult to
plug into TTCN3 easily or bad read,write and maintainability of TTCN3 [17].
Design
The first step is to drop Excel and replace it by a plain textual notation. This solves problems with versioning, comparability and vendor lock-in.
The design of the language and the implementation strategy evolved in different iterations. An earlier approach considered the development of a completely new DSL including
elements of the current language. The idea was overthrown because the language elements
were far away from being complete for defining a usable language. A replacement syntax
for parts which are by now defined in Python would have to be specified. This would have
lead to a reinvention of expression and condition syntax.
The second idea was based on the principle of embedding discussed in 2.1.8. Some
examples of successfully embedding a language in Python exist [14, 56]. Using available
Python constructs the special test functionality should be represented. This approach was
more successful as a large number of test language constructs could be represented with a
clever usage of the Python language. Different changes were applied to the framework to
support an embedded test language.
• The Step function was removed and its functionality like error handling and event
queue checking is implemented in an service call proxy. A service call looks like a
simple method call.
79
• The use of the Service Manager is now optional and a service does not depend on it
anymore. It can be utilized without the complete framework. Moreover it is possible
to import services directly with a Python import statement.
• Services are made easily start- and stoppable by a single method call. Currently
enabled triggers are frozen while a service is stopped and resumed on restart. Furthermore different instances of a service can exist while it is parametrized at startup.
• Explicit instantiation of queuevariables was removed as queuevariables are always
bound to a trigger. Creating a trigger now always return a queuevariable. It is still
possible to attach a queuevariable to an existing trigger.
• An often used construct was the wait statement14 accompanied by a single timeout
event. The intended behavior was like a simple system sleep parametrized by the
amount of time. Thus a new sleep command is introduced capturing the semantic in
a more compact notation.
• The wait construct was extended. Now it is possible to have a wait not terminated by
the first occurring event. It can be exactly stated which events have to occur before
a wait terminates. They can be connected by logic statements. Also guarding events
with conditions can be accomplished easily.
In listing 5.8 an excerpt from generated Python code is shown followed by listing 5.9
showing the embedded approach. Moreover the second listing has more functionality than
the former. For demonstration a simple string is printed and a trigger which can recognize
a crack on the audio line-in of a sound card is started.
However this approach suffers from some of the typical problems encountered using
embedding. Not all of the constructs can be mapped in s satisfying way to Python constructs, especially those whose concepts are very different from concepts offered in Python
(wait). Also it is difficult to modify the error handling and to process exceptions invisibly
to the DSL user. Finally the Python environment does not know about the test constructs
and treats them like ordinary Python code, which is not always the intended behavior (in
the editor for example),
The final solution is developed with the help of Frodo. As the goal was not to reinvent
everything, the new language is based on the results of the embedding step. To support
domain specific constructs directly inside Python, Python has to be modified. The idea is
reusing the Python grammar and transforming it into a grammar model. Subsequently the
grammar model is modified by injecting test specific constructs. These test specific constructs are mapped to an extended domain model. But the back-end is still the modified
PyTAF framework. It does not understand the extended domain model. So a semantic
mapping from test specific constructs to Python constructs and to PyTAF calls is defined.
To execute a test case, its textual representation is parsed and the domain model is created, which in turn id transformed into a Python model. The Python model could then be
executed on top of Python and the PyTAF framework.
14
An event loop with a timeout.
80
import s y s , os , t h r e a d i n g
2 sys . p a t h . append ( r ’ . . ’ )
import random
4 from c c u import ∗
from c c u . Check import ∗
6 from c c u . E x p r e s s i o n s import ∗
from t h i r d p a r t y e x t e n s i o n s import ∗
8 from u t i l import ∗
10 c l a s s T e s t S e q u e n c e ( T e s t S e q u e n c e B a s e . T e s t S e q u e n c e B a s e ) :
def _ _ i n i t _ _ ( s e l f , oServiceManager ) :
12
s e l f . AudioService = oServiceManager . g e t S e r v i c e ( ’ AudioService ’ )
s e l f . crack = B a s e F u n c t i o n a l i t y . QueueVariable ( )
14
TestSequenceBase . TestSequenceBase . _ _ i n i t _ _ ( s e l f , oServiceManager )
16
def runTestSequence ( s e l f ) :
s e l f . s t e p ( funcAny = s e l f . A u d i o S e r v i c e . say , i L o g L i n e =None , o E r r o r H a n d l e r C l a s s =None ) ( ( " h e l l o w o r l d " , ) )
18
20
22
24
26
28
s e l f . bWaiting = True
s e l f . bTimeOut = F a l s e
t i m e r = t h r e a d i n g . Timer ( 5 , s e l f . t i m e o u t )
timer . s t a r t ()
while s e l f . bWaiting :
i f s e l f . bTimeOut :
s e l f . bWaiting = False
i f s e l f . bWaiting :
s e l f . _wait ( )
s e l f . checkQueue ( )
30
32
s e l f . s t e p ( funcAny = s e l f . A u d i o S e r v i c e . say , i L o g L i n e =None , o E r r o r H a n d l e r C l a s s =None ) ( ( " a f t e r 5 s e c o n d s " , ) )
s e l f . s t e p ( funcAny = s e l f . Audio . E n a b l e T r i g g e r , i L o g L i n e =None , o E r r o r H a n d l e r C l a s s =None )
( ( s e l f . c r a c k , " 2000 h z T r i g g e r " , " C r a c k T r i g g e r " , "ONCE" , 2 0 0 0 , ) )
34
36
38
40
42
44
46
48
50
s e l f . bWaiting = True
s e l f . bTimeOut = F a l s e
t i m e r = t h r e a d i n g . Timer ( 2 0 0 , s e l f . t i m e o u t )
timer . s t a r t ()
while s e l f . bWaiting :
i f s e l f . bTimeOut :
s e l f . bWaiting = False
else :
o C o n d i t i o n = C o n d i t i o n ( ’ s e l f . hz . l a s t V a l u e ( ) = = 0 ’ )
i f oCondition . Eval ( g l o b a l s ( ) , l o c a l s ( ) ) :
s e l f . _oLogger . i n f o ( " Crack d e t e c t e d " )
s e l f . bWaiting = False
timer . cancel ()
i f s e l f . bWaiting :
s e l f . _wait ( )
s e l f . checkQueue ( )
52
54
i f __name__== ’ __main__ ’ :
oMain = TAF . TAF ( [ ] , { } )
oMain . Run ( T e s t S e q u e n c e )
Listing 5.8: Example of generated code for TestDSL
These steps allow to define constructs and behavior that could not be easily realized
with the embedded approach.
• A wait construct is integrated into Python. It can wait for the arrival of events which
are enqueued in queuevariables. Logical operations like AND and OR are supported.
Also a guard condition can be formulated. In general the design is similar to the alt
statement in OCCAM or in TTCN3. Listing 5.10 contains an example of this new
statement.
• An easier logging and sleep statement.
• Direct embedding of a subset of YAML as configuration language.
• The user does not need to manually import services anymore. The used services are
derived from the service call syntax.
81
1 from s e r v i c e s . A u d i o S e r v i c e import ∗
3 # s t a r t s t h e a u d i o s e r v i c e w i t h a s c a n n i n g r a t e o f 1 0 0 , and c a l l s a d e m o n s t r a t i o n method
AudioService ( 1 0 0 ) . say ( " h e l l o world " )
5 Sleep (5)
AudioService ( 1 0 0 ) . say ( " a f t e r 5 seconds " )
7 AudioService ( 1 0 0 ) . shutdown ( )
9 # s t a r t s t h e A u d i o s S e r v i c e w i t h o u t p a r a m e t e r s , c a l l s t h e s a y mehtod a s y n c r o u n o u s
a y n c r e s u l t = AudioService . say . async ( " h e l l o world asyncron " )
11
# w a i t s f o r t h e a y n c c a l l t o be f i n i s h e d , a t l e a s t one s e c o n d a t m o s t f i v e s e c o n d s
13 Wait ( Or ( T i m e o u t ( 5 ) , And ( a s y n c r e s u l t , T i m e o u t ( 1 ) ) ) . w a i t ( )
15 A u d i o S e r v i c e . s h u t d o w n ( )
17 # r e s t a r t s t h e s e r v i c e , c r e a t e a t r i g g e r and a s s i g n i t t o t h e h z q u e u e v a r i a b l e
hz = A u d i o S e r v i c e . e n a b l e C y c l i c H z T r i g g e r ( " 200 hz d e t e c t o r " , 2 0 0 )
19
# log the frequence d e t e c t i o n
21 hz . a c t i o n = lambda hz : l o g ( s t r ( hz ) + " 200 hz f r e q u e n z e d e t e c t e d : " + hz )
23 # c r e a t e s a t r i g g e r w h i c h t e r m i n a t e s t h e w a i t
c r a c k = AudioService . e n a b l e C r a c k T r i g g e r ( " Crack d e t e c t o r " , t e r m i n a t e s W a i t = True )
25
# log a detected crack
27 c r a c k . a c t i o n = lambda msg : l o g ( " C r a c k d e t e c t e d ! " )
29 # w a i t s f o r 200 s e c o n d s o r u n t i l a c r a c k i s d e t e c t e d
Wait ( Or ( hz , c r a c k , T i m e o u t ( 2 0 0 ) ) ) . w a i t ( )
Listing 5.9: Example of the embedded Python test language
• No explicit global exception handler is needed anymore. Start over functionality is
now implicitly generated in comparison to the embedding approach.
To get a better understanding of the extensions, the domain metamodel and the grammar
model should be examined.
a y n c r e s u l t = AudioService . say . async ( " h e l l o world asyncron " )
2 wait :
on t i m e o u t 5 :
4
l o g " Async c a l l d i d n o t r e t u r n i n t i m e "
break
6
on a s y n c r e s u l t and t i m e o u t 1 :
l o g " e v e r y t h i n g ok "
8
break
10
# create triggers
12
hz200 = A u d i o S e r v i c e . e n a b l e C y c l i c H z T r i g g e r ( " 200 hz d e t e c t o r " , 2 0 0 )
14 hz300 = A u d i o S e r v i c e . e n a b l e C y c l i c H z T r i g g e r ( " 300 hz d e t e c t o r " , 3 0 0 )
c r a c k = AudioService . e n a b l e C r a c k T r i g g e r ( " Crack d e t e c t o r " )
16
s t a r t t i m e = time . time ( )
18
wait :
20
on c r a c k :
log " Crack d e t e c t e d "
22
break
on t i m e o u t 2 0 0 :
24
l o g "No c r a c k i n 200 s e c o n d s "
break
26
on hz200 and hz300 :
g u a r d t i m e . t i m e ( ) > s t a r t t i m e + 10
28
l o g " 200 hz and 300 hz were d e t e c t e d "
Listing 5.10: Example of the new wait statment
Implementation
To implement the extended Python language (TestDSL), a number of steps are taken. At
first a formal description of its grammar understandable by Frodo is needed. Luckily for
82
Python 2.4 such a formal description exists inside the official source repository15 . Frodo
currently only supports the reading of ANTLR grammars but the provided grammar has a
custom format similar to the EBNF (PyEBNF from now).
2
module p y g 2 a n t l r ;
c r e a t e OUT: AntlrMM from IN : PythonGrammar ;
# AntlrMM = F r o d o Grammar Metamodel
4
r u l e GrammarToParserClass
6 {
from s o u r c e : PythonGrammar ! Grammar
8
t o t a r g e t : AntlrMM ! Grammar ( c l a s s D e f i n i t i o n s <− S e q u e n c e { p a r s e r } ) ,
p a r s e r : AntlrMM ! P a r s e r S p e c ( i d <− ’ P y t h o n 2 5 P a r s e r ’ , r u l e s <− s o u r c e . r u l e s )
10 }
12
rule TranslateRules
{
14
from s o u r c e : PythonGrammar ! R u l e
t o t a r g e t : AntlrMM ! R u l e ( i d <− s o u r c e . name , b l o c k <− a B l o c k ) ,
16
a B l o c k : AntlrMM ! B l o c k ( a l t e r n a t i v e s <− S e q u e n c e { a A l t } ) ,
a A l t : AntlrMM ! A l t e r n a t i v e ( e l e m e n t s <− s o u r c e . e x p r s )
18 }
20
22
24
26
28
30
rule AlternativeTransform
{
from s o u r c e : PythonGrammar ! A l t e r n a t i v e E x p r
t o e l e m e n t : AntlrMM ! Element ,
e b n f : AntlrMM ! EBNF ( b l o c k <− s u b b l o c k ) ,
s u b b l o c k : AntlrMM ! B l o c k −−( a l t e r n a t i v e s <− s o u r c e . a l t e r n a t i v e s −> c o l l e c t ( e | t h i s M o d u l e . newAlt ( e ) ) )
do
{
s u b b l o c k . a l t e r n a t i v e s <− s o u r c e . a l t e r n a t i v e s −> c o l l e c t ( e | t h i s M o d u l e . e l e m e n t T o A l t e r n a t i v e ( e ) ) ;
thisModule . c r e a t e C h i l d ( element , source , ebnf ) ;
}
}
32
r u l e RuleRef
34 {
from s o u r c e : PythonGrammar ! R u l e R e f E x p r
36
t o e l e m e n t : AntlrMM ! Element ,
r e f e r e n c e : AntlrMM ! R u l e R e f ( r e f e r e n c e d R u l e <− s o u r c e . r e f )
38
do
{
40
thisModule . c r e a t e C h i l d ( element , source , r e f e r e n c e ) ;
}
42 }
Listing 5.11: Exerpt from the Python grammar metamodel to Frodo metamodel transformation
Two solution are possible: introducing support for PyEBNF into Frodo or creating
a Python grammar metamodel, parsing the grammar, creating elements conforming this
metamodel and then transforming it into the Frodo grammar metamodel. The second solution is chosen because at the development time of the DSL the Frodo grammar metamodel
was still not finished. Meanwhile creating a ECore metamodel for PyEBNF grammars was
feasible in a short amount of time. The PyEBNF is then parsed with a custom written
ANTLR parser creating the according model elements16 . This parser can parse the specification for the Python 2.4 language and also Python 2.5 because they are defined with
the same PyEBNF grammar. If the grammar format does not change, switching to a new
Python version can be done automatically, at least for the grammar.
To be able to work with the grammar model it needs to be transformed from the Python
PyEBNF grammar metamodel to Frodo’s grammar metamodel. This transformation is done
15
Similar to EBNF. http://svn.python.org/view/python/trunk/Grammar/
The PyEBNF grammar can be implemented as a Frodo DSL. However as Frodo and the TestDSL were
developed in parallel, Frodo was not matured enough to pursue this approach from the beginning. Yet this is
future work and could be part of a major refactoring.
16
83
with ATL (listing 5.11).
Certain problems arise in and after this step. First, the Python grammar does not contain
a formal token description but to be able to use the grammar, a lexer which constructs the
tokens referenced in the grammar is needed. The solution is reuse again: a Python lexer
grammar is available17 for ANTLR and can therefore be used as a grammar model within
Frodo. Another transformation, this time defined in Python instead of ATL because only
small changes are introduced, combines the lexer grammar model and the parser grammar
model.
The second problem lies in the unified grammar model and arises if a parser is directly
generated from it. In contradiction to an official statement18 the original Python grammar is
not LL(1). A generated ANTLR parser can not decide on three productions employing finite look ahead (listing 5.12). Increasing the look ahead does not solve the problem because
it is impossible to match the rules using a top-down19 approach. Inside the CPython20 implementation custom code deals with this case. To allow the generation of a working parser
another transformation is introduced. It transforms the three ambiguous rules so they can
be matched with a finite look ahead. All other rules can be matched by a maximum look
ahead of two.
After these corrective transformations the parser which is generated from the grammar
model is able to parse every available Python program21 : it successfully parses the Python
language test suite. However no actions are associated with the recognition of rules at the
moment.
From successful parsing a Python AST should be created. The AST elements should
also be represented by model elements conforming to an AST metamodel. Again the available AST metamodel elements are automatically derived from an available specification22 .
In the next step the existing Python grammar model is annotated using a top-down visitor
with actions to instantiate the matching AST metamodel elements starting at the root rule.
An implicit derivation of the model elements to create was impossible in this case, as the
AST differs to much from the grammar.
The parser which is generated from the grammar model is now able to construct a
AST model. Finally a last transformation from this model to a real AST is applied. The
real AST is then execute on top of the Python runtime after successful parsing. To verify
the implementation the parser was tested against the standard test suite and successfully
17
Small modifications are made to the lexer grammar because it only supported Python 2.3.
http://www.python.org/dev/peps/pep-3099/
19
ANTLR generates a top-down parser.
20
The first and major Python implementation is written in C and often referred to as CPython, because in
the meantime a .Net and a Java Python implementation exist (IronPython and Jython).
21
With one restriction. The only supported charset is UTF.
22
http://svn.python.org/projects/python/trunk/Tools/compiler/ast.txt
Python 2.4 contains this specification and also implements classes to create, modify and execute a Python
AST. However inside the C implementation a concrete syntax tree is used not an AST nor the defined AST
classes. This changed in Python 2.5. The compiler is now built upon a AST. Yet the classes used to represent
this AST are not the classes available in the Python 2.4 standard library rather than a new development. To
make this case even more difficult, the AST classes inside the standard library of Python 2.4 were updated to
Python 2.5 and are still included. However they are not used in the compiler. The classes used in the compiler
can currently only be instantiated from C. A unification in the future would be useful and remove redundancy.
18
84
executed all tests.
1
arglist_orginal
3
a r g l i s t _ r e w r i t e n : a r g u m e n t ( ’ , ’ a r g u m e n t )∗ [ ’ , ’ [ ’∗ ’ t e s t [ ’ , ’ ’∗∗ ’ t e s t ] | ’∗∗ ’ t e s t ] ]
| ’∗ ’ t e s t [ ’ , ’ ’∗∗ ’ t e s t ] |
’∗∗ ’ t e s t
: ( a r g u m e n t ’ , ’ )∗ ( a r g u m e n t [ ’ , ’ ] |
’∗ ’ t e s t [ ’ , ’ ’∗∗ ’ t e s t ] | ’∗∗ ’ t e s t )
5
# F i r s t rule i s not matchable with f i n i t lookahead
7 # Rewritten rule i s matchable with f i n i t lookahead
# Elements in square b r a c k e t s are o p t i o n a l
Listing 5.12: Example of unmatchable rule with finite lookahead and a solution.
Now as the Python parser is in place, the domain model for the TestDSL is specified. This domain model includes the new concepts which should be introduced to Python
such as the wait construct and extends the Python AST metamodel. Moreover transformations from domain specific elements to a set of Python AST elements are defined (a
simple example is the transformation of the model element representing log “something”
to the AST call element Logger.log with the sub AST for parameters (currentLoggingMode,timestamp,”something”) attached). These transformations can also modify the current AST model. The Python AST elements represent the semantic of a domain model
element and can be simple transformations into a call or complex instructions orchestrating
the PyTAF framework.
Having specified the TestDSL domain model and therefore the specific model elements
the current Python grammar model is modified. The modification introduces new rules to
the grammar programmatically. These rules contain the productions used to recognize the
textual representation of domain model elements and also the actions to create those and
attach them to the Python AST model resulting in a mixed model. The drawback of this
approach is, that the developer has to know which Python rule has to be extended.
A parser generated from this grammar model is able to parse the TestDSL which is
Python including the domain specific constructs though the instantiated model is not executable in this form.
Finally to allow execution, the transformations resulting from the domain model to
Python AST metamodel mapping is applied, turning the mixed model into a pure Python
AST model. This model and therefore the DSL can be executed.
To allow error reporting and later debugging every Python AST and TestDSL model
element stores its corresponding line and column number in the textual representation.
Also a simple Eclipse based editor is generated from the TestDSL grammar model.
Results
It could be argued that the new language is a GPL like Python, however it provides clearly
domain specific constructs. It is advised not to use all available Python features (for example leaving out classes or decorators is advisable). Moreover they could be easily removed
from the language. The reason for avoiding this is of practical nature. In most cases the basic Python expression syntax, lists and functions and the additional test specific constructs
are sufficient to specify a test case, yet in rare cases advanced features may be needed.
Compared to the original Excel DSL the new TestDSL offers many advantages. Most
of the named problems are solved such as the ability to associate a Python exception with
85
a DSL element. Furthermore it is easy to introduce23 semantic checks because everything
is a model, for example with OCL. Also early prototypical debugging is available. With
more precise error messages the TestDSL can be considered as usable.
Frodo allowed the reuse of many existing artifacts and reduced the development time
for the TestDSL. Also the artifact forming the TestDSL can be further reused, for implementing new language features, for analysis or model based test generation. The TestDSL
proves that Frodo fulfills some of the initially formulated requirements like reusability and
a decreased development effort. Ease of use and language evolution support can be improved further.
Language evolution support was a requirement for Frodo but is currently not directly
supported inside the framework and for that reason neither for the TestDSL. It could be
implemented by defining transformations between different versions of the domain metamodel and preserving the grammar models for each version. So even an automatic translation from an older TestDSL version to a newer one is possible provided that they still have
commonalities.
23
Using an external tool.
86
Part III
Conclusions
87
Chapter 6
Future work
“The best way to predict the future is to invent it.”1
In the following and short overview about possible future work is presented, first for Frodo and then for the test automation language.
The combination of DSL development with an Model-Driven Engineering approach is still
an ongoing field of research. Many gaps still exist and Frodo fills some of them. Nevertheless the framework is far from being a complete solution and improvements on both sides
theoretical foundations and practical implementation are possible.
Furthermore the model-driven TestDSL can be enhanced in different directions which
maybe a better domain model or a totally different syntax front-end like TTCN3 or a defined subset of it.
6.1
DSL Framework
6.1.1
Theoretical outlook
The framework proposes different improvements to the model-driven development of domain specific languages. For example the derivation of an generative grammar from the
grammar metamodel. This derivation was not completed in this thesis and no explicit theoretical foundation was developed. Yet such a foundation in the context of model-driven
grammars could be useful, not only for the proposed framework.
Another aspect is the grammar metamodel. The metamodel is currently polluted by
ANTLR specific elements. Refactoring this model can improve its reusability and make
it more independent of Frodo. Moreover the model does not include generally applicable
elements usable to describe parser error handling, recovery or domain specific error reporting. The error handling implementation differs from parser generator to parser generator,
finding an expressive abstraction can result in comparable domain specific error reporting
for the generated parsers. One more possible improvement could be an algorithm for automatically determining the power of the needed parser (i.e. LL(0), LL(1)) and selecting a
1
Theodore Hook
88
certain generator automatically. This can be accomplished by a configuration of the target
platform / language so an exactly matching generator is selected.
To save development effort an implicit by name mapping from domain model to grammar rules was specified. Using this approach the flexibility of the grammar is constraint.
Finding better conventions for the mapping can mitigate some of the constraints and would
be an interesting further work.
An idea worth to examine is an special metamodel aware grammar with advanced
implicit mapping by convention abilities and the possibility to execute transformation or
model modifications directly from the grammar.
If an explicit mapping is used the actions are defined in the target language. Finding
a way to specify explicit actions target language independent could improve reuse even
more. A starting point is the Kermeta language.
Finally it can be very useful to examine model checking for grammar models. Different Model-Driven Engineering tools can be used therefore but at first it must be stated what
should be checked. Interesting properties are validity and consistency after a modification or transformation, whether a grammar model is ambiguous given a specific class (like
LL(1)) or if problem occur in the implicit mapping between grammar model and domain
model.
6.1.2
Practical outlook
Frodo is a prototype implementation of the proposed framework and therefore not fully
complete. Different directions for further improvements can be taken.
First, the only supported parser generator is ANTLR at the moment. To get some of
the described benefits of a tool agnostic grammar metamodel support for other parser backends can be added. To allow this step the grammar metamodel should be freed of any
ANTLR related construct like exception handling.
Another idea is the implementation of the grammar parser with the help of Frodo as every grammar is itself a DSL. This would unify the parser generator grammar parser creation
and allows the model-driven modification for example adopting to a new parser generator
version.
Furthermore Frodo showed preliminary generation of different language tools. Generating a working debugger is especially intriguing and was not accomplished in this thesis.
Also the integration with the environments can be smoothed. Currently many scripts need
to be called manually for example the execution of ATL transformations.
As a demonstration implementation Frodo showed already some of the potential lying
in a MDE approach to DSL development, yet numerous other features could be integrated.
6.2
TestDSL
The test language can also be subject of further improvements. The domain metamodel
is at the moment a mixture of the Python AST model and domain specific constructs. It
could be useful to express more language elements directly in the domain model. For
example service calls are at the moment ordinary Python calls and therefore only a Python
89
AST element in the model. This would allow even easier model-based generation of test
case for input data because Python AST elements are merely needed for the expression
language.
One use case for the model-based generation is the transformation of existing Excel test
cases. A temporary ECore metamodel including an Excel parser which creates conforming
model elements already exists for the old test automation language. A working transformation would allow the reuse of already written test cases. As such a transformation is rather
trivial, it was not included.
Another direction which can be explored is offering another new front end to the test automation framework. While the new TestDSL was driven by practical considerations using
a more formalized language like TTCN3 could offer a new class of benefits. Test engineers
already used to TTCN3 could easily create test cases still using the existing framework.
Furthermore TTCN3 is the only language test specification and implementation language
which is standardized at the moment and also different research project try to integrate new
ideas like TPT2 for continuous stream testing. Because of Frodos reuse capabilities existing
TTCN3 grammars could be used like the ANTLR grammar from the TRex3 project.
Because the TestDSL is now based on a model, examining model-based test generation
is also interesting and could reduce manual testing efforts greatly as first experiments with
model-based generation of MMI tests showed at the BMW Group.
Finally also the usage of the language itself in practice will guide its further development and should therefore be monitored. Concepts which fit the needs of the test developer
should be improved, unnecessary constructs dropped and new useful constructs added.
2
3
Time Partion Testing[120]
A TTCN3 language refactoring tool.
90
Chapter 7
Summary
“Even for the physicist the description in plain language will be a criterion
of the degree of understanding that has been reached.”1
The final chapter gives a survey of what has been done, what has been learned and draws
conclusions. Furthermore some of the major contributions are discussed.
7.1
Survey
This thesis discusses the model-driven development of domain specific languages, proposed a model-driven DSL framework -Frodo- and showes some of the possible benefits
with the help of two examples. At the start, a broad overview about the two major technical
research areas is given.
Domain specific languages were explained, including a discussion about their advantages, different development patterns and a description of supporting tools and frameworks.
The second major topic was Model-Driven Engineering, a new software engineering approach based on models. To understand MDE, important terms (metamodel, technological space), key concepts (transformations, DSMLs) and technologies were elaborated. A
new approach tries to combine DSL development and MDE technology resulting in modeldriven DSL frameworks like TCS/AMMA and XText/oAW which are directly related work
to this thesis and were explained, too.
The main part of this thesis proposed a framework following a model-driven approach
like the previous named frameworks. It should fulfill requirements such as reusability,
a decreased development effort for DSL, usability of the generated DSL and support for
language evolution. Frodo, the implementation of the proposal showed what kind of
DSL/MDE tools could be used for model-driven DSL development (like ANTLR or ATL)
and how to combine them. Moreover the architecture leads to a model based separation of
concerns and makes language artifact exchangeable with other MDE tools and frameworks.
With the help of Frodo and language specification artifacts a DSLs could be created.
The current framework implementation supports the reuse of existing ANTLR grammars
and can create a grammar model out of them. Frodo allows nearly every imaginable textual
1
Werner Karl Heisenberg
91
representation2 for a domain model through explicit or implicit mapping from grammar
elements to model elements. The implicit mapping helps to reduce the DSL development
effort but is restricted to domain models similar to the texutal representation. Frodo also
increases the DSL usability by the automatic generation of an Eclipse based editor. The
more interesting goal of generating a debugger was not reached in a general applicable
way. Yet some prototypical experiments show the possibility of debugger generation using
model and grammar aspects.
A complex language implemented with Frodo is the TestDSL, a replacement for the
currently used test automation language specified in Excel sheets which was analyzed in
this thesis, The Excel language has some design flaws. It misses a profound language
foundation and it offers no error reporting or exception mapping between DSL and test
execution language. The new TestDSL solves most of these problems and mitigates the
rest.
While developing the TestDSL, different approaches for DSL implementation were applied. At first embedding, resulting in a cleaner language directly embedded into the Python
host language. Moreover useful modifications to the test automation framework which is
controlled by the language were a result. This approach improved upon the existing language but not all feature could be embedded natively in Python.
With Frodo the missing features were implemented as a superset of Python. In this
process heavy reuse of existing artifacts like the Python grammar or the Python AST specification was made. A large number of practical experience was put into the TestDSL
which is not only applicable in the automotive domain. Also in comparison to TTCN3,
the language itself is better readable and writable. Admittedly, TTCN3 is more than just
a language. Beyond the language, features like error reporting and exception mapping between DSL and the back-end were implemented in a model-driven way with Frodo. The
generated TestDSL editor completes the new DSL.
This more complex example showed that it is possible to develop a sophisticated domain specific language with Frodo. Also through reuse of Python and its artifacts the
development time was decreased. Nevertheless the framework user still needs expertise in
language development and understanding of the modeling concepts and the target domain.
Creating reusable and easily extendable artifacts is possible with Frodo, the goal of eased
DSL development still needs improvements.
Choosing a model-driven approach proved very useful, as it leaded to improved reuse
and a large number of leverageable tools. Yet at the moment no broad understanding of
complicated terms like metametamodel is available which slows the adaption of a modeldriven process.
7.2
Contributions
As final words, some of the main contributions of this thesis should be enumerated.
At first a comprehensive overview about the topic model-driven DSL development and
its realization with other topics such as DSLs in general and MDE was given. Secondly
2
Must be recognizable by a context free grammar.
92
a framework which eases the development of such languages was presented including a
proposed architecture and explanation why certain parts are needed. This was done based
on a list of specific requirements.
On the practical side, Frodo, a prototype implementing some of the proposed functionality, was developed, and can serve as guidance for further implementations. This prototype also contains reusable artifacts like an ECore implementation for the Python language
which is, at least as known to the author, the only other implementation then the Java/EMF
one. Furthermore a usable grammar metamodel and an ANTLR parser for transforming
ANTLR grammars into the metamodel and back to their textual representation was developed, For example a refactoring tool for grammar could easily reuse these components. At
least as known to the author non of the other framework allows the direct reuse of existing
grammars.
Besides the ANTLR grammar metamodel a complete Python grammar metamodel is
available and can be a starting point for further modifications or extensions to the Python
language as a base for DSLs. Without modifications Python 2.4 can be parsed and executed
on a Python 2.4 or 2.5 back-end.
Finally the TestDSL language and its generated editor can be considered a contribution allowing easier formulation of tests including a domain specific error handling and
language checking.
To some extent a number of jigsaw pieces were added to the ongoing quest for an
“engineering discipline for grammarware” [106].
93
Appendix A
Fundamental Modeling Concepts
The notation of FMC was used at several places in this thesis. As it is not as widely
known as UML, it is introduced by a short citation and a reference sheet for the used block
diagrams. [108, 183]
“The Fundamental Modeling Concepts (FMC) primarily provide a framework for the comprehensive description of software-intensive systems. It is
based on a precise terminology and supported by a precise graphical notation
optimized for human comprehension. Modeling we call the intellectual activity of creating a model of some system with the goal to capture its essential
structures necessary to understand its (existing or planned) behaviour (internal and to its environment) and to describe these structures in a comprehensive
way. Against that background, didactics take an important position in that approach.”
94
Figure A.1: FMC block diagram example
95
Figure A.2: FMC block diagram reference sheet
96
Appendix B
Statement of Academic Honesty
Hiermit erkläre ich an Eides statt, dass ich die vorliegende Arbeit selbständig nur unter
Benutzung der in der Arbeit angegebenen Literatur angefertigt habe.
Martin Karlsch, Berlin den 12. Januar 2007
97
Bibliography
[1] ANTLR Grammar Repository. URL http://www.antlr.org/grammar/
list. [Online; accessed 19-January-2007].
[2] The Catalog of Compiler Construction Tools.
URL http://catalog.
compilertools.net. [Online; accessed 10-December-2006].
[3] The Generic Eclipse Modeling System, 2006.
URL http://www.dre.
vanderbilt.edu/~jules/gems.htm. [Online; accessed 19-January-2007].
[4] TokTok, the DSTC’s implementation of the OMG’s HUTN standard. URL http:
//www.dstc.edu.au/Research/Projects/Pegamento/TokTok/.
[5] ATL Transformations.
URL http://www.eclipse.org/m2m/atl/
atlTransformations/. [Online; accessed 19-January-2007].
[6] Aditya Agrawal. Graph Rewriting And Transformation (GReAT): A Solution For
The Model Integrated Computing (MIC) Bottleneck. In 18th IEEE International
Conference on Automated Software Engineering, 2003.
[7] Marcus Alanen. EMF/ECORE Integration in the Coral Modeling Framework. In
Proceedings of the NWUML’06: The 4th Nordic Workshop on UML and Software
Modelling, 2006.
[8] Marcus Alanen and Ivan Porres. Coral: A Metamodel Kernel for Transformation
Engines. In Proceedings of the Second Europena Workshop on Model Driven Architecture (MDA), number 17, pages 165–170, Canterbury, Kent CT2 7NF, United
Kingdom, Sep 2004. University of Kent.
[9] E. Allen, D. Chase, V. Luchangco, J.-W. Maessen, S. Ryu, G. Steele, and S. TobinHochstadt. The Fortress language specification version 0.707. Technical report, Sun
Microsystems, 2005.
[10] José M. Alvarez, Andy Evans, and Paul Sammut. Mapping between Levels in the
Metamodel Architecture. In UML ’01: Proceedings of the 4th International Conference on The Unified Modeling Language, Modeling Languages, Concepts, and
Tools, pages 34–46, London, UK, 2001. Springer-Verlag. ISBN 3-540-42667-1.
[11] Daniel Amyot, Hanna Farah, and Jean-Francois Roy. Evaluation of Development
Tools for Domain-Specific Modeling Languages. In SAM, pages 183–197, 2006.
98
[12] Matthias Anlauff. XASM - An Extensible, Component-Based ASM Language. In
ASM ’00: Proceedings of the International Workshop on Abstract State Machines,
Theory and Applications, pages 69–90, London, UK, 2000. Springer-Verlag. ISBN
3-540-67959-6.
[13] I. Attali, C. Courbis, P. Degenne, A. Fau, J. Fillon, C. Held, D. Parigot, and
C. Pasquier. Aspect and XML-oriented Semantic Framework Generator SmartTools,
2001.
[14] J. Aycock. Compiling little languages in Python. In Proc. 7th Int. Python Conf.,
Houston, TX, 1998.
[15] Jonthan Bachrach and Keith Playford. The Java syntactic extender (JSE). In OOPSLA ’01: Proceedings of the 16th ACM SIGPLAN conference on Object oriented
programming, systems, languages, and applications, pages 31–42, New York, NY,
USA, 2001. ACM Press. ISBN 1-58113-335-9.
[16] Jason Baker and Wilson C. Hsieh. Maya: multiple-dispatch syntax extension in Java.
In PLDI ’02: Proceedings of the ACM SIGPLAN 2002 Conference on Programming
language design and implementation, pages 270–281, New York, NY, USA, 2002.
ACM Press. ISBN 1-58113-463-0.
[17] Paul Baker, Shiou Loh, and Frank Weil. Model-Driven Engineering in a Large Industrial Context - Motorola Case Study. In MoDELS, pages 476–491, 2005.
[18] A. Basu, M. Hayden, G. Morrisett, and T. von Eicken. A Language-Based Approach
to Protocol Construction. In Proc. ACM SIGPLAN Workshop on Domain Specific
Languages, January 1997.
[19] Don Batory, Bernie Lofaso, and Yannis Smaragdakis. JTS: tools for implementing domain-specific languages. In Proceedings Fifth International Conference on
Software Reuse, pages 143–153, Victoria, BC, Canada, 2–5 1998. IEEE.
[20] Ira D. Baxter, Christopher Pidgeon, and Michael Mehlich. DMS: Program Transformations for Practical Scalable Software Evolution. In ICSE ’04: Proceedings of the
26th International Conference on Software Engineering, pages 625–634, Washington, DC, USA, 2004. IEEE Computer Society. ISBN 0-7695-2163-0.
[21] Kent Beck and Cynthia Andres. Extreme Programming Explained: Embrace
Change (2nd Edition). Addison-Wesley Professional, 2004. ISBN 0321278658.
[22] Oren Ben-Kiki, Clark Evans, and Brian Ingerson. YAML Ain’t Markup Language,
Version 1.1. 2004.
[23] J. L. Bentley. Programming Pearls. Addison-Wesley, 1989.
[24] L. Bichler. Codegeneratoren für MOF-basierte Modellierungssprachen. PhD thesis,
Universität der Bundeswehr München, Neubiberg, 11 2004.
99
[25] Judith Bishop. Multi-platform user interface construction: a challenge for software
engineering-in-the-small. In ICSE ’06: Proceeding of the 28th international conference on Software engineering, pages 751–760, New York, NY, USA, 2006. ACM
Press. ISBN 1-59593-375-1.
[26] E. Bjarnason. APPLAB – A Laboratory for Application Languages. Technical
report, Lund Institute of Technology, 19961996.
[27] Per Bjesse, Koen Claessen, Mary Sheeran, and Satnam Singh. Lava: hardware
design in Haskell. In ICFP ’98: Proceedings of the third ACM SIGPLAN international conference on Functional programming, pages 174–184, New York, NY,
USA, 1998. ACM Press. ISBN 1-58113-024-4.
[28] Xavier Blanc, Marie-Pierre Gervais, and Prawee Sriplakich. Model Bus: Towards
the Interoperability of Modelling Tools. In MDAFA, pages 17–32, 2004.
[29] M. Bloos. Echtzeitanalyse der Kommunikation in KfZ Bordnetzen auf Basis des CAN
Protokolls. PhD thesis, TU München, 1999.
[30] Claus Brabrand, Mads Vanggaard, and Michael I. Schwartzbach. The metafront System: Extensible Parsing and Transformation. In Proc. 3rd ACM SIGPLAN Workshop
on Language Descriptions, Tools and Applications, LDTA ’03. ACM, April 2003.
[31] P. Braun and F. Marschall. The Bi-directional Object-Oriented Transformation Language. Technical report, Technische Universität München, 2003.
[32] Martin Bravenboer and Eelco Visser. Concrete syntax for objects: domain-specific
language embedding and assimilation without restrictions. In OOPSLA ’04: Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pages 365–383, New York, NY,
USA, 2004. ACM Press. ISBN 1-58113-831-9.
[33] Frederick P. Brooks. The Mythical Man-Month, chapter No Silver Bullet - Essence
and Accident in Software Engineering, pages 177–203. Addison Wesley Longman
Inc., New York, 15., aufl. edition, 2001.
[34] S. A. Brown, C. E. Drayton, and B. Mittman. A description of the APT language.
Commun. ACM, 6(11):649–658, 1963. ISSN 0001-0782.
[35] Manfred Broy. Challenges in automotive software engineering. In ICSE ’06: Proceeding of the 28th international conference on Software engineering, pages 33–42,
New York, NY, USA, 2006. ACM Press. ISBN 1-59593-375-1.
[36] Manfred Broy, Michael Deißenböck, and Markus Pizka. A holistic approach to
software quality at work. In 3rd World Conference for Software Quality (3WCSQ),
September 2005.
[37] Frank Budinsky, David Steinberg, Ed Merks, Raymond Ellersick, and Timothy
Grose. Eclipse Modeling Framework. Addison Wesley Professional, 2003.
100
[38] L. Burgy, C. Consel, F. Latry, J. Lawall, N. Palix, and L. Réveillère. Language
Technology for Internet-Telephony Service Creation. In IEEE International Conference on Communications, JUNE 2006. URL http://phoenix.labri.fr/
publications/talks/icc06.pdf.
[39] Jean Bézivin. On the Unification Power of Models. Software and System Modeling (SoSym), 4(2):171–188, 2005. URL http://www.lina.sciences.
univ-nantes.fr/Publications/2005/Bé05b.
[40] Jean Bézivin. On the unification power of models. Software and Systems Modeling,
4(2):171–188, May 2005. URL http://www.metapress.com/link.asp?
id=XN50242535640K10.
[41] Jean Bézivin and Olivier Gerbé. Towards a Precise Definition of the OMG/MDA
Framework. In ASE ’01: Proceedings of the 16th IEEE international conference
on Automated software engineering, page 273, Washington, DC, USA, 2001. IEEE
Computer Society.
[42] Jean Bézivin, C. Brunette, R. Chevrel, Frédéric Jouault, and Ivan Kurtev. Bridging
the Generic Modeling Environment (GME) and the Eclipse Modeling Framework
(EMF). In OOPSLA Workshop on Best Practices for Model Driven Software Development, 2005. URL http://www.lina.sciences.univ-nantes.fr/
Publications/2005/BBCJK05.
[43] Jean Bézivin, Guillaume Hillairet, Frédéric Jouault, Ivan Kurtev, and William Piers.
Bridging the MS/DSL Tools and the Eclipse Modeling Framework. In Proceedings of the International Workshop on Software Factories at OOPSLA 2005, San
Diego, California, USA, 2005. URL http://softwarefactories.com/
workshops/OOPSLA-2005/Papers/Bezivin.pdf.
[44] Jean Bézivin, Fabian Büttner, Martin Gogolla, Frédéric Jouault, Ivan Kurtev, and
Arne Lindow. Model Transformations? Transformation Models! In MoDELS,
pages 440–453, 2006.
[45] Jean Bézivin, Jean-Marie Favre, and Bernhard Rumpe. Introduction to GaMMa
2006 first international workshop on global integrated model management. In
GaMMa ’06: Proceedings of the 2006 international workshop on Global integrated
model management, pages 1–3, New York, NY, USA, 2006. ACM Press. ISBN
1-59593-410-3.
[46] Stefano Ceri, Piero Fraternali, and Aldo Bongio. Web Modeling Language
(WebML): a modeling language for designing Web sites. Computer Networks (Amsterdam, Netherlands: 1999), 33(1–6):137–157, 2000.
[47] Philippe Charles, Julian Dolby, Robert M. Fuhrer, Stanley M. Sutton Jr., and Mandana Vaziri. SAFARI: a meta-tooling framework for generating language-specific
IDE’s. In OOPSLA Companion, pages 722–723, 2006.
101
[48] Shigeru Chiba. A Metaobject Protocol for C++. In ACM Conference on ObjectOriented Programming Systems, Languages, and Applications (OOPSLA’95), SIGPLAN Notices 30(10), pages 285–299, Austin, Texas, USA, October 1995.
[49] C. Consel, H. Hamdi, L. Réveillère, L. Singaravelu, H. Yu, and C. Pu. Spidle: A
DSL approach to specifying streaming applications. Technical Report RR1282-02,
LaBRI, Bordeaux, France, October 2002.
[50] James R. Cordy. The TXL source transformation language. Sci. Comput. Program.,
61(3):190–210, 2006. ISSN 0167-6423.
[51] Corinna Cortes, Kathleen Fisher, Daryl Pregibon, and Anne Rogers. Hancock: a
language for extracting signatures from data streams. In KDD ’00: Proceedings of
the sixth ACM SIGKDD international conference on Knowledge discovery and data
mining, pages 9–17, New York, NY, USA, 2000. ACM Press. ISBN 1-58113-233-6.
[52] Douglas Crockford. JSON: The Fat-Free Alternative to XML, 2006. URL http:
//www.json.org/xml.html. [Online; accessed 19-December-2006].
[53] Krzysztof Czarnecki and Simon Helsen. Classification of Model Transformation
Approaches. In OOPSLA 2003 Workshop on Generative Techniques in the context
of Model Driven Architecture, oct 2003. URL http://www.softmetaware.
com/oopsla2003/czarnecki.pdf.
[54] Ricardo de Almeida Falbo, Giancarlo Guizzardi, and Katia Cristina Duarte. An
ontological approach to domain engineering. In SEKE ’02: Proceedings of the 14th
international conference on Software engineering and knowledge engineering, pages
351–358, New York, NY, USA, 2002. ACM Press. ISBN 1-58113-556-4.
[55] Joel de Guzman. Spirit Parser Framework, 2006. URL http://spirit.
sourceforge.net/distrib/spirit_1_8_3/libs/spirit/index.
html. [Online; accessed 5-January-2007].
[56] Jan Decaluwe. MyHDL: a python-based hardware description language. Linux J.,
2004(127):5, 2004. ISSN 1075-3583.
[57] Verband der Automobilindustrie. Auto Annual Report, 2006. URL http://www.
vda.de/en/service/jahresbericht/files/VDA_2006_en.pdf.
[58] A. van Deursen and P. Klint. Domain-Specific Language Design Requires Feature
Descriptions. Journal of Computing and Information Technology, 10(1):1–17, 2002.
URL http://www.cwi.nl/~arie/papers/fdl/fdl.pdf.
[59] P. Devanbu and J. Poulin, editors. Modular Domain Specific Languages and Tools,
1998. IEEE Computer Society Press.
[60] D. Di Ruscio, F. Jouault, I. Kurtev, J. Bézivin, and A. Pierantonio. Extending
AMMA for Supporting Dynamic Semantics Specifications of DSLs. Technical report, LINA, 2006.
102
[61] E. W. Dijkstra. Notes on structured programming. In O.-J. Dahl, C. A. R. Hoare,
and E. W. Dijkstra, editors, Structured Programming, pages 1–82. 1972.
[62] BMW Group Information and Communication Systems division. Python Test Automation Framework, 2006.
[63] Dominic Duggan. A Mixin-Based, Semantics-Based Approach to Reusing DomainSpecific Programming Languages. In ECOOP ’00: Proceedings of the 14th European Conference on Object-Oriented Programming, pages 179–200, London, UK,
2000. Springer-Verlag. ISBN 3-540-67660-0.
[64] Sven Efftinge and Markus Völter. oAW xText: A framework for textual DSLs. In
Eclipse Summit 2006 Workshop: Modeling Symposium, 2006.
[65] Conal Elliott, Sigbjoern Finne, and Oege De Moor. Compiling embedded languages.
J. Funct. Program., 13(3):455–481, 2003. ISSN 0956-7968.
[66] Eric Evans. Domain-Driven Design: Tacking Complexity In the Heart of Software.
Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2003. ISBN
0321125215.
[67] Rickard E. Faith, Lars S. Nyland, and Jan F. Prins. KHEPERA: A System for Rapid
Implementation of Domain Specific Languages. pages 243–256.
[68] Jean M. Favre. Towards a Basic Theory to Model Model Driven Engineering. Workshop on Software Model Engineering, October 2004.
[69] Jean-Marie Favre. Foundations of Meta-Pyramids: Languages vs. Metamodels –
Episode II: Story of Thotus the Baboon1. In Jean Bezivin and Reiko Heckel, editors,
Language Engineering for Model-Driven Software Development, number 04101 in
Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany, 2005.
[70] Jean-Marie Favre. Foundations of Model (Driven) (Reverse) Engineering : Models
– Episode I: Stories of The Fidus Papyrus and of The Solarus. In Jean Bezivin and
Reiko Heckel, editors, Language Engineering for Model-Driven Software Development, number 04101 in Dagstuhl Seminar Proceedings. Internationales Begegnungsund Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany, 2005.
[71] Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, and Shriram Krishnamurthi. Fostering Little Languages. Dr. Dobb’s Journal, March 2004. URL
http://www.ddj.com/documents/ddj0403a/.
[72] Raphael A. Finkel. Advanced Programming Language Design. Addison-Wesley
Longman Publishing Co., Inc., Boston, MA, USA, 1995. ISBN 0805311912.
[73] Frédéric Fondement, Rémi Schnekenburger, Sébastien Gérard, and Pierre-Alain
Muller. Metamodel-Aware Textual Concrete Syntax Specification. Technical report, 2006.
103
[74] Martin Fowler. Language Workbenches: The Killer-App for Domain Specific Languages? May 2005. URL http://www.martinfowler.com/articles/
languageWorkbench.html.
[75] William Frakes, Ruben Prieto-Diaz, and Christopher Fox. DARE: Domain analysis
and reuse environment. Ann. Softw. Eng., 5:125–141, 1998. ISSN 1022-7091.
[76] Didier Vojtisek Cyril Faucher Franck Fleurey, Zoé Drey. Kermeta language - Reference manual, 2007.
[77] Steve Freeman and Nat Pryce. Evolving an embedded domain-specific language in
Java. In OOPSLA ’06: Companion to the 21st ACM SIGPLAN conference on Objectoriented programming systems, languages, and applications, pages 855–865, New
York, NY, USA, 2006. ACM Press. ISBN 1-59593-491-X.
[78] Erich Gamma, Richard Helm, Ralph Johnson, and John Vliaides. Design Patterns,
Elements of Reusable Object-Oriented Software. Addison-Wesley Profeaional Computing Series. Addison-Wesley, Boston, San Francisco, New York, 1995. ISBN 0201-63361-2.
[79] Anna Gerber and Kerry Raymond. MOF to EMF: there and back again. In eclipse
’03: Proceedings of the 2003 OOPSLA workshop on eclipse technology eXchange,
pages 60–64, New York, NY, USA, 2003. ACM Press.
[80] ETAS GmbH.
LABCAR-AUTOMATION V2.1, 2006.
URL http:
//de.etasgroup.com/products/labcar/software_in_detail.
shtml#automation. [Online; accessed 26-December-2006].
[81] Adele Goldberg. Why Smalltalk? Commun. ACM, 38(10):105–107, 1995. ISSN
0001-0782.
[82] Jack Greenfield and Keith Short. Software factories: assembling applications with
patterns, models, frameworks and tools. In OOPSLA ’03: Companion of the 18th
annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pages 16–27, New York, NY, USA, 2003. ACM Press.
ISBN 1-58113-751-6.
[83] R. Grønmo and J. Oldevik. An Empirical Study of the UML Model Transformation
Tool (UMT). In Proc. of the 1st Int. Conf. on Interoperability of Enterprise Software
and Applications, 2005.
[84] James R. Groff and Paul N. Weinberg. SQL, the complete reference. Osborne /
McGraw-Hill, 1999. ISBN 0-07-211845-8.
[85] Bernd Hardung, Thorsten Külzow, and Andreas Krüger. Reuse of software in distributed embedded automotive systems. In EMSOFT ’04: Proceedings of the 4th
ACM international conference on Embedded software, pages 203–210, New York,
NY, USA, 2004. ACM Press. ISBN 1-58113-860-1.
104
[86] Jan Hendrik Hausmann. Dynamic Meta Modeling - A Semantics Description Technique for Visual Modeling Languages. PhD thesis, University of Paderborn, 2005.
[87] David Hearnden, Kerry Raymond, and Jim Steel. Anti-Yacc: MOF-to-Text. In
EDOC ’02: Proceedings of the 6th International Enterprise Distributed Object
Computing Conference, pages 200–211, Washington, DC, USA, 2002. IEEE Computer Society. ISBN 0-7695-1742-0.
[88] P.R. Henriques, M.J.V. Pereira, M. Mernik, M. Lenic, J. Gray, and H. Wu. Automatic generation of language-based tools using the LISA system. IEE Proceedings
- Software, 152(2):54–69, 2005. URL http://link.aip.org/link/?IPS/
152/54/1.
[89] CAP Gemini Innovation Hewlett Packard, Matra Marconi Space. Domain Analysis
Method. Deliveable D3.2B, PROTEUS ESPRIT project 6086, 1994.
[90] Ellis Horowitz. Programming Languages: A Grand Tour. Computer Science Press,
Rockville, Maryland, third edition, 1987.
[91] M. Huisman. The calculation of a polytypic parser. Master’s thesis, Utrecht University, 1996. Master Thesis.
[92] Capers Jones. Programming Languages Table, 1996. URL http://www.
theadvisors.com/langcomparison.htm. [Online; accessed 05-January2007].
[93] Frédéric Jouault and Jean Bézivin. KM3: a DSL for Metamodel Specification. pages
171–185, 2006. URL http://www.sciences.univ-nantes.fr/lina/
atl/www/papers/KM3-FMOODS06.pdf.
[94] Frédéric Jouault and Jean Bézivin. On the Specification of Textual Syntaxes for
Models. In Eclipse Modeling Symposium, Eclipse Summit Europe 2006, 2006.
[95] Frédéric Jouault and Ivan Kurtev. On the Architectural Alignment of ATL and
QVT. In ACM Symposium on Applied Computing (SAC), model transformation track, 2006. URL http://www.lina.sciences.univ-nantes.fr/
Publications/2006/JK06.
[96] Frédéric Jouault, Jean Bézivin, and Ivan Kurtev. TCS: a DSL for the specification of
textual concrete syntaxes in model engineering. In GPCE ’06: Proceedings of the
5th international conference on Generative programming and component engineering, pages 249–254, New York, NY, USA, 2006. ACM Press. ISBN 1-59593-237-2.
[97] Basim M. Kadhim and William M. Waite. Maptool - Supporting Modular Syntax Development. In CC ’96: Proceedings of the 6th International Conference on
Compiler Construction, pages 268–280, London, UK, 1996. Springer-Verlag. ISBN
3-540-61053-7.
105
[98] Audris Kalnins, Janis Barzdins, and Edgars Celms. Model Transformation Language
MOLA. In MDAFA, pages 62–76, 2004.
[99] Samuel N. Kamin. Programming languages: an interpreter-based approach.
Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1990. ISBN
0-201-06824-9.
[100] K.C. Kang, S.G. Cohen, J.A. Hess, W.E. Novak, and A.S. Peterson. FeatureOriented Domain Analysis (FODA) Feasibility Study. Technical Report CMUSEI90-TR-21, Software Engineering Institute, CMU, November 1990.
[101] Hayato Kawashima and Katsuhiko Gondow. Experience with ANSI C Markup Language for a Cross-Referencer. In HICSS ’03: Proceedings of the 36th Annual Hawaii
International Conference on System Sciences (HICSS’03) - Track 9, page 324.3,
Washington, DC, USA, 2003. IEEE Computer Society. ISBN 0-7695-1874-5.
[102] Brian W. Kernighan. The C Programming Language. Prentice Hall Professional
Technical Reference, 1988. ISBN 0131103709.
[103] Gregor Kiczales, Jim des Rivieres, and Daniel G. Bobrow. The Art of the Metaobject
Protocol. Cambridge, Mass., 1991.
[104] Holger M. Kienle. Using smgn for Rapid Prototyping of Small Domain-Specific
Languages. 36(9):64–73, September 2001. ISSN 0362-1340.
[105] Anneke Kleppe, Jos Warmer, and Wim Bast. MDA Explained. The Model Driven
Architecture: Practice and Promise. Addison-Wesley, 2003.
[106] Paul Klint, Ralf Lämmel, and Chris Verhoef. Toward an engineering discipline for
grammarware. ACM Trans. Softw. Eng. Methodol., 14(3):331–380, 2005. ISSN
1049-331X.
[107] J. W. Klop. Term rewriting systems. pages 1–116, 1992.
[108] Andreas Knopfel, Bernhard Grone, and Peter Tabeling. Fundamental Modeling
Concepts: Effective Communication of IT Systems. John Wiley & Sons, May 2006.
ISBN 047002710X.
[109] Eugene Kohlbecker, Daniel P. Friedman, Matthias Felleisen, and Bruce Duba. Hygienic macro expansion. In LFP ’86: Proceedings of the 1986 ACM conference
on LISP and functional programming, pages 151–161, New York, NY, USA, 1986.
ACM Press. ISBN 0-89791-200-4.
[110] Dimitrios S. Kolovos, Richard F. Paige, and Fiona A.C. Polack. Model comparison:
a foundation for model composition and model transformation testing. In GaMMa
’06: Proceedings of the 2006 international workshop on Global integrated model
management, pages 13–20, New York, NY, USA, 2006. ACM Press. ISBN 1-59593410-3.
106
[111] G. Kramler, G. Kappel, T. Reiter, E. Kapsammer, W. Retschitzegger, and
W. Schwinger. Towards a semantic infrastructure supporting model-based tool integration. In GaMMa ’06: Proceedings of the 2006 international workshop on
Global integrated model management, pages 43–46, New York, NY, USA, 2006.
ACM Press. ISBN 1-59593-410-3.
[112] G. Kramler, G. Kappel, T. Reiter, E. Kapsammer, W. Retschitzegger, and
W. Schwinger. Towards a semantic infrastructure supporting model-based tool integration. In GaMMa ’06: Proceedings of the 2006 international workshop on
Global integrated model management, pages 43–46, New York, NY, USA, 2006.
ACM Press. ISBN 1-59593-410-3.
[113] Charles W. Krueger. New methods in software product line practice. Commun.
ACM, 49(12):37–40, 2006. ISSN 0001-0782.
[114] Thomas Kühne. What is a Model? In Jean Bezivin and Reiko Heckel, editors,
Language Engineering for Model-Driven Software Development, number 04101 in
Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany, 2005.
[115] Ivan Kurtev and Klaas van den Berg. MISTRAL: A Language for Model Transformations in the MOF Meta-modeling Architecture. In MDAFA, pages 139–158,
2004.
[116] Ivan Kurtev, Jean Bezivin, , and Mehmet Aksit. Technological Spaces: An Initial
Appraisal. In CoopIS, DOA’2002 Federated Conferences, Industrial track, Irvine,
USA, 2002.
[117] Ivan Kurtev, Jean Bézivin, Frédéric Jouault, and Patrick Valduriez. Model-based
DSL frameworks. In OOPSLA ’06: Companion to the 21st ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages
602–616, New York, NY, USA, 2006. ACM Press. ISBN 1-59593-491-X.
[118] Michael Lawley and Jim Steel. Practical Declarative Model Transformation with
Tefkat. In MoDELS Satellite Events, pages 139–150, 2005.
[119] A. Ledeczi, M. Maroti, A. Bakay, G. Karsai, J. Garrett, C. Thomason, G. Nordstrom,
J. Sprinkle, and P. Volgyesi. The Generic Modeling Environment. In Workshop on
Intelligent Signal Processing, Budapest, Hungary, volume 17, May 2001.
[120] Eckard Lehmann. Time Partition Testing - Systematischer Test des kontinuierlichen Verhaltens von eingebetteten Systemen. PhD thesis, Technischen UniversitÄat
Berlin, 2003.
[121] Michael R. Levy. Web programming in guide. Softw. Pract. Exper., 28(15):1581–
1603, 1998. ISSN 0038-0644.
[122] Holloway C. M. Lfm2000 - Fifth NASA Langley Formal Methods Workshop. Technical report, 2000.
107
[123] Alanen Marcus and Porres Ivan. Difference and Union of Models. Technical Report
527, TUCS, April 2003.
[124] Stephen J. Mellor and Marc Balcer. Executable UML: A Foundation for ModelDriven Architectures. Addison-Wesley Longman Publishing Co., Inc., Boston, MA,
USA, 2002. ISBN 0201748045. Foreword By-Ivar Jacoboson.
[125] Tom Mens and Pieter Van Gorp. A Taxonomy of Model Transformation. Electronic
Notes in Theoretical Computer Science, 152:125–142, March 2006.
[126] Marjan Mernik, Jan Heering, and Anthony M. Sloane. When and how to develop domain-specific languages. ACM Comput. Surv., 37(4):316–344, December
2005. ISSN 0360-0300. URL http://portal.acm.org/citation.cfm?
id=1118890.1118892.
[127] Microsoft. Microsoft Domain Specific Language Tool. URL http://msdn.
microsoft.com/vstudio/DSLTool.
[128] Sachio Mizuta and Runhe Huang. Automation of Grid Service Code Generation
with AndroMDA for GT3. In AINA ’05: Proceedings of the 19th International
Conference on Advanced Information Networking and Applications, pages 417–420,
Washington, DC, USA, 2005. IEEE Computer Society. ISBN 0-7695-2249-1.
[129] Pierre-Alain Muller and Michel Hassenforder. HUTN as a Bridge between ModelWare and GrammarWare. In WISME Workshop, MODELS / UML’2005, Montego
Bay, Jamaica, 2005.
[130] Bonnie A. Nardi. A small matter of programming: perspectives on end user computing. MIT Press, Cambridge, MA, USA, 1993. ISBN 0-262-1405305.
[131] J. M. Neighbors. Draco: a method for engineering reusable software systems. pages
295–319, 1989.
[132] James Neighbors. Software Construction Using Components. PhD thesis, University
of California Irvine, 1980.
[133] Inc. Object Management Group. UML 2.0 Specification. http://www.uml.
org/UML2.0, 2003.
[134] Inc. Object Management Group. Human-Usable Textual Notation, v1.0, 2004. URL
http://www.omg.org/technology/documents/formal/hutn.htm.
[135] Inc. Object Management Group. Meta Object Facility (MOF) 2.0 Core Specification. Object Management Group, Inc., 2004. URL http://www.omg.org/
cgi-bin/apps/doc?ptc/03-10-04.pdf.
[136] Inc. Object Management Group. Meta Object Facility (MOF) 2.0 Query/View/Transformation Specification - Final Adopted Specification. Object Management Group,
Inc., 2005. URL http://www.omg.org/docs/ptc/05-11-01.pdf.
108
[137] Inc. Object Management Group. SysML Specification 1.0. Object Management Group, Inc., 2006. URL http://www.omg.org/cgi-bin/doc?ptc/
06-05-04.
[138] Martin Odersky, Philippe Altherr, Vincent Cremet, Burak Emir, Sebastian Maneth,
Stéphane Micheloud, Nikolay Mihaylov, Michel Schinz, Erik Stenman, and
Matthias Zenger. An Overview of the Scala Programming Language. Technical Report IC/2004/64, École Polytechnique Fédérale de Lausanne, 1015 Lausanne, Switzerland, 2004. URL http://scala.epfl.ch/docu/files/
ScalaOverview.pdf. Revised June 3, 2005.
[139] Didier Parigot. Towards domain-driven development: the smartTools software factory. In OOPSLA ’04: Companion to the 19th annual ACM SIGPLAN conference on
Object-oriented programming systems, languages, and applications, pages 37–38,
New York, NY, USA, 2004. ACM Press. ISBN 1-58113-833-4.
[140] Terence Parr. A Functional Language For Generating Structured Text. URL http:
//www.cs.usfca.edu/~parrt/papers/ST.pdf. 2006.
[141] Terence J. Parr and Russell W. Quong. ANTLR: A Predicated-LL(k) Parser Generator. Software Practice and Experience, 25(7):789–810, 1995.
[142] Octavian Patrascoiu. Model Transformations in YATL. Studies and Experiments.
Technical Report 3-04, March 2004. URL http://www.cs.kent.ac.uk/
pubs/2004/1863.
[143] Jurgen Vinju Paul Klint, Tijs van der Storm. Term Rewriting Meets Aspect Oriented
Programming. In Aart Middeldorp, Vincent van Oostrom, Femke van Raamsdonk,
and Roel C. de Vrijer, editors, Processes, Terms and Cycles: Steps on the Road to
Infinity, Essays Dedicated to Jan Willem Klop, on the Occasion of His 60th Birthday,
volume 3838 of Lecture Notes in Computer Science. Springer, 2005.
[144] Luis Pedro, Levi Lucio, and Didier Buchs. Principles for System Prototype and Verification Using Metamodel Based Transformations. In RSP ’06: Proceedings of the
Seventeenth IEEE International Workshop on Rapid System Prototyping (RSP’06),
pages 10–17, Washington, DC, USA, 2006. IEEE Computer Society. ISBN 0-76952580-6.
[145] Mikael Peltier. MTrans, a DSL for Model Transformation. In EDOC ’02: Proceedings of the Sixth International ENTERPRISE DISTRIBUTED OBJECT COMPUTING Conference (EDOC’02), page 190, Washington, DC, USA, 2002. IEEE
Computer Society. ISBN 0-7695-1656-4.
[146] Muller Pierre-Alain. Weaving Executability into Object-Oriented Meta-Languages.
In MODELS/UML’2005, pages 264–278. Springer. URL http://www.irisa.
fr/triskell/publications/testbib/triskellbib/Muller05a.
109
[147] Ruben Prieto-Diaz. Domain analysis: an introduction. SIGSOFT Softw. Eng. Notes,
15(2):47–54, 1990. ISSN 0163-5948.
[148] Arend Rensink. Subjects, Models, Languages, Transformations. In Jean Bezivin and
Reiko Heckel, editors, Language Engineering for Model-Driven Software Development, number 04101 in Dagstuhl Seminar Proceedings. Internationales Begegnungsund Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany, 2005.
[149] Armin Rigo and Samuele Pedroni. PyPy’s approach to virtual machine construction.
In OOPSLA ’06: Companion to the 21st ACM SIGPLAN conference on Objectoriented programming systems, languages, and applications, pages 944–953, New
York, NY, USA, 2006. ACM Press. ISBN 1-59593-491-X.
[150] Vincent Rosener and Denis Avrilionis. Elements for the definition of a model of
software engineering. In GaMMa ’06: Proceedings of the 2006 international workshop on Global integrated model management, pages 29–34, New York, NY, USA,
2006. ACM Press. ISBN 1-59593-410-3.
[151] Davide Di Ruscio, Frédéric Jouault, Ivan Kurtev, Jean Bézivin, and Alfonso Pierantonio. A Practical Experiment to Give Dynamic Semantics to a DSL for Telephony
Services Development. Technical Report 06.03, LINA, 2006. URL http://www.
lina.sciences.univ-nantes.fr/Publications/2006/RJKBP06a.
[152] Andrea Schauerhuber, Manuel Wimmer, and Elisabeth Kapsammer. Bridging existing Web modeling languages to model-driven engineering: a metamodel for
WebML. In ICWE ’06: Workshop proceedings of the sixth international conference on Web engineering, page 5, New York, NY, USA, 2006. ACM Press. ISBN
1-59593-435-9.
[153] Douglas C. Schmidt, Hans Rohnert, Michael Stal, and Dieter Schultz. PatternOriented Software Architecture, Volume 2: Patterns for Concurrent and Networked
Objects. John Wiley & Sons, Inc., New York, NY, USA, 2000. ISBN 0471606952.
URL http://portal.acm.org/citation.cfm?id=557786.
[154] Eric C. Schnarr, Mark D. Hill, and James R. Larus. Facile: a language and compiler
for high-performance processor simulators. In PLDI ’01: Proceedings of the ACM
SIGPLAN 2001 conference on Programming language design and implementation,
pages 321–331, New York, NY, USA, 2001. ACM Press. ISBN 1-58113-414-2.
[155] A. Schürr, A. Winter, and A. Zündorf. PROGRES: Language and Environment. In
H. Ehrig, G. Engels, H. Kreowski, and G. Rozenberg, editors, Handbook on Graph
Grammars and Computing by Graph TransformationApplications, Languages, and
Tools, volume 2, pages 487–550. World Scientific, Singapur, 1999.
[156] Ed Seidewitz. What Models Mean. IEEE Softw., 20(5):26–32, 2003. ISSN 07407459.
110
[157] Hana Sevikova, Alan Borning, David Socha, and Wolf-Gideon Bleek. Automated
testing of stochastic systems: a statistically grounded approach. In ISSTA ’06: Proceedings of the 2006 international symposium on Software testing and analysis,
pages 215–224, New York, NY, USA, 2006. ACM Press. ISBN 1-59593-263-1.
[158] M. Simos and J. Anthony. Weaving the Model Web: A Multi-Modeling Approach
to Concepts and Features in Domain Engineering. In ICSR ’98: Proceedings of the
5th International Conference on Software Reuse, page 94, Washington, DC, USA,
1998. IEEE Computer Society. ISBN 0-8186-8377-5.
[159] Emin Gün Sirer and Brian N. Bershad. Using production grammars in software testing. In PLAN ’99: Proceedings of the 2nd conference on Domain-specific languages,
pages 1–13, New York, NY, USA, 1999. ACM Press. ISBN 1-58113-255-7.
[160] Eric J. Smith. Codesmith: Template based code generation, 2006. URL http:
//www.codesmithtools.com. [Online; accessed 19-December-2006].
[161] Diomidis Spinellis.
Notable Design Patterns for Domain Specific Languages.
Journal of Systems and Software, 56(1):91–99, February 2001.
ISSN 0164-1212. URL http://www.dmst.aueb.gr/dds/pubs/jrnl/
2000-JSS-DSLPatterns/html/dslpat.html.
[162] Vijayan Sugumaran, Sooyong Park, and Kyo C. Kang. Software product line engineering: Introduction. Commun. ACM, 49(12):28–32, 2006. ISSN 0001-0782.
[163] Alistair Sutcliffe and Nikolay Mehandjiev. End-user development: tools that empower users to create their own software solutions. Communications of the ACM.
SPECIAL ISSUE: End-user development, 47(9), September 2004. See Table of Contents.
[164] Michiaki Tatsubori, Shigeru Chiba, Kozo Itano, and Marc-Olivier Killijian. OpenJava: A Class-Based Macro System for Java. In OORaSE, pages 117–133, 1999.
[165] Richard N. Taylor, Will Tracz, and Lou Coglianese. Software development using
domain-specific software architectures: CDRl A011 a curriculum module in the SEI
style. SIGSOFT Softw. Eng. Notes, 20(5):27–38, 1995. ISSN 0163-5948.
[166] Scott Thibault, Renaud Marlet, and Charles Consel. Domain-Specific Languages:
From Design to Implementation Application to Video Device Drivers Generation.
Software Engineering, 25(3):363–377, 1999.
[167] Dave Thomas, David Hansson, Leon Breedt, Mike Clark, Thomas Fuchs, and Andrea Schwarz. Agile Web Development with Rails (The Facets of Ruby Series). Pragmatic Bookshelf, July 2005. ISBN 097669400X.
[168] Andrea Valerio, Giancarlo Succi, and Massimo Fenaroli. Domain analysis and
framework-based software development. SIGAPP Appl. Comput. Rev., 5(2):4–15,
1997. ISSN 1559-6915.
111
[169] Mark van den Brand, Arie van Deursen, Jan Heering, H. A. de Jong, Merijn de Jonge,
Tobias Kuipers, Paul Klint, Leon Moonen, Pieter A. Olivier, Jeroen Scheerder, Jurgen J. Vinju, Eelco Visser, and Joost Visser. The ASF+SDF Meta-environment: A
Component-Based Language Development Environment. In Computational Complexity, pages 365–370, 2001.
[170] Mark van den Brand, Bas Cornelissen, Pieter Olivier, and Jurgen Vinju. TIDE: A
Generic Debugging Framework - Tool Demonstration. Electronic Notes in Theoretical Computer Science, 141(4):161–165, 2005.
[171] Frank van der Linden, editor. Proceedings of the Second International ESPRIT
ARES Workshop on Development and Evolution of Software Architectures for Product Families, London, UK, 1998. Springer-Verlag. ISBN 3-540-64916-6.
[172] Arie van Deursen and Paul Klint. Little languages: little maintenance? In 62,
page 17. Centrum voor Wiskunde en Informatica (CWI), ISSN 1386-369X, 30 1997.
[173] Ghica van Emde Boas. Fantastic, unique, UML tool for the Java environment
(FUUT-je) (poster session). In OOPSLA ’00: Addendum to the 2000 proceedings of
the conference on Object-oriented programming, systems, languages, and applications (Addendum), pages 109–110, New York, NY, USA, 2000. ACM Press. ISBN
1-58113-307-3.
[174] Frank van Harmelen and Deborah L. McGuinness.
Feature Synopsis for
OWL Lite and OWL, July 2002. URL http://www.w3.org/TR/2002/
WD-owl-features-20020729/. W3C Working Draft.
[175] Gergely Varro, Andy Schurr, and Daniel Varro. Benchmarking for Graph Transformation. In VLHCC ’05: Proceedings of the 2005 IEEE Symposium on Visual
Languages and Human-Centric Computing (VL/HCC’05), pages 79–88, Washington, DC, USA, 2005. IEEE Computer Society. ISBN 0-7695-2443-5.
[176] Todd L. Veldhuizen. C++ Templates as Partial Evaluation. In Partial Evaluation
and Semantic-Based Program Manipulation, pages 13–18, 1999.
[177] Eelco Visser. Stratego: A Language for Program Transformation based on Rewriting
Strategies. System Description of Stratego 0.5. In A. Middeldorp, editor, Rewriting
Techniques and Applications (RTA’01), volume 2051 of Lecture Notes in Computer
Science, pages 357–361. Springer-Verlag, May 2001.
[178] Konrad Voigt. Eine praxisnahe Evaluierung von QVT am Beispiel von UML und
XML. URL http://www2.informatik.hu-berlin.de/Institut/
struktur/systemanalyse/zweckverband/qvt_uml_xml_rev.pdf.
March 2006.
[179] Robert Wagner. Developing Model Transformations with Fujaba. In Holger Giese
and Bernhard Westfechtel, editors, Proc. of the 4th International Fujaba Days 2006,
Bayreuth, Germany, volume tr-ri-06-275 of Technical Report, pages 79–82. University of Paderborn, September 2006.
112
[180] Qian Wang and Gopal Gupta. Rapidly prototyping implementation infrastructure of
domain specific languages: a semantics-based approach. In SAC ’05: Proceedings
of the 2005 ACM symposium on Applied computing, pages 1419–1426, New York,
NY, USA, 2005. ACM Press. ISBN 1-58113-964-0.
[181] David A. Watt. Programming language concepts and paradigms. Prentice-Hall,
Inc., Upper Saddle River, NJ, USA, 1990. ISBN 0-13-728874-3.
[182] David M. Weiss and Chi Tau Robert Lai. Software product-line engineering: a
family-based software development process. Addison-Wesley Longman Publishing
Co., Inc., Boston, MA, USA, 1999. ISBN 0-201-69438-7.
[183] Siegfried Wendt and Frank Keller. FMC: An Approach Towards ArchitectureCentric System Development. In Proceedings of 10th IEEE Symposium and Workshops on Engineering of Computer Based Systems, Huntsville Alabama USA (2003),
pages 173–182, April 2003. Proceedings of 10th IEEE Symposium and Workshops
on Engineering of Computer Based Systems.
[184] Wikipedia.
Semi-Thue-System — Wikipedia, Die freie Enzyklopädie,
2006.
URL http://de.wikipedia.org/w/index.php?title=
Semi-Thue-System&oldid=19764219. [Online; accessed 5-March-2007].
[185] Wikipedia. List of compiler-compilers — Wikipedia, The Free Encyclopedia,
2007. URL http://en.wikipedia.org/w/index.php?title=List_
of_compiler-compilers&oldid=105714730. [Online; accessed 14February-2007].
[186] David Wile. Lessons learned from real DSL experiments. Sci. Comput. Program.,
51(3):265–290, 2004. ISSN 0167-6423.
[187] Hui Wu, Jeff Gray, Suman Roychoudhury, and Marjan Mernik. Weaving a debugging aspect into domain-specific language grammars. In SAC ’05: Proceedings of
the 2005 ACM symposium on Applied computing, pages 1370–1374, New York, NY,
USA, 2005. ACM Press. ISBN 1-58113-964-0.
[188] Eric Van Wyk, Lijesh Krishnan, Derek Bodin, and Eric Johnson. Adding domainspecific and general purpose language features to java with the java language extender. In OOPSLA ’06: Companion to the 21st ACM SIGPLAN conference on
Object-oriented programming systems, languages, and applications, pages 728–
729, New York, NY, USA, 2006. ACM Press. ISBN 1-59593-491-X. doi: http:
//doi.acm.org/10.1145/1176617.1176696.