Université catholique de Louvain (UCL)
Belgian Laboratory of Computer-Human Interaction (BCHI)
Place des Doyens, 1
B-1348 Louvain-la-Neuve (Belgium)
A MDA-Compliant Environment for
Developing User Interfaces of
Information Systems
Presented by
Jean Vanderdonckt
[email protected]
Head of BCHI Lab, http://www.isys.ucl.ac.be/bchi
CAiSE'2005 Keynote address - Porto, June 16, 2005
1
« Everything you can imagine is real » (Picasso)
« Everything you model can be turned into a real interface »
CAiSE'2005 Keynote address - Porto, June 16, 2005
2
Outline
1.
2.
3.
4.
5.
Conceptual modeling of user interfaces
Forward engineering
Reverse engineering
Lateral engineering
Conclusion
CAiSE'2005 Keynote address - Porto, June 16, 2005
3
1.1 Why is conceptual modeling of
user interfaces desirable?

“The presentation layer outside the scope of CSCD”
(Antoni Olivé)
– A domain model is certainly not a presentation model, but
there could be a mapping between

UIs remained for many years the poor parent of
conceptual modeling and software engineering
– It emerged during late 80’s
– Still growing


The complexity and the variety of user interfaces (UI)
is dramatically increasing
The paradigm of « one system fits all » is no longer
valid
CAiSE'2005 Keynote address - Porto, June 16, 2005
4
1.1 Diversity of users

Traditional users

People with disabilities
– Visual, motor, cognitive
CAiSE'2005 Keynote address - Porto, June 16, 2005
5
1.1 Variety of tasks
[Forrester Research, Inc., 2003]

Users want to determine their path on
each platform
CAiSE'2005 Keynote address - Porto, June 16, 2005
6
1.1 Heterogeneousness of platforms
CAiSE'2005 Keynote address - Porto, June 16, 2005
7
1.1 Multiplicity of contexts of use
Location
Role
Device
Experience
Sporting
Multimedia Travel programme
Working
Travel booking site
Powerful interface for complex
operations
Travelling
Booking notification
Everywhere connectivity for
simple data exchange
Family
TV is multi-media family device
#1
CAiSE'2005 Keynote address - Porto, June 16, 2005
8
1.1 Consequence

Proliferation of developments
Platform #1
Platform #2
Platform #3
Platform #4
Application 1
UI #1
UI #2
UI #3
UI #4
Application 2
UI #5
UI #6
UI #7
UI #8
Application 3
UI #9
UI #10
UI #11
UI #12
Application 1
Application 2
Application 3
Platform model #1
Platform #1
Platform model #2
Platform #2
Platform model #3
Platform #3
Platform model #4
Platform #4
UI model #1
UI model #2
UI model #3
CAiSE'2005 Keynote address - Porto, June 16, 2005
9
1.1 Consequence

Why is it difficult?
– For the designer:
• Multiplicity of contexts of use
• No systematic design method
– For the user:
• Poor usability of resulting UI Fold & Drop technique: P. Dragicevic
• UI not adapted to the genuine context of use
– For the developer:
• Increase of development and maintenance costs
• Increasing development complexity
• Little of no factoring out of common elements
CAiSE'2005 Keynote address - Porto, June 16, 2005
10
1.2 What does it cover?

Conceptual modeling should cover
– Presentation: external manifestation of the
UI that is perceivable to the user (static)
– Dialog: ordering in time and space of
operations performed by the user/system
(dynamic)
– Modalities of interaction
• Graphical
• Vocal
• Haptic, tactile
CAiSE'2005 Keynote address - Porto, June 16, 2005
11
1.2 What does it cover?
Platform
Environment
User
Domain
Task
CAiSE'2005 Keynote address - Porto, June 16, 2005
12
1.3 Abstraction #1: the concrete UI

Starting point: FUI = Final User Interface
– Mark-up languages: HTML, WML, cHTML
– Programming languages: Visual Basic, Java

Many manifestations of the same object

Abstraction with respect to the toolkit
–
–
–
–
Presentation aspects
Events: generating, passing, controlling
Attributes: external (e.g., font) vs internal (e.g. state)
Primitives: life cycle
CAiSE'2005 Keynote address - Porto, June 16, 2005
13
1.3 Abstraction #1: the concrete UI
[Vanderdonckt & Bodart, 1993]
IO created
IO displayed
IO activated
IO deactivated IO undisplayed IO destroyed
PR_IO_Deactivate
PR_IO_Create PR_IO_Display PR_IO_Activate
PR_OI_Erase PR_OI_Destroy
PR_IO_Activate
PR_IO_ChangeAttribute

Definitions
– Original: Abstract Interaction Object = abstraction of the same
Concrete Interaction Objects independently of their
underlying presentation and dialog
– We have introduced this abstraction in 1993!
– Current: Concrete Interaction Object = abstraction of
interaction objects with respect to the underlying toolkit
CAiSE'2005 Keynote address - Porto, June 16, 2005
14
1.3 Abstraction #1: the concrete UI
[Limbourg, 2004]

CUI Model
CIO
graphicalCIO
graphicalContainer
graphicalIndividualComponent
Each graphicalCIO has
specific properties such
as isVisible, isEnabled,
fgColor, and bgColor.
 Each graphicalCIO is
sub-typed into one of
the two possible
categories:
graphicalContainer, and
graphicalIndividualCom
ponent
GrafiXML
CAiSE'2005 Keynote address - Porto, June 16, 2005
15
1.3 Abstraction #2: the abstract UI

Different CIOs can be used for the same purpose, but
with different interaction modalities

Definition
– Abstract Container = set of Abstract Individual Component
– AIC = abstraction of CIOs of the same type, but independently
of any interaction modality
– Abstract User Interface (AUI) = decomposition into AC+AIC
CAiSE'2005 Keynote address - Porto, June 16, 2005
16
1.3 Abstraction #2: the abstract UI
[Limbourg, 2004]
CAiSE'2005 Keynote address - Porto, June 16, 2005
17
1.3 Abstraction #2: the abstract UI
[Montero et al., 2005]

Notation: based on canonical abstract prototypes:
Larry Constantine, Helmut Windl, James Noble, &
Abs.container
Lucy Lockwood: http://www.forUse.com
A. component
input
output
navigation
control
select
IdealXML
CAiSE'2005 Keynote address - Porto, June 16, 2005
18
1.3 Abstraction #3: the task
[Paternò et al., 1997]

Task = set of actions carried out by a user in a given context to reach a
goal
 Logical decomposition of task into sub-tasks
 Temporal ordering: LOTOS operators
–
–
–
–
–
–
–
–
–
–
–
–
T1 >> T2
T1 [ ]>> T2
T1 |> T2
T1 [ ] T2
T1 [> T2
T1 |=| T2
T1*
T1{n}
T1 ||| T2
T1 |[x]| T2
[T]
T
Enabling
Enabling + information passing
Suspend/resume
Choice
Disabling (e.g. Form submit)
Independence (any order, but finished)
Iteration
Finite iteration
Concurrency
Concurrency + information passing
Optional
Recursion
CAiSE'2005 Keynote address - Porto, June 16, 2005
CTTE Editor
19
1.3 Abstraction #3: the task

Task definition = action + object
– Action types
• Acquire, render, modify, publish, compute,
derive,…
– Object types:
• Element, list, table, collection, compound,…
CAiSE'2005 Keynote address - Porto, June 16, 2005
20
1.3 Abstraction #3: the task
[Limbourg, 2004]
CAiSE'2005 Keynote address - Porto, June 16, 2005
21
1.3 Abstraction #4: the context of use
[Cameleon project, 2004]

Context of use= triple (U,P,E) where
– U is a user model: from cognitive psychology
– P is a platform model: subset of UAProf (W3C)
– E is an environment model: from ubiquitous
computing
Pick & Drop technique: J. Rekimoto
CAiSE'2005 Keynote address - Porto, June 16, 2005
22
1.3 Abstraction #4: the context of use
[Limbourg, 2004]
CAiSE'2005 Keynote address - Porto, June 16, 2005
23
1.3 Mapping the models
[Montero et al., 2005]
These mappings can be established:
triggers (tg):
{
,
updates (up):
x
observes (ob):
x
isExecutedIn (ex):
x
manipulates (ma): {
CAiSE'2005 Keynote address - Porto, June 16, 2005
}x
,
}x
24
1.3 Mapping the models
[Limbourg, 2004]

Mapping the models with a mapping
model (!!)
CAiSE'2005 Keynote address - Porto, June 16, 2005
25
1.4 What do we have so far?
Task &
Domain
Abstract User
Interface (AUI)
Concrete User
Interface (CUI)
Final User
Interface (FUI)
Method triggered: download a file
Object: computer file
Task
Classes
Modality-independent
Abstract Individual Component
Control AIC
Digital control AIC
Toolkit-independent
Concrete Interaction Object
HTML push button
Code
Graphical 2D push button
Windows push button
Physical control AIC
Graphical 3D push button
OSF/Motif
XmButton
VRML97/X3D
button
Physical push button
Software
key
Function
key
<input type=submit value=“Download" name=
Rendering
Download
CAiSE'2005 Keynote address - Porto, June 16, 2005
26
1.4 What do we have so far?
S=Source context of use
User S Platform S Environment S
Task and
Domain S
UsiXML
supported
model
Abstract user
Interface S
Concrete user
Interface S
UsiXML
unsupported
model
Reification
Final user
Interface S
CAiSE'2005 Keynote address - Porto, June 16, 2005
27
2. Forward engineering
[Limbourg, 2004]
2.1 Task and domain models
 2.2 From T&D to AUI

– Model-to-model transformation

2.3 From AUI to CUI
– Model-to-model transformation

2.4 From CUI to FUI
CAiSE'2005 Keynote address - Porto, June 16, 2005
28
2.1 Task and domain models
CAiSE'2005 Keynote address - Porto, June 16, 2005
29
2.1 Task and domain models
[Limbourg, 2004]
CAiSE'2005 Keynote address - Porto, June 16, 2005
30
2.1 Task and domain models
CAiSE'2005 Keynote address - Porto, June 16, 2005
31
2.2 From T&D to AUI
[Limbourg, 2004]

Definition of AUI structure
– Which objects should be logically grouped?

Definition of AIC types
– Which « functionnality » should assume AICs and
what data do they manipulate ?

Definition of spatio-temporal arrangement
– How should AIC be arranged in space and/or in
time ?

Definition of dialog control
– What is the valid flow of action on AIOs ?
CAiSE'2005 Keynote address - Porto, June 16, 2005
32
2.2 From T&D to AUI
[Limbourg, 2004]

Definition of AUI structure
NAC
LHS
RHS
::=
NAC
LHS
RHS
::=
CAiSE'2005 Keynote address - Porto, June 16, 2005
33
2.2 From T&D to AUI
[Limbourg, 2004]

Definition of AIC types
NAC
LHS
RHS
::=
AC = Abstract Component
AIC = Abstract Individual Component
CIC = Concrete Interaction Component
CAiSE'2005 Keynote address - Porto, June 16, 2005
34
2.2 From T&D to AUI
[Limbourg, 2004]

Definition of spatio-temporal
arrangement
NAC
LHS
RHS
::=
CAiSE'2005 Keynote address - Porto, June 16, 2005
35
2.2 From T&D to AUI

All transformations are in UsiXML
– Each model = instance of meta-model
– Each model = graph as instance of graph
type
– Each model-to-model transformation =
• graph transformation
• Set of productions
CAiSE'2005 Keynote address - Porto, June 16, 2005
36
2.2 From T&D to AUI
[Limbourg, 2004]

Typed model-to-model transformation
Meta-Meta-Model
Graph Structure
is instance of
Meta-Model
Our Meta-Model
Meta-Model Subset 1
Meta-Model Subset 2
e.g., Task+Domain Model
e.g., Concrete UI Model
Uses language
is instance of
is instance of
Initial UI Model
Transformation Rule
Resultant UI Model
e.g., MyTaskAndDomainModel
Our transformation catalog
e.g., MyConcreteUIModel
CAiSE'2005 Keynote address - Porto, June 16, 2005
37
2.2 From T&D to AUI

TransformiXML
CAiSE'2005 Keynote address - Porto, June 16, 2005
38
2.3 From AUI to CUI
[Limbourg, 2004]

Definition of CUI structure
– Which AIC is a window?

Definition of CIO type
– Which « widget » should represent which AIO ?

Definition of placement
– What layout can be specified between CIOs
– Definition of navigation
– Which container can be started or closed from which
container?

Definition of dialog control
– What is the dialog local to each CIO?
– What is the valid flow of action on CIOs?
CAiSE'2005 Keynote address - Porto, June 16, 2005
39
2.3 From AUI to CUI
[Limbourg, 2004]

Definition of CIO structure
NAC
LHS
RHS
::=
NAC
LHS
RHS
::=
::=
CAiSE'2005 Keynote address - Porto, June 16, 2005
40
2.3 From AUI to CUI

Definition of placement
NAC
[Limbourg, 2004]
LHS
RHS
::=
CAiSE'2005 Keynote address - Porto, June 16, 2005
41
2.4 From CUI to FUI

Model-to-code transformation
– By rendering (interpretation)
• Java (InterpiXML)
• Tcl/tk (QtkiXML)
• Flash (FlashiXML)
– By code generation
• HTML
• Visual Basic
CAiSE'2005 Keynote address - Porto, June 16, 2005
42
2.4 What do we have so far?
S=Source context of use
User S Platform S Environment S
Task and
Domain S
UsiXML
supported
model
Abstract user
Interface S
Concrete user
Interface S
UsiXML
unsupported
model
Reification
Abstraction
Final user
Interface S
CAiSE'2005 Keynote address - Porto, June 16, 2005
43
3. Reverse engineering
3.1 From FUI to CUI
 3.2 From CUI to AUI, task & domain

CAiSE'2005 Keynote address - Porto, June 16, 2005
44
3.1 From FUI to CUI
[Bouillon & Vanderdonckt, 2004]

Do you have the source code?
– Markup languages (e.g., HTML): static code
analysis (ReversiXML)

Do you have the compiled code?
– Programming languages (e.g., compiled C):
resource file extraction and static code analysis

Do you have the running application?
– Video analysis: computer vision

Do you have only the documentation?
– Image analysis: image processing
CAiSE'2005 Keynote address - Porto, June 16, 2005
45
3.1 From FUI to CUI
[Bouillon & Vanderdonckt, 2004]

ReversiXML
CAiSE'2005 Keynote address - Porto, June 16, 2005
46
3.2 From CUI to AUI, task & domain
[Limbourg, 2004]

Code-to-model and model-to-model
– CUI to AUI
NAC
LHS
RHS
::=
CAiSE'2005 Keynote address - Porto, June 16, 2005
47
3.2 From CUI to AUI, task & domain
Re-engineering
= reverse +
forward
 Possible reinterpretation
by QtkiXML

[Bouillon & Vanderdonckt, 2004] CAiSE'2005 Keynote address - Porto, June 16, 2005
48
3.2 What do we have so far?
S=Source context of use
User S Platform S Environment S
Task and
Domain S
UsiXML
supported
model
Abstract user
Interface S
Concrete user
Interface S
UsiXML
unsupported
model
Reification
Abstraction
Final user
Interface S
CAiSE'2005 Keynote address - Porto, June 16, 2005
49
3.2 What do we want to get?
[Cameleon project, 2004]
S=Source context of use
User S Platform S Environment S
T=Target context of use
User T Platform T Environment T
http://www.plasticity.org
Task and
Domain S
Task and
Domain T
UsiXML
supported
model
Abstract user
Interface S
Abstract user
Interface T
Concrete user
Interface S
Concrete user
Interface T
UsiXML
unsupported
model
Reification
Abstraction
Final user
Interface S
Final user
Interface T
CAiSE'2005 Keynote address - Porto, June 16, 2005
Reflexion
Translation
50
3.3 Lateral engineering

Definition = model-to-model
transformation applied at the same level
of abstraction
– Same context of use
– Across various contexts of use

Examples
– Simple CUI adaptation: widget replacement
– Design-time vs run-time adaptation
CAiSE'2005 Keynote address - Porto, June 16, 2005
51
Example 1: widget replacement (1)
[Limbourg et al., 2004]
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cuiModel name="MyModel">
<version modifDate="2004-03-24T17:09:17.402+01:00" xmlns="">7</version>
<authorName xmlns="">Youri</authorName>
<window height="500" width="600" name="Formulaire (2/5)" id="window_1">
<box relativeHeight="100" name="box1_0" id="box1_0">
<box type="vert" name="boxTodo" id="boxTodo">
...
<box type="horiz" name="box_2_2_2_1" id="box_2_2_2_1">
<textComponent defaultContent="Sexe" isBold="true" id="label_2"/>
<radioButton groupName=“grupo01" defaultContent="Femme"
defaultState="false" id="radiobutton_0"/>
<radioButton groupName="grupo01" defaultContent="Homme"
defaultState="true" id="radiobutton_1"/>
</box>
...
</box>
</box>
</window>
</cuiModel>
Excerpt for an usiXML CUI specification.
CAiSE'2005 Keynote address - Porto, June 16, 2005
52
Example 1: widget replacement (2)
CAiSE'2005 Keynote address - Porto, June 16, 2005
53
Example 1: widget replacement (3)
The UsiXML graph before applying any rule
CAiSE'2005 Keynote address - Porto, June 16, 2005
54
Example 1: widget replacement (4)
[Limbourg et al., 2004]
Rule 1: Create a new comboBox with the same
id and name as the name of the group of
radioButtons.
NAC
RHS
LHS
CAiSE'2005 Keynote address - Porto, June 16, 2005
55
Example 1: widget replacement (5)
Rule 1: Create a new comboBox with the same
id and name as the name of the group of
radioButtons.
The usiXML graph after aplying the first rule
CAiSE'2005 Keynote address - Porto, June 16, 2005
56
Example 1: widget replacement (6)
[Limbourg et al., 2004]
Rule 2: Convert every radioButton within the
group “x” into an item for the comboBox “x”, we
have just created.
LHS
RHS
::=
CAiSE'2005 Keynote address - Porto, June 16, 2005
57
Example 1: widget replacement (7)
Rule 2: Convert every radioButton within the
group “x” into an item for the comboBox “x”, we
have just created.
The usiXML graph after aplying the second rule
CAiSE'2005 Keynote address - Porto, June 16, 2005
58
Example 1: widget replacement (8)
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cuiModel name="MyModel">
<version modifDate="2004-03-24T17:09:17.402+01:00" xmlns="">7</version>
<authorName xmlns="">Youri</authorName>
<window height="500" width="600" name="Formulaire (2/5)" id="window_1">
<box relativeHeight="100" name="box1_0" id="box1_0">
<box type="vert" name="boxTodo" id="boxTodo">
...
<box type="horiz" name="box_2_2_2_1" id="box_2_2_2_1">
<textComponent defaultContent="Sexe" isBold="true" id="label_2"/>
<comboBox id="comboBox001" name="label_3" isDropDown="true">
<item id="radiobutton_0" name="radiobutton_0" defaultContent="Femme"/>
<item id="radiobutton_1" name="radiobutton_1" defaultContent="Homme"/>
</comboBox>
...
</box>
</box>
</window>
</cuiModel>
Excerpt from the final transformated usiXML specification
CAiSE'2005 Keynote address - Porto, June 16, 2005
59
Example 1: widget replacement (9)
CAiSE'2005 Keynote address - Porto, June 16, 2005
60
Example 2: Adaptation

Two forms of adaptation
– Adaptability: user-initiated
– Adaptivity: system-initiated

Four steps: user, system, third party, combination
–
–
–
–

Detection
Computation
Decision
Execution
2 Adaptivity forms
– UI remain the same: vectorial UIs
– UI change at run-time: plastic UIs
CAiSE'2005 Keynote address - Porto, June 16, 2005
61
Example 2: Plastic UI
[Grolaux, Van Roy, Vanderdonckt, 2002]
CAiSE'2005 Keynote address - Porto, June 16, 2005
62
Example 3: multimodal UI
[Stanciulescu, Limbourg, Michotte, Vanderdonckt, 2005]
Need for additional abstraction
 Additional transformation rules

CAiSE'2005 Keynote address - Porto, June 16, 2005
63
Example 4: Crazy UIs
[Molina, Vanderdonckt, Gonzalez, 2005]

Use the same model
– for 3D UI: Cube
– for migration across platforms in virtual
reality: MigriXML
CAiSE'2005 Keynote address - Porto, June 16, 2005
64
5. Conclusion

A multi-path support of MDA
TransformiXML
IdealXML
FlashiXML
Rendering
UsiXML
models: task,
domain
Graph
transformations
UsiXML model:
UsiXML model:
Graph
Concrete user
Abstract user
transformations
interface
interface
Generative
programming
Final user
interface
VisualiXML
Derivation rules
KnowiXML
GrafiXML
VisiXML
SketchiXML
FormiXML
CAiSE'2005 Keynote address - Porto, June 16, 2005
ReversiXML
65
5. Conclusion

Conceptual modeling of UIs
–
–
–
–
–
–
Separation of concerns is feasible
Correlability of models is required
It is possible to have multiple abstractions on demand
Not all models should be involved
Everything is in UsiXML
Extreme modeling or extreme engineering
• Everything is in the model: relatively easy
• Model-to-model transformation: harder
• Model-to-code generation: hardest

Industrial practise
– Multi-path development
– Sketching of UI
CAiSE'2005 Keynote address - Porto, June 16, 2005
66
Future trend: sketching?
CAiSE'2005 Keynote address - Porto, June 16, 2005
[Coyette & Vanderdonckt, 2005]
67
I would like to dedicate this talk to

Prof. Em. François Bodart (Univ. of Namur, BE) for
– Inoculating me the virus of (meta-)modeling and structured
design of information systems
– Sharing with me his vision about user interfaces and
interactive information systems
CAiSE'2005 Keynote address - Porto, June 16, 2005
68
I would like to thank

BCHI team members for their involvement and their constant
effort in the UsiXML initiative among others
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Laurent Bouillon: ReversiXML, UsiXML
Benoît Collignon: PlastiXML
Adrien Coyette: SketchiXML
Murielle Florins: Graceful degradation
Monica Gemo: multi-platform UIs and annotations
Juan Gonzalez: 3D UsiXML
Donatien Grolaux: detachable UIs, DistriXML
Josefina Guerrero: UIs for workflows
Quentin Limbourg: multi-path development, UsiXML, all tools
Céline Mariage: MetroWeb, guidelines
Benjamin Michotte: GrafiXML, TransformiXML, UsiXML
José Pascual Molina: MigriXML, VUIToolkit
Francisco Montero: IdealXML
Adrian Stanciulescu: ModaliXML, UsiXML
Daniela Trevisan: augmented reality
…
CAiSE'2005 Keynote address - Porto, June 16, 2005
69
I would like to thank
João Falcão e Cunha, Oscar Pastor,
Nuno Jardim Nunes
 The CAiSE Advisory Committee

– Janice Bubenko
– Collette Roland
– Arne Solvberg
CAiSE'2005 Keynote address - Porto, June 16, 2005
70
… and you for your attention
Free download and register
USer Interface eXtensible Language
http://www.usixml.org
SIMILAR, the European task force making
user interfaces similar to human-to-human communication
http://www.similar.cc
Home Page of BCHI Lab
http://www.isys.ucl.ac.be/bchi
CAiSE'2005 Keynote address - Porto, June 16, 2005
71