towards a ubiquitous
semantics of interaction
Alan Dix
Lancaster University
formalism in HCI
adolescence
or …
rigormortis
formalism in HCI
• blocks to uptake?
– expertise
– utility
– tools
– cost
– fear
• use only where value exceeds cost
– safety critical, ?generic widgets?
ubiquity
• lots of simple devices
• but complex context:
– lots of users
– mobile
– context dependent behaviour
• good for FM
ubiquity
simple devices
complex context
good for formal methods!
models
discrete
continuous
• most common
• good for GUI
• familiar notations
• very few
• VR, ubicomp
• simpler semantics
what do they mean?
notation
fever!
notations
• lots of them
• why?
– cult of notation in computer science
– differing paradigms and problems
– tractability
– familiarity
myriad notations
• problems
– communication
– education
– tools
– meaning and satisfaction
?? moving to ubiquity ??
routes
myriad
notations
interoperability
translation
shared
semanitics
UML-ishness
single notation
one size fits all
transformation
model 1
(notation A)
model 2
(notation B)
Galois connection
common semantics?
model 1
model 2
model 1
model 2
complete meta-semantics
common semantics?
model 1
model 2
shared sub-semantics
external phenomena
trace / scenario
issues
advantages
problems
•
•
•
•
• not obvious
obvious
physical/real
common
relates to real time
– e.g. CCS/CSP
•
•
•
•
internal structure
binding to real
granularity/instants
real time
meta-model
Z
model
*
x
instances
x
a
Y
b
c
a
happenings
x
b
a
x
a
Y
b
c
time
event stream
model
P  abP | acP
instances
a – b – a – b – a – c – a – b …
happenings
<
<
<
<
<
<
<
time
turntaking & states
some
state
action
response
other
state
s0 – a1 – r1 – s1 – a2 – r2 – s2 – …
one happening or two?
mapping
model 1
P  abP | acP
<
<
<
<
<
model 2
*x
a
<
<
Z
Y
b
c
interstices
<
<
?
<
<
NO
<
<
status and events
A
S1
S2
A
S1 – A – S2 – A – S1 – A – S2 ….
hierarchies – task analysis
0.
DSV-IS talk
1.
prepare
2.
talk
3.
discussion
2.1
introduction
2.2
show slide
2.3
conclusions
3.1
question
3.2
answer
0. DSV-IS talk
1. prepare
2. talk
2.1 intro 2.2 sld 1 2.2 sld 1 2.3 conc
3. discussion
3.1 Q 3.2 A 3.1 Q 3.2 A
time
layers – UI events
None
Word
Single
Double
Dotted
application level
dialogue level
lexical level
change selection format
underline is ‘word’
mouse activity on menu
subscript
size is 10
finished!
click
radio btn
font size menu
click OK
continuous
• status–event analysis
• CS hybrid system literature
• TACIT – hybrid Petri Nets
& continuous interactors
• Wuther???? – systems theory
common features
• actions
– at events, discrete changes in state
• interstitial behaviour
– between events, continuous change
other continuous issues
• granularity issues – do it today
• two timing – infinitely fast times
• temporal gestalt – words, gestures
summary
• harder than it looks!
… but easier than pairwise matching
• consequences:
less sloppy semantics
clear semantics  shared tools
foundation …