Project ‘aire’
and Semantic Networks
Seminar ‘Multimodal Rooms’
Jochen von Spiczak

content:
overview
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
- project ‘aire’
· overview / interesting examples
- semantic networks
· introduction
· knowledge representation in IEs
· description of SNs,
especially in intelligent rooms
· realization of SNs
· conclusion / future work
slide 2 / 40

content:
project ‘aire’
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- ‘aire’ ~
· Agent-based Intelligent Reactive
Environments
- MIT Artificial Intelligence Lab
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 3 / 40
- examining perceptual computing
systems and applications

content:
overview
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- tools for collaboration support:
· supporting interactions among people,
spaces and mobile devices
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
coll. support
project ‘aire’
slide 4 / 40

content:
overview
overview
project ‘aire’
- overview
- examples
semantic networks
- software infrastructure
- introduction
· communication
- knowledge
representation in IE
· resource management
- SN in general
· device control
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
coll. support
infrastructure
project ‘aire’
slide 5 / 40

content:
overview
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- human-computer interfaces
· new interfaces for people to interact
with their environments
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
coll. support
infrastructure
project ‘aire’
slide 6 / 40
HCI

content:
overview
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- ‘aire’ spaces
· large range of form factors and sizes
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
conference rooms,
personal workspaces,
transitional spaces
coll. support
infrastructure
project ‘aire’
slide 7 / 40
HCI

content:
examples
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
conference room,
personal workspace,
transitional spaces
coll. support
infrastructure
project ‘aire’
slide 8 / 40
HCI

content:
examples
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- ‘Metaglue’
· Java-based agent infrastructure
- SN in general
· providing communication and
levels of abstraction
- SN in intelligent
rooms
· base of the rest of the project
- knowledge
representation in IE
- realization of SN
- example of a SN
- conclusion /
future work
summary
coll. support
infrastructure
project ‘aire’
slide 9 / 40
HCI

content:
examples
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
coll. support
infrastructure
project ‘aire’
slide 10 / 40
HCI

content:
examples
overview
project ‘aire’
- overview
- examples
- visit this very day:
semantic networks
- introduction
- knowledge
representation in IE
http://www.ai.mit.edu/projects/aire
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
coll. support
infrastructure
project ‘aire’
slide 11 / 40
HCI

content:
overview
project ‘aire’
semantic networks
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 12 / 40
- one outstanding aspect of the
‘aire’ project
- knowledge representation as a
fundamental design question
· semantic networks as such a
representation

content:
introduction
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- ever-increasing amounts of data
about users
- knowledge
representation in IE
· who they communicate with,
- SN in general
· their location,
- SN in intelligent
rooms
- realization of SN
· visited web pages, ...
- example of a SN
- conclusion /
future work
summary
slide 13 / 40
- aim:
· quick, correct assumptions about what
the user would like to do next

content:
overview
project ‘aire’
knowledge representation in IE
- overview
- examples
semantic networks
- five different roles that KRs play
- introduction
· surrogates for the real world
- knowledge
representation in IE
» easily/less easily
identifiable entities
· ontological
commitments
- SN in general
» KR imposes
a filter on what the environment
of
reasoning
· theories
making
can
seedeductions
and act onbased on their layout
» KR defines
how inferences are made by the
for
computation
· media
» example of camera-/microphone-signal
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
» important for reasoning about items and
reasoning system
» KR must
able toexpression
operate in an efficient
· media
forbe
human
» constraints
of inferences
which are
manner viakinds
computational
processes
» KR
represents
the human view
of the
world
allowed
or recommended
by the
system
» without taking up inordinate quantities of
» must
simple or
todisk
use space
time, be
memory,
» turn into human-readable representations
slide 14 / 40

content:
overview
project ‘aire’
knowledge representation in IE
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
- much literature about developing
contextual systems for IE
- problems of other approaches
· interpretation and inferencing is done
by the application
· application must collate and assemble
disparate pieces of data
· no cohesive representation of the
gained knowledge
slide 15 / 40

content:
overview
project ‘aire’
SN in general
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 16 / 40
- need for a representation that…
· can store information on objects
· can easily track and follow the
relationships between them
- introducing semantic networks as
one such representation

content:
SN in general
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- concept of semantic networks
· information represented in a directed
graph:
- SN in general
» nodes  objects
- SN in intelligent
rooms
» links
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 17 / 40
 relationships

content:
SN in general
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- variety of node and link types
· nodes just like…
- knowledge
representation in IE
» people,
- SN in general
» meetings,
- SN in intelligent
rooms
» documents,
- realization of SN
» issues,
- example of a SN
- conclusion /
future work
summary
slide 18 / 40
» discourse items,
» commitments,…

content:
SN in general
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- variety of node and link types
· links just like…
- knowledge
representation in IE
» relationship between people,
- SN in general
» responsibilities for objects (groups/people),
- SN in intelligent
rooms
» hierarchies between contributors,
- realization of SN
» attendance to meetings,
- example of a SN
- conclusion /
future work
summary
slide 19 / 40
» interest for objects, …

content:
overview
project ‘aire’
SN in general
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 20 / 40
- further link types:
· endpoints for links not limited to storage
nodes
· can also be created to other links
- allows more complex interactions
in the data model

content:
overview
project ‘aire’
SN in general
- overview
- examples
semantic networks
- easy to discover…
- introduction
· information about a particular object
- knowledge
representation in IE
· relationships between different objects
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 21 / 40
- little information stored in nodes
· network relies more on the links to define
the relationships

content:
overview
project ‘aire’
SN in general
- overview
- examples
semantic networks
- some inherent advantages
- introduction
· adding data is highly straightforward
- knowledge
representation in IE
· changing data is highly localized
- SN in general
· making inferences is fast and easy
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 22 / 40

content:
overview
project ‘aire’
SN in general
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 23 / 40
- further advantages
· great flexibility in terms of the data
which is captured and interpreted
· extremely efficient
· easy to bring into human-readable form

content:
overview
project ‘aire’
SN in intelligent rooms
- overview
- examples
semantic networks
- introduction
- several areas for using a SN-based
representation
- knowledge
representation in IE
· user knowledge
- SN in general
· meeting management
- SN in intelligent
rooms
- realization of SN
· location management
- example of a SN
- conclusion /
future work
summary
slide 24 / 40
- SN as an appropriate and
valuable piece of infrastructure

content:
overview
project ‘aire’
SN in intelligent rooms
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
- user knowledge
· knowledge of the users and
the individual spaces they work with
· simplistic level
» simple set of objects, spaces and
the user’s current location
» queries like “Where is Steve located?”
· system as an “intelligent assistant”
» far more information and relations must be
included
» more complex queries like
“Who is Joe’s superior?”
slide 25 / 40

content:
overview
project ‘aire’
SN in intelligent rooms
- overview
- examples
semantic networks
- meeting management
- introduction
· capturing meetings as the occur
- knowledge
representation in IE
· typical information
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 26 / 40
» agenda topics,
» action items,
» supporting and dissenting arguments,
» documents such as presentations or
web references, …
· can be linked to a video or audio
capture of the meeting

content:
overview
project ‘aire’
SN in intelligent rooms
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
- location management
· location information important for IEs
· especially true when integrating
hand-held devices
» require knowledge of the current location
» own coordinate framework of each system
· SN as a location infrastructure
» integration and translation between different
coordinate systems
» specify own location descriptions,
such as “at home”
slide 27 / 40

content:
overview
project ‘aire’
realization of SNs
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- technology of SNs deployed within
the project ‘aire’
- current implementation
- SN in intelligent
rooms
· works on top of a SQL database
- realization of SN
· easily extendable to use…
- example of a SN
- conclusion /
future work
» object-oriented databases
summary
» resource description framework (RDF)
back-ends
slide 28 / 40

content:
overview
project ‘aire’
realization of SNs
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 29 / 40
- access to the semantic network
· using the Metaglue agent infrastructure
of project ‘aire’
· several agents used to access the SN
» MeetingModel agent
» GUI agent
» other viewing/editing agents
· network query language

content:
overview
project ‘aire’
realization of SNs
- overview
- examples
semantic networks
- introduction
- semantic network itself
· provides only basic interface methods
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
semantic network
slide 30 / 40

content:
overview
project ‘aire’
realization of SNs
- overview
- examples
semantic networks
- MeetingModel agent
- introduction
· serves merely as an interface to SN
- knowledge
representation in IE
· abstracts out some of the lookup tasks
- SN in general
· broadcast of any changes
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
MeetingModel
semantic network
slide 31 / 40

content:
overview
project ‘aire’
realization of SNs
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- network query language
· allows user or agents to specify queries
in a formal language
· definition of rules in a syntax similar to
Prolog
- realization of SN
- example of a SN
- conclusion /
future work
summary
MeetingModel
query language
semantic network
slide 32 / 40

content:
realization of SNs
overview
project ‘aire’
- overview
- examples
semantic networks
- GUI agent
- introduction
· tree-structure view of stored information
- knowledge
representation in IE
· creation/editing of information nodes
- SN in general
· currently the primary conduit for
information to get into the SN
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
GUI agent
MeetingModel
query language
semantic network
slide 33 / 40

content:
overview
project ‘aire’
- overview
- examples
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- example of a SN
- conclusion /
future work
summary
slide 34 / 40
realization of SNs

content:
overview
project ‘aire’
realization of SNs
- overview
- examples
semantic networks
- introduction
- other viewing/editing agents
· providing different views of the data:
- knowledge
representation in IE
» web interfaces,
- SN in general
» graph networks,
- SN in intelligent
rooms
» specialized meeting views which simply
present the agenda or commitments, …
- realization of SN
- example of a SN
- conclusion /
future work
summary
GUI agent
MeetingModel
…
query language
semantic network
slide 35 / 40

content:
example of a SN
overview
project ‘aire’
- overview
- examples
face tracking
Mercedes SLK
semantic networks
- introduction
- knowledge
representation in IE
speech recognition
expert-in
- SN in general
room 225
expert-in
expert-in
works-in
assisted
- SN in intelligent
rooms
rest
Jochen
McDonough
- realization of SN
- example of a SN
Rogina
Stiefelhagen
- conclusion /
future work
bores-to-death
currently-doing
responsible-for
responsible-for
responsible-for
summary
multimodal rooms
seminar talk
belongs-to
slide 36 / 40

content:
example of a SN
overview
project ‘aire’
- overview
- examples
face tracking
Mercedes SLK
semantic networks
- introduction
- knowledge
representation in IE
room 225
speech recognition
expert-in
- SN in general
expert-in
expert-in
works-in
assisted
- SN in intelligent
rooms
rest
Jochen
McDonough
- realization of SN
- example of a SN
Rogina
Stiefelhagen
- conclusion /
future work
bores-to-death
currently-doing
responsible-for
responsible-for
responsible-for
summary
multimodal rooms
seminar talk
belongs-to
- “In
room
225.”
“John
McDonough.”
slide 37 / 40

content:
example of a SN
overview
project ‘aire’
- overview
- examples
face tracking
Mercedes SLK
semantic networks
- introduction
- knowledge
representation in IE
speech recognition
expert-in
- SN in general
room 225
expert-in
expert-in
works-in
assisted
- SN in intelligent
rooms
rest
Jochen
McDonough
- realization of SN
- example of a SN
Rogina
Stiefelhagen
- conclusion /
future work
bores-to-death
currently-doing
responsible-for
responsible-for
responsible-for
summary
multimodal rooms
seminar talk
belongs-to
slide 38 / 40

content:
overview
project ‘aire’
conclusion / future work
- overview
- examples
semantic networks
- introduction
- seems to be the most appropriate
knowledge representation for IEs
- knowledge
representation in IE
· flexibility of data
- SN in general
· adding/changing data highly localized
- SN in intelligent
rooms
- realization of SN
· inference generation extremely efficient
- example of a SN
- conclusion /
future work
summary
- future work
· integration within other projects
· automatic sensor-based information
recognizing
· incorporate user preference information
slide 39 / 40

content:
summary
overview
project ‘aire’
- overview
- examples
- project ‘aire’
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- semantic networks
· one approach to KR
· inherent advantages in general…
- example of a SN
· … and in the context of IEs
- conclusion /
future work
· realization
summary
» implementation
» access hierarchy
· example
· most appropriate approach to KR
slide 40 / 40

content:
summary
overview
project ‘aire’
- overview
- examples
- project ‘aire’
discussion
semantic networks
- introduction
- knowledge
representation in IE
- SN in general
- SN in intelligent
rooms
- realization of SN
- semantic networks
· one approach to KR
…?
· inherent advantages in general…
- example of a SN
· … and in the context of IEs
- conclusion /
future work
· realization
summary
» implementation
» access hierarchy
· example
· most appropriate approach to KR
slide 40 / 40