Agents on the
Semantic Web
David De Roure
Intelligence Agents Multimedia
Dept of Electronics and Computer Science
University of Southampton, UK
[email protected]
http://www.ecs.soton.ac.uk/~dder
Overview




The Pervasive Information Fabric
Agents state of play
Why agents need metadata
Onward to ontologies
Semantic Web Technologies November 2000
2
Background
This talk is based on the experience of building
several applications (research prototypes) involving
agents. These include:
 Conceptual
hypermedia and ontologies
 Context-aware hypermedia linking
 Information middleware for pervasive computing
 Collaborative filtering systems
 ‘Content based’ navigation of multimedia (images and
music)
We have also developed an Agent framework for
Distributed Information Management
Semantic Web Technologies November 2000
3
Background
Current activities include:
 Mixed
reality adaptive information systems
 Advanced knowledge technologies
 Disappearing computer initiative project
 Simulation of very large scale distributed
systems
 Grid computing (information/knowledge grid)
Semantic Web Technologies November 2000
4
The Pervasive
Information Fabric
The Web of the past…




Is mainly a document dissemination Web
Can only link to multimedia
Assumes a traditional Web browser
Has static hyperstructure
Semantic Web Technologies November 2000
6
The Web of the future is…







Multimedia, including temporal media
Mobile (with different style of working)
Adaptive and open e.g. XLink
Collaborative
Automated (machine-to-machine e.g. XML)
Semantic (of course!)
And…
Semantic Web Technologies November 2000
7
Semantic Web Technologies November 2000
8
…pervasive






Embedded internet
e.g. The Disappearing Computer Initiative
Large numbers of devices
Ad hoc networking (some support from IPv6)
Systems need to be self organising
Don’t wait for 1000s of bluetooth devices before
exploring scalability issues!
We call the middleware the Pervasive Information
Fabric
Semantic Web Technologies November 2000
9
…virtual



Interface is 3D worlds, telepresence, VR
‘click’ on object, query has spatial context
Visualisation of results?
For example,abstract 3D midi visualisaton with links
Semantic Web Technologies November 2000
10
Semantic Web Technologies November 2000
11
Scenario 1 – this meeting!



Our devices communicate using wireless/ad hoc
networking, publishing information resources and
associated metadata
A hyperstructure (web) is created on-the-fly,
enabling us to navigate our local information space.
Links derived from metadata, metadata derived
from documents; also bookmarks.
When a message comes in (e.g. mobile phone call)
it is routed appropriately to minimise invasiveness
Semantic Web Technologies November 2000
12
Scenario 2 – the musician





Musician walks on stage with bluetooth guitar
Musical devices on stage are located automatically,
musician chooses to control appropriate
instrument(s)
Plays a few notes, musical score appears on
private display
More…the musician appears in virtual world, other
musicians there too, playing virtual instruments,
audible in physical world
(AR meets VR)
Semantic Web Technologies November 2000
13
Scenario 3 – the information grid

Group of experts deciding whether to make
changes to production line in manufacturing
organisation
Use multiple simulations to investigate
Processes farmed out across clusters (server
farms) on WAN
Visualise results locally, collaboratively
Compare with results from previous runs

(Is this The Web?)




Semantic Web Technologies November 2000
14
Agents
State of Play
Software agents


A buzzword for over 5 years now!
Classic early papers:
 ‘Agents
that reduce work and information overload’
1994
 ‘Intelligent Agents: Theory and Practice’ 1995


The paradigm of weak agency is widely accepted,
especially in the Web area
There is a convenient subcategorisation into:
 Personal
and information agents
 Multiagent systems
Semantic Web Technologies November 2000
16
Personal information assistants





Personal assistants that collaborate with the user at
the user interface
Learning by ‘watching over their shoulders’,
building and maintaining model of user
Believable agents (cf traditional media)
Movie/CD/book/document recommender agents
exist, though with fairly weak user models
For ‘people’, read ‘businesses’
Semantic Web Technologies November 2000
17
Multiagent systems





Whole is greater than sum of the parts
Requires agent communication languages (eg
KQML, FIPA ACL)
Ontology required – to agree the terms to be
exchanged in communications
Agent frameworks have emerged
Few large scale systems exist
Semantic Web Technologies November 2000
18
Nwana’s appraisal
“The ontology issues has always been considered
secondary to other issues such as cooperation,
negotiation, formalisation and logics for beliefs,
desires and intentions, etc. ... This problem is at
the core of the agent interoperability issue – is it
reasonable to expect knowledge and cooperation
level interoperability without a significant degree
of ontological sophistication of the agents
concerned?”
Nwana and Ndumu, A Perspective on Software Agents Research, BT
Labs
Semantic Web Technologies November 2000
19
My appraisal


XML, RDF(S) are useful infrastructure
Need also to address…
– putting the ‘multi’ into multiagent!
 Security – not just an add-on
 Performance
 Real systems!
 Scalability
Semantic Web Technologies November 2000
20
Current work





Collaboration and negotiation between agents
Market-based models, e.g. auctions
See http://www.ecs.soton.ac.uk/~nrj
Applications in e-commerce but also telecoms
Agents in the PIF; e.g. briefing room scenario –
routing information in right format to right device at
right time, taking account of security and
invasiveness issues
Semantic Web Technologies November 2000
21
Agents and metadata
Agents…

Use metadata to find resources and work with them
They also create and maintain metadata

For example, our ‘query by humming’ system

 MIDI
data gathered from net
 Tidied
 Channels classified and indexed
 Queries routed by index servers
 Results presented (e.g. SMIL)

There is also metadata associated with the agents!
Semantic Web Technologies November 2000
23
Agents…recommender systems






MEMOIR, an early agent-based recommender
system
Logged trails in object-oriented database
Users could ask ‘who else has looked at these
documents?’ and ‘what else did they look at?’
Later used keywords from docs in trails (and
bookmarks) to model users
Can now search and present results with a notion
of context
Through COHSE, will be able to navigate concept
space
Semantic Web Technologies November 2000
24
The Web
UI Agent
Web Browser
Facilitator(s) and Agent Server(s)
Memoir Agent
Link Server
Image DBase
Wrapper 1
Resource 1
Wrapper n
Resource m
Semantic Web Technologies November 2000
25
Open Hypermedia
Link database
Separable
hyperstructure
Note the
direction
of this
arrow!
documents
Semantic Web Technologies November 2000
26
Hyperstructure as metadata




Open hypermedia introduces separable
hyperstructure, e.g. as supported through XLink
Southampton model introduces reuseable
separable hyperstructure, which can be applied to
new documents
Agents used for link resolution
Agents build link databases and maintain them
Semantic Web Technologies November 2000
27
A case for streaming metadata?

Three kinds of multimedia streams:
Media on demand
 Live, one way
 Two way


Live metadata may be created by:
Producer (e.g Big Brother)
 Video segmentation and classification
 Annotation



There are multiple simultaneous flows of data, from
multiple sources
Metadata needs to come from upstream in
production process!
Semantic Web Technologies November 2000
28
Onward to ontologies
Agents using the Semantic Web

Scenarios revisited:
 Workshop
scenario. Use ontology for our domain of
interest. (And for IST?) Multimedia ontology for
delivery of multimedia content.
 Musician. Use ontology for navigating the musical
information space. What about information about
musical devices? Creation of metadata for new
compositions.
 Information grid. Ontologies for manufacturing and
organisation. What about computational resources?


Also need to find agents, and to communicate with
them
Hence working with multiple, distributed,
Semantic Web Technologies November 2000
30
ontologies.
SoFAR



SoFAR (the Southampton Framework for Agent
Research) is a versatile multi-agent framework designed for
Distributed Information Management tasks.
SoFAR embraces the notion of proactivity as the
opportunistic reuse of the services provided by other agents,
and provides the means to enable agents to locate suitable
service providers.
SoFAR combines some ideas from the distributed
computing community with the performative-based
communications used in many agent systems:
communications in SoFAR are based on the
startpoint/endpoint paradigm, which is the foundation of
Nexus, the communication layer at the heart of the
Computational Grid.
Semantic Web Technologies November 2000
31
Index of /distrib/sofar024/ontology
Name
Parent Directory
actions/
base/
fohm/
infrastructure/
metadata/
multimedia/
system/
web/
Last modified
16-Nov-2000 22:45
16-Nov-2000 22:45
16-Nov-2000 22:45
16-Nov-2000 22:45
16-Nov-2000 22:46
16-Nov-2000 22:45
16-Nov-2000 22:46
16-Nov-2000 22:45
16-Nov-2000 22:45
Size Description
–
Apache/1.3.9 Server at www.sofar.ecs.soton.ac.uk Port 80
Semantic Web Technologies November 2000
32
HyStream example

Agents deal with multimedia streams
ACL handles session control, synchronisation,
linking
Publish-subscribe

Example predicates:


ContainsContour(music, contour, time)
 Relay(multicast_address1, multicast_address2)


The mediadata-metadata distinction becomes
blurred, e.g. when features extracted from
multimedia documents
Semantic Web Technologies November 2000
33
Agents supporting the Semantic Web


Metadata, vocabularies, thesauri, ontologies are
‘stuff’ in the information space
Note distinction between:
 Automation
of tasks, i.e. computer-to-computer
interaction (a goal of XML et al)
 Ontology capture and design tools involving humans

Agents help with both; it’s the first that really helps
agents (and is supported by current web
technologies)
Semantic Web Technologies November 2000
34
The (distributed) intelligence



Currently agents ‘wrap’ existing inference engines
Agent Based Computing is an appropriate
paradigm to work in complex world with multiple
ontologies, fragments, multiple inferencing engines
We anticipate further decomposition into multiple
inferencing components
Semantic Web Technologies November 2000
35
Future work at Southampton





Ontology support for agent collaboration and
negotiation
Ontologies and hypermedia (COHSE, Pervasive)
Use of OIL/DAML
Instantiating application-neutral ontologies for
agent infrastructure
Agents supporting the knowledge lifecycle
Semantic Web Technologies November 2000
36
Summary
Summary




The fabric of the Web is changing
DIM agents eat metadata for breakfast
XML and RDF(S) support agent-agent interaction
Agents not only use but also automate the
construction and maintenance of metadata and
ontologies
[email protected]
Semantic Web Technologies November 2000
38
Motivation
“There will always be plenty of things to
compute in the detailed affairs of millions of
people doing complicated things”
“Consider a future device for individual use, which is a sort of
mechanized private file and library. It needs a name, and to coin
one at random, “memex” will do. A memex is a device in which
an individual stores all his books, records, and communications,
and which is mechanized so that it may be consulted with
exceeding speed and flexibility. It is an enlarged intimate
supplement to his memory” – Bush, July 1945
Semantic Web Technologies November 2000
39