Temporal Aspects of
Intelligent Multimedia Retrieval
and Presentation
Zhisheng Huang, Jacco van Ossenbruggen, and
Lynda Hardman
CWI
The Netherlands
26 Oct 2005
I2RP Meeting
Outline
• Motivation and Background
• Temporal approaches for intelligent
multimedia retrieval and presentation
• Temporal reasoning in the Semantic
Web
• Research Plan
• Past work (by Zhisheng)
• Demo
26 Oct 2005
I2RP Meeting
Time and Multimedia
Documents and Presentation
• Time is one of the most crucial aspects of
multimedia documents and presentation.
• Audio, video, animation clips are presented with
time goes,
• Image archivies are usually displayed with
timelines,
• Text information describes events, time
instants, intervals/periods,
• Presentation involves a series of actions
which occur in time.
26 Oct 2005
I2RP Meeting
Temporal Aspects and Their
Description
• Time instant: year, date, time clock
• Time intervals and events: golden age,
Monet’s born,
• Temporal Modifiers: after, before, in,
during,
• Temporal operators: always in the past,
sometimes in the past, never in the past, in
the future, now, at present
• Action operators: doing, acting, …
26 Oct 2005
I2RP Meeting
Semantics of Time
Three Main Approaches:
1. The Quantitative Time Approach: time
is a collection of time instants and
intervals
2. The Temporal Logic Approach: time
fluent is considered as state transitions
3. The Dynamic Logic Approach: time
fluent is considered as results of taking
actions
26 Oct 2005
I2RP Meeting
Temporal Retrieval vs.
Temporal Reasoning
• Temporal retrieval: find entities which
satisfy a temporal specification.
• Temporal reasoning: check if a
temporal property holds.
• Temporal retrieval can be achieved
by converting answers with “yes/no”
into answers with entities.
26 Oct 2005
I2RP Meeting
Quantitative Time
• Time interval: Show all work which involve
Monet in years 1850-1890;
• Time instant: Show all Dutch impressionist
work in 1926;
• Time instant with time modifier: Show all
Dutch impressionist work after 1926;
• Event: Show all Dutch impressionist work in
year Monet died;
• Event with time modifier: Show all Dutch
impressionist work after Monet died;
26 Oct 2005
I2RP Meeting
Temporal Logics
• Linear time vs. branching time
• Temporal operators:
•
•
•
•
•
•
•
26 Oct 2005
always-in-the-past,
sometimes-in-the-past,
never-in-the-past,
at-present,
always-in-the-future,
Until
Since
I2RP Meeting
Time in Temporal Logics
• Temporal retrieval: show the first
Dutch impressionist work (in this art
archive). ( Defined in terms of never-inthe-past operator and the at-present
operator)
• Temporal reasoning: Is it never the case
that a Cubist work occurred earlier than
the first impressionist work ?
26 Oct 2005
I2RP Meeting
Dynamic Logics
• State vs. Action
• A state corresponds with a time instant
• An action is a set of state pairs which
correspond with the starting state and
the ending state of the action.
• Dynamic operators:
• [a]p: after doing action a, the property p
always holds;
• <a>p: after doing action a, the property
p may holds;
26 Oct 2005
I2RP Meeting
Dynamic Actions
• Dynamic action operators:
•
•
•
•
•
26 Oct 2005
sequential actions,
parallel actions,
action choice,
state checking
conditional action
I2RP Meeting
Time in Dynamic Logics
• Temporal Presentation: a sequential
execution of actions
• Temporal retrieval: a search action with
temporal specification
• Temporal reasoning: model checking with
dynamic actions
26 Oct 2005
I2RP Meeting
Temporal Reasoning in
the Semantic Web
• Temporal reasoning with RDF/RDFS
data
• Temporal Ontologies
• Time in OWL
26 Oct 2005
I2RP Meeting
Research Plan
• Compare the three temporal approaches
• Investigate their expressibility on intelligent
multimedia retrieval and presentation
• Quantitative time reasoning for image
retrieval and presentation
• Temporal retrieval and reasoning with the
support of the Semantic Web technology.
26 Oct 2005
I2RP Meeting
Some Past Work (by
Zhisheng)
• Temporal reasoning for multi-version
ontologies (2005)
• PAMELA: an intelligent presentation
embodied agent (2004)
• Facial expressions for embodied agents in
STEP (2004)
• XSTEP: a markup Language for embodied
agents (2003)
• STEP: a scripting language for embodied
agents (2002)
• WASP: Web Agent Support Program (2001)
26 Oct 2005
I2RP Meeting
Pamela: Demo
26 Oct 2005
I2RP Meeting