StreetTIVO
Arjen P. de Vries
[email protected]
11/1/2005
StreetTIVO
N5: Semantic Access
• Main goal:
– Improve search system engineering efficiency
• Approach:
– Given
• Declarative specification of collection, background
information, and context
• Declarative specification of retrieval model
– Generate a parameterised search system
11/1/2005
StreetTIVO
Collection
+
Search Strategy

System
11/1/2005
StreetTIVO
TRECVID example
• A query
– //VideoSegment[
about(./transcript,
“airplane taking off”) and
about(./keyframe,
airplane-in-sky.jpg)]
• and a retrieval model specification
… [avg(
[log((0.9*psd(s,d)) +
(0.1*ps(s))) | s<$NSamples])
| d<$NDocs ] …
11/1/2005
StreetTIVO
Spiegle Prototype
Retrieval model specification (RAM)
+
Collection schema (XML)
+
Cirquid XML-IR (TIJAH)
+
MonetDB/XQuery
11/1/2005
StreetTIVO
Waterland Demo
• Demonstration system querying “De
Digitale Voorziening” at the Dutch
broadcasters
11/1/2005
NEXI.cgi
MonetDB/
XQuery
TIJAH
4 VIDEO
(IR engine)
StreetTIVO
11/1/2005
StreetTIVO
let $NEXI_QUERY:="//segment[about(.,metro)]"
let $ranking := doc(fn:concat($NEXI_CGI,$NEXI_QUERY))
let $vids := distinct-values($ranking//Result/@docID)
TIJAH’s NEXI query
let $vranking :=
MonetDB/XQuery
to language
present
element Vranking {
~ XPath+about
result
fragments
grouped by
for $v in $vids
let $vshots := $ranking//Result[@docID=$v] video
let $vscore := for $f in $vshots/@score return $f cast as xs:double
let $MAXvscore := max($vscore)
let $MINvscore := min($vscore)
order by $MAXvscore descending
return element Vid {
attribute id {$v},
attribute score {$MAXvscore},
for $s in $vshots[@score > $MINvscore]
return $s
}
}
11/1/2005
StreetTIVO
StreetTIVO
• Golden demo
“A demo not seen
anywhere else”
•Co-operation between projects:
•N1: Multimedia Features
Twente, UvA, Delft
•N3: Ambient Data Management
CWI, Philips, Twente
•N5: Semantic Access
CWI, Twente
11/1/2005
StreetTIVO
StreetTIVO
PVR
+ MultimediaN Features
• Teletekst
• ASR (automatic speech recognition)
• Scene- and/or shot-segmentation
• Detecting repeated objects
• Tracking a library of famous people
• [your fav “detector” here…]
+ Information Retrieval
11/1/2005
StreetTIVO
StreetTIVO
Challenge: feature detection takes long time
Idea: P2P  Collaborative Multimedia Indexing
Work distribution
Result exchange
Fault tolerance
11/1/2005
StreetTIVO
StreetTIVO
CWI Seed farm
10 workstations
11/1/2005
StreetTIVO
StreetTIVO
Work
distribution
StreetTIVO
aims for the world-wide
Result exchange
Fault tolerance
MythTV
community (starting in NL).
CWI Seed farm
10 workstations
AmbientDB
Scalability and robustness!
11/1/2005
StreetTIVO
StreetTIVO Architecture
AmbientDB
P2P
database
XQuery/embedded
StreetTIVO
module
SOAP/XQuery
TIJAH
4 VIDEO
(IR engine)
Multimedia
Feature
Detection
Extending the search functionality
corresponds to installing a new
detector and updating the TIJAH 4
VIDEO XQuery queries
11/1/2005
StreetTIVO
Key Persons
Thijs Westerveld
(CWI) Multimedia search
Matthijs Mourits
(CWI) mythTV boxes
Jennie Zhang
(CWI) P2P data management
Djoerd Hiemstra
(CTIT) TIJAH
Roeland Ordelman
(CTIT) ASR (Dutch)
YOUR NAME IS WELCOME HERE!!!
11/1/2005
StreetTIVO