MonetDB/XQuery
Technology Preview 1
Stefan Manegold
Centrum voor Wiskunde en Informatica
Amsterdam
http://monetdb.cwi.nl/ - http://pathfinder-xquery.org/
European Pathfinder Team
• CWI, Amsterdam
(Netherlands)
– Peter Boncz, Stefan Manegold, Sjoerd Mullender
• University of Twente
(Netherlands)
– Maurice van Keulen, Jan Flokstra
• University of Konstanz (Germany)
– Torsten Grust, Jens Teubner, Jan Rittinger
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Results: Performance (1)
XMark benchmark, 110 MB: MonetDB/XQuery vs. X-Hive & Galax
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Results: Performance (2)
XMark benchmark, 1.1 GB: MonetDB/XQuery vs. X-Hive
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Story
• XQuery Example
• Relational XQuery
– System Architecture
– XML Encoding
• Science & Reseach
• Scalability
• Outlook
– Conclusions
– Roadmaps
– Release & References
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
XQuery Example
• For each author, return number of books and receipts
for books published in the past 2 years, ordered by name
let $cat := fn:doc(“www.bn.com/catalog.xml”),
(:Documents:)
$sales := fn:doc(“www.publishersweekly.com/sales.xml”)
for $author in distinct-values($cat//author)
(:Grouping:)
let $books := $cat//book[@year >= 2003 and author = $author],(:Sel.:)
$receipts := $sales/book[@isbn = $books/@isbn]/receipts
order by $author
return
(:Ordering:)
<sales>
{ $author }
(:XML Construction:)
(:Join:)
<count> { fn:count($books) } </count> (:Aggregation:)
<total> { fn:sum($receipts) } </total>
</sales>
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
XQuery Example
• For each author, return number of books and receipts
for books published in the past 2 years, ordered by name
let $cat := fn:doc(“www.bn.com/catalog.xml”),
Documents
$sales := fn:doc(“www.publishersweekly.com/sales.xml”)
for $author in distinct-values($cat//author)
Grouping
let $books := $cat//book[@year >= 2003 and author = $author],Sel.
$receipts := $sales/book[@isbn = $books/@isbn]/receipts
order by $author
return
Ordering
<sales>
{ $author }
XML Construction
Join
<count> { fn:count($books) } </count> Aggregation
<total> { fn:sum($receipts) } </total>
</sales>
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
XQuery Systems: 2 Approaches
• Existing “native” XML/XQuery systems are
built from scratch
– Galax, Saxon, …
– X-Hive, Tamino, …
– (Still have to) re-invent optimization technology
• Our approach:
– Build XQuery system on top of an RDBMS
– Leverage mature relational technology
to achieve efficient XQuery processing
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Architecture
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
XML in an RDBMS: XPath Accelerator
Node-based relational encoding of XQuery's data model
0<a>
<a>
1<b>
<b>
2<c/>
2<c/>0
<c/>
</b>
</b>1
3<d/>
3<d/>2
<d/>
4<e>
<e>


5<
< f>
6<g/>
6<g/>3
<g/>
7<h/>
7<h/>4
xx <h/>

</f>5
>
8<i>
<i>
9<j/>
9<j/>6
<j/>
</i>
</i>7
</e>
</e>8
</a>
</a>9




f/following:
f/descendant:
f/preceeding:
f/ancester:

SELECT * FROM
SELECT * FROM
SELECT * FROM
SELECT * FROM
pre_post WHERE pre > f.pre AND post > f.post
pre_post WHERE pre > f.pre AND post < f.post
pre_post WHERE pre < f.pre AND post < f.post
pre_post WHERE pre < f.pre AND post > f.post
Similar queries for all 13 XPath axes
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Science & Research
• More research lead to more optimization
– Join Recognition
– Embedded XPath processing
– Order Awareness
• Various scientific publications
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Results: Scalability (3)
Unsurpassed scalability
• Standard Opteron PC, 8GB RAM, 64-bit Linux
• Can process 11GB documents!
Mostly linear scaling with document size
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Conclusions
• Relational approach
 Works
 Is fast
 Is scalable
• Crucial Optimizations
– Join recognition
– Embedded XPath processing
– Order awareness
• Research turned into open-source release
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Roadmap
• 30.05.05: MonetDB/XQuery 4.8/0.8 “Mercurius”
– Developers Release / Technology Preview 1
• 30.09.05: MonetDB/XQuery 4.10/0.10 “Venus”
– Student Release / Technology Preview 2
– XUpdate, More Optimization
• 30.12.05: MonetDB/XQuery 4.12/1.12
“Mars”
– Final Release
– Application Programming Interfaces
– End-User Front-Ends
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Open Source Release & References
• MonetDB + Pathfinder on SourceForge
– Mozilla-like License
• MonetDB homepage
– http://monetdb.cwi.nl/
• Pathfinder homepage
– http://pathfinder-xquery.org/
• Developers website
– http://sf.net/projects/monetdb/
You are welcome to join the MonetDB/XQuery team!
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31.5.2005
Results: Performance (4)
XMark
Q
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Galax
0.06
0.03
0.14
0.22
0.05
1.30
2.68
0.16
113.23
1.74
2.62
1.44
0.03
1.92
0.02
0.03
0.06
0.07
1.17
0.28
11 MB
X-Hive MDB/XQ
0.05
0.37
0.07
0.45
0.28
0.65
0.08
0.10
0.05
0.13
0.02
1.07
0.03
1.57
0.14
0.85
0.20
32.25
0.80
5.28
0.18
98.91
0.14
23.39
0.07
0.10
0.17
0.72
0.09
0.03
0.11
0.03
0.07
0.09
0.04
0.08
0.11
0.67
0.24
0.11
Galax
0.72
0.31
1.76
2.91
0.63
13.29
30.01
2.12
DNF
18.61
DNF
DNF
0.66
99.53
0.20
0.46
0.82
0.73
14.73
2.98
11 GB
1.1 GB
110 MB
X-Hive MDB/XQ X-Hive MDB/XQ MDB/XQ
13
1.2
9.9
0.41
1.29
25
2.4
33.0
0.30
1.75
126
12.5
25.1
1.51
5.66
36
3.8
18.1
0.45
1.00
11
1.2
20.7
0.16
0.90
3
0.3
178.1
0.05
10.17
4
0.4
278.4
0.07
24.84
208
10.4
49.1
0.75
3.51
289
12.9
0.87 DNF
12280.66
1882
55.0
5.31 DNF
442.37
872.5 DNF
3.48 DNF
19927.29
150.7 DNF
1.66 DNF
5100.19
13
1.3
12.9
0.22
1.03
959
13.7
110.2
1.40
11.16
16
1.7
10.6
0.28
0.49
18
1.8
10.9
0.26
0.52
26
2.8
11.8
0.30
0.85
9
0.9
14.8
0.13
0.64
88
5.3
254.5
0.55
12.15
50
4.9
24.6
0.62
1.40
XMark performance in seconds: MonetDB/XQuery vs. Galax & X-Hive
Stefan Manegold
MonetDB/XQuery
HollandOpen, Amsterdam 31-5-2005