OAS Contribution
• The ACDS
• The new XCATDB based on Saada
Laurent MICHEL
Christian MOTCH
François Xavier PINEAU
Observatoire Astronomique de Strasbourg (Fr)
Saada: http://astro.u-strasbg.fr/websaada
XCATDB: http://amwdb.u-starsbg.fr/jacds
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The ACDS
• Delivered Products
– FITS catalogue extractions
• X-ray source counterparts
• Archival sources in the XMM field of view
– Graphical products (PDF)
• Finding charts
• Rosat Images
• Catalogue plots
– HTML product
• Browsing pages for ACDS data with many links to CDS resources
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The ACDS
Simbad
(Strasbourg France)
Pipeline
(Leicester)
NED
(Caltech USA)
ACDS:
• 2 Sparcs 400 Mhz
• 1Gb RAM
• 8 Agents in //
• Correlation computing
• Some remote server accesses (Aladin)
• A lot of computation
Vizier at CADC
(Victoria Canada)
Vizier at Cambridge
(UK)
Vizier at CFA
(Harvard USA)
• A lot of remote server accesses
• A few computation
• File product generation
Vizier at CDS
(Strasbourg France)
Aladin
(Strasbourg France)
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The ACDS
• Actual Throughput (8 process in //)
– 3300 sec /dataset
– (8*3600*24)/3300 = 209 datasets/day
• Most part of the time is
pure CPU (graphical prd)
8%
20%
– IDL, Distiller….
– Easy to improve
• new hardware
72%
Cross Correlation
Graphical Products
Other Tasks
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The ACDS
•
Operation issues
– Support the pipeline throughput (but not more)
• > 2200 datasets processed since October 13th
–
157 pending this morning due to a 2 days Aladin shutdown
• No need to improve the hardware
– CDS database upgrades
• Vizier is being moved on PSQL without serious consequences for us
• Major Simbad to be release late 2006
–
New error definition (no effect for standard users)
– External failures
• Remote server shutdown
• Corrupted products returned (cache miss-synchronization after crash)
– Internal failures
• Empty Rosat images/wrong PPS index
• Hardware failure
–
Spare available/never used
• Miss-configuration
• Holidays
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The ACDS
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The ACDS
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The XCATDB
• The XCATDB is not a data repository but a tool to
make science
X Source 1
Arch Source 1
X Source 2
Arch Source 2
X Source 3
Arch Source 3
Arch Source 4
Arch Source 5
• N to M persistent relationship
– Difficult to implement in an efficient way for complex queries
– Possibility of doing more than with dynamic cross-matches
• Data-mining feature
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
Saada in a few Words
• Saada: An Astronomical Database generator
– Making automatic the process
of building a database
– Hosting heterogeneous datasets
– Highlighting scientific content
– Publishing personal data into
the VO
• Databases are installed on local machines
– Any Linux/Windows/Mac box
– Tomcat 5.xx
– PostrgreSQL 8.xx
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
The XCATDB
• Based on Saada
– Queries expressed in SaadaQL
– Data loader
• How to build quickly millions of correlation links
• How to restore archival source uniqueness
Source X
Arch Source 1
Source X
Source X
Arch Source 2
Source X
Source X
Arch Source 1
Source X
Arch Source 1
Arch Source 2
– GUI
• Layout inspired from the 1XMM XCatDB
• Architecture based on native Saada GUI
• Prototype based on RC2 data open for the community
– http://amwdb.u-strasbg.fr/jacds
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
2nd Release of the XMM Catalogue
Native Saada Data-loader
150,000 X-ray sources
In 2800 observations
Implemented with
Saada relationships
Specific Module
1,000,000 links
Native Saada Data-loader
1,400,000 archival sources
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
Demo
A simplified example: Looking for
High Mass X-ray Binaries
kT = 5 – 50keV
HR3 = (F2-4.5keV – F1-2keV) / (F2-4.5keV + F1-2keV)
HR4 = (F4.5-12keV– F2-4.5keV) / (F4.5-12keV + F2-4.5keV)
< 1.5keV
Hardness ratio range
for X-ray binaries
Demo
Hardness ratio limit for stars
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
HR Condition
With bright counterparts
Without WR counterparts
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
Using UCDs in queries
• Selecting X-Rays sources by correlation patterns on 200
classes of X-Ray sources requires to handle a lot of metadata
select
select
select
select
select
………….
oidsaada
oidsaada
oidsaada
oidsaada
oidsaada
from
from
from
from
from
Arch_9037AEntry
Arch_7181AEntry
Arch_9032AEntry
Arch_9015AEntry
Arch_9031AEntry
where
where
where
where
where
(
(
(
(
(
_Flux > 1 e-13 )
_Flux2 > 1e-16 )
_FX > 1 e-16 )
_fX > 1e-16 )
_Fx > 1 e-13 )
• Solution: Expressing queries using UCDs and Units
[phot.flux;em.X-ray] > 1e-16 [W/m2]
Demo
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
UCD Issue
[UCD]>x1 and [UCD]<x2
(att1>x1 or att2>x1) and (att1<x2 or att2<x2)
(att1>x1 and att1<x2) or (att2>x1 and att2<x2)
or
(att1>x1 and att2<x2) or (att2>x1 and att1<x2)
Usually makes no sense
Solution: Using unambiguous operators: [UCD]In(x1,x2)
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
UCD Issue (continued)
• Query language issue: using a simple syntax
– One operator for one UCD
• Query execution
– Editable execution plan
• UCD attribution: a sensible issue
– Using multi-words UCDs as often as possible
– Limit the columns accessible per UCDs
– Take care with units
• Existence
• Name: known by the converter
• Result presentation: another sensible issue
– The client must understand how resources have been selected and accessed
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
XCATDB: Status and Prospects
• Data loader
– Correlation setup works fine
– Other products can be loaded with the native data-loader
• GUI
– HTML form improvements
– Display of constrained values
• Interoperability
–
–
–
–
External Xmatch in SaadaQL (F.X. Pineau)
Connection with ESAC resources (XSA, VOSpec)
Connections with CDS clients (VO portals)
VO protocols
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]
Thank You for your Attention
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saada
http://amwdb.u-strasbg.fr/jacds
[email protected]