“The VO in Australia”
Melbourne Nov. 28/29 2002
 What is the AVO?
 How did it develop - Grid computing –
particle physics
 Current status of International VO
projects (http://www.ivoa.net)
 Role for Australian Astronomy?
 Opportunities & challenges
20 Nov, 2002
Virtual Molonglo Observatory
1
What is the Virtual
Observatory?
 NOT one project or the Web
 Distributed CPU – AVO, NVO, ASTROGRID
 Distributed data – images, catalogues,
spectra, simulations & models
 Distributed software – assorted acronyms
 Resource broker, road map, nodes
20 Nov, 2002
Virtual Molonglo Observatory
2
What’s it all about?
Grid computing deals with
coordinated resource sharing and
problem solving in dynamic, multiinstitutional virtual organisations.
The resources are compute power,
software, data and collaboration
tools.
20 Nov, 2002
Virtual Molonglo Observatory
3
Some statistics on doubling
times




Computing power (Moore’s law): 18 mths
Bandwidth (Nielsen’s law): 20 mths
Data archive size: 12 mths
Number of websites: 9 mths
20 Nov, 2002
Virtual Molonglo Observatory
4
Challenges & responses




Slow CPU growth distributed computing
Limited BW
information hierarchies
Limited storage
distributed data
Data diversity
interoperability
 SOLUTION: GRID COMPUTING
20 Nov, 2002
Virtual Molonglo Observatory
5
Technical Update




Big commitment in Europe & USA
Wide applications – business & science
VO-compliance & VO-table
Issues of access, security, universal
querator, resource broker
20 Nov, 2002
Virtual Molonglo Observatory
6
Role of Australian Astronomy
 Workshop focus on data and tools
 Examples of current possibilities
 Challenges and opportunities
20 Nov, 2002
Virtual Molonglo Observatory
7
Functional Requirements: A First Draft
• Immediate processing of data from sensors (all s)
• Formats for raw data in sensor databases
• Transparent access to all databases
• Correlation of data sets across databases
• Facilitation and acceleration of the scientific
method using all databases
20 Nov, 2002
AVO Project Management
Virtual Molonglo Observatory
8
Gavin Thoms
27 November 2002
1. AAO & the IVOA - Strategy




Build/continue alliances with key groups
Assist in development of VO standards
Build VO-compliance into data & products
Facilitate development of analysis tools
20 Nov, 2002
Virtual Molonglo Observatory
9
20 Nov, 2002
Virtual Molonglo Observatory
10
The Way forward: ARC grant
for 2003 (1.5FTE@AAO)
 Incorporate 2dF survey into VO-table
(milestone: demo at IAU GA)
 Integrate 2dF spectra & catalogue server
(milestone: end 2003)
 VO-compliance for 6dF from start
(milestone: April 2003)
 Route map for AAO VO-compliance
(milestone: end 2003)
20 Nov, 2002
Virtual Molonglo Observatory
11
2. Contribution from the
Molonglo Observatory
 Image availability - data calibration &
quality
 Source catalogues – integrity and
interpretation
 What is raw data? Case study at 408 MHz
20 Nov, 2002
Virtual Molonglo Observatory
12
Response Classification with a
Decision Tree
Blue ellipses - Sources
Red ellipses - Artefacts
20 Nov, 2002
Virtual Molonglo Observatory
13
Current data pipeline






Automated observations
Manual transport of data (CDs) to Sydney
Customised analysis software programs
Image archive & source catalogue
Processed data back to Molonglo & Web
Resource intensive
20 Nov, 2002
Virtual Molonglo Observatory
14
3. Machine Learning
techniques
 Goal – multiwavelength correlations
 Problem – database mismatches
 Traditional methods – closest position &
other information
20 Nov, 2002
Virtual Molonglo Observatory
15
RADIO: HIPASS 21cm survey
OPTICAL: SuperCOSMOS
(B)
(A)
Y
X
10 arc min error diameter
The correlation problem: which is the radio source?
20 Nov, 2002
Virtual Molonglo Observatory
16
Use Machine Learning
 Data vectors from catalogues
 Radio: RA, Dec, velocity, velocity width, flux
 Optical: (RA, Dec, B,R,I mags, shape)N
 Training sets
 Optical counterparts with measured velocities
 Machine learning
 Support Vector Machine
 Use all parameters for the classification: new physics?
 Quadratic programming problem, so unique solutions
20 Nov, 2002
Virtual Molonglo Observatory
17
4. Future: direct image analysis
 Handwritten postcode recognition
 US Postal Service database: each digit 16×16 pixels
 7,300 training patterns, 2,000 test patterns
 Classifier
% Error
 Decision tree
16.2
 5-layer neural net
5.1
 Support vector machine
4.1
 Human
2.5
 Direct analysis of optical pixel data?
 Established for morphological galaxy classification
 Too many pixels for radio identification problems?
20 Nov, 2002
Virtual Molonglo Observatory
18
5. Example element of
e-Astronomy Australia
Build a pipeline processor (running aips++)
to process radio synthesis data from ATCA
archive on the fly
 User can choose parameters of image
 Field centre
 Field size
 Optimise algorithm for science question being
asked
 Can use latest version of calibration algorithm
 Expert users can tweak parameters
20 Nov, 2002
Virtual Molonglo Observatory
19
Goals of e-Astronomy Australia
 Survey and archive data from Australian
telescopes available to all IVO users
 Prospects to put full ATCA archive online
 Set up datagrid and compute grid to
give Australian astronomers access to
IVO resources
 Help develop techniques, protocols, etc
for the IVO
20 Nov, 2002
Virtual Molonglo Observatory
20
6. Tools – new and used
 FITS – successful data format – keep?
 Astronomy co-ordinate systems – several
in use – IAU working group
 VOtable – flexibility, greater complexity,
incorporate current protocols
20 Nov, 2002
Virtual Molonglo Observatory
21
7. New multicolour Survey
 Imaging survey with Great Melbourne
Telescope
 A TRAGEDY!
20 Nov, 2002
Virtual Molonglo Observatory
22
Discussion: paradigm for a
small country
1. Identify strengths or special roles in the
international context
2. Identify any major international partners
gains from the involvement
3. Identify gains for the small country from
involvement in the project
4. Identify a realistic niche for a significant
contribution
5. If any of 1- 4 are missing, withdraw!
20 Nov, 2002
Virtual Molonglo Observatory
23
Challenges & Opportunities
 Continue training of future astronomers
 Need resources to maintain and upgrade
databases & fund future instruments
 Cross-discipline collaborations
 Maintain role in observational science
 FIND A NICHE!
20 Nov, 2002
Virtual Molonglo Observatory
24
Where to now?
 LIEF grant for 1 year – new grants?
 Raise visibility in Europe, USA programs
 Cross discipline links – herbarium, medical
centre, particle physics
 Identify areas of contribution to
international VO – spectroscopy?
 http://www.aus-vo.org (David Barnes)
20 Nov, 2002
Virtual Molonglo Observatory
25
Conclusions
GOAL: To develop tools, data and
organisational structures to facilitate
international collaborations and individual
research on multidimensional archives
operating as a VO.
20 Nov, 2002
Virtual Molonglo Observatory
26