Other Services in Hyrax
Other Services in Hyrax
•
•
•
•
HTML: Web-page form interface
ASCII: Data responses
Info: Metadata as formatted HTML
Catalogs: Directory and THREDDS
responses
• GridFTP: Direct access to the BES
HTML Form Interface
• The Form interface provides a uniform way to access data
• It’s quite primitive because it must work for all data - even data
with no semantic metadata
• When a constraint is formed with this interface, the subsetting
must be done using only syntactic information like the physical
size of an array.
• This interface is part of every Hyrax and most other DAP
servers, although there are some slight variations among
different implementations
• It provides a way to view the structure of the data and form a
constraint so that a complete URL can be cut and pasted into
another application
ASCII
• ASCII is the lowest common denominator for
data since many programs can read it
• Hyrax has a handler that transforms the DAP
data response (binary data with structural
metadata) into a CSV ASCII response.
• You can use a spreadsheet to read data
– Try it!
Find some data in the form interface
Build a constraint with the form (remember
this is ASCII we’re talking about, so data
volume is a consideration!) and then hit
‘Get ASCII’
FireFox puts the new data into a tab, other
browsers open a separate window…
Try it with a spreadsheet; type (or paste)
the URL into the ‘Location’ box. Then hit
‘OK.’ This example is using Kspreadsheet
from the Virtual machine; Excel works too.
Data come back in the spreadsheet cells.
You may be prompted to supply information
Depending on the spreadshhet
Info
• The DDS (structural) and DAS
(semantic) metadata responses can be
combined and returned as an HTML
page
• Add the extension ‘.info’ to the DAP
URL
• Try it…
Catalogs
• OPeNDAP directory is a kind of catalog
– Conceptually based on the HTML directory pages
– Provides a click-to-navigate interface
– Different servers implement it differently and this
has created problems with uniformity for clients
• THREDDS provides a framework for
cataloging data sources
– Provides a way to list several different access
protocols
– Listings iin XML
Catalogs, cont.
• To get the OPeNDAP directory listing from an
old server, end the URL in a slash (/)
• To get this from Hyrax, use either a slash or
‘contents.html’
• To get a THREDDS catalog from any DAP
server, use ‘catalog.xml’ for the XML listing or
‘catalog.html’ for the HTML version
– The HTML THREDDS catalog supports a click-tonavigate interface like the old OPeNDAP directory
• Try it
THREDDS examples
• Switch Hyrax to the THREDDS HTML view:
Choose the HTML view
The THREDDS HTML view
• The top-level THREDDS catalog on our test server defines a
single data root directory (SVN Test Data Archive)
• This illustrates how THREDDS can be used to control the view
of data presented by the server
• Use ‘catalog.xml’ in place of ‘catalog.html’ to get the catalog
data in an XML document.
Traverse the links to find data
THREDDS data set page
• THREDDS catalogs can list more than one access mechanism here we see on the DAP, but WCS, WMS, et c., are other
possibilities
GridFTP
• Hyrax does not actually support GridFTP but
the BES does.
• The BES was originally developed as a DAPcapable server for the Earth Systems Grid
project (now ESG II)
• The BES can be run standalone in
appropriate environments and the GridFTP
protocol can be used to connect to it.
• Using GridFTP, DAP requests can be made
and DAP responses transferred to ESG
clients.
Numerical Grid Computations within BES
•
CSAC - Community Specro-polarimetric Analysis
Center
•
Two part problem;
1.
2.
•
•
•
Compute the so called Stokes vector for a given vector
parameter space P using a model F(x,y,λ; P)
Correct P to improve the fit of the theoretical signal to the
observed signal using non-linear optimization
The purpose is to determine a vector parameter
space for which the model produces the best fit in
the Least Square sense.
The complete problem is solved with a Numerical
Library called MERLIN
Very computational Intensive, once took up to 6
hours now takes 16 minutes.
High Performance Solution
• Create multiple versions of the computing
node waiting for data to crunch
• Create a central place (controller) to browse
spectra- polarimetry data inventories and
launch inversions upon user’s request.
• Let the controller decide how to distribute the
problem.
• Requires multiple identical processes waiting
to be called for duty.
Distributing the Work
GRID
BES Server 1
Client
APACHE
TOMCAT
MODULE
topology
BES Server 2
.
.
.
.
.
.
BES Server 10
Linux Dualcore
Xeon 64-bit
processor with 10
Terabyte RAID
Merlin Inversion
Engines
20 64-bit AMD Opteron processors running on Solaris 10. Can also plug in MacPro EightCore machines