International Planetary Data Alliance
PDAP Query Language
September 14, 2011
Topics
• Overview
• PDS4 Search Protocol
-
Search Service Design
Query Syntax
Parameters
Response
• Mapping PDAP to PDS4
-
PDS4 Overview
Resource Class vs. Product Class
Compulsory Parameters
Additional Suggestions
September 14, 2011
PDS/PDAP Query Language
2
IPDA Query Language Analysis
Project Description
As part of the PDAP definition, the use of
a more complex query language than the
keyword=parameter has been considered
a useful extension. At the same time, the
control of the metadata fields returned in
a PDAP query could produce a more
powerful protocol. The definition of a
query language for planetary data
requires a previous analysis.
September 14, 2011
PDS/PDAP Query Language
3
Overview
• With version 1.1, PDAP is progressing with
suggestions incorporated from CDPP/IRAP and
NASA/PDS.
• This presentation focuses on the
characteristics of the PDS4 Search Protocol.
- Some of these aspects have been or are planned for
incorporation into PDAP.
- Others are may be candidates for future
incorporation.
• Also covers how the current PDAP specification
maps to the PDS4 data model.
September 14, 2011
PDS/PDAP Query Language
4
PDS4 Search Protocol
Overview
• The Search Service is a deployable component
that accepts queries for data and returns a set
of matching results.
• Provides the public interface to the metadata
contained in the PDS federated registries.
• Generation of search indices from registry
metadata supports multiple query formats and
is tailor-able for customized search interfaces.
September 14, 2011
PDS/PDAP Query Language
6
Search Aspects
• Discovery
- Focuses on discovering content, whether at the
catalog or product level.
- Facilitated by support for search parameters and
paged result sets.
• Link
- Focuses on passing search parameters from one
service to another.
- Deployment of the Search Service facilitates
parameter passing and integration.
• Access
- Focuses on retrieval of product files.
September 14, 2011
PDS/PDAP Query Language
7
Design Considerations
• Protocol implemented as a REST-based
interface over HTTP.
• For discovery, supports return of paged results
in a defined structure (e.g., XML or JSON).
• For access, supports return of products and
product packages.
• The architecture allows support for other
query languages (e.g., IPDA’s Planetary Data
Access Protocol (PDAP)).
September 14, 2011
PDS/PDAP Query Language
8
Architecture
• Search indexes
built from
multiple sources.
• Allows for
annotation of
archive metadata.
• Customizable for
a disciplinespecific search
interface.
September 14, 2011
PDS/PDAP Query Language
9
Query Syntax
• Simple
- Support HTTP parameters, similar to PDAP.
- Simplifies interfaces and minimizes encoding.
- Example:
• …?target=mars & investigation=mro
• Advanced
- Based on the Lucene/Solr query syntax and
encapsulated in a single parameter.
- Supports wild carding, relational and logical
operators, ranges, grouping and term search.
- Example:
• …?query=target:mars AND investigation:mro
• …?query=target_type:satellite AND “search term”
September 14, 2011
PDS/PDAP Query Language
10
Parameters
• Query related (common):
- Instrument Name and Type, Instrument Host Name,
Investigation Name, Observing System Name, Start
and Stop Time, Target Name and Type
- Identifier (with version), Title, Product Class
• Query related (extended):
- Node and Mission specific parameters
• Metadata result related:
- Return Type, Page Size, Max Results, Start Record
• Data result related:
- Transform, Package
September 14, 2011
PDS/PDAP Query Language
11
Response
• Result Set Structures
- Support VOTable for science-based applications.
- Support a lighter weight structure in XML and/or
JSON for web-based applications.
• General Features
- Support paging and managing result set size.
- Fully utilize HTTP protocol features:
• Population of mime type, previous and next result set
URIs.
• Map errors to HTTP error codes.
September 14, 2011
PDS/PDAP Query Language
12
Mapping PDAP to PDS4
PDS4 Overview
• All artifacts in PDS4 are considered products.
- Bundles, collections, observational data, documents,
etc.
- Each product has a logical identifier (URN) and
version.
• urn:nasa:pds:PHX:SSI:RAW:PHX_TAU451_030_20081203A
• PDS3 Data Sets are closer in content to
Bundles rather than Collections.
- Though the mapping can be one to many or many to
one depending on the Discipline Node interpretation.
September 14, 2011
PDS/PDAP Query Language
14
Resource Class vs.
Product Class
• PDAP’s Resource Class defines the granularity
of the result set and includes values of:
- Data Set, Product, Map Projected, etc.
• PDS4’s Product Class can be used in a similar
manner.
• The following are example mappings:
- DATA_SET
• Product_Bundle and Product_Data_Set_PDS3
- PRODUCT
• Product_Observational which is a parent class for the
image and table classes.
- MAP_PROJECTED
• A subset of the Product_Observational subclasses.
September 14, 2011
PDS/PDAP Query Language
15
Compulsory Parameters
• Direct mapping in PDS4:
- INSTRUMENT_NAME, INSTRUMENT_TYPE,
START_TIME, STOP_TIME, TARGET_NAME,
TARGET_TYPE
• Indirect mapping in PDS4:
- DATA_SET_ID
• In the case where product_class is equal to
“Product_Data_Set_PDS3”, this parameter is
equivalent to logical_identifier and data_set_id.
- PRODUCT_ID
• In PDS4, everything is a product insinuating that this
parameter is equivalent to logical_identifier.
September 14, 2011
PDS/PDAP Query Language
16
Additional PDAP Suggestions
• The addition of paging support is welcomed.
- It would be useful if its existence in the metadata
response was elevated to “SHOULD” instead of “MAY”
be present.
- Without this information the client is left to guess with
respect to whether there are more results.
• The response supports one data access reference
for a product.
- It would be useful if more than one data access
reference were allowed in the metadata response.
- Would allow for retrieval of label and data files
separately.
- Would also allow for the product to be served from
multiple product services.
September 14, 2011
PDS/PDAP Query Language
17
Additional PDAP Suggestions
REST-Specific Enhancements
• Allow for HTTP Accept header to determine the
type of response (e.g. Accept: application/xvotable+xml).
• When specifying lists of query parameters
repeat param=value to follow standard
practice (e.g. param=value1&param=value2
NOT param=value1,value2)
• Map errors to HTTP error codes.
• Consider allowing for a JSON response as it is
more compact than XML.
September 14, 2011
PDS/PDAP Query Language
18
Questions/Comments