IOOS National Glider Data
Assembly Center
Metadata, Archiving and Community Tools
April 6, 2017
14:00 – 15:00 EST
Topics
• QARTOD QC
• Quality Metadata and NCEI Archiving
Process
• Development of Community Tools
Data Quality Control
• Supplied by:
– Data provider
– DAC
• Criteria
– For all QARTOD CTD parameters/variables, if
the variable:ancillary_variables attribute
contains variables, assume QC has been
implemented by the data provider
– If variable:ancillary_variables is left blank,
QARTOD tests will be applied by the DAC and
appended to the data provider NetCDF files.
Improving Metadata
• Archiving Goal: “to preserve data and associated
information for future users…”
• “To accomplish this goal, the expert knowledge of
the data provider must be acquired and preserved
with the data”
https://www.ncei.noaa.gov/archive
Metadata is only as good as you make it. This
will require additional time (although not as
much as you’d think)
Common Metadata Issues
•
•
•
•
•
•
•
Glider names
Incorrect WMO id
Provider Name
Project
Summary
Acknowledgment
CF Standard Names
Glider Name
• Use the name provided by the data
provider on the deployment registration
page
• Capital or lowercase, exactly as entered
on the deployment registration page
WMO ID
• WMO ids are not required for DAC
submission and NCEI archiving
• If specified, must match the WMO id that has
been previously assigned to that glider
deployed in the same WMO id region
• Only near real-time datasets with valid WMO
ids are transmitted to GTS
• Older deployments that do not include a
WMO id are still accepted by the DAC and
archived by NCEI
Institution/Provider Names
• Required global attribute
• Use the provider name you would like,
but consistency matters.
• Institution/Provider name should not
contain the street address
• Archiving and searching for
deployments by provider name requires
consistency
Institution/Provider Examples
• Coastal Ocean Observation Lab,
Rutgers University, USA
• Rutgers University
• Coastal Ocean Observation Lab,
Rutgers University, 71 Dudley Road,
New Brunswick, NJ 08901
• RU
Project Name
• Required global attribute
• Project should contain the official
project name and acronym, if
applicable.
Project Examples
• “TEMPESTS: The Experiment to
Measure and Predict East Coast STorm
Strength”
• “SECOORA: SouthEast Coastal Ocean
Observing Regional Association”
• “This is where the project should go”
Summary
• The summary attribute is used by NCEI
as the official description of the dataset.
This is what everyone sees when they
search NCEI for glider datasets and,
utlimately, how the dataset is identified.
• A useful summary or abstract
explaining the purpose of the
deployment, the type of data included in
the file and any additional information
deemed useful to the data user.
Summary Examples
•
“Seaglider La Push NANOOS line, Washington. Glider covering a 200km SWNE transect off La Push (Olympic Peninsula), Washington, as part of the
Northwest Association of Networked Ocean Observing Systems (NANOOS).
Glider deployments complement two NANOOS moorings off La Push at the
onshore (NE) end of the transect, and provide observations on dissolved oxygen
and oceanographic conditions in this dynamic upwelling region that interacts
with waters in the Salish Sea. Operated by the University of Washington Applied
Physics Laboratory (UW-APL) and funded by NOAA via the U.S. Integrated
Ocean Observing System (IOOS), with previous support from the Murdock
Charitable Trust and the UW.”
•
•
“This is a summary of the Originator, Group, Mission or whatever”
“Glider profile data from <Insert University Here>.”
Acknowledgment
• Acknowledges funding agencies
https://gliders.ioos.us/map/
• 2 ways to provide acknowledgement:
– “Attribution” input element on deployment
registration page
– “Acknowledgment” global attribute in data
provider NetCDF files
• Regardless of which method is used, no need
to start with “Funding provided by…”. Just
state the funding agencies or any other
parties.
CF Standard Names
•
http://cfconventions.org/Data/cf-standard-names/42/build/cf-standard-name-table.html
• Standard name indicates correct
spelling
• For NCEI archiving and CF-compliant
purposes, these names have to be
properly spelled. If they are not, the are
not “standard” names.
Community Tools
• IOOS NetCDF compliance checker:
https://github.com/ioos/compliance-checker
• IOOS Glider DAC plugin:
https://github.com/ioos/cc-plugin-glider
• GCOOS/SECOORA Slocum Toolbox
https://github.com/SECOORA/GUTILS
– Process binary or ascii Slocum glider files to IOOS Glider DACcompliant NetCDF files
– Python 2 or 3 compliant
– Currently does not provide QC, but support is planned
– Need to build raw seaglider and spray glider (others?) raw data file
readers to provide support for these glider types (Need example
files from the community).
– Plan is for pull request merge by April 14.
• Future webinar and/or python notebooks on usage
IOOS DAC 2017 Goals
• Acceptance of delayed mode datasets
• Acceptance of native glider data files
and creation of IOOS Glider DACcompliant NetCDF files
• GTS proof
What are the priorities from the
community?