Preservation Strategies:
What do long-term archives do with my
data?
Jeff Arnfield
NOAA’s National Climatic Data Center
Version 1.0
Review Date
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
Overview
• A long term archive’s primary goal is “to protect and serve”
by ensuring that its data holdings, include the metadata
necessary for discovery and interpretation, remain intact,
available and usable.
• Ensuring the long term integrity and viability of data is an
active, ongoing process. A long term archive manages the
details for you.
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
Define roles and responsibilities
• Data archives work with data submitters to concisely define
their respective expectations and responsibilities for the
body of data to be archived. Such an accord is often
documented in a submission agreement.
• The archive’s requirements for data format, content and
packaging are clearly enumerated. The archive will also
require the metadata necessary to characterize, read,
advertise and properly interpret your data.
• Limitations on the data’s distribution and use are explicitly
spelled out in these submission agreements.
• Plans for versioning, if updates are envisioned, will usually
also be addressed during the planning process.
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
Ingest, identify and store
• Your data must be archived using a non-proprietary format
to ensure that it may be read in the future. The archive will
work with you to ensure your data uses such a format.
• The archive may ensure that your data are received intact
using checksums or other techniques.
• The data archive will assign a unique identifier to your data,
providing a stable way to reference it.
• In addition to the metadata you provide, the archive may
generate data management metadata for your submission.
• Your data will be securely stored to ensure accessibility and
its long term integrity.
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
Preserve data integrity
• Your content must be protected against corruption and
modification. File checksums and other techniques help
ensure that data files remain intact and unaltered.
• Storage devices and media can fail due to equipment
problems, and an entire location may be compromised by
natural disaster. A data archives usually maintains backup
copies of data at an offsite location to ensure its holdings
survive such events.
• Archives plan for and manage migration of data to new
media, storage architectures and formats to avoid
obsolescence and help prevent media failure.
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
Advertise and provide access
• Archives use metadata to permit your data to be
discovered, acquired and interpreted.
• In addition to employing in-house discovery and access
tools, archives often publish metadata to various
clearinghouses, portals and registries. Such resources can
make your data available to a much broader community.
• Archives support access requests to your data by
packaging and delivering it, along with associated
metadata.
• A data archive will control access to its holdings, enforcing
any distribution restrictions or limitations specified in the
submission agreement.
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
References and Resources
• Van den Eynden, V., Corti, L., Woollard, M., Bishop, L., &
Horton, L. (2011.). Managing and sharing data : best
practices for researchers. Colchester: University of Essex.
UK Data Archive. www.dataarchive.ac.uk/media/2894/managingsharing.pdf
• Reference Model for an Open Archival Information System
(OAIS)
http://public.ccsds.org/publications/archive/650x0b1.PDF
Preservation Strategies: What do long-term archives do with my data?; Version 1.0, Reviewed xx/xx/xxxx
Other Relevant Modules
• Planning for longer term perservation
• Work with your archive early and often
• Options for archiving your data
• Data transfer and submission agreements
• Intro to the OAIS Reference Model
• Responsible Use: Data restrictions