reference GAEL-ICF-TN-060.2
GAEL
Consultant
issue 2 revision 3
date 13/04/2000
page 1 of 32
Development of a Baseband
Data Archive Interchange
Format
ICF Concept Technical Note
name
function
company
prepared by
Serge RIAZANOFF
project manager
Gael Consultant
approved by
authorized by
Gian Maria PINNA
project manager
ESA – ESRIN
date
signature
GAEL
Consultant
Development of a baseband data archive
interchange format
reference GAEL-ICF-TN-060.2
issue 2 revision 3
ICF Concepts technical note
date 13/04/2000
page 2 of 30
Document Status Sheet
Issue
Date
Comments
Author
1.0
22/10/1999
Issue as formal ‘Development of a Baseband Data Archive
InterChange Format’ project deliverable
A. Mbaye
2.0
25/01/2000
Decisions taken during the meeting held at ESRIN the
26/10/1999.
S. Riazanoff
2.1
10/03/2000
Decisions taken during the meeting held at Gael Consultant the S. Riazanoff
10/03/2000.
2.2
27/03/2000
Decisions taken by e-mail exchange on 27/03/2000. Version
sent to ATT members before meeting London April 2000.
S. Riazanoff
2.3
13/04/2000
Review of document during CEOS WGISS meeting held at
London 11-14 April 2000.
S. Riazanoff
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 3 of 30
Contents
DOCUMENT STATUS SHEET........................................................................................................................... 2
CONTENTS ........................................................................................................................................................... 3
1
INTRODUCTION ......................................................................................................................................... 5
1.1
SCOPE AND PURPOSES .............................................................................................................................. 5
1.1.1. Rationale ............................................................................................................................................... 5
1.1.2. Place of this document in the project .................................................................................................... 5
1.1.3. Archive Format Technical Note............................................................................................................. 6
1.1.4. Basic Principles ..................................................................................................................................... 6
1.1.5. Archive and InterChange Formats ........................................................................................................ 6
1.1.6. Limitations ............................................................................................................................................. 6
1.1.7. Document Overview .............................................................................................................................. 6
1.2
APPLICABLE AND REFERENCE DOCUMENTS ............................................................................................. 7
1.2.1. Applicable documents ............................................................................................................................ 7
1.2.2. Reference documents ............................................................................................................................. 7
1.2.3. Reference documents – Archive formats ................................................................................................ 7
1.2.4. Reference documents – Information description methodology .............................................................. 8
1.3
ABBREVIATIONS AND ACRONYMS ............................................................................................................ 8
1.4
DEFINITIONS........................................................................................................................................... 10
2
CEOS ICF CONCEPTS.............................................................................................................................. 11
2.1. INTERCHANGE CONTEXT ........................................................................................................................ 11
2.1.1. From Earth Observation to end-users ................................................................................................. 11
2.1.2. Mission-dependent exchanges ............................................................................................................. 12
2.1.3. Telemetry processing levels ................................................................................................................. 12
2.1.4. Data formats and configuration control .............................................................................................. 12
2.2. DATA EXCHANGE SCENARII ................................................................................................................... 14
2.2.1.
Interchange Driven by Export Facility .......................................................................................... 14
2.2.2.
Interchange Driven by Import Facility .......................................................................................... 15
3
CEOS ICF HIGH LEVEL REQUIREMENTS ........................................................................................ 17
3.1. DIFFERENCES WITH ARCHIVING REQUIREMENTS .................................................................................... 17
3.1.1. Memory constraints ............................................................................................................................. 17
3.1.2. Run-time constraints ............................................................................................................................ 17
3.1.3. Strong portability constraint ............................................................................................................... 17
3.1.4. Released physical reliability ................................................................................................................ 17
3.1.5. Direct access to archive ...................................................................................................................... 17
3.1.6. External references .............................................................................................................................. 18
3.1.7. Impact of software changes ................................................................................................................. 18
3.1.8. Sensor Description for Earth Observing Dynamic .............................................................................. 18
3.1.9. Open Archival Information System ...................................................................................................... 18
3.2. HIGH LEVEL REQUIREMENTS .................................................................................................................. 19
4.
CEOS ICF DETAILED REQUIREMENTS ............................................................................................. 20
4.1.
BASEBAND INTERCHANGE UNIT (BIU) .................................................................................................. 20
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 4 of 30
4.2. BASEBAND INTERCHANGE MEDIUM ....................................................................................................... 20
4.2.1. Multi-media exchange ......................................................................................................................... 20
4.2.2. File naming.......................................................................................................................................... 20
4.2.3. Use of tapes and data packaging ......................................................................................................... 20
4.2.4. Independence from medium and limitations of operating system ........................................................ 20
4.2.5. Data segmentation ............................................................................................................................... 21
4.3. BIU ORGANIZATION............................................................................................................................... 21
4.3.1. File structure ....................................................................................................................................... 21
4.3.2. File order ............................................................................................................................................. 21
4.4. METADATA MANAGEMENT .................................................................................................................... 21
4.4.1. Metadata Organization........................................................................................................................ 21
4.4.2. Language for Metadata Description ................................................................................................... 21
4.4.3. DED organization ............................................................................................................................... 22
4.4.4. Mission-dependent section .................................................................................................................. 23
4.5. SIGNAL DATA MANAGEMENT ................................................................................................................ 24
4.5.1. No redundancy..................................................................................................................................... 24
4.5.2. No encryption ...................................................................................................................................... 24
4.5.3. CRC and Checksum ............................................................................................................................. 24
4.5.4. No compression ................................................................................................................................... 24
4.5.5. PN decoded.......................................................................................................................................... 24
4.5.6. Data is frame synchronized ................................................................................................................. 24
4.5.7. Chronological order ............................................................................................................................ 24
4.5.8. Byte alignment ..................................................................................................................................... 24
4.5.9. Signal data record ............................................................................................................................... 25
4.6. CEOS ICF VERSION MANAGEMENT ...................................................................................................... 25
4.6.1. Minor/major versions .......................................................................................................................... 25
4.6.2. DED and software availability ............................................................................................................ 25
APPENDIX A – BASEBAND DATA CONCEPT COMPLIANCE MATRIX .............................................. 26
A.1.
A.2.
INTRODUCTION ....................................................................................................................................... 26
COMPLIANCE MATRIX ............................................................................................................................ 26
APPENDIX B – CEOS ICF DATA ENTITY DICTIONARY ......................................................................... 28
B1.
B.2.
INTRODUCTION ....................................................................................................................................... 28
HIERARCHICAL REPRESENTATION .......................................................................................................... 28
Development of a baseband data archive
interchange format
GAEL
Consultant
ICF Concepts technical note
1
INTRODUCTION
1.1
Scope and Purposes
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 5 of 30
This document serves as a description of the Baseband Data InterChange Format Concepts for the
Committee on Earth Observation Satellites (CEOS). It has been reviewed by the Archive Task Team of
the CEOS Working Group on Information Systems and Services (WGISS) Data during the ATT Meeting
held at London, April 2000.
1.1.1. Rationale
The development of a Baseband Data Archive Interchange Format (CEOS ICF) comes from a real need to
exchange data between different archiving facilities, which do not employ the same processing systems.
CEOS ICF shall be an intermediate format that all archive centers can interpret. Facilities should translate
their own format into CEOS ICF to export their archive data and shall translate CEOS ICF into their own
format in order to import other facility data (Figure 1.a).
Export Facility
Archive
Format
Import Facility
CEOS ICF
Archive
Format
Figure 1.a: Representation of data exchange through CEOS ICF Format
1.1.2. Place of this document in the project
« Archive Formats Technical Note » document
« ICF Concepts Technical Note » document
- High level requirements
- Detailed requirements
- Data Entity Dictionary (Entries)
Archive Format
document
Archive
Format
documents
Archive
Format
FRED
document
Archive Format
MDPS
document
GERALD
VEXCEL
« CEOS ICF » document
- Format description
- Data Entity Dictionary (Full description)
ICF Translator demonstration software
Deliverable of this project
Figure 1.b: Documents roadmap.
Archive
Data
FRED
MDPS
Metadata
Description
Information
Methodology
Description
Information
Methodology
Description
Information
DEDSL
Methodology
Description
PVL
Methodology
ODL
XML
GAEL
Consultant
Development of a baseband data archive
interchange format
reference GAEL-ICF-TN-060.2
ICF Concepts technical note
issue 2 revision 3
date 13/04/2000
page 6 of 30
1.1.3. Archive Format Technical Note
The feasibility of such interchange format was assessed through a preliminary analysis (R-1). This last
was undergone within the same current project. It showed that:
- information commonalties between existing archive formats are enough to allow a transfer of a
consistent amount of data,
- differences of structure and syntax do not prevent the information exchange. Obviously, some losses are
possible but storing extra information in separated structures should overcome most of them.
Moreover, this preliminary analysis highlighted the fact that it is possible to complete efficiently a target
format as long as not only source format is employed but also all necessary external sources of
information. Indeed, the exchange does not concern solely archive formats, it involves the source and
target whole information systems.
1.1.4. Basic Principles
The basic principles of CEOS ICF may be expressed with the following points:
- it shall settle a strong formalism for archive metadata which shall be clearly understood, easily
employed and hopefully widely accepted by the community,
- it shall also provide a flexible and appropriate structure which allow data transfer as fast (in terms of
development time) and inexpensive as possible,
- finally, it shall foresee and favor the development of powerful translation software.
These points shall be developed in the followings.
1.1.5. Archive and InterChange Formats
The importance of data exchange has been clearly underlined within the “Baseband Data Concept”
document. This document was already considering possible differences between goals of data archive and
data exchange, these differences will be more detailed within section 2.
In the same time, strong requirements of the “Baseband Data Concept” aimed to preserve the original data
will lead to state high level requirements.
1.1.6. Limitations
Even if the proposed hierarchical data organization could allow considering other type of data, the present
analysis only concerns 2-Dimensional images that represent the largest amount of data presently
processed by Earth Observation missions.
1.1.7. Document Overview
Scope of this document is to provide the ATT meeting members with the maximum material allowing to
assess high level principles with regard to their direct implications in data organization. The next
document that will detail “CEOS InterChange Format” it-self will be a strict application of the detailed
requirements of the present document after having been released by the ATT meeting.
“1. INTRODUCTION” is this section. It will contain a detailed list of acronyms and definitions that
should try to realize a consensus within the user community because they will be used when defining the
CEOS ICF Data Entity Dictionary.
“2. CEOS ICF CONCEPTS”. This section summarizes the context in that interchange takes place. The
first part of this section describes the acquisition, archiving and production interdependencies. The second
part analyzes the procedures for setting and controlling definitions by the groups having the necessary
authority. The last part presents two philosophies that could be adopted for baseband data interchange.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 7 of 30
“3. CEOS ICF HIGH LEVEL REQUIREMENTS”. After having underlined the main differences between
the requirements dedicated to baseband data archiving and the ones dedicated to interchange of these
data, this section lists a series of high level requirements that should govern exchange of baseband data
archive.
“4. CEOS ICF DETAILED REQUIREMENTS”. High level considerations listed in the previous section
will generate more detailed requirements. The final “CEOS ICF” document will be a straightforward
implementation of the detailed requirements laid down in this section.
“APPENDIX A – BASEBAND DATA CONCEPT COMPLIANCE MATRIX”. Compliance with
Baseband Data Concepts is the highest recommendation for ICF development. A compliance matrix will
guaranty that detailed requirements developed in section 4 fulfill this adherence.
“APPENDIX B – CEOS ICF DATA ENTITY DICTIONARY”. This section presents the first
hierarchical organization of data entities that will be kept in metadata file.
1.2
Applicable and Reference Documents
1.2.1. Applicable documents
A-1
AEF99/002/GMP - Development of a Baseband Data InterChange Format v1.0 ESA/ESRIN - 7 June 1999.
A-2
Baseband Data Concept White Paper - Draft Version - 06 May 1999 - ATT CEOS WGISS.
A-3
GAEL-ICF-MM-060.4 - Minutes of Meeting - ESRIN - 26/10/1999.
A-4
GAEL-ICF-MM-060.5 - Minutes of Meeting - GAEL Consultant – 10/03/2000.
1.2.2. Reference documents
R-1
GAEL-ICF-TN-060.1 - Archive Format Technical Note - Issue 1, Version 2 – 10/03/2000 GAEL Consultant.
R-2
CCSDS 100.0-G-1 – Telemetry – Summary of Concept and Rationale – CCSDS – December
1987.
1.2.3. Reference documents – Archive formats
F-1
DG-MA-50-6897 – Framed Raw Expanded data Product Specification, Issue/Rev. 2/10 –
MDA – 30 Oct. 1998.
F-2
C447-ST-73-103-CN – SPOT Archive GERALD Format Specification, Ed.1 Rev.0 –
27/10/1997 - CNES.
F-3
MDPS Systems – Multi Satellite data Processing Systems – Transcribed Data Format on
DLT – Release 4.1.1. – 11 February 1999 - ACS.
F-4
VX-STF-001 – Vexcel level 0 Processor (SKY) – Telemetry data Format v.1.0 – 26/07/1999 –
VEXCEL Corporation.
F-5
Landsat 7 System – Data Format Control Book (DFCB) Volume IV – Wideband Data –
Revision L – 11 June 1999 – NASA/GSFC.
F-6
S-IF-O/E-10-SI – SPOT to Direct Receiving Station Interface Description – 01/12/1997 –
SPOT IMAGE.
GAEL
Consultant
F-7
Development of a baseband data archive
interchange format
reference GAEL-ICF-TN-060.2
ICF Concepts technical note
issue 2 revision 3
date 13/04/2000
page 8 of 30
ER-IS-ESA-GS-002 – ERS-2 to Ground Segment Interface Specification, Issue 1.1 02.12.1993 – ESA/ESTEC.
1.2.4. Reference documents – Information description methodology
1.3
M-1
CCSDS 647.0-R-2.6 - Data Entity Dictionary Specification Language (DEDSL) Abstract
Syntax - October 1999 – CCSDS.
M-2
FGDC-STD-001-1998 - Content Standard for Digital Geospatial Metadata - Federal
Geographic Committee – June 1998.
M-3
FGDC-STD-???-???? - Content Standard for Digital Geospatial Metadata: Extensions for
Remote Sensing Metadata - Federal Geographic Committee – Internal Committee Draft,
version 0.2 – September 8, 1999.
M-4
Draft, A Strawman Proposal for a Standard Sensor Description Format of Earth Observing
Dynamic Sensors - 20.09.1998 – Mike BOTTS, prepared for CEOS WGISS.
M-5
CCSDS 650.0-R-1 - Reference Model for an Open Archival Information System (OAIS) - June
1999 - CCSDS.
M-6
CEOS.WGISS.DS.TN01 - Guidelines on Standard Formats and Data Description Languages
Version C Draft - 03.09.1997 - CEOS WGISS Guidelines.
M-7
Alain MICHARD – XML Language et applications - Eyrolles – 1999.
M-8
St Laurent, Cerami - Building XML Applications - Mc Graw-Hill – 1999.
M-9
Michael MORRISON, et al. – XML Unleashed – Sams Publishing – 2000.
M-10
Planetary data System Standards Reference – Jet Propulsion laboratory
URL: http://pds.jpl.nasa.gov/stdref – Updated 10/11/95 – Browsed 24/02/2000.
M-11
Robert Orfali, Dan Harkey - Client/Server Programming with JAVA and CORBA - Wiley
Computer publishing – 1998.
–
Abbreviations and Acronyms
This list is a collection of terms found within the referenced documents. When an abbreviation has been
found within only one document its unique reference is reported inside parenthesis.
API
Application Programming Interface
ASCII
American National Standards Institute
ATT
Archive Task Team
BER
Bit Error Rate
BIL
Band Interleaved by Line
BIP
Band Interleaved by Pixel
BSQ
Band Sequential
CADU
Channel Access Data Unit (delimited by Synchronization codes)
CCD
Charged-Coupled Device
CCF
Compatible Computer Format
CCSDS
Consultative Committee for Space Data Systems
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
CEOS
Committee on Earth Observation Satellite
CNES
Centre National d’Etudes Spatiales
CONI (F-4)
Tagged ASCII File
CRC
Cyclic Redundancy Check
DCS (F-4)
Direct Capture System
DED
Data Entity Dictionary
DEDSL
Data Entity Dictionary Specification Language
DLT
Digital Linear Tape
DPCM
Differential Pulse Code Modulation
EAF
Existing Archive Format
ECC
Error Correcting Code
EO
Earth Observation
EOF
End Of File
EOR
End Of Record
EOT
End Of Tape
EROS
Earth Resource Observation Systems
ESA
European Space Agency
FTP
File Transfer Protocol
GB
GigaByte(s)
HDDT
High Density Data Tape
HR
High Rate (Data)
ICF
Baseband Data Archive InterChange Format
IDHT (F-8)
Instrument Data Handling on Transmission Subsystem
IEEE
Institute of Electrical and Electronic Engineers
MIR
Moyen Infra-Rouge
NASA
National Aeronautics and Space Agency
NASDA
National Space Development Agency of Japan
NORAD
North American Aerospace Defense Command
OBR
On-Board Recorder (or On-Board Recording (F-4))
ODL
Object Description Language
OGRC
On-Ground Range Compression
PCD
Payload Control Data
PN (or PRN)
Pseudorandom noise
PRF
Pulse Response Frequency
PVL
Parameter Value Language
SAR
Synthetic Aperture Radar
SPOT
Satellite Pour l’Observation de la Terre
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 9 of 30
GAEL
Consultant
1.4
Development of a baseband data archive
interchange format
ICF Concepts technical note
SW
Software
SWST
Sample Window Start Time
TBC
To Be Confirmed
TBD
To Be Defined
URL
Uniform Resource Locator
UTC
Universal Time Coordinated
VCDU
Virtual Channel Data Unit
WGISS
Working Group on Information Systems and Services
WRS
World Reference System
XML
eXtensible Markup Language
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 10 of 30
Definitions
Many words or groups of words are found within the various documents with sometimes slightly different
meaning. The following list of definitions will be used consistently throughout this document.
format
Description of the way data are organized.
frame
Telemetry unit.
segment
Period of time in which sensor is operating in a single mode.
GAEL
Consultant
2
Development of a baseband data archive
interchange format
reference GAEL-ICF-TN-060.2
issue 2 revision 3
ICF Concepts technical note
date 13/04/2000
page 11 of 30
CEOS ICF CONCEPTS
Scope of this section is threefold:

Analyze the present context of the various tasks going from the data acquisition to their distribution
to end-users. Baseband data interchange is precisely located with regard to the other activities:
acquisition, archiving and production.

Identify the organizations having the knowledge and the authority to define the form and contents of
the information involved in baseband data interchange.

Set out two strategies that could be adopted for baseband data interchange: -a “interchange driven by
export facility” and –a more modern “interchange driven by import facility”.
2.1. Interchange context
This section addresses two of the major difficulties of baseband data interchange:
1.
What are exactly the pre-processing levels of telemetry that are achieved in existing systems?
2.
Who has authority to define and control the contents and meaning of the various data entities that can
be found in existing archives and in the interchange format?
2.1.1. From Earth Observation to end-users
Space segment
OBR
original signal
data
Transmission
subsystem
Ground segment
instruments
transmitted telemetry
CEOS ICF
Ground segment
Export CEOS
ICF subsystem
Import CEOS
ICF subsystem
baseband
archive
Earth
Receiving
subsystem
received
telemetry
Archiving
subsystem
Production
subsystem
end-user
product
Figure 2.1.a.: Data flow diagram.
Original signal data
are the outputs of sensors when satellite processes acquisition within the
reception area of the ground station or data played back from On-Board
Recorder(s) (OBR).
Transmitted telemetry
is the serial stream of bits than have been arranged in a transmission format.
CCSDS has settled standards for this transmission like the organization in
CADU’s and transfer frames.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 12 of 30
Received telemetry
Transmitted telemetry in bitstream format possibly corrupted during its
transfer and filtered by front-end sub-system.
Baseband archive
After having been pre-processed (see here below), telemetry is arranged in an
archive format (FRED, MDPS, GERALD, VEXCEL) and stored onto highdensity media in computer compatible format.
End-user product
Upon work ordering, data are extracted from the archive, processed and rearranged in an intermediate format (see level 0 of SPOT or Wideband of
Landsat 7) or a distribution format for end-users.
2.1.2. Mission-dependent exchanges
Some satellite operating agencies have already set-up an exchange policy to fulfill their proper needs.
 Direct Receiving Stations of the SPOT network shall be able to produce level 0 scenes in CAP (or
CRIS few years ago) format. This product is very close the first distributed processing level (1A) that
only performs radiometric corrections with respect to level 0.
In this way, SPOT IMAGE may process scenes that have been received by other ground segments.
 For Landsat 7 mission, NASA has issued a Wideband data format (F-5) that defines the output of LPS
(Landsat processing System). This format seems to be the one used to archive 0R data at the EROS
Data Center. A note entitled “Landsat 7 Computer Compatible Raw Wideband exchange Data Format
Control Book, version 1.0” has been issued by USG in June 1999 stating the way archives will be
exchanged between the EROS Data Center and the international ground stations.
At the contrary of SPOT, exchange concerns complete subintervals and not only single scenes.
2.1.3. Telemetry processing levels
Details of the way FRED, MDPS, GERALD and VEXCEL process telemetry before archiving is
analyzed in section “3.1. Validation of Image Data Exchange” of “Archive Format Technical Note”
(R-1).
This study applied to the three missions/instruments (Landsat ETM+, SPOT 1,2,3,4 XS,XI,P/M and ERS
SAR HR OGRC) shows that formats may be:

very close: example of ERS SAR, where the raw telemetry is almost left unchanged by FRED,
MDPS or VEXCEL, or

very different: example of SPOT, where FRED keeps the original telemetry and GERALD preprocesses data until a level 0 format.
Nevertheless, all the formats process base pre-processing: -PN decoding, -downlinked channels are
interleaved, -play-back data are inverted, -Red Salomon corrections are applied, -DPCM is decompressed
and –data are almost always unpacked into bytes.
Even for the more elaborated level 0 format, changes seem to be limited to a re-arrangement of the
transmitted data and are reversible.
2.1.4. Data formats and configuration control
Scope of this section is to identify the various format types and the organizations in charge of controlling
them (see figure 2.1.b.).
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
Satellite Operating
Agency
archive
format 1
issue 2 revision 3
date 13/04/2000
page 13 of 30
Product Distribution
Agency
Archive
Processor
Designer 1
telemetry
format
reference GAEL-ICF-TN-060.2
Archive
Processor
Designer 2
ICF format
archive
format 2
end-user
format
CEOS WGISS
Standardization
bodies
Figure 2.1.b.: Data format and configuration control responsibilities.
Telemetry format
This format has been issued by the satellite operating agency (ESA for ERS, NASA
for Landsat, CNES for SPOT, NASDA for JERS…) and distributed to the ground
stations within a document often called “Satellite to Ground Station Interface
Specifications”. This reference document defines the way data are transmitted from
satellite and possibly the way auxiliary data useful for further processing are
transmitted from Mission Operation Center. This document is unique and contents
the exhaustive list of data entities that are handled by the mission.
Archive formats
These formats are normally designed by the companies, sometimes in conjunction
with the operating agency, that have developed complete systems able to receive,
archive and produce end-user data for one or more missions, or by the operating
agency. There is no standard to state at which processing level shall stay the archive
(for example, a raw telemetry may be archived in FRED instead data in GERALD
archive are close the end-user representation). CEOS has proposed the Baseband
Data Concepts as the baseline for the archive data.
End-user formats
In the majority of the cases, end-user format are defined by the satellite Operating
Agency it-self. But, in the same way distribution is sub-contracted to subsidiaries or
other organizations, end-user formats are sometimes defined by these third-part
organizations (example of SPOT IMAGE that produces and distributes images from
the satellite managed by the CNES). Because the tendency is to distribute addedvalue products with always less auxiliary data, these end-user formats provide few
data entities to be kept in CEOS ICF. Only the knowledge of the way end-user
products are generated allow to determine which data entity shall be kept in CEOS
ICF to reach the same performances as for data directly received from the station.
CEOS ICF is developed and will be managed by CEOS WGISS. As shown before, it is essential to
maintain a close collaboration with the following experts:

Mission experts – These people are the more qualified to determine the list of data entities (name,
definition, default value, allowed values, mandatory/optional status…) that concern their mission.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 14 of 30

Archive format experts – These people can define which data are required in input of their archive to
allow further processing and which information is required to ensure the long-term preservation of
data.

Standard experts – Designed from the standards, ICF format shall be maintained under a strict
configuration control by an organization that will stay in close collaboration with the standardization
bodies.
2.2. Data Exchange Scenarii
The interchange operation may be seen in two manners that lead to two different implementation
procedures and two types of application tools :
-
interchange driven by export facility,
-
interchange driven by import facility.
2.2.1.
Interchange Driven by Export Facility
The archiving facility develops a defined translator where available information is introduced into ICF
version of its own dataset. The information may come from the archive format but also from external
source. The goal of this task is to fill as much as possible the CEOS ICF format (Figure 2.2.1).
When another facility requires those data, the exporting operations are :
-
select the requested dataset and the respective data from respective auxiliary archive,
-
convert the information into the CEOS ICF,
-
dump resulting data onto a medium or transfer it through network.
When the import facility receives the data, the importing operations are :
-
convert the CEOS ICF dataset into the import format,
-
record the resulting data set onto the archive medium,
-
record the default value when required information is not available,
-
keep interchange metadata file on a separated server.
More sophisticated operations such as the following ones may be added :
-
the export facility shall send first the current CEOS ICF DED for expertise. Comparison with
equivalent files of the import facility shall lead to the exchange losses assessment before effective
transfer ,
-
it can undergo some reprocessing or some quality control of resulting data with the current archiving
system,
-
it can add to the interchange metadata file some data from CEOS ICF, some precision, some extra
comment, data which were not recorded.
Development of a baseband data archive
interchange format
GAEL
Consultant
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 15 of 30
Export Facility
AUXILIARY
ARCHIVE
ARCHIVE
PRODUCT
EXTERNAL
SOURCES
ICF WRITER
ICF
PRODUCT
ICF READER
AUXILIARY
ARCHIVE
ARCHIVE
PRODUCT
OPERATOR
Import Facility
Figure 2.2.1 : Representation of exchange driven by the export facility
2.2.2.
Interchange Driven by Import Facility
The idea of this exchange is to avoid transferring information which shall not be required by the import
facility. The export shall make a request according to CEOS ICF formalism of information it needs.
When available, import facility shall convert it into CEOS ICF formalism.
Those exchanges shall be implemented through Request Broker Architecture which will allow real time
transfer of information (M-112).
Development of a baseband data archive
interchange format
GAEL
Consultant
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 16 of 30
EXPORT FACILITY
AUXILIARY
ARCHIVE
ARCHIVE
PRODUCT
ICF READER
ICF READER
EXTERNAL
OPERATOR
ICF READER
Format Request Broker
ICF WRITER
ICF WRITER
FACILITY
ARCHIVE
SERVER
PRODUCT
SOURCES
Import Facility
Figure 4.2 : Representation of exchange driven by the import facility
GAEL
Consultant
3
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 17 of 30
CEOS ICF HIGH LEVEL REQUIREMENTS
3.1. Differences with archiving requirements
Scope of this section is to resume the major objective differences between requirements suitable for
archiving from those suitable for interchange.
3.1.1. Memory constraints
Interchange will concern only a part of existing archives. Although archiving is a systematic process
performed every day whatever being the quality of gigabytes of data, interchanges are often punctual
activities that normally concern only small amount of data of particular interest.
As a consequence, data of ICF shall not use compression techniques nor sophisticated physical
organization on singular high-density support that are not foreseen in the external archive formats.
3.1.2. Run-time constraints
At present, archiving is processed in near real-time. In a next future, we may imagine that telemetry
resulting from direct capture will be processed on-the-flight onto high-density media without intermediate
storing on local disks. It can also be an off-line process performed during transcription operations but
normally the format is kept the same as the real-time one, in order to ensure compatibility.
Interchange is processed off-line and export facility has time to re-arrange data into a more convenient
format. As a consequence, ICF may require more processing to possibly re-arrange and process baseband
data archives.
An other impact concerns the sequencing of data. Although some information (for example, the amount
of data, their quality, the time interval…) may only be assessed once the complete acquisition is
performed, CEOS ICF can compile a metadata file that will contain all this information and position this
file before the data themselves for sequential access media (tapes or electronic links).
3.1.3. Strong portability constraint
Whereas the existing archives are often thought as internal storage unit within the acquisition and
production system, CEOS ICF shall guaranty a perfect portability between heterogeneous systems.
ICF shall be multi-media: tapes of whatever density, disk representation and electronic transfer.
3.1.4. Released physical reliability
Durability of acquired data is a critical issue for archiving. As shown in the study of existing archive
formats (R-1), these systems use mechanisms (duplication of data, redundancy checking, periodical
transcriptions) to guaranty such durability in front the aging of supports.
At the contrary, CEOS ICF shall not consider this issue. The medium that will be used to exchange the
data will be often occasional. If the data cannot be read from the destination system, exchange will be
processed again.
3.1.5. Direct access to archive
Archive formats are almost always designed together with the production system that will extract the data
to generate end-user products. Mechanisms are set up to allow easier and/or faster access to the archive
from the production system.
This issue is not a requirement for CEOS ICF.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 18 of 30
3.1.6. External references
In almost all the studied archive formats (R-1), codes are used to identify some data entities (mission,
instrument, processing level, transmission mode…). These codes or mnemonics are understandable by the
production system and allows to spare memory space in archive.
Because CEOS ICF shall not consider memory constraints and shall be self-consistent, completeness, use
of external references (mnemonic tables, databases, external functions providing metadata…) shall be
avoided, in favor of a more explicit representation.
3.1.7. Impact of software changes
Archives have been designed for long term. Changes in format may lead to difficulties when reading data
written with the previous version or long time ago.
At the opposite, interchange activities are limited in time. Exchanged data in CEOS ICF are ephemeral
and has not to be kept. Possible changes of CEOS ICF will therefore have a less critical impact.
3.1.8. Sensor Description for Earth Observing Dynamic
Our intention was to use the sensor definitions present in document “A Strawman Proposal for a
Standard Sensor Description Format of Earth Observing Dynamic Sensors” (M-4). Unfortunately, these
definitions cannot be retained for the following reasons:

These sensor definitions do not match (or very few) the information and the formalism found within
the archives (see R-1). Scope of interchange format is to deal with data entities actually present in
source and target formats, and is not to introduce new information.

Sensor definitions of M-4 proposal are aimed to provide the necessary (and homogeneous)
information that would enable further geolocation of the acquired data. Importance of this issue is to
be considered when designing a new (or universal) archive format.
3.1.9. Open Archival Information System
In the same way as previous section, a possible contribution of principles and statements laid down in
“Reference Model for an Open Archival Information System (OAIS)”(M-5) has been assessed. An
interesting concept is defined in section “2.2.2. Information Package Definition” that introduces the
Preservation Description Information (see here below).
Preservation Description Information (PDI) is divided into four types of preserving information:
 Provenance that describes the source of Content Information (primary target of the preservation).
 Context describes how the Content Information (CI) relates to other information outside the
Information Package (CI+PDI).
 Reference provides one or more identifiers, or systems of identifiers, by which the Content
Information may be uniquely identified.
 Fixity that provides a wrapper, or protective shield, that protects the Content Information from
undocumented alteration (for example use of check sum).
Some information may be found in section “5.1.3.4. Transformation” that introduces the concepts of
Reversible Transformation and Non-Reversible Transformation. This section concerns principally the
transcription activity.
Interoperability between OAIS archives is examinated in section “6.1. Technical Levels of Interaction
Between OAIS Archives”. This section identifies the four following schemas of collaboration:
-independent archives, -cooperating archives, -federated archives, and –archives with shared functional
areas.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 19 of 30
3.2. High level requirements
A good format for interchanging baseband data archive shall satisfy the three major constraints:
1.
Portability – The format shall be suitable for a wide variety of storage and exchange technologies.
Format shall be simple and managed by standard softwares.
2.
Exhaustiveness – All the elements shall be kept that will allow post-processing’s until a reliable enduser product.
3.
Opening – The format shall be open for further introduction of new mission/instrument until new
acquisition technologies.
GAEL
Consultant
4.
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 20 of 30
CEOS ICF DETAILED REQUIREMENTS
4.1. Baseband Interchange Unit (BIU)
Interchange will concern only one segment (also called sub-segment (F-2) or sub-interval (F-5)) of one
satellite at time. A segment is a period of acquisition during that the physical characteristics of the
platform stay invariant (same spectral mode, same pulse frequency, same viewing angle…). All these
parameters will be stored with only one value within the attached metadata file. It is therefore not
foreseen to describe a list of products (catalog) as metadata of the exchange.
This interchange unit (sometimes called dataset or datatake) is called Baseband Interchange Unit
(abbreviation BIU). Data of one BIU are stored in a collection of files (at least two: 1 metadata file + 1
signal data file).
When the export facility desires to export more than one BIU on the same physical support, they will be
sequentially stored until the physical end of recording mark is encountered (EOT for tape, directory
contents for disk files and communication closing for electronic transfer).
4.2. Baseband Interchange Medium
4.2.1. Multi-media exchange
CEOS ICF has been basically foreseen to describe data stored on disk files. Nevertheless, exchange of
baseband data may use the following types of supports:

Disk files – series of files under the same directory,

Tapes (DLT, Exabyte, DAT…) – series of sequential files separated by EOF mark and terminated by
the EOT mark,

Electronic transfer – series of disk files transferred using an appropriate file transfer protocol.
4.2.2. File naming
All the files of a same baseband interchange unit shall have the same prefix. This prefix is compound
from values of mandatory fields identifying at least the mission, instrument, station, and acquisition
date/time start/stop for the segment.
The suffix shall identify the type of data (signal or metadata).
4.2.3. Use of tapes and data packaging
One or more tape(s )may be used to exchange BIU. In this case, standard archiving tools (UNIX:tar or
gzip, cpio, dd; VMS:BACKUP; Windows: zip…) will be agreed between export and import facilities.
4.2.4. Independence from medium and limitations of operating system
Interchange is very often an occasional interface involving two organizations that should agree about the
type of medium and possible limitations due to their operating systems. To reduce risks of
misunderstanding, CEOS ICF shall allow to adopt the lowest profile of performances.
Disk representation – File size: Size of files on disk may not exceed 231 bytes (approximately 2
Gigabytes) because many operating systems (or versions of operating systems) do not allow file sizes
greater than this limit (for example, the standard UNIX “tar” command deals with files whose size is less
than 2 GB). When a file is greater, it may be split in a series of 2 GB files whose names record the
sequence for further concatenation.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 21 of 30
4.2.5. Data segmentation
An other way to reduce the size of data shall be given allowing to export only a part of the acquired
segment. Metadata shall record the elements (time/frame start/stop) allowing to accurately locate the part
of segment being exchanged.
4.3. BIU Organization
4.3.1. File structure
To maintain CEOS ICF as simple as possible, an entire BIU will basically contain

one unique Metadata file, and

one or more signal data files.
Other files (for example a “Tape Volume Label File”) could accompagn the BIU in the medium but these
files are not part of the ICF specification. In order to allow ICF readers to clearly identify the metadata
file, its first line will always contain a statement identifying it-self.
4.3.2. File order
The metadata file will always stand in first position within BIU on sequential access media like tapes or
electronic transfer.
4.4. Metadata Management
4.4.1. Metadata Organization
Metadata are stored within groups of information organized in a hierarchical way. This organization shall
allow to add a new group (for example for a new mission or instrument) under a certain node of this
hierarchy, without changing the rest of the format.
4.4.2. Language for Metadata Description
The language to be used for the description of metadata should fulfill the following technical
requirements:
a)
Strict use of ASCII.
b) Hierarchical description – Language shall allow to describe groups compound of one or more
(sub)groups and terminal leaves.
c)
Base data types – Integer, float, double, boolean and string data types shall be available and numerical
values can be managed with the largest number of significant bits allowed by the intrinsic
representation.
d) User’s data type – Programmer may built-in its proper data type (for example a UTC date-time)
naming it for further use. Description of these user’s data type will be joined to the BIU. CEOS ICF
shall also takes into account the possibility that certain information shall not be described by their
content but optionally by the location where they can be found. It can refer to a document or a server
which can be respectively accessed through a reference or an URL address. The possibility to transfer
blocks of information may be sometimes required.
e)
Signal data schema – Language shall allow to define the structure of the signal data record within the
relative metadata file. In such a way, import translation software could dynamically read and process
signal data records.
GAEL
Consultant
f)
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 22 of 30
Lists – Language shall allow to represent lists with a variable and unlimited number of items. The
empty list can be declared or can be considered as a default value.
g) Default value – Language shall allow to set a default value when no definition has been found in
metadata file.
h) Undefined value – Language shall allow to declare that no value is known for a data entity.
i)
Mandatory/optional – Language shall allow to state that value for data entity is mandatory or optional.
The language parser should be able to warn user when a data entity considered as mandatory has not
been found with a value.
j)
Value range or enumeration – Language shall allow to define a range or a list of allowed values.
k) Conditional description - Language should allow to consider description for a data entity only if one
or more conditions are fulfilled getting the value of one or more other data entities. For example data
entities of the LANDSAT group have not to be defined if the mission is ERS…
In addition this language should fulfill the following operational requirements:
l)
Parser able to ingest the description shall be Off-the-Shelf or is available freeware on Internet.
m) Language shall be maintained and a version should emerge considered as a standard.
All these requirements will be collated with regard to the description languages already available
(DEDSL, ODL, PVL, XML…). The data entity dictionary entries contained in Appendix B will be fully
codified and documented using the selected language to stand within the “CEOS ICF” document.
4.4.3. DED organization
DED is organized in a hierarchical way and includes the following main sections at the first level of this
hierarchy.
Entity title
Short description
ICF_VERSION
Information found in this section shall identify the version of the dictionary
to be used to parse this metadata file.
INTERCHANGE
Information included in this section identifies the physical volume (see
4.3.1) and the BIU contents. These last elements will be used to generate
the metadata and signal data file names.
PROCESSING
This section contains the history information allowing to maintain the
traceability of data through the main activities: -RECEIVING,
-PRE_PROCESSING (TBC), -ARCHIVING and –EXPORTING.
PLATFORM
All the information generated on-board the platform is included in this
section.

ACQUISITION and ON_BOARD_COUNTER allow to precisely set
the viewing period and the conversion between both dating systems.

SEGMENT includes the identifiers (orbit, track, segment start/stop and
image line number) of the granule.

ORBITAL_BULLETIN retains parameters used by NORAD TLE
(Two-Line Elements) to propagate the orbit.

EPHEMERIS and ATTITUD are the state vectors registered on-board
and transmitted in telemetry.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 23 of 30
SIGNAL_DATA
This section describes the way signal data files (one or more) are organized
and, in particular, the design of the record field per field.
QUALITY
All the information that will form a Quality bulletin is reported in this
section. This bulletin is divided in three sub-sections relative to the various
critical steps: -ACQUISITION, -TRANSMISSION, -PRE_PROCESSING.
The number of observed defects (dead detectors, out of range radiometry,
synchronization loss, bit errors, missing lines) is reported. In addition, for
some of these defects, a list of indices (or locations) for that a defect has
been observed may be reported.
The complete list of CEOS ICF data entities is reported in Appendix B.
4.4.4. Mission-dependent section
An attempt has been made to try to unify the various missions / instruments according to their technical
characteristics. Using a hierarchical approach, each node includes attributes that are relevant only to itself and to its sub-tree (see figure 4.4.4).
SENSOR
SCANNER
 DETECTOR_NUMBER
 SCANNING_PERIOD
 …
PUSH_BROOM
 DETECTOR_NUMBER
 SAMPLING_PERIOD
 …
MATRIX
 VERTICAL_DETECTOR_NUMBER
 HORIZONTAL_DETECTOR_NUMBER
 …
SAR
 PULSE_REPETITION_FREQUENCY
 …
TO BE COMPLETED
Figure 4.4.4.: Sensor characteristics tree.
Unfortunately, it seems difficult and not always easy to understand to pursue in this way.
We suggest an alternative solution, in that all the attributes specific (i.e.; not already foreseen in the
CEOS ICF DED) to a certain mission / instrument would be stored under a SPECIFIC section (example
“SPECIFIC_SPOT” or “SPECIFIC_LANDSAT_ETM”...).
This specific section will have the following characteristics:

These sections will be located at the first hierarchical level, i.e. at the same level as the standard
sections listed in 4.4.3.

This specific section will be defined with the same formalism as for the standard section.

Name and contents of this section will be agreed in close collaboration with the agency operating this
satellite. Organization in charge of CEOS ICF configuration control would design a first proposal of
hierarchy deriving from the information found within the “Satellite to Ground Station” document(s).
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 24 of 30
4.5. Signal Data Management
4.5.1. No redundancy
CEOS ICF shall not introduce duplication of data neither redundancy allowing to check a particular value
from values found in other fields.
4.5.2. No encryption
CEOS ICF shall not introduce encryption. If the input data is encrypted and if at least one of the
destination archives expect no encrypted data in input, data shall be decrypted.
4.5.3. CRC and Checksum
When an error has been detected within the transmitted frame and if a correction mechanism is available,
correction shall be applied..
4.5.4. No compression
Signal data shall not be compressed except in the following cases:

Compression is defined inside the transmitted frame and would require sophisticated processing to
decompress it.

Compression is processed by a standard tape driver.

File based lossless compression is applied by using standard, well agreed, SW tools to increase the
transfer throughput, for example over the network.
4.5.5. PN decoded
Like for all the studied formats (see R-1), transmitted data shall be PN-decoded.
4.5.6. Data is frame synchronized
Loss of synchronization will lead to replace corrupted bytes (or lines) by 0 value.
Bit slip/bit insertion shall be corrected. No “guard-bytes” (F-5) shall be kept within the signal records.
4.5.7. Chronological order
When data are time-reversed (case of playback from on-board recorders), bytes and bits will be reversed
to restore a chronological acquisition.
4.5.8. Byte alignment
Bytes are aligned on major and minor (case of Landsat) frames.
CEOS ICF supports both (TBC):

the native representation as found within the telemetry, and

byte alignment per pixel, i.e. when number of bits is extended to match a multiple of 8 (example of
ERS SAR data for that the 5-bits used in telemetry are increased to 8 bits).
A flag will be present in metadata file to indicate if the byte-alignment per pixel has been processed or
not. When this alignment has been processed, the original number of bits within the native representation
will be recorded.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 25 of 30
4.5.9. Signal data record
In order to allow an unique description of all the records containing signal data, length of records shall
match the smallest repeated pattern within the transmitted telemetry.
For example, the fact that the first transmitted frame of ERS OGRC HR format (figure 4.4.2.3.-1 of F-7)
differs from the 28 following ones would oblige to make coincide one CEOS ICF signal data record with
29 ERS frames (record length = 7424 bytes = 29 x 256 bytes).
In this way, it would be possible to describe the signal data record (see “RECORD_DESCRIPTION” tag)
from fields identifying the –synchronization code, -IDHT-ID, -IDHT header, -auxiliary data, and –
measurement data.
4.6. CEOS ICF Version Management
To maintain its durability, CEOS ICF shall be managed by an institutional Configuration Control Board
(CCB). Indeed, the increasing number of missions together with the development of new techniques make
necessary to envisage upgrades of this format.
Scope of this section is to state some first principles for the management of new releases.
4.6.1. Minor/major versions
CEOS ICF organization is principally based on the metadata description. This choice provide a light way
to introduce new mission / instrument just adding new group(s) and data entities. Such change shall lead
to increase the minor version number of CEOS ICF.
All the other changes (deletion of group(s) or data entities, change in hierarchy, change of the type of data
entity, change of the mandatory/optional status of a data entity…) shall lead to increase the major version
number of CEOS ICF.
In its primary release, after acceptance of “CEOS ICF” document, the first version v1.1 will allow to
process at least Landsat ETM+, SPOT 4 P/Xi/XS and ERS SAR missions/instruments.
4.6.2. DED and software availability
All the versions of the Data Entity Dictionary and possibly the software (tools, API’s) allowing to parse,
analyze and generate the metadata file will be made available on a site of CEOS WGISS (TBC).
CEOS ICF version number (minor and major) shall be recorded as first data entity within metadata file.
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 26 of 30
APPENDIX A – BASEBAND DATA CONCEPT COMPLIANCE MATRIX
A.1. Introduction
One of the highest user’s requirement of applicable document A1 is “R-9) The ICF format shall be based
on the CEOS Baseband Data concept.”.
Scope of this section is to guarantee that requirements laid down in this document are fully compliant
with Baseband Data Concepts (A2).
A.2. Compliance Matrix
Baseband Data Concepts (A2) requirements
CEOS ICF requirements
High level requirements
BDC.1
Baseband data should be able of reconstructing the
original sensor output data stream.
Signal data management
BDC.2
Baseband Data should remove encoding techniques
used for transmission (PN, CRC).
4.5.3.
4.5.5.
But Synchronization codes and zero fill can be kept.
BDC.3
Telemetry stream should be frame synchronized and
byte aligned.
4.5.6.
4.5.8.
BDC.4
De-multiplexing of the original serial stream that
separates the video data from all the rest is
mandatory.
BDC.5
Bit/byte reversal of OBR data is to be considered
mandatory.
4.5.7.
BDC.6
Removal of duplicate data is mandatory.
4.5.1.
BDC.7
Removal of bad data is mandatory.
4.5.3.
BDC.8
Ordering of data in time forward order is mandatory.
4.5.7.
BDC.9
Proper time tagging of the stored data is mandatory.
4.4.3.
PLATFORM section
4.4.3.
QUALITY section
BDC.10 All data gaps should be detected and occurrences
should be recorded in the metadata.
Metadata management
BDC.11 The metadata will include information on the satellite, 4.4.3.
sensor and mission, identification of facilities and
processors involved in the formation of product.
INTERCHANGE,
PROCESSING and
PLATFORM sections
BDC.12 The metadata should also include times of various
events such as acquisition time, down link time and
transcription time.
4.4.3.
PROCESSING section
BDC.13 Other mandatory information: -special configuration,
-mode of operation of sensor (on-board gain mode),
-use of OBR, -use of relay station.
4.4.4.
SPECIFIC section
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 27 of 30
BDC.14 Metadata will include methods for the identification of 4.4.3.
the information granules (ex. WRS).
PLATEFORM.
SEGMENT section
BDC.15 Baseband data information granule should be clearly 4.4.3.
identified and time-tagged (on-board clock or another
secure time source).
PLATEFORM.
ACQUISITION section
BDC.16 PCD and on-board calibration information should be
included in the mandatory metadata for Baseband
Data.
PLATFORM.
EPHEMERIS and
ATTITUD sections
4.4.3.
Calibration (TBD)
BDC.17 In the case the Baseband Data are time-tagged with
the on-board clock, a method for converting it into
universal time should be made available.
4.4.3.
PLATEFORM.
ACQUISITION and
ON_BOARD_COUNT
ER sections
BDC.18 It should be possible to associate the baseband Data
granules with the (possibly) latest set of ephemeris
elements, satellite attitude data and sensor pointing
data.
Times of signal data and
ephemeris shall be
within the same
referential.
BDC.19 History of Baseband Data transcription can be
4.4.3.
optionally recorded. History should include –data
source, -version of processing/capture system, -facility
identification, -format version, -data transmission
quality, -data gap information.
PLATFORM.
PROCESSING section.
Development of a baseband data archive
interchange format
GAEL
Consultant
ICF Concepts technical note
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 28 of 30
APPENDIX B – CEOS ICF DATA ENTITY DICTIONARY
B1.
Introduction
Name of entities will respect the following requirements:

Language to be used is the US American.

All the letters of the name will only use upper-case characters [A,Z] and the underscore character (_)
to separate words.

Use of abbreviations shall be minimized (example “NUMBER” in place of “NR”).

Use of plural with letter “S” shall be minimized (example “ATTITUDE_NUMBER” in place of
“NUMBER_OF_ATTITUDES”).
B.2. Hierarchical representation
CEOS_ICF
ICF_VERSION
MINOR
MAJOR
INTERCHANGE
PHYSICAL_ORGANIZATION
VOLUME_NUMBER
VOLUME_INDEX
VOLUME_NAME
SATELLITE_ID
FAMILY_NAME
NUMBER
CODE
SENSOR_ID
FAMILY_NAME
NUMBER
CODE
ACQUISITION
START_DATE
START_TIME
STOP_TIME
PROCESSING
RECEIVING
DATE
START
STOP
FACILITY
COUNTRY
AGENCY
STATION
FACILITY
SOFTWARE_RELEASE
FORMAT_SYNCHRONIZER_DECOMMUTATOR
NAME
RELEASE
PREPROCESSING
SOFTWARE
NAME
RELEASE
ARCHIVING
Optional
Optional
Development of a baseband data archive
interchange format
GAEL
Consultant
ICF Concepts technical note
DATE
START
STOP
EXPORTING
DATE
START
STOP
FACILITY
COUNTRY
AGENCY
STATION
FACILITY
SOFTWARE_RELEASE
FILE_NAME_RADIX
PLATFORM
ACQUISITION
TIME_START
TIME_STOP
ON_BOARD_COUNTER
COUNTER_START
COUNTER_STOP
SEGMENT
ORBIT_NUMBER
TRACK_NUMBER
SEGMENT_START
(time)
SEGMENT_STOP
(time)
ORBITAL_BULLETIN
PRODUCED_BY
UTC_DATE
ORBITAL_INCLINATION
RIGH_ASCENSION_NODE
ECCENTRICITY
ARGUMENT_OF_PERIGEE
MEAN_ANOMALY
MEAN_MOTION
DRAG_COEFFICIENTS
NUMBER
LIST
EPHEMERIS
PRODUCED_BY
NUMBER
LIST
TIME
POSITION
X
Y
Z
VELOCITY
X
Y
Z
ATTITUDE
ANGLE
NUMBER
LIST
TIME
YAW
ROLL
PITCH
VELOCITY
NUMBER
LIST
TIME
YAW
ROLL
PITCH
SIGNAL_DATA
NUMBER
LIST
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 29 of 30
GAEL
Consultant
Development of a baseband data archive
interchange format
ICF Concepts technical note
ORGANIZATION
DATA_SIZE
FILE_NUMBER
RECORD_LENGTH
FIRST_FILE_RECORD_LENGTH
FILE_NAME_RADIX
BYTE_ORDER
(LSBF,MSBF)
BYTE_ALIGNMENT
IS_PROCESSED_FLAG
BITS_PER_PIXEL
NATIVE
CURRENT
DATA
INTERLEAVING_INDICATOR
BITS_PER_SAMPLE
SAMPLES_PER_BLOCK
CHANNELS_PER_BLOCK
BLOCKS_PER_RECORD
RECORD_DESCRIPTION
FIELD
NUMBER
LIST
NAME
DEFINITION
BYTE_START
BYTE_STOP
OCCURRENCE
CODING_TYPE
DATA_TYPE
QUALITY
TRANSMISSION
SYNCRONIZATION
PATTERN
LOSS
NUMBER
BIT
ERROR_NUMBER
ERROR_RATE
FRAME
PROCESSED_NUMBER
ERROR_NUMBER
SPECIFIC_LANDSAT7_ETM
…
SPECIFIC_SPOT1234
…
QUALITY
ACQUISITION
DEAD_DETECTOR_NUMBER
OUT_OF_RANGE_RADIOMETRY
PREPROCESSING
MISSING_LINE
NUMBER
REPORTED
NUMBER
LIST
…
SPECIFIC_ERS_SAR_OGRC
…
reference GAEL-ICF-TN-060.2
issue 2 revision 3
date 13/04/2000
page 30 of 30
(BINARY,ASCII)
(INTEGER,FLOAT,DOUBLE,STRING)
Optional