GOME NO2 data product
HDF data file user manual
Folkert Boersma and Henk Eskes
23 April 2002
Contents
Content of the no2trackyyyymmdd.hdf files
3
The use of the averaging kernel
6
IDL sample code to read the ozone fields
7
2
Content of the no2trackyyyymmdd.hdf files
The no2trackyyyymmdd.hdf file contains NO2 data for one day. These files are organised as follows:
• SDS Global Attributes (contains information about the file)
• Vdata (the actual data)
SDS Global Attributes
An example of the SDS Global Attributes is given below. Data_created_by refers to the version of
the retrieval-assimilation software that was used to generate the NO2 column data.
SDS Global Attributes
Name
Value
__________________________________________________________________________
Author
H.J. Eskes, K.F. Boersma
Affiliation
KNMI (Royal Netherlands Meteorological Institute)
Email
[email protected], [email protected]
Data_created_by
TM3-NO2A, version 1
Creation_date
8 Mar 2003
Unit_of_NO2_column
1e15 molecules/cm2
VData
The Vdata tables contain the data product, and consists of:
• pressure_grid
• NO2_ymmddttt (contains the main NO2 retrieval data for one GOME track)
• GEO_ymmddttt (contains all geometric data associated with retrievals in this track)
• ANC_ymmddttt (contains all ancillary data associated with retrievals in this track)
The VData contains only one pressure_grid. This table defines the hybrid pressure levels at which
the averaging kernel is provided. The VData table attribute also provides a recipe to convert the level
constants a_lev, b_lev and surface pressure p_surf into the hybrid level pressures p (in Pascal). The
equation is:
p = a_lev + p_surf * b_lev
(1)
The variable p_surf is stored in the NO2_ymmddttt table: the surface pressure is provided for each
GOME pixel separately.
The VData may contain between 0 and 15 (maximum number of satellite orbits in one day) NO2\_ymmddttt,
GEO\_ymmddttt and ANC\_ymmddttt tables. The table NO2\_ymmddttt is accompanied by a VData Table
Attribute that contains the name of the track, and the start- and end-time (year, month, day, hour,
minutes, and seconds):
3
VData Table Attributes
Name
Value
__________________________________________________________________________
track_identifier
70201035
start_time
1997, 2, 1, 3, 57, 43
end_time
1997, 2, 1, 4, 40, 45
The main data product table is NO2\_ymmddttt and it contains 15 columns. They are:
date
time
lon
lat
vcd
sigvcd
vcdtrop
sigvcdt
vcdstrat
sigvcds
fltrop
psurf
sigvcdak
sigvcdtak
kernel
Date of GOME retrieval (yyymmdd)
Time of measurement ((h)hmmss)
Centre longitude of pixel
Centre latitude of pixel
Retrieved total vertical column density (in 1e15 molec. cm-2)
Error in the total vertical column density (in 1e15 molec. cm-2)
Retrieved tropospheric vertical column density (in 1e15 molec. cm-2)
Error in the tropospheric vertical column density (in 1e15 molec. cm-2)
Assimilated stratospheric vertical column density (in 1e15 molec. cm-2)
Error in the stratospheric vertical column density (in 1e15 molec. cm-2)
Flag that indicates if the tropospheric retrieval is meaningful;
0 = yes, -1 = no
Surface pressure of the pixel (in Pa)
Error in total vertical column density when averaging kernel information
is used, in 1e15 molec. cm-2 (without profile error contribution)
Error in tropospheric vertical column density when averaging kernel information
is used, in 1e15 molec. cm-2
Averaging kernel vector, corresponding to the kernel values at the
pressure levels as defined above
The geolocation data is stored in GEO\_ymmddttt, and it contains 6 columns:
sza
vza
raa
ssc
loncorn
latcorn
satellite solar zenith angle
satellite viewing zenith angle
satellite relative azimuth angle
GOME subset counter, 0 = West, 1 = Center, 2 = East, 3 = Backscan
longitudes of the four corners of the pixel
latitudes of the four corners of the pixel
4
Additional retrieval information is provided by the ancillary data table ANC\_ymmddttt, and it contains
10 columns:
scd
amf
amftrop
amfgeo
scdstr
clfrac
cltpres
albclr
crfrac
ltropo
slant column density, in 1e15 molec. cm-2 (from IUP-Heidelberg)
total air-mass factor used to compute vcd (= scd / amf)
tropospheric air-mass factor used to compute vcdtrop (= [scd-scdstr] / amftrop )
geometrical air-mass factor
stratospheric slant column density (= amfgeo * vcdstrat )
cloud fraction from FRESCO
cloud top pressure from FRESCO
surface albedo for clear part of the pixel from TOMS/GOME database
cloud radiance fraction, i.e. percentage of light coming from the cloudy
part of the scene
index of pressure level where tropopause occurs (2K/km level)
5
The use of the averaging kernel
Users will generally have a range of interests in employing the data. We can basically distinguish two
applications of the NO2 data sets.
(1) For plotting purposes, qualitative comparisons, and when no additional source of information on NO2
is available: In these cases the users will be mainly interested in the total NO2 column (vcd) and its
error estimate (sigvcd), or in the tropospheric column (vcdtrop) and its error estimate (sigvcdt). The
NO2 columns vcd and vcdtrop are the best estimate of the total and tropospheric column. However,
it is important to note that these retrieved quantities are sensitive to assumptions concerning the NO2
profile shape.
(2) For comparisons with measured NO2 profiles, with chemistry-transport models, or when the data is
assimilated into a chemistry model: In these cases the users should combine the retrieved columns with
the averaging kernels (AK) provided. The averaging kernel also gives a very useful interpretation of the
data product: it is the sensitivity of the GOME measurement to NO2 at a given altitude.
The averaging kernel vector provides the relation between the actual NO2 profile x and the retrieved
quantity y (either vcd or vcdtrop),
y =A·x
(2)
with A the averaging kernel vector, specified at pressure levels as described in the previous section. The
actual NO2 profile x should consist of partial columns of NO2 , in 1015 molecules cm−2 .
For data assimilation purposes A is the observation operator that converts the NO2 model forecast xf
into a forecast of the GOME retrieved column y f .
For detailed comparisons with chemistry models one should first interpolate the model field to the
longitude/latitude and time of the GOME observation. Next, this collocated model profile is multiplied
by the averaging kernel to result in a model prediction of the observation. This can be directly compared
with for instance vcd as retrieved from the GOME observation.
For these calculations, the error in y reduces to sigvcdak, which is smaller than sigvcd, since uncertainties on the vertical NO2 profile are no longer contributing. Note that sigvcdak combines both
measurement errors (in y) and averaging kernel errors.
The application of the averaging kernel involves an interpolation to the pressure grid on which the
averaging kernel is defined. This should be done with some care: abrupt changes in the AK will occur
especially near the cloud top pressure.
For tropospheric retrievals (with y now the tropospheric column), equation 2 reduces to:
y = Atrop · xtrop
(3)
where Atrop is the averaging kernel for the tropospheric column, defined as:
Atrop = A ·
amf
amftrop
(4)
Here xtrop is the profile defined for the tropospheric levels only (levels up to level number ltropo as
specified in ANC\_ymmddttt). The pressure at level ltropo corresponds to the pressure of the layer in
which the tropopause occurs according to the WMO 1985 temperature tropopause criterion.
6
IDL sample code to read the NO2 fields
The following IDL program reads an HDF data file and stores the data in a structure no2.
pro readNO2colHDF
;------------------------------------------------------------------; Read data from GOME HDF file and stores in structure "no2"
;
;"maxorbits"
maximum number of orbits in a file
;"maxpix"
maximum number of pixels in an orbit
;
; Folkert Boersma, KNMI, April 2003
;------------------------------------------------------------------maxorbits = 15
maxpix
= 2500
file
= ’no2track19970201.hdf’
; Define structure for GOME data
no2 = { $
norbits : 0, $
npix
: intarr(maxorbits), $
a_lev
: make_array(19,/float,value=-999), $
b_lev
: make_array(19,/float,value=-999), $
date
: make_array(maxorbits,maxpix,/int,value=-999), $
time
: make_array(maxorbits,maxpix,/int,value=-999), $
lon
: make_array(maxorbits,maxpix,/float,value=-999.9), $
lat
: make_array(maxorbits,maxpix,/float,value=-999.9), $
vcd
: make_array(maxorbits,maxpix,/float,value=-999.9), $
sigvcd
: make_array(maxorbits,maxpix,/float,value=-999.9), $
vcdtrop
: make_array(maxorbits,maxpix,/float,value=-999.9), $
sigvcdt
: make_array(maxorbits,maxpix,/float,value=-999.9), $
vcdstrat : make_array(maxorbits,maxpix,/float,value=-999.9), $
sigvcds
: make_array(maxorbits,maxpix,/float,value=-999.9), $
fltrop
: make_array(maxorbits,maxpix,/int,value=-999.9), $
psurf
: make_array(maxorbits,maxpix,/int,value=-999.9), $
sigvcdak : make_array(maxorbits,maxpix,/float,value=-999.9), $
sigvcdtak : make_array(maxorbits,maxpix,/float,value=-999.9), $
kernel
: make_array(maxorbits,maxpix,19,/float,value=-999), $
sza
: make_array(maxorbits,maxpix,/float,value=-999.9), $
vza
: make_array(maxorbits,maxpix,/float,value=-999.9), $
raa
: make_array(maxorbits,maxpix,/float,value=-999.9), $
ssc
: make_array(maxorbits,maxpix,/int,value=-999.9), $
loncorn
: make_array(maxorbits,maxpix,4,/float,value=-999), $
latcorn
: make_array(maxorbits,maxpix,4,/float,value=-999), $
scd
: make_array(maxorbits,maxpix,/float,value=-999), $
amf
: make_array(maxorbits,maxpix,/float,value=-999), $
amftrop
: make_array(maxorbits,maxpix,/float,value=-999), $
7
amfgeo
scdstr
clfrac
cltpres
albclr
crfrac
ltropo
:
:
:
:
:
:
:
make_array(maxorbits,maxpix,/float,value=-999),
make_array(maxorbits,maxpix,/float,value=-999),
make_array(maxorbits,maxpix,/float,value=-999),
make_array(maxorbits,maxpix,/float,value=-999),
make_array(maxorbits,maxpix,/float,value=-999),
make_array(maxorbits,maxpix,/float,value=-999),
make_array(maxorbits,maxpix,/int,value=-999)}
$
$
$
$
$
$
id = hdf_sd_start(file,/read)
; The HDF_SD_FILEINFO procedure retrieves the number of datasets and
; global attributes in an HDF file.
hdf_sd_fileinfo,id,datasets,attributes
help,datasets,attributes
iret = 0
for i=0,attributes-1 do begin
hdf_sd_attrinfo,id,i,name = name, data = d
command = name+’=d’
iret = execute(command)
print, ’Retrieved Attribute:’,name,’ ’,d
endfor
hdf_sd_end,id
; Open file and initialize vdata reading
file_id=hdf_open(file,/read)
vd_id = -1
vd_handle = -1
end_of_file=0
vds = hdf_vd_lone(file_id)
nvds = n_elements(vds)
if (nvds eq 0) then begin
print, ’ERROR: No vdatas found in file’
stop
endif
; Loop over vdatas
for i=0,attributes+nvds-1 do begin
vd_id = hdf_vd_getid(file_id,vd_id)
vd_handle=hdf_vd_attach(file_id,vd_id,/read)
hdf_vd_get,vd_handle,nfields=nf,name=vd_name,count=count,fields=fields
; Read pressure fields a_lev and b_lev into variables
if (strmid(vd_name,0,4) eq ’pres’ ) then begin
for j=0,nf-1 do begin
hdf_vd_getinfo,vd_handle,j,name=fieldname,size=size,type=type
name = strcompress(fieldname,/remove_all)
iret = execute(’nread=hdf_vd_read(vd_handle,’+$
8
name+’,fields="’+fieldname+’")’)
if (iret ne 1) then begin
print,’Error dataset’,i
stop
endif
endfor
no2.a_lev(0:18)
no2.b_lev(0:18)
endif
= transpose(a_lev)
= transpose(b_lev)
if (strmid(vd_name,0,4) eq ’NO2_’ ) then begin
if (vd_name ne ’start_time’ and vd_name ne ’end_time’) then begin
track_date = long64(strmid(vd_name,strlen(vd_name)-8,8))
track_date = track_date * 1000 + 19900000000000
endif
; Check whether orbit fits in data structure
if (no2.norbits ge maxorbits) then begin
print, ’ERROR: more orbits in file than’, maxorbits
stop
endif
if (count gt maxpix) then begin
print, ’ERROR: orbit’, no2.norbits+1,’ has more pixels (’, $
count, ’ ) than’, maxpix
stop
endif
; Read fields into variables
for j=0,nf-1 do begin
hdf_vd_getinfo,vd_handle,j,name=fieldname,size=size,type=type
name = strcompress(fieldname,/remove_all)
iret = execute(’nread=hdf_vd_read(vd_handle,’+ $
name+’,fields="’+fieldname+’")’)
if (iret ne 1) then begin
print,’Error dataset’,i
stop
endif
endfor
; Add variables to structure
no2.date(no2.norbits,0:count-1)
no2.time(no2.norbits,0:count-1)
no2.lon(no2.norbits,0:count-1)
no2.lat(no2.norbits,0:count-1)
no2.vcd(no2.norbits,0:count-1)
no2.sigvcd(no2.norbits,0:count-1)
no2.vcdtrop(no2.norbits,0:count-1)
no2.sigvcdt(no2.norbits,0:count-1)
no2.vcdstrat(no2.norbits,0:count-1)
=
=
=
=
=
=
=
=
=
9
date
time
lon
lat
vcd
sigvcd
vcdtrop
sigvcdt
vcdstrat
no2.sigvcds(no2.norbits,0:count-1)
no2.fltrop(no2.norbits,0:count-1)
no2.psurf(no2.norbits,0:count-1)
no2.sigvcdak(no2.norbits,0:count-1)
no2.sigvcdtak(no2.norbits,0:count-1)
no2.kernel(no2.norbits,0:count-1,*)
no2.npix(no2.norbits)
=
=
=
=
=
=
=
sigvcds
fltrop
psurf
sigvcdak
sigvcdtak
transpose(kernel)
count
endif
if (strmid(vd_name,0,4) eq ’GEO_’) then begin
; Check whether orbit fits in data structure
if (no2.norbits ge maxorbits) then begin
print, ’ERROR: more orbits in file than’, maxorbits
stop
endif
if (count gt maxpix) then begin
print, ’ERROR: orbit’, no2.norbits+1,’ has more pixels (’, $
count, ’ ) than’, maxpix
stop
endif
; Read fields into variables
for j=0,nf-1 do begin
hdf_vd_getinfo,vd_handle,j,name=fieldname,size=size,type=type
name = strcompress(fieldname,/remove_all)
iret = execute(’nread=hdf_vd_read(vd_handle,’+ $
name+’,fields="’+fieldname+’")’)
if (iret ne 1) then begin
print,’Error dataset’,i
stop
endif
endfor
; Add variables to structure
no2.sza(no2.norbits,0:count-1)
no2.vza(no2.norbits,0:count-1)
no2.raa(no2.norbits,0:count-1)
no2.ssc(no2.norbits,0:count-1)
no2.loncorn(no2.norbits,0:count-1,*)
no2.latcorn(no2.norbits,0:count-1,*)
=
=
=
=
=
=
sza
vza
raa
ssc
transpose(loncorn)
transpose(latcorn)
endif
if (strmid(vd_name,0,4) eq ’ANC_’) then begin
; Check whether orbit fits in data structure
if (no2.norbits ge maxorbits) then begin
print, ’ERROR: more orbits in file than’, maxorbits
stop
endif
10
if (count gt maxpix) then begin
print, ’ERROR: orbit’, no2.norbits+1,’ has more pixels (’, $
count, ’ ) than’, maxpix
stop
endif
; Read fields into variables
for j=0,nf-1 do begin
hdf_vd_getinfo,vd_handle,j,name=fieldname,size=size,type=type
name = strcompress(fieldname,/remove_all)
iret = execute(’nread=hdf_vd_read(vd_handle,’+$
name+’,fields="’+fieldname+’")’)
if (iret ne 1) then begin
print,’Error dataset’,i
stop
endif
endfor
; Add variables to structure
no2.scd(no2.norbits,0:count-1)
no2.amf(no2.norbits,0:count-1)
no2.amftrop(no2.norbits,0:count-1)
no2.amfgeo(no2.norbits,0:count-1)
no2.scdstr(no2.norbits,0:count-1)
no2.clfrac(no2.norbits,0:count-1)
no2.cltpres(no2.norbits,0:count-1)
no2.albclr(no2.norbits,0:count-1)
no2.crfrac(no2.norbits,0:count-1)
no2.ltropo(no2.norbits,0:count-1)
no2.norbits
=
=
=
=
=
=
=
=
=
=
=
scd
amf
amftrop
amfgeo
scdstr
clfrac
cltpres
albclr
crfrac
ltropo
no2.norbits+1
endif
hdf_vd_detach, vd_handle
endfor
hdf_close, file_id
end
References
[1] H.J. Eskes and K.F. Boersma, Averaging kernels for DOAS total-column satellite retrievals, Atmos.
Chem. Phys. Discuss., 3, 895-910, 2003.
11