EEA/NSV/13/002 – ETC/ICM
Revising the WISE SoE Transitional,
Coastal, and Marine waters
data collection models
- streamlining SoE data models and reporting process -
Prepared by:
Irene del Barrio (EEA)
Marek Staroň (EEA)
Jørgen Nørrevang Jensen (ETC/ICM-ICES)
Anita Künitzer (ETC/ICM-UFZ)
1
Contents
1
Purpose of WISE- State of the Environment Transitional, Coastal, and Marine
waters dataflows revision ...........................................................................................3
2
Why are the data needed? - Products derived from the WISE SoE marine dataflows
......................................................................................................................................3
3
Revision background of WISE-TCM and WISE-2 ......................................................4
4
Eionet consultation on the dataflows revision ..........................................................4
5
Results of WISE-TCM and WISE-2 revision ...............................................................5
5.1
Main changes ....................................................................................................5
5.2
Reporting monitoring sites under WISE-5 ..........................................................7
5.3
Reporting classification systems under WISE-2 .................................................8
5.4
Reporting riverine inputs and direct discharges under WISE-1 (Emissions) .......8
5.5
Changes in codelists ..........................................................................................8
5.6
Streamlining of the data processing and quality control......................................9
6
Reporting permissions (reporter roles) .....................................................................9
7
Providing support to data reporters ........................................................................ 10
Annexes .............................................................................................................................. 11
ANNEX I. Results of the Eionet consultation ............................................................... 11
ANNEX II. Summary of main changes in the tables ..................................................... 11
a)
Water quality .................................................................................................... 11
b)
Riverine inputs and direct discharges............................................................... 13
2
1
Purpose of WISE- State of the Environment Transitional,
Coastal, and Marine waters dataflows revision
As announced in 2015, EEA has undertaken a review of the WISE-SoE dataflows that collect data on
transitional, coastal and marine waters (WISE-SoE marine dataflows), namely the reporting
obligations WISE-TCM and WISE-2. This review has been conducted to simplify data collection
specifications in order to increase the consistency of reported data and to ensure consistency with
the WISE-SoE freshwater dataflows. The review has encompassed the underlying data model, and
reporting requirements.
The WISE-SoE marine dataflows are managed through the Eionet NRC coastal, marine, maritime
network. However, the WISE-TCM dataflow overlaps with similar dataflows established under
Helcom and OSPAR. To avoid double reporting, the EEA has adopted a practice where data reported
under the Regional Sea Conventions (RSCs) mechanisms are incorporated into the WISE-TCM by ICES.
This is possible due to the open access data policy of these conventions. The revised data model has
considered the data models used for the Helcom and OSPAR flows so that this practice can continue,
and the changes will not affect the existing reporting obligations under the RSCs.
Until now, there were two WISE-SoE marine dataflows: 1) the WISE-TCM (stations on water quality,
flux stations, nutrients in seawater, hazardous substances in seawater, sediment and biota, direct
discharges, riverine inputs and associated pressures in the catchments) and 2) WISE-2 (monitoring
stations for biological quality elements and ecological quality ratios of the biological quality elements
in transitional and coastal waters).
As mentioned, both dataflows included the position of monitoring stations, which under the new
conceptual model will be collected under a different obligation: WISE-5 (for Spatial data). Regarding
the riverine inputs and direct discharges, they have been moved to WISE-1 (Emissions), since the
information collected was on loads arriving to the environment, which fits in this dataflow. Finally,
the measurements of nutrients and hazardous substances collected under the WISE-TCM will be now
requested under WISE-6, where all the data will be entered in one table: DisaggregatedData.
2
Why are the data needed? - Products derived from the WISE
SoE marine dataflows
The WISE-TCM is the information basis to produce 3 of the EEA’s marine indicators, 2 of them being
part of the EEA Core Set of Indicators (CSI):



Nutrients in transitional, coastal and marine waters (CSI 021/MAR 005): part of the CSI, and
updated every 2 years, last published in March 2015: http://www.eea.europa.eu/data-andmaps/indicators/nutrients-in-transitional-coastal-and-3/assessment
Chlorophyll in transitional, coastal and marine waters (CSI 023/MAR 006): part of the CSI, and
updated every 2 years, last published in March 2015: http://www.eea.europa.eu/data-andmaps/indicators/chlorophyll-in-transitional-coastal-and-2/assessment
Hazardous substances in marine organisms (MAR 001): part of the marine set of indicators,
and also updated every 2 years. Last published in March 2015:
http://www.eea.europa.eu/data-and-maps/indicators/hazardous-substances-in-marineorganisms/hazardous-substances-in-marine-organisms-1
As well, new indicators on biological data are under development, based on WISE-2 data.
3
EEA also publishes “Waterbase - Transitional, coastal and marine waters”, which is a publicly
available database including data on stations and measurements in transitional, coastal and marine
waters, as well as Direct discharges and Riverine inputs: http://www.eea.europa.eu/data-andmaps/data/waterbase-transitional-coastal-and-marine-waters-11
3
Revision background of WISE-TCM and WISE-2
As mentioned above, one of the main reasons for the revision was the need to streamline with the
freshwater WISE-SoE dataflows. The aim was to simplify the WISE SoE data collection specifications,
to enable better harmonisation and interlinking of WISE SoE data, and to progressive align with
INSPIRE requirements. As part of the freshwater revision it was found advantageous to decouple
reporting of information regarding monitoring stations from the reporting of measurements, and this
has been taken forward in the revised WISE-TCM and WISE-2 data models.
Furthermore, it will support the automation of the data processing phase. To this effect, quality
control procedures will be automated to the extent possible. These procedures will be supported by
the Reportnet QC functionalities, and other procedures and routines.
The revision also focused on identifying:





Key parameters (parameters used for the indicators or for quality control)
Redundant parameters
Poorly filled tables and fields
Options for merging tables (e.g. nutrients and hazardous substances in the 3 matrices)
Options for deleting tables (e.g. PressuresTCM)
To the extent possible, the terminology used in table titles and fields was streamlined with the
terminology used for WFD (e.g. monitoring sites instead of monitoring stations) or the INSPIRE spatial
data themes.
4
Eionet consultation on the dataflows revision
The model revised during 2015 (which intented to merge WISE-TCM with WISE-2) was opened to
Eionet consultation, together with Quality Fact Sheets developed for each country from 18/02-30/04:
http://forum.eionet.europa.eu/nrc-marine-coastal-and-maritime/library/2016-consultations/reviewsoe-tcm-data-collection-model
The countries were requested to review the proposed changes to the WISE SoE TCM data collection
model and to comment on them. Table 1 shows the low feedback received and Annex I includes an
embedded file with the specific comments addressed.
Table 1 Countries feedback during the Eionet consultation
Country
Estonia
Netherlands
Belgium
Turkey
Spain
Ireland
Poland
Date
22/03/2016
25/03/2016
28/04/2016
28/04/2016
29/04/2016
29/04/2016
29/04/2016
QFS
x
x
x
x
x
x
Model
x
4
Country
Date
Sweden
Poland
Sweden
29/04/2016
30/04/2016 x
06/05/2016 x
5
QFS
Model
x
Results of WISE-TCM and WISE-2 revision
After the Eionet consultation, countries comments were taken into account to face the final revision
of the data models, together with further changes that had been addressed in the freshwaters
dataflows. A decision was taken on keeping WISE-TCM and WISE-2 separated.
5.1
Main changes
The final results of the revision are the following:
Common to WISE-TCM and WISE-2:

The stations (monitoring sites) will have to be reported under WISE-5 (see section 5.2), which
is a continuos data-flow, with no specific calls (datasets can be submitted whenever needed).
It is intented to have a stable set of stations, so that they are not anymore reported on a
yearly basis, but just updated when necessary. A registry of stations will be created, that will
serve to quality check that monitoring sites identifiers provided in the time series data
correspond to the monitoring sites in the registry.
Main changes in WISE-TCM:

Measurements of nutrients and chlorophyll a (in water) and hazardous substances in the
three matrices (water, sediment and biota) will be reported under the same table:
DisaggregatedData. This single table constitutes the revised version of the dataflow collecting
information on water quality. It has been baptized WISE-6, and it is the younger cousin of
WISE-4, under which the information on water quality for freshwaters is collected.

Discharges tables (riverine inputs and direct discharges) have been moved to WISE-1.

The Pressures table and fields that were not used or redundant have been removed.
Main changes in WISE-2:

The classification system for TC waters is now considered a metadata table
(BiologyEQRClassifProcedure), and will only need to be reported in the first data-call (2016),
and then updated when changed.

A ClassificationSystem vocabulary has been created with all the methods to be included in
the forecoming Intercalibration Decision
The following changes refer exclusively to TCM elements:

Salinity (which was an attribute in tables NutrientsSeawater_Disagg and
HazSubstSeawater_Disagg) and OrganicCarbon (which was an attribute in table
5
HazSubstSediment_Disagg), are no longer included as attributes, but are expected to be
reported as Determinands (both included in the revised Determinands codelist).

Measurements in sediments are expected in Dry weight, and measurements in Biota are
expected in Wet weight. Therefore DryWetRatio is no longer required. PercentMoisture has
been added, to allow the conversion of the wet weight to dry weight.

For some of the elements, codelists developed within the freshwaters revision have been
implemented (observedPropertyDeterminandCode, resultUom, resultObservationStatus)

For some of the elements, codelists have been developed (procedureAnalysedMatrix, that
integrates Tissues and sediment fraction) or codes have been revised (parameterSpecies)

Some elements have been deleted, because they were very poorly filled (basis, grain type,
integrated depth, thermocline depth, halocline depth, method_Chla, sampler)

Some elements have been combined into one (sediment top and sediment bottom) or are
now requested under Remarks (PlatformCode_SHIPC)
So the WISE-SoE marine data will be reported from now on under 4 different dataflows (Table 2):
WISE-1 (riverine inputs and direct discharges), WISE-2 (ecological quality ratios per monitoring site
and per water body, and methods used), WISE-5 (monitoring stations on water quality, biology and
flux stations for the estimation of riverine inputs) and WISE-6 (water quality in water, sediment and
biota). Note here the coordination needs of the Marine NRCs with the Freshwater NRCs on emissions
regarding the riverine inputs and direct discharges reporting.
Table 2 WISE SoE dataflows collecting transitional, coastal and marine data
WISE-TCM
WISE-2
Until
2015
Stations (incl. flux
stations),
nutrients in
seawater,
hazardous
substances in
seawater,
sediment and
biota, riverine
inputs, direct
discharges,
pressures in
catchements
From
2017
N/A
WISE-1
WISE-5
WISE-6
Stations, EQRs per N/A
monitoring
station, EQRs per
water body,
methods
N/A
N/A
EQRs per
monitoring
station, EQRs per
water body,
methods
Stations (named
MonitoringSites)
incl. WISE-6,
WISE-2 and Flux
stations (WISE-1)
Nutrients in
seawater,
hazardous
substances in
seawater,
sediment and
biota
Riverine inputs,
direct discharges
Note: N/A: not applicable
The revised tables and elements of the data-models are described in the Data Dictionary:
6
Table 3 Links to WISE SoE TCM revised dataflows definitions
WISE-1: http://dd.eionet.europa.eu/datasets/3180
WISE-6: http://dd.eionet.europa.eu/datasets/3363
WISE-2: http://dd.eionet.europa.eu/datasets/3306
WISE-5: http://cdr.eionet.europa.eu/help/WISE_SoE/wise5
Again, it should be noted that for countries opting to report via Helcom and OSPAR, these changes
will have no effect. The data reported to the RSCs will be transformed by ICES into a structure that
matches the EEA data model. The ICES DOME database will not be influenced by these changes.
5.2
Reporting monitoring sites under WISE-5
The old WISE SoE TCM data flow operated with 3 different types of stations: StationsTCM (for water
quality measurements), flux stations (for riverine discharges estimations) and BiologyStations (for
WFD biology measurements in transitional and coastal waters). In the revised model, monitoring
sites will be reported under the WISE Spatial data flow (WISE-5), following the MonitoringSite
schema, which has been developed for WFD reporting, and adapted for the WISE SoE reporting of
both freshwaters and marine stations.
The schema and GIS guidance to be used for the reporting of monitoring sites under WISE-5 can be
found in the WISE-5 help page (link in Table 3).
The legacy stations have been transformed to the new monitoring site data-model and will be used
to create the registry of monitoring sites to be used for QC, in which also the monitoring sites
reported under the WFD (in 2010 and 2016) will be included. From the registry, an extraction of the
Eionet legacy stations will be done and shapefiles will be made available to countries in Forum, in
order to check if the information is correct. Otherwise, they would have to generate an updated
shapefile and upload the correspondent GML to CDR (conversion tool available in WISE-5 help page).
For the reporting of the feature of interest within the monitoring site model, the option
‘euMarineRegionCode’ has been added, which in the case of countries reporting under WFD, should
only be used where the station falls beyond the territorial waters. The codes to be used within such
scheme are those from the Marine regions and subregions vocabulary:
http://dd.eionet.europa.eu/vocabulary/msfd/regions/view. For the Atlantic and the Mediterranean
regions, the subregion code is preferred, whenever possible.
For any spatial analysis using the marine regions and subregions, the layers of the marine regions and
subregions listed in the MSFD are available at: https://www.eea.europa.eu/data-andmaps/data/msfd-regions-and-subregions.
For those using the option to report via OSPAR or Helcom, there will be no changes. Instead the data
reported to the Conventions will be transformed by the ICES data center into a structure that
matches the EEA data model. The ICES DOME database will not be influenced by these changes, nor
the existing ICES reporting formats.
7
5.3
Reporting classification systems under WISE-2
The classification procedure information under WISE-2 has to be reported only once (2017), and
updated whenever necessary in successive calls.
A vocabulary of classification systems has been developed for WISE-2 data, including all those
methods that are to be published in Annexes I and II of the forecoming Intercalibration Decision.
Coordination with the JRC has taken place so that every classification system is included.
Whenever the classification system used is not in the vocabulary, it has to be reported under the field
Remarks, which will serve to update the vocabulary in regards of future calls.
5.4
Reporting riverine inputs and direct discharges under WISE-1 (Emissions)
The revised riverine inputs and direct discharges tables now include some common elements with
the Emissions table in WISE-1, such as ‘Emissions unit of measure’ (which is a vocabulary), ‘Emissions
value’ or ‘Observation status flag’. Regarding the former ‘Method of Estimation’, it has been renamed
to ‘Details of discharge estimate’.
In the case of Direct Discharges, the ‘Discharge Type’ has been replaced by ‘Source of point emissions
or of diffuse emissions ‘, which includes the codes of the former (among many others): total
industrial discharges (I), total urban discharges (U) and total point sources discharges (PT).
5.5
Changes in codelists
The revision of the data-model has included the necessary changes in the Data Dictionary and the
QA/QC rules to be applied. Within these changes, there are relevant ones related to the codelist of
determinands, as well as the spatial reference data (mainly how to report spatial identifiers).
With regard to the determinands, the common codelist developed for the freshwaters will be used.
This was an outcome of the content related review1. The WISE SoE codelist has been harmonised
with the WFD code list.
The common codelist includes nutrients, hazardous substances, and supportive determinands.
Whenever available, CAS (Chemical Abstract Service) registry numbers are used as identifiers. If a
determinant does not have a CAS number (e.g. total nitrogen) then an EEA-generated code is
provided.
A separate codelist is kept for biological determinands. However, currently, it only includes the
biological quality elements (BQEs) that are reported using ecological quality ratios (EQR).
Other codelists were also reviewed, e.g. the Units of Measure codelist which now follows the Unified
Code for Units of Measure (UCUM) in order to facilitate the data exchange and avoid ambiguous
representations of the units, while keeping human readability. UCUM is also used as the
recommended standard representation for units of measure for Observations and Measurements
(O&M) in OGC standards and INSPIRE specifications, which should facilitate its adoption also at
national level.
1
See document “Content related SoE review- Maintenance and content development of data flows
(SoE and WFD) and its annexes”, presented in and finalised after the EIONET Freshwater Workshop 2015:
http://forum.eionet.europa.eu/nrc-eionet-freshwater/library/eionet-workshops/copenhagen-freshwatereionet-workshop-2015/copenhagen-freshwater-eionet-workshop-2015/background-documents/soe-contentreview-document-and-annex-20151115_final-after-country-consultation
8
5.6
Streamlining of the data processing and quality control
An additional goal to the review process is to improve the data processing at the EEA and reduce the
time and effort required to publish updated European data sets and indicators.
This goal requires the automation of various steps of the data processing phase, which is only
feasible if the data being delivered conforms to the data collection model.
The quality control procedures have thus undergone a technical review. Extended QC procedures
have been implemented at Reportnet level, so that Data Providers can receive better feedback upon
delivery of an envelope.
The QC procedures at Reportnet level screens for different types of situations:
BLOCKER
A critical error. The envelope can not be released. Normally, a blocker is an error in the format of the
file, or in the structure or content of the data. Such a critical error makes it impossible for the
delivery to be harvested and integrated into the European database. The envelope can only be
released if every incorrect file is removed and replaced by corrected files.
ERROR
A non-critical error. The envelope can be released, but part of its content may be excluded from the
European database (or be marked as having low reliability). Data Reporters are strongly advised to
correct the non-critical errors. If the automated QC returned errors, a clarification or a resubmission
may be requested by the Data Client, when the data is processed and the final feedback is added to
the envelope. The delivery scoring evaluation may be reduced if errors are present.
WARNING
An issue that may be an error. Data Reporters are advised to check the correctness of the records or
values that raised the warning. The envelope can be released. If the automated QC returned
warnings, a clarification may be requested by the Data Client, when the data is processed and the
final feedback is added to the envelope.
INFO
Other issues related to the quality of the data. The envelope can be released. A clarification may be
requested by the Data Client, when the data is processed and the final feedback is added to the
envelope.
OK
The automatic QC did not detect quality issues. The envelope can be released.
6
Reporting permissions (reporter roles)
The lists of allowed reporters for the dataflows are:
WISE-2: http://www.eionet.europa.eu/ldap-roles/?role_id=reportnet-awp-wise2-reporter
WISE-6: http://www.eionet.europa.eu/ldap-roles/?role_id=reportnet-awp-wise6-reporter
Whenever an update of the lists is needed, please contact [email protected]
9
7
Providing support to data reporters
Support requests can be sent by email to [email protected] and will be registered
in a ticketing system that allows an efficient and transparent treatment of all requests.
The system can also be accessed via https://helpdesk.EIONET.europa.eu/otrs/customer.pl using the
EIONET credentials.
Please note that emails sent directly to data managers will not be registered or processed, so only the
above email must be used. It must also be emphasized that any corrections to the reported data must
be delivered via Reportnet, that they can be processed into the European data sets. Data sent via email
or to the helpdesk system cannot be processed, and will ultimately not be incorporated in the
databases.
Help pages are also available in CDR for all WISE SoE dataflows in the following link:
http://cdr.eionet.europa.eu/help/WISE_SoE
10
Annexes
ANNEX I. Results of the Eionet consultation
Microsoft Excel
Worksheet
ANNEX II. Summary of main changes in the tables
a) Water quality
Table
Element name under the old model (WISE-TCM)
Change
Element name under the revised model (WISE-6)
All
CountryCode
deleted
All
NationalStationID
change of definition
All
Year-month-day-time
phenomenonTimeSamplingDate
All
SampleID
sampleIdentifier
All
Determinand_Nutrients/DeterminandHazSubsName
vocabulary created
(=freshwaters)
All HS
CASNumber
deleted
All
Unit_NutrientsSeawater/Unit_HazSubs
vocabulary created
(=freshwaters)
All
LOD_LOQ_Flag
resultQualityObservedValueBelowLOQ
All
Concentration
resultObservedValue
Nutrients and
HS in seawater
SampleDepth
parameterSampleDepth
Observations
Included in monitoringSiteIdentifier
monitoringSiteIdentifier
observedPropertyDeterminandCode
Included in
observedPropertyDeterminandCode
resultUom
11
Table
Element name under the old model (WISE-TCM)
Change
Nutrients
seawater
IntegratedDepth
deleted
Filled in only 0,2 % of times
Nutrients and
HS in seawater
ThermoPos
deleted
Filled in only 7 % of times
Nutrients and
HS in seawater
HaloPos
deleted
Filled in only 7 % of times
Nutrients and
HS in seawater
Salinity
deleted
To be reported as Determinand (it
has been included in the revised
Determinands codelist)
PlatformCode_SHIPC
deleted
To be reported under Remarks
Method_Chl-a
deleted
Filled in only 8 % of times
HS in sediment
Sampler
deleted
Not used for indicators
HS in sediment
BottomDepth
deleted
To be provided under the
MonitoringSite data-model
HS in sediment
SedimentTop
HS in sediment
SedimentBottom
HS in sediment
OrganicCarbon
deleted
HS in sediment
and biota
DryWetRatio
deleted
HS in sediment
and biota
Basis
deleted
HS in sediment
GrainType
deleted
HS in sediment
Fraction
included in a new
vocabulary
Nutrients
seawater
Nutrients
seawater
Element name under the revised model (WISE-6)
combined with
SedimentBottom
combined with
SedimentTop
Observations
parameterSedimentDepthSampled
parameterSedimentDepthSampled
to be reported as Determinand
(included in the revised
Determinands codelist)
measurements in sediments are
expected in Dry weight, while
measurements in Biota are
expected in Wet weight
Measurements in sediments are
expected in Dry weight, and
measurements in Biota are
expected in Wet weight.
Filled in only 4 % of times
procedureAnalysedMatrix
12
Table
Element name under the old model (WISE-TCM)
Change
Element name under the revised model (WISE-6)
HS in biota
Species
new coding
parameterSpecies
HS in biota
Tissues
included in a new
vocabulary
procedureAnalysedMatrix
HS in biota
Fat
parameterFat
HS in biota
Exlip
parameterExtractableLipid
HS in biota
Lipid
parameterLipid
new field
parameterMoisture
new field
monitoringSiteIdentifierScheme
new field
parameterWaterBodyCategory
new field
procedureLOQValue
new field
resultObservationStatus
Observations
GBIF code to be used (to
accomodate freshwaters species in
the next future)
b) Riverine inputs and direct discharges
Table
Element name under the old model (WISE-TCM)
Change
Element name under the revised model (WISE-1)
Observations
DD & RI
Country Code
deleted
DD
Water Body ID
change of definition
DD
Water Body Name
deleted
Not needed for analysis
DD
Sea Area Name
deleted
Requested for the monitoring sites
(featureOfInterestIdentifier to be
Included in monitoringSiteIdentifier
Water body identifier
13
Table
Element name under the old model (WISE-TCM)
Change
Element name under the revised model (WISE-1)
Observations
filled in with marine region or
subregion code)
DD
Regional Sea Name
deleted
Similar as Sea Area Name
DD
Marine Convention Area
deleted
Similar as Sea Area Name
DD & RI
Year
DD
Discharge Type
changed to
vocabulary on
emissions sources
DD & RI
Name of Input or Discharge Determinand
vocabulary created
Determinand code
for water quality to be
used (=freshwaters)
DD & RI
Chemical Abstract Service Number
deleted
DD & RI
Unit of Measurement of Input or Discharge
vocabulary created
(=freshwaters)
DD & RI
Estimate
deleted
DD & RI
Method of Estimation
renamed
Details of discharge estimate
DD & RI
Estimated Load
renamed
Emissions value
DD & RI
Remark
DD
Reference period (year)
Source of point emissions or of diffuse emissions
Included in
observedPropertyDeterminandCode
Emissions unit of measure
Average estimate is requested from
now on
Remarks
new
Water body identifier scheme
14
Table
Element name under the old model (WISE-TCM)
DD
RI
RI
National Station ID
Change
Element name under the revised model (WISE-1)
new
Observation status flag
change of definition
Monitoring site identifier
new
Monitoring site identifier scheme
15
Observations