Auditing data management system
Bruno Deprez
Data Audit officer
SPC, Oceanic Fisheries Program
What is data management audit?
• Definition: “…a formal, often periodic examination and checking of
data or procedures to verify their correctness…”
• “Audit” term is often used in financial and accounting sectors
– To evaluate organization and people
– To validate the official financial results of an organization before
publication
• Fishing data management audit purpose is different. The final
objective is to improve the quality level of data entered in country
in order to make the studies based on this data easier and better
• To achieve this objective, we need to ensure the correctness and
the coverage of the data, throughout systemic and non-systemic
audit
• This audit is only applied to logsheet data
Objectives of Data Audit
• To ensure that data are standardized
– Almost guaranteed by the use of TUFMAN system to enter data
– Basic checks on the database to make sure it is used properly
•
To improve the quality and coverage of data available at national
level and to the WCPFC
– Data quality audit is applied to position/time, catch and effort
– Coverage audit tries to identify missing logsheets
– Both use extensively the VMS data and samples of paper logsheets
• To improve data collection in country
– Existence of data registration system
– Timeliness and frequency of provision of logsheet data from
companies
– Level of liaison/cooperation with fishing companies to retrieve
logsheets
– Existence of a filing system to easily retrieve hard-copy or scanned
logsheets
The Data Quality Audit Tool
• Stand-alone application to be linked to your Tufman database
• Comes with a set of tests separated in three categories :
– Position/Time
• The tests will verify the accuracy of logsheets times and positions
• E.g. Average speed between sets, Missing sets, Overlapping trips, etc
– Catch
• The tests will verify the accuracy of the catch part of logsheets
• E.g. By Catch
– Effort
• The tests will verify the accuracy of the effort part of logsheets
• Discrepancies in number of hooks
• Generates pre-formatted Excel reports with test status and list of warning
and error database entries
• Comes with an embedded GIS option in order to visualize any trip on the
map
• The list of tests is not static and new ones can be easily added throughout
the audit process
• Some basic automated correction could be also applied on demand
Example : The average speed between sets
Example of a test: Use of limits
The speed limits for a longliner in the tool are set as follows :
– Raise a warning if the speed is above 11 kts
– Raise an error if the speed is above 15 kts
Example : The average speed between sets
Example of a test: The detail part
• 324 knots is obviously wrong
• The error is on the longitude entered for Feb 7th
– 01733E is wrong, it should be something around 173E (this
points ends up in Namibia, Africa)
– Need to get back to the paper logsheet and see what was
the right value
Use of GIS feature on error
• This trip error is obvious
• It is not always the case, and viewing the trip itself can help
finding some answers
• Let’s take a look at this example
Use of GIS feature on error
Use of GIS feature on error
Use of GIS feature on warning
• Previous example was from the error report. Let’s take one
from the warning report (much less obvious)
• The SOLANDER 6 has been steaming at 11-12 knots between
3 positions, it is above the warning threshold
• Here the use of the GIS feature can help to ensure there is a
position error on the logsheet
Use of GIS feature on warning
Use of GIS feature with VMS
• In order to help the decision to modify or not a point, the
VMS reconciliation can be used (when applicable)
• This process is explained in details next on this presentation
• Its main purpose is to compare the logsheets with the trips
generated from VMS and find reconciliations
Use of GIS feature with VMS
Data Collection Audit
• During in-country visit, the following aspect of data
management will be checked
– Data Registration
• E.g any potential hard-copy logsheet going missing prior to
data entry or submission to regional agency
– Hard-copy data management
• E.g does the responsibility matrix for hard-copy data
management cover all work and match actual
responsibilities
– Database System
• E.g Are there any database post data-entry checking
• TUFMAN logsheets will be checked against randomly selected
hard-copy logsheets
In-Country Data Audit Output
• The output of an audit consists in
– The audit conclusion containing the audit results
• For each part of the audit (quality, coverage and data collection),
the detailed status : FAILED, WARNING or SUCCESS
• The global audit status
• The summing up of the actions to be undertaken
– The details of quality audit
• For each quality test (average speed between sets, inland
positions, etc)
• The warnings and errors to be modified in Tufman
– The detail of data collection
• Comments about each logsheet manually checked
• Comments about how to improve data collection
– The detail of logsheet coverages per year, gear and flag
• To be generated in Tufman using the new report
Systemic Data Audit
• Tufman Reconciliation report is analysed during in-country
visit
• The same tool is used internally at SPC to compare the same
VMS data with the logsheets sent by the member countries to
SPC
• Every quarter, an export will be sent to countries to highlight
gaps in the logsheets received at SPC
• Three main reasons for a gap
– Data received in-country from fishing company but not
sent to SPC
– Data not received in-country from fishing company
– Wrong VMS trip