Data Quality Strategy and Action Plan 2008/09
DATA QUALITY STRATEGY 2008/09
1.
INTRODUCTION
1.1
Consistent, high-quality, timely and comprehensive information supports good
decision-making and improved service outcomes. Performance information is
increasingly being used by external bodies to assess our performance, often as an
alternative to inspection, and this trend is expected to continue. The new
Comprehensive Area Assessment will be accompanied by an external audit
approach moving away from checking calculations and systems reports towards a
more challenging scrutiny of systems controls.
1.2
The Audit Commission has carried out two annual reviews of Westminster’s data
quality arrangements and has found that the council’s management arrangements
and application of good data quality standards are sound.
1.3
This Strategy outlines the standards required throughout the data collection
process, from setting up systems to collect and report performance information to
the eventual publication of a robust set of performance data which is accurate and
fit for external scrutiny.
1.4
The strategy also sets out the key principles for maintaining and improving data
quality across the Council for key performance information:
 Clear roles and responsibilities – arrangements are in place at all levels of
the organisation to ensure good quality data is used to manage and report on
performance
 Clear definitions - relevant staff know which PIs are produced from the
information they input and how they are defined
 Good systems and processes – to implement and maintain effective systems
and processes to secure data quality
 Effective data use – ensuring that data supporting performance information is
used to manage and improve the delivery of services.
1.5
The strategy is accompanied by an Action Plan to address some areas of
improvement, particularly in relation to a small number of performance indicators.
The Strategy will be updated annually and the Action Plan monitored six monthly.
This Strategy is accessible to all staff via the Council’s intranet and communicated
to partners and contractors through Data Quality champions (Business Planners). A
Data Quality Seminar will be held annually with the Data Quality Champions.
1.6
For the purposes of this strategy ‘data’ covers key performance information used to
support monthly and annual performance monitoring and statutory returns to
external bodies. This includes Local Area Agreement/Best Value Performance
Indicator/National Indicator Set data, financial information and other key
performance indicators that feed the corporate performance monitoring process.
With the move towards a new performance framework in line with CAA
requirements, we will apply these data quality standards to local indicators also.
-1-
Data Quality Strategy and Action Plan 2008/09
1.7
2.
2.1
The council is currently investigating the streamlining of management/performance
storage and collection through a centralised management information system and it
is proposed that over time standard data quality principles will be applied to all data
contained with the new system.
CLEAR ROLES AND RESPONSIBILITIES
There is collective responsibility for data quality, but it is necessary to be clear about
what actions and responsibilities are allocated to specific individuals and teams in
order to implement this strategy. Data quality responsibility at all levels of the
organisation is highlighted in the chart below.
Internal Data Quality Responsibilities
Fran Beasley
Director Policy and Performance
Corporate Lead
Corporate Management Board
Corporate champion
Departmental Lead
Performance Improvement Team
Departmental Service Heads
Departmental DQ Champions
Departmental Business Planners
/ Business planners
Departmental data
officers
Individual Data Providers
Cllr MelvynCaplan
Chairman Audit and Performance Committee
Internal audit – Support function
Officer/ Member
Champion
Cabinet Members
All Staff
Senior Manager and Member roles
2.2
The Council uses data appropriately to inform service planning, review and
development. It has long-established processes in place for using performance and
data information to support service developments. These include data and
performance review at local service level through to Chief Officers and Members.
There are processes in place for reviewing the accuracy and quality of data
submitted prior to review.
2.3
Senior officer responsibility is led by the Chief Executive and the Director of Policy
and Performance has responsibility for monthly performance monitoring and
meeting best value external data requirements. The Chair of the Audit and
Performance Committee takes the lead on the Member side. The Corporate
Management Board receive monthly Performance Monitoring Reports and sign off
relevant statutory performance documents which ensures a robust approach and
challenge exists to drive data quality as well as encouraging a culture of good
quality data. There is also an element of challenge within the reporting regime,
which involves the Chief Executive and Leader asking performance questions of
Cabinet Members and Chief Officers. The answers to these questions are fed into
the next months reporting cycle, creating a cycle of ongoing performance monitoring
and improvement.
-2-
Data Quality Strategy and Action Plan 2008/09
Service Heads
2.4
Service Heads are responsible for reviewing and authorising monthly, quarterly and
annual performance returns.
Business Planners
2.5
The Council’s Business Planners are also dedicated departmental data quality
champions who promote data quality policies and report on compliance whilst also
ensuring that data quality issues are fully integrated into the Councils Business
Plans.
2.6
Data Quality champions are required to ensure that:
 users are adequately trained
 there is security of access/amendment
 periodic tests of the integrity of data are undertaken
 information management and support is available to users
 system upgrades are made where necessary (including to accommodate
amendments to PI definitions)
 the system meets managers’ information needs
 the system can produce adequate audit trails
 actions recommended by the external auditors are implemented
 a set of written procedures exists for the purpose of extracting performance
information within the system form
 a business continuity plan for the system exists to protect vital records and data.
Corporate Performance Management Team
2.7
This Team:
 monitors council wide performance
 encourage accurate and prompt collection and publication of data
 promote the value of performance measurement
 ensure that data quality is fully integrated into the council’s planning, monitoring
and reporting processes
 ensure that local indicators are relevant to the measurement of and reporting of
outcomes expected
 ensure that data quality is fully integrated into the council’s planning, monitoring
and reporting processes
 may instruct internal audit to investigate data quality issues
 advises departments and internal audit of new and amended performance
indicators so that data quality processes can be set up/amended
 check and chase up data returns and ensure that appropriate explanations are
provided for performance exceptions
Data Collection Staff
2.8
Data quality is the responsibility of every member of staff entering, extracting or
analysing data from any of the Council’s information systems. All officers should
know how their day-to-day job contributes to the calculation of performance
indicators and how lapses can either lead to errors or delays in reporting, both of
which limit our ability to manage performance effectively. At a departmental level
-3-
Data Quality Strategy and Action Plan 2008/09
roles and responsibilities are clearly defined and documents and bespoke training
provided to ensure staff have the necessary skills in relation to data quality.
2.9
The development needs of staff members with responsibility for data quality are
reviewed alongside other staff development needs alongside the staff appraisal
process. Bespoke data quality training is provided as required for staff with data
quality responsibilities. Training will be provided at departmental level and
monitored by managers.
2.10 Responsibility for data quality and authorisation of performance indicators is clearly
defined in job descriptions. Departments are encouraged to ensure that suitable
appraisal targets and paragraphs in job descriptions are included, given the level of
involvement staff have in the PI process. Data quality standards are also extended
to partners and contractors as appropriate. In particular there are requirements for
partners to provide timely and accurate performance information and clear
responsibilities for data quality.
Internal Audit
2.11
The Head of Audit and Risk ensures that an element of data quality is routinely
included in normal service audits. Internal Audit carry out an annual advisory data
quality review, reported to the Corporate Management Board.
2.12
The review is risk based and investigates indicators based on the following criteria:
 new indicators
 indicators more than 15% above or below target
 indicators more than 15% above or below the previous years’ performance
 previous issues in calculation or complexity
 indicator identified by the Audit Commission as high importance
2.13 The confidence in the internal data quality review has led to a decreasing amount of
time spent on PI data quality checks by external auditors.
3.
CLEAR DEFINITIONS
3.1
As outlined above, all officers know how their day-to-day job contributes to the
calculation of PIs and how lapses can either lead to errors or delay reporting, both
of which limit our ability to manage performance effectively.
3.2
This means that everyone has an understanding of any PIs affected by the data
they contribute. A basic grasp might be, for example, knowledge of what the
numerator and denominator is, and where there are any important technical
guidelines (for example, the exclusion of certain cases).
3.3
CPA indicators and National Indicators have nationally set definitions. It is
important that every detail of the definition is applied. This ensures that data is
recorded consistently, allowing for comparison over time, and national
benchmarking.
3.4
Where we are setting local PIs we ensure that we have established a clear
definition, and that there are systems available to collect and report the data in an
-4-
Data Quality Strategy and Action Plan 2008/09
agreed format. In particular, we are clear about target and outturn figures and
ensure they are SMART (Specific, Measurable, Achievable, Reliable and Timely)
3.5
Every PI has a named officer who is responsible for collecting and reporting the
information. This ensures that there is consistency in the application of definitions
and use of systems for providing the data. Each named officer will be kept up to
date of any changes in definition that may occur from time to time.
3.6
Particular attention needs to be paid to data provided by external sources. A
number of PIs are calculated using information provided by other agencies and the
Council’s intention must be to work alongside them to ensure that such data is
accurate.
3.7
When entering into Contracts or Service Level Agreements with other service
providers it is essential that, wherever relevant, there is a requirement to provide
timely and accurate performance information. And that there is a clear, commonly
shared understanding of their responsibilities for data quality and how we will be
checking the information they provide.
3.8
It might not always be possible to alter existing agreements so that external service
providers are fully committed to providing an agreed quality of performance data. In
this case, the data must be treated as high-risk and thought must be given to
establishing a system of checks and measures to ensure that we are confident
about the accuracy of this data. When carrying out checks on such information it is
important that this is documented and signed off by the relevant officer.
3.9
Some important performance information – for example, crime statistics – is
provided directly to the Council by the Police. In the event of concerns about the
integrity of externally-provided information, the Council’s intention is to work with
other agencies constructively wherever possible to provide assurance and rectify
any problems identified. A system of checks and sampling should be established to
ensure that we are confident about the accuracy of the information.
3.10 Responsibility for data verification will lie within services, but the Performance
Management Team can offer advice and guidance about verification procedures.
4.
GOOD SYSTEMS AND PROCESSES
4.1
The Council has established systems for the reporting of performance and data.
Performance information is used routinely to inform service developments.
Reporting to chief and senior officers and Members includes exception reporting
across all services.
4.2
There are established processes for ensuring the accuracy and reliability of data.
Guidance (via System Forms and Working Paper summaries) is issued annually to
departments on procedures necessary to ensure that high quality external and
statutory audit submissions are made. These will be further consolidated in a series
of workshops on the subject, to be repeated as necessary to ensure that the
Council continually improves data quality standards.
-5-
Data Quality Strategy and Action Plan 2008/09
4.3
When information is presented for external audit, the lead officer and Service
Manager must review Working Papers to confirm that the definition has been
followed, the calculations are correct and the indicator is supported by a full audit
trail.
4.4
A completed System Form must be accompanied by full documentation and a clear
audit trail must be compiled for any PI presented for external audit. This needs to
include:
 a calculation, based on indicator definition and further guidance, crossreferenced to supporting information
 system notes
 explanation of any variance from the previous year
 documentation supporting any allowable estimates, sampling, or any
apportionments made
 supporting information (e.g. spreadsheet, database etc) or at least a full
description of where the supporting information is kept
4.5
The Council has procedures in place which show clear audit trails are evidenced.
Risks are assessed and incorporate mitigating actions, which are followed up, to
ensure implementation. All data is subject to senior approval prior to external
reporting as outlined in the flow chart below.
6
Data Use
-6-
Data Quality Strategy and Action Plan 2008/09
5.
EFFECTIVE DATA USE
5.1
A review of current performance should be a standing item at all team meetings.
Individual contribution to performance indicators will form part of the staff appraisal
process.
5.2
The latest service performance information should be displayed where appropriate
within the department on notice boards and accessible to all staff.
5.3
Actions to improve poor performance are included in annual Business Plans and
monthly performance reports. This includes both actions to improve performance to
meet agreed targets and corrective actions to address performance exceptions
identified by data returns. These actions are included in regular business plan
monitoring and will be reported to members.
5.4
To ensure that issues can be addressed at the earliest opportunity, service heads
will endeavour to ensure to monitor performance at more frequent intervals than
demanded by the performance monitoring timetable. Where required, supporting
performance indicators will be developed to ‘drill down’ the issue into underlying
causes (e.g. by area, team or individual).
For more information
Contact: Una McCarthy or Nick Lowe, Corporate Performance Team
Visit the wire: http://wirecms/policycomms/Performance/performance/dataquality/
-7-
Data Quality Strategy and Action Plan 2008/09
ACTION PLAN 2007/08
Actions
The data quality strategy 2007/08 should be adopted by the CMB as soon
as possible.
The Council's action plan should be regularly monitored to ensure that data
quality objectives are achieved.
Formal training on data quality requirements to be provided to departmental
data quality champions, key staff and the Chair of the Audit and
Performance Committee. (Via annual business planning seminar)
Produce periodic updates to proactively inform staff of the results of their
efforts in ensuring data quality (performance matters bulleting, wire site and
monthly performance reporting)
The Council should undertake a 100% check of pedestrian crossing sites to
ensure that BV165 calculations are correct.
Checks should be made to ensure that the spreadsheet used to calculate
BV212 agrees to the raw data taken from the housing management system
BV215a should be compiled using the BVPP guidance.
To ensure that BV215 is fairly stated, the Council should ensure that third
party contractors are subject to the same controls when recording lighting
work. This includes using correctly the confirm database to record all
repairs.
-8-
Responsible
officer/
department
Corporate
Performance
Team
Corporate
Performance
Team
Corporate
Performance
Team
Corporate
Performance
Team
Martin Low
When
Rosemary
Westbrook
Martin Low
March 2008
Martin Low
March 2008
Comments
February 2008
March 2008
September
2008
October 2007
13 March 2008
Ongoing
Ongoing
March 2008
Completed