A Methodology Guide
United Kingdom collaborative clinical audit of
health care for children and young people
with suspected epileptic seizures
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
1
Introduction and Ethos
This guide aims to outline the chosen methodology and process for Epilepsy12. This guide
should you understand how the project will work and some of the rationale behind the
methodology.
The audit ethos is:
 to facilitate and resource local audit
 to be both practical and meaningful
 to consider results for services as a whole rather than individuals
 to enable local epilepsy services and networks to ‘own their own data’ and prioritise ‘their
own strategies’ to evolve and improve
 to provide data regarding quality of care against national recommendations with the
ability to compare results to other cohorts in a standardised way.
 that league tables are not a meaningful way to represent the audit’s results
 to strengthen roles and relationships within existing epilepsy networks
 to learn from the audit process such that the audit strategy evolves over time
What are the aims of the audit?
1. To facilitate health providers and commissioners to measure and improve quality of care
for children and young people with seizures and epilepsies.
2. To contribute to the continuing improvement of outcomes for those children, young
people and their families.
Which services in the UK are eligible to participate?
The audit will examine care across the whole UK including England, Northern Ireland,
Scotland and Wales. All paediatric services that employ NHS paediatricians that request
EEGs and are involved with the care of children and young people with seizures or
epilepsy are invited to participate. The audit will be conducted principally around hospital
and community services functioning at a secondary tier level. For hospitals with tertiary
roles it will be any secondary care aspects within their service that will be the main focus of
the audit.
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
2
What is an ‘audit unit’?
The UK is divided into defined 200 ‘audit units’ which are based on existing clinical
networks, affiliations and pathways of care that form 20 “audit regions” across the UK.
Each audit unit will have a defined ‘link Paediatrician’ and component paediatric services.
Most audit units will comprise groupings of both hospital and community paediatric
services. Each audit unit is linked with one or more tertiary paediatric services with whom it
has developed pathways of care. Each tertiary service has a defined lead or ‘link paediatric
neurologist’.
When will the audit begin?
The audit is being undertaken over a 3 year period which began in 2009 and comprises
several phases. The start date for 21 early adopters audit units is January 2011 using the
1st February as the census date. The start and census date for the rest of the UK is 1st May
2011.
What is census day?
This is the key point in time defined by the project team which determines when an audit
unit can become ‘live’ via the web tool and begin to enter data. It is the date which is
considered by the audit unit team when describing attributes of the audit unit’s services. It
also determines the various dates which identify the target cohort for the audit. All children
and young people included in the audit will have presented to paediatric service at least 12
months prior to census day. The census day for ‘early adopter’ audit units will be 1st
February 2011. The census day for the rest of the UK will be 1st May 2011.
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
3
What is being measured?
The audit will allow the audit unit to measure quality of care in three separate domains.
1. Service Descriptors
2. Clinical audit and Performance Indicators
3. User Experience
1. Service Descriptors
Each audit unit lead will complete a service descriptor questionnaire via the web tool to
describe the epilepsy service within their audit unit as it was on census day.
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
4
2. Clinical Audit and Performance Indicators
The web tool will prompt the clinical audit process which may be undertaken by a number
of professionals from within the audit unit.
a)
The audit unit will generate a letter to the relevant EEG department asking for their
help in identifying all children and young people referred from their services for an
EEG within the defined 12 month time period.
b)
The audit unit will filter these patients by application of inclusion and exclusion criteria
to establish the target cohort.
c)
Case notes for the target cohort will be analysed by the audit unit and relevant
questions answered for each patient.
Inclusion criteria:

First EEG during defined 12 month period.

The child has a ‘first paediatric assessment’ for the ‘paroxysmal episode or episodes’
during defined 6 month time period.

Child is older than 1 month and younger than 16 years at ‘first paediatric
assessment’.

The EEG was prompted by the patient having one or more afebrile paroxysmal
episodes.
Exclusion criteria:

All ‘paroxysmal episodes’ in question were diagnosed as ‘febrile seizures’.

The patient has had a paediatric assessment previously for similar episode or
episodes or epilepsy prior to first paediatric assessment

All the paroxysmal episodes that the patient had were acute symptomatic seizures or
occurred within a week of a traumatic head injury

The patient’s care was permanently transferred to a secondary paediatric service
outside the ‘audit unit’ boundaries or an adult service during the year after first
paediatric assessment
3. User Experience
Audit units will be supported such that an anonymous user experience survey is
undertaken:
a) A cohort will be identified for each audit unit. This will be all children and young people
commenced on regular anti epilepsy drugs and not deceased.
b) A paper questionnaire and cover letter will be sent to each family by the audit unit
team.
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
5
c) Anonymous questionnaires will be returned in pre-paid envelope to the RCPCH project
team or questionnaire answered online.
Who will carry out the audit?
The audit will be led at local level by the audit unit ‘link Paediatrician’. For those taking part
it should feel like a typical departmental audit except that the audit is supported by a web
tool and central RCPCH team. The audit is constructed such that the audit unit lead can
choose to nominate other professionals to take part. For example, the audit department
may help with ascertainment, junior doctors may help with case note review and a
departmental secretary may help with the user experience questionnaire. It is likely that
different units will achieve the audit in different ways depending on local arrangements.
The audit will be supported centrally by the RCPCH Epilepsy12 project team throughout
the three year project.
How will the web tool work?
The web tool has been designed specifically to support the Epilepsy12 audit project.
Defined health professionals within each audit unit will be given individual access rights to
their audit unit within the web tool. The web tool will support many aspects of the audit and
the majority of data collection will be done through online forms. Help will be available to
support this process.
Will patient identifiable information be collected?
The audit has exemption from section 251 of the NHS Act 2006 (formally section 60) which
has been granted by the National Information Governance Board (formerly PIAG). Users
will be asked to submit the child’s NHS number and date of birth into the web tool. These
will be encrypted upon entry.
Where will the data be stored and how will it be secured?
Data will be stored and held securely in compliance with Data Protection and Caldicott
Guardian principles. Only users registered with audit will be able to enter data and will only
be able to so through the use of a unique username and password. Further data security
policies are available from the project team.
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
6
What are the 12 performance indicators?
This is the term given to defined quality parameters determined from the clinical
questionnaire. Each performance indicator (PI) correlates with specific SIGN and NICE
recommendations and has a defined method of calculation. We are encouraging services
to examine their own scores alongside other provider’s scores. It may not be optimal for a
service to score 100% as patients and circumstances differ and not all children fit with all
models of care. Target standards will not be set in this audit cycle.
There are 12 performance indicators:
1.
% with evidence of input by Paediatrician with expertise in Epilepsy by 1 year.
2.
% with evidence of input by, or referral to, an Epilepsy Specialist Nurse by 1 year.
3.
% diagnosed epileptic seizures meeting referral criteria with evidence of referral to or
discussion with a Paediatric Neurologist by 1 year.
4.
% with evidence of appropriate clinical assessment at first paediatric assessment.
5.
% with a diagnosis of epilepsy maintained by 1 year.
6.
% diagnosed as epilepsy with evidence of seizure classification by 1 year.
7.
% diagnosed as epilepsy with evidence of epilepsy classification by 1 year.
8.
% having ‘appropriate’ EEG.
9.
% with epilepsy and indications for neuroimaging having neuroimaging by 1 year
10. % with convulsive seizures having 12 lead ECG by 1 year.
11. % females >12 years old commenced on epilepsy medication with evidence of
discussion regarding pregnancy or contraception related issues.
12. %
commenced
on
carbamazepine
with
absence
of
contraindications
for
carbamazepine.
How were the performance indicators chosen?
The Epilepsy12 team selected meaningful measures that can be mapped to NICE and
SIGN with a clear rationale. These act as broad indicators of good practice and can be
reasonably defined and achieved by the methodology of retrospective case note analysis.
How is misdiagnosis being considered?
The Epilepsy12 team has considered how to obtain a measure of misdiagnosis using a
retrospective case note methodology. The chosen approach is different from actual
misdiagnosis rates. To measure misdiagnosis in a gold standard way would need
independent clinical review for each child. This is impractical and not appropriate for what
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
7
is intended. The Epilepsy12 national audit will explore "whether there was any documented
evidence of withdrawal of diagnosis of epilepsy by 12 months after first paediatric
assessment".
Will there be ‘league tables’?
‘League tables’ will be considered as an inappropriate and non-meaningful approach to
displaying results.
Will audit unit’s results be published?
In the long term all audit unit’s data will be available within the public domain.
What will happen if an audit unit has difficulties
completing the audit?
It is hoped that all paediatric services within the UK will participate in the audit. The project
team will provide support to audit units in order to help them undertake the three domains.
It is acknowledged that not all audit units may achieve participation and complete all three
audit domains. Degree of participation and data completeness will be reported for all
participating and non-participating audit units.
Is the audit measuring clinical outcomes?
Epilepsy outcomes are difficult to measure objectively within the current health service
structures and methodology of the Epilepsy12 audit. Meaningful outcomes measurements
are particularly difficult in epilepsies as they are a heterogeneous group of long term
conditions with a wide and varied impact beyond just their seizure component. Clinical
outcomes based on ‘syndrome specific seizure freedom rates’ are being piloted within the
clinical domain of this audit.
Where can I find out more?
The RCPCH project team can be contacted throughout the project:
Rita Ranmal, Project Manager
Royal College of Paediatrics and Child Health, 5-11 Theobalds Road, London, WC1X 8SH
Tel: 020 7092 6167.
Email: [email protected]
Go to www.rcpch.ac.uk/epilepsy12 for ongoing further information regarding the project.
Royal College of Paediatrics and Child Health
Version 2.1 January 2011
8