Liberating the NHS: Transparency in Outcomes
Consultation Response from the
Royal College of Paediatrics and Child Health
Principles
1. Do you agree with the key principles which will underpin the development of
the NHS Outcomes Framework (page 10)?
Yes – largely. But, what is missing from this document is sufficient detail on the
implementation of the framework. Stating that the Secretary of State will use the
framework to hold the NHS to account at a national level is straightforward.
However, what matters in practice is how the framework will be used at a more local
level, for instance at the level of GP consortia. Regarding this, the document appears
somewhat confused. On the one hand, it states that “It will not be used as a tool to
performance manage providers of NHS care.” (Section 1.15) At the same time,
though, it also states that “the Department of Health will support the development of
metrics that allow intra-UK comparisons to be made.” (Section 2.23) It remains
unclear, then, how the framework will be used at the level of consortia – if at all. If it
not used at that level, but merely at the national level, then we would suggest it will
make no practical difference to how the NHS operates. It will report practice, but not
influence it. If it is used at a local level, then adjustments for local circumstances /
case-mix will be required. This is a highly complex task, and it is for precisely this
reason that in many aspects of NHS practice, proxy process and structural metrics are
more appropriate (see next point).
2. Are there any other principles which should be considered?
We believe that greater attention should be paid to the importance of proxy process
and structure measures where no suitable outcome measures are available. The
consultation document acknowledges that at a local level these may be appropriate.
However, we would argue that for some services, even at a national level, the most
relevant measure is not an outcome but a structure or process.
Immunisation rates and provision of safeguarding services would be two such
examples – one process, one structural. In neither case, even at the national level, are
there any meaningful outcome measures.
3. How can we ensure that the NHS Outcomes Framework will deliver more
equitable outcomes and contribute to a reduction in health inequalities?
By making measures of equality (e.g. relative difference in mortality across socioeconomic groups) part of the outcome framework to be used at both a national and
local level. We believe that greater attention should have been paid to equality
measures throughout this document.
4. How can we ensure that where outcomes require integrated care across the
NHS, public health and/or social care services, this happens?
By insisting on integrated commissioning for these services, and by developing
pathway tariffs for them. We are concerned that that this framework was not
developed in parallel with the public health outcomes framework and alongside
whatever framework is used to measure the outcomes of social services and children’s
services. For some years, the RCPCH has been calling for a greater degree of
alignment in the goals of regulators, commissioners and providers. There is a
significant risk of continuing such lack of coherence by isolating the NHS
contribution to outcomes in the way this framework does. In particular, many of the
health outcomes of children are necessarily dependant on a range of providers that
cross sectors, and therefore it is essential that robust cross-sectorial outcome measures
are developed for this cohort.
Five domains
5. Do you agree with the five domains that are proposed in Figure 1 (page 14) as
making up the NHS Outcomes Framework?
They are one acceptable way to stratify outcomes, though having said that if we focus
too much on mortality, then given the overall low mortality rates in children, it is
inevitable that children’s services will suffer as a result.
6. Do they appropriately cover the range of healthcare outcomes that the NHS is
responsible for delivering to patients?
One of the unintentional problems of creating these domains is that many children’s
conditions would fit into all five categories. Cystic Fibrosis, for instance, is a long
term condition with acute episodes of illness, where significant safety issues apply,
and which frequently leads to premature death. This framework forces us to create
five different outcome measures for this single condition, and if only one of these is
kept for inclusion, it may not be the most appropriate one. There should have been
some mechanism to enable outcome measures to be defined that cover the whole
pathway of care. We are of course aware that NICE quality standards cover such
pathways, but we are concerned about those conditions that because they are
relatively low volume are unlikely to become of the 150 quality standards. Cystic
Fibrosis would be an example.
Structure
7. Does the proposed structure of the NHS Outcomes Framework under each
domain seem sensible?
There is an admirable logic to the way the framework is constructed. However, we are
concerned that the basket of indicators may not be sufficiently comprehensive. If this
is correct, there is a danger that commissioners will focus on certain high volume,
high cost conditions to the relative disregard of others. Children’s services are likely
to do poorly in such a situation as in terms of cost their health services do not
comprise a sufficiently large amount of expenditure. Such an approach, though, would
be to the detriment of the long term health of our population. Children’s ill-health
stores up a legacy that lasts a lifetime. Therefore, in the long run it makes sense to
invest heavily in children’s health now, even if the benefits will not be seen for some
time.
Domain 1 - Preventing people from dying prematurely
8. Is ‘mortality amenable to healthcare’ an appropriate overarching outcome
indicator to use for this domain? Are there any others that should be
considered?
The measure as it stands takes no account of years of life lost, and some adjustment to
this should be considered.
9. Do you think the method proposed at paras 3.7-3.9 (page 20) is an appropriate
way to select improvement areas in this domain?
If we are going to focus on certain conditions, then this methodology is acceptable.
However, by only focussing on certain conditions, it is inevitable that other conditions
will receive relatively little attention. A more comprehensive approach is required that
takes into account all causes of mortality in an appropriately equitable manner.
10. Does the NHS Outcomes Framework take sufficient account of avoidable
mortality in older people as proposed in para 3.11 (page 21)?
Possibly. But it does not take sufficient account of mortality in childhood, as there is
no adjustment for years of life lost.
11. If not, what would be a suitable outcome indicator to address this issue?
12. Are either of the suggestions at para 3.13 (pages 21) appropriate areas of focus
for mortality in children? Should anything else be considered?
As well as adjusting all mortality for years of life lost, we would propose including:
 perinatal / infant mortality rates
 all respiratory deaths, 0-18 years
 deaths associated with chemotherapy
 survival outcomes for acute leukaemias
 number of adolescents dying from suicide that have had contact with NHS in
previous three months
 deaths from meningitis
 traumatic brain injury - post-admission mortality.
Domain 2 - Enhancing the quality of life for people with long-term conditions
13. Are either of the suggestions at para 3.19 (page 24) appropriate overarching
outcome indicators for this domain? Are there any other outcome indicators
that should be considered?
Neither of the proposed indicators adequately captures outcomes for children or
young people (see our proposal in question 14 below).
14. Would indicators such as those suggested at para 3.20 (page 24) be good
measures of NHS progress in this domain? Is it feasible to develop and
implement them? Are there any other indicators that should be considered for
the future?
As possible overarching indicators, we would propose:
 Child-specific PROM for basket of conditions (Asthma, Epilepsy, Cystic
Fibrosis, neurodisabilities, diabetes, rheumatic conditions, Attention Deficit
Hyperactivity Disorder, Autism, Inflammatory Bowel Disease)
 School absence due to the condition for basket of conditions (Asthma,
Epilepsy, cystic fibrosis, neurodisabilities, diabetes, rheumatic conditions,
ADHD, Autism, Inflammatory Bower Disease)
15. As well as developing Quality Standards for specific long-term conditions, are
there any cross-cutting topics relevant to long-term conditions that should be
considered?
An absolute essential for this is child protection. There exists no single, suitable
outcome measure to assess safeguarding services and the only appropriate means by
which to ensure high quality safeguarding services is a NICE quality standard that
specifies the levels of service required across the whole pathway.
The proportion of children with long term conditions with multi-disciplinary care
plans should also be considered as a cross-cutting measure.
The quality of ‘transitional care’ from adolescent to adult services should also be
considered as should the development of self-management skills in adolescents (i.e.
their ability to manage their own condition).
Domain 3 - Helping people to recover from episodes of ill health or following injury
16. Are the suggestions at para 3.28 (page 27) appropriate overarching outcome
indicators for this domain? Are there any other indicators that should be
considered?

Unplanned returns to urgent / emergency care within 48 hours of being seen in
urgent / emergency care – children only.
We believe that this indicator better captures the scenario, common in paediatrics,
where a child is seen in urgent / emergency care, but the severity of their illness is not
recognised, and so they are not admitted, only to return later and be admitted in a
more unwell state.
17. What overarching outcome indicators could be developed for this domain in
the longer term?
18. Is the proposal at paras 3.30-3.33 (page 28-29) a suitable approach for
selecting some improvement areas for this domain? Would another method be
appropriate?
This is an appropriate methodology if the intention is to collect outcomes for the most
common acute conditions requiring hospital admission. However, it is not an effective
methodology to assess the overall quality of care in respect of acute conditions in
childhood. As a measure of primary and secondary care, we would propose the
indicator given above – namely:

Unplanned returns to urgent / emergency care within 48 hours of being seen in
urgent / emergency care – children only.
As a measure of in-hospital care, we would propose:
 Number of cardiac arrests, respiratory arrests and acute life threatening events
for children that have been in the hospital more than 1 hour (excluding those
taking place in Paediatric Intensive Care / Neonatal Intensive Care)
19. What might suitable outcome indicators be in these areas?



emergency hospital admissions for children with gastroenteritis
emergency hospital admissions for children with lower respiratory tract
infections
number of admissions of children in diabetic keto-acidosis
Domain 4 - Ensuring people have a positive experience of care
20. Do you agree with the proposed interim option for an overarching outcome
indicator set out at para 3.43 (page 32)?
No. The current surveys take no account of children’s experience of care (or their
parents) and a new survey must be developed to address this. This is an area in which
we have considerable experience. It is important to appreciate that the use of PREMs
with children and young people and their parents / carers is relatively underdeveloped
compared to its use in adult healthcare.
In our own work on feverish illness in children, we have already used questionnaires
with parents / carers which explored their experience of urgent care provision. We are
continuing this work by developing a PREM which will explore parents / carers and
children's experience of acute illness in any urgent or emergency care setting, and
additional ones which will address children’s experience of epilepsy and diabetes
management. The intention is to make these PREMs available nationally once the
pilot phases have been completed. Given that acute illnesses represent a significant
proportion of all childhood illness, there is no reason why the urgent / emergency care
PREM could not be utilised by the DH in domain 4 of its outcome framework. It
could certainly feature, for instance, in the emergency care dashboard that is currently
being developed.
It is also worth noting that BLISS have developed a questionnaire for parents of
babies on neonatal units and it may be the case that this could be used in assessing
their experience of patient care.
Clearly, further work is required to develop appropriate PREMs that can be used by
children and young people themselves. However, our extensive experience of working
with children and young people in clinical settings, and through our engagement with
child health advocates in our youth advisory panel, the RCPCH is in a unique position
to advise on this work.
21. Do you agree with the proposed long term approach for the development of an
overarching outcome indicator set out at para 3.44 (page 32-33)?
Possibly – as long as child and parent / carer specific questionnaires are developed,
and contribute to the overall indicator. It is important to note that such questionnaires
will need to be stratified according to the age group of the children and young people.
Adolescents will require a different approach to those under 12 for instance. An
important aspect of this is how we will reach and so gain the views of the nonrespondents. This is especially important for young people as those who don’t initially
respond are often those most in need of healthcare. It is also important that such
questionnaires address the physical characteristics of care settings, and whether they
are age appropriate. When children and young people have been surveyed on their
experience of care, the things that bother them are often not those that concern
clinicians e.g. the nature of the waiting area, the food available and how nice the
receptionists were.
22. Do you agree with the proposed improvement areas and the reasons for
choosing those areas set out at para 3.45 (pages 33-34)?
Yes. In particular, the focus on children and young people.
23. Would there be benefit in developing dedicated patient experience Quality
Standards for certain services or client groups? If yes, which areas should be
considered?
Our view is that patient experience measures should generally be incorporated into
condition / pathway specific Quality Standards rather than stand alone as some kind
of generic standard.
24. Do you agree with the proposed future approach for this domain, set out at
paras 3.52-3.54 (pages 36-37)?
Yes – as long as the work around child and parent / carer specific PREMs is pursued,
including the require age stratification.
Domain 5 - Treating and caring for people in a safe environment and protecting
them from avoidable harm
25. Do you agree with the proposed overarching outcome indicator set out at para
3.58 (page 38)?
Yes.
26. Do you agree with the proposed improvement areas proposed at para 3.63
(page 39-40) and the reasons for choosing those areas?
We would wish to see added:
 Medication errors due to weight calculation errors
 Use of an early warning score in children
 Number of paediatric drug errors (all causes)
 Central line associated problems - % of infections expressed as line days.
Number of lines removed for infection
 Number of extravasation injuries and severity.
General Consultation Questions
27. What action needs to be taken to ensure that no-one is disadvantaged by the
proposals, and how do you think they can promote equality of opportunity and
outcomes for all patients and, where appropriate, NHS staff?
The key to this is to ensure comprehensive cover of the measures. All attempts at
measurement have unintended consequences, and it is inevitable that those areas of
clinical practice which are not covered by this framework will be likely to receive less
attention than they merit. Very careful thought, then, needs to be given as to how
these measures will be used to ensure that such unintended consequences are
minimised.
28. Is there any way in which the proposed approach to the NHS Outcomes
Framework might impact upon sustainable development?
29. Is the approach to assessing and analysing the likely impacts of potential
outcomes and indicators set out in the Impact Assessment appropriate?
30. How can the NHS Outcomes Framework best support the NHS to deliver best
value for money?
By giving it sufficient flexibility that clinicians do not distort their own priorities in
order to achieve the particular goals set out here.
By focussing on early interventions in children to ensure a decreased burden of illness
in later life.
31. Is there any other issue you feel has been missed on which you would like to
express a view?
A focus on outcomes is admirable and we support it. However, it must be balanced by
the recognition that appropriate outcome measures do not exist in all areas of clinical
practice. So, the framework is of value, but it must be used in a sufficiently flexible
manner.
In addition, it is hard to comment further on these proposals until we know precisely
how they will be used at a local, GP consortia level.
ANNEX A: Identifying Potential Outcome Indicators
Potential indicators
32. What are the strengths and weaknesses of any of the potential outcome
indicators listed in Annex A with which you are familiar?
33. Are other practical and valid outcome indicators available which would better
support the five domains?
Please see our attached paper, which has previously been sent to the DH: Outcomes
Framework – Proposals for DH
34. How might we estimate and attribute the relative contributions of the NHS,
Public Health and Social Care to these potential outcome indicators?
It is our view that you should not attempt to. There exist in paediatrics complex, long
term conditions which require input from a range of professionals. What matters for
these children (and their parents / carers) is that their outcomes improve, not which
particular service has led to that improvement. It is a false economy in such situations
to try and make that calculation and it does not help the children in question.
Principles for selecting indicators
35. Are the principles set out on pages 48 and 49 on which to select outcome
indicators appropriate? Should any other principles be considered?
The principles are valid, but if applied too rigidly could become inappropriate. For
instance, HbA1C is not especially meaningful to the public, but it is a highly
important measure that has repeatedly been shown to equate with outcomes that are
meaningful to the public. If these principles were applied too rigidly then HbA1C
might be excluded when in reality it is a very useful measure to assess the quality of
care in diabetes.
ANNEX 1: Equity and Excellence Outcomes Framework
Proposals by the Royal College of Paediatrics and Child Health
Overarching
Indicator
Domain 1:
Mortality
Improvement
Area
Possible Outcome
Indicator
Infant Health
Perinatal / infant
mortality rates
Respiratory
disease
Childhood
cancers
All respiratory deaths, 018
Deaths associated with
chemotherapy
Acute
Leukaemias
Suicide
Survival outcomes
Meningitis
Traumatic Brain
Number of adolescents
dying from suicide that
have had contact with
NHS in previous three
months
Deaths from meningitis
prior to hospital
admission / in-hospital
deaths
Post-admission mortality
Associated
NICE Quality
Standard
Neonatology
Asthma
Possible Data source / data comments
National Centre for Health Outcomes
Development, Centre for Maternal and
Child Enquiries, Neonatal Data Analysis
Unit, National Patient Safety Agency,
Office for National Statistics, Child Death
Overview Panels (CDOPs)
Annual deaths, HQIP audit of asthma
deaths. CDOPs
Annual deaths. Might need to link via
NHS nos to HES to distinguish those
associated with chemo. CDOPs
Annual deaths. CDOPs
Contact with primary care not available,
so may be difficult.
CDOPs
General Practice Research Database, HES
Deaths in hospital or following discharge
Injury
- either HES or annual deaths.
Domain 2:
LTC
Overarching
Indicator
Improvement
Area
Child-specific
PROM for
basket of
conditions
(Asthma,
Epilepsy,
Cystic
Fibrosis,
neurodisabilitie
s, diabetes,
rheumatic
conditions,
ADHD,
Autism,
Inflammatory
Bower
Disease)
Basket of LTC
School absence due to
(eg. asthma,
condition
diabetes,
epilepsy,
neurodisability,
cystic fibrosis,
rheumatic
conditions,
ADHD / Autism,
Inflammatory
Bower Disease)
Asthma
Diabetes
Possible Outcome
Indicator
emergency hospital
admissions for children
with asthma
range of outcome
measures for diabetes in
children (cholesterol, BP,
HbA1c,
microalbuminuria, longterm micro- and
macrovascular
complications)
Associated
NICE Quality
Standard
Asthma,
Diabetes,
Epilepsy,
Arthritis,
Autism,
Cerebral Palsy,
Possible Data source / data
comments
Asthma
HES/Disease Management
Information Toolkit (DMIT)
Diabetes
Possibly primary care data or new
child health dataset.
Child and Maternal Health
Observatory tates that this data is
available
Neonatal care
Epilepsy
Arthritis
Sickle Cell
Disease
Disability
IBD
two year outcomes (rates Neonatology
of impairment /
disability)
percentage of ‘Status
Epilepsy
epilepticus’ admissions of
children on
anticonvulsants. Number
of admissions due to
epilepsy or epilepsy
related morbidity eg
injuries; days/year free of
seizures
1.Children with uveitis /
poor visual outcome
National Neonatal Audit Programme,
Neonatal Data Analysis Unit
2. Young adults needing
joint replacement
Indicators for sickle cell
in children: pain
management; avoidance
of serious complications
eg. stroke.
Proportion of children
with multi-disciplinary
care plans
Proportion of patients
requiring steroids >3/12
in last year, growth
2. HES - but only for those having had
replacement, not those waiting
Possibly from HES depending on
quality of diagnosis coding.
Acute pain
management
DMIT but would not be able to link to
prescribing data to know if
anticonvulsants used.
1.Possibly from HES depending on
quality of diagnosis coding.
Unlikely to be able to obtain any
datasets to support this as most will be
seen in primary care. Though possibly
RCP audit.
Overarching
Indicator
Domain 3:
Acute
Illness
Improvement
Area
Unplanned
Urgent /
returns to
emergency care
urgent /
emergency
care within 48
hours of being
seen in urgent /
emergency
care – children
only.
Acute Life
Threatening
Events
Gastroenteritis
Respiratory
Diabetes
Possible Outcome
Indicator
Associated
NICE Quality
Standard
Possible Data source / data comments
Unplanned returns to
urgent / emergency care
within 48 hours of being
seen in urgent /
emergency care –
children only.
Frequent flyers to A&E or Emergency
admission to hospital linked on NHSno
Number of cardiac
arrests, respiratory arrests
and acute life threatening
events for children that
have been in the hospital
more than 1 hour
(excluding those from
Paediatric Intensive Care
/ Neonatal Intensive
Care)
emergency hospital
admissions for children
with gastroenteritis
emergency hospital
admissions for children
with lower respiratory
tract infections
number of admissions of
Times not recorded on HES so would not
be able to determine if been in for 1 hour
previous to arrest.
HES
HES
Diabetes
HES but dependent on coding of DKA
children in diabetic
ketoacidosis (DKA)
Domain 4 – PREMS – Patient Reported Experience Measures
The use of PREMs with children and young people and their parents / carers is relatively underdeveloped compared to its use in adult healthcare.
Nevertheless, in its analysis of the literature, and in its own direct experience, the Royal College of Paediatrics and Child Health is developing
significant expertise in this area.
We are aware, for instance, of the innovative use of interactive animated questionnaires and cartoon characters in the Alder Hey developed
national paediatric toolkit. In our own work on feverish illness in children, we have already used questionnaires with parents / carers which
explored their experience of urgent care provision. We are continuing this work by developing a PREM which will explore parents / carers and
children's experience of acute illness in any urgent or emergency care setting, and additional ones which will address children’s experience of
epilepsy and diabetes management. The intention is to make these PREMs available nationally once the pilot phases have been completed. Given
that acute illnesses represent a significant proportion of all childhood illness, there is no reason why the urgent / emergency care PREM could not
be utilised by the DH in domain 4 of its outcome framework. It could certainly feature, for instance, in the emergency care dashboard that is
currently being developed.
It is also worth noting that BLISS have developed a questionnaire for parents of babies on neonatal units and it may be the case that this could be
used in assessing their experience of patient care.
Clearly, further work is required to develop appropriate PREMs that can be used by children and young people themselves. However, our
extensive experience of working with children and young people in clinical settings, and through our engagement with child health advocates in
our youth advisory panel, the RCPCH is in a unique position to advise on this work.
Overarching
Indicator
Domain 5:
Safety
Improvement
Area
Possible Outcome
Indicator
Children’s
prescribing
Medication errors due to
weight calculation errors
Data not available, should be held locally
(clinical audit)
Children –
preventable
deterioration
Hospital
acquired
infections
Drug errors
Use of an early warning
score
Data not available, should be held locally
(clinical audit)
?appropriate measure
Potentially from HES dependent on
coding.
Number of paediatric
drug errors (all causes)
% of infections expressed
as line days. Number of
lines removed for
infection
Number of extravasation
injuries and severity.
Data not available, should be held locally
(clinical audit)
Data not available, should be held locally
(clinical audit)
Central line
associated
problems
Extravasation
injuries
Associated
NICE Quality
Standard
Possible Data source / data comments
Data not available, should be held locally
(clinical audit)
NICE Quality Topics
These are the most important topics that the RCPCH would suggest that NICE develop quality standards around in addition to neonatology,
which has already been developed and is awaiting approval from the National Quality Board. They are in order of priority.












Safeguarding
Asthma
Epilepsy
Allergic Disease
Bronchiolitis
Meningitis
Early developmental impairment
Acute pain management
Headache
Cerebral palsy
Autism
Diabetes