The Implementation field team six years on:
approaches to engagement and evaluating
impact
Val Moore, Implementation Programme Director
Jane Moore, Implementation Consultant
The NICE field team function, est 2006
• Strategic advice and context to help senior
management teams in NHS, public health and social
care organisations work with NICE guidance
• Problem solving by sharing examples of how
organisations and teams work together to implement
guidance and use NICE quality standards
• Advice on how to use NICE implementation support
products, NICE pathways and NICE evidence services
• Feedback from the field and recommendations to NICE
on improvement opportunities
Annual visits to over 650 top teams or
opinion formers and 100 meetings/events
Stephen Stericker
Annie Coppel
Deborah O’Callaghan
Chris Connell
Jane Moore
Steve Sparks
Stephen Judge
Themed engagement to meet needs
“NICE influences clinical practice heavily – there are few
instances where clinicians disagree with NICE
recommendations” Trust Medical Director
“ I didn't realise that NICE
did so much". Chair of LA
scrutiny committee
What the users say
“NICE is a ‘way of life’ for the
Trust particularly in relation to
‘We take NICE very seriously’.
“If NICE does a
high cost drugs” Specialist Trust
All commissioning plans Medical
try to Director
consultation, it is one
build NICE guidance in. The
of those
PCT include NICE guidance in
organisations that
the quality schedules of their
really does seem to
contracts and also within
"NICE guidance is like a listen" Head of
CQUIN’. Director of
bible. It tells you what
governance
Commissioning
you should be doing ....
and is a very useful tool
in changing practice".
Head of Midwifery
Evaluating field team impact
• We have consistently evaluated our activities (see
annual reports), but inherent difficulties with identifying
impact, and have relied on proxy measures of success
• As a new system for commissioning health services
develops across England, we reviewed evidence around
effective implementation activities and evaluating impact
• This led to innovative approaches to engagement and
improved methods of evaluating impact
Actions
• Revised field team implementation strategies and activities
to fit better with new system of health commissioning
(less counting visits, more doing what counts!)
• Appointed an external evaluator to conduct a survey with
field team “clients”
• New success criteria:
– outcomes focused
– owned by the whole of NICE
– three year incremental objectives for external engagement
activities
Interviews and survey
• Field contacts
– Interviews with 19 field contacts
– Online survey of field team contacts, with 195
contacts completing questionnaires (response
rate = 39.6%)
• NICE directorate representatives
– Interviews with representatives from all six
directorates
Evaluation objectives
• Client satisfaction with the support received
• Meeting needs of clients
• Detect increased knowledge of NICE products and
resources
• Behaviour change
• Internal satisfaction with field team feedback
• Opportunities to improve the services offered by the
field team
Taking all things into consideration, how satisfied are you
with the help and support that you receive from your NICE
Implementation Consultant?
What people found most helpful
A personal link to NICE
70%
Explains things clearly to me
47%
70%
that might be helpful
Helps keep me up to date with
developments
53%
28%
Acts as sounding board for ideas
Feedback conduit from us to NICE
43%
Shares examples of what other
orgs are doing
43%
Advises on how to get involved
with guidance in development
30%
31%
Always available if have question
Other
11%
0%
Base: All respondents (195)
25%
50%
75%
100%
New engagement criteria example
Sector
Health
Audience
Objective
NHS England AreaTo use NICE to inform their
Teams
commissioning of primary
and specialist services and
to encourage CCGs to
apply it to their
commissioning activities .
2013 -14
A named lead for NICE
identified by 80% of ATs
and contact established
by local IC .
Health
Clinical
Commissioning
Groups
That CCGs routinely
incorporate NICE
guidance, advice and
standards into their
commissioning processes .
Health
Strategic
Clinical
Networks
To use NICE to inform
commissioning and service
review activities .
A lead for NICE identified
for 80% of CCGs and
regular communication
maintained as agreed
with the lead. 20
examples of CCGs
working with NICE QS
identified.
A named contact for
NICE identified for 80%
of SCNs and a meeting
with the local IC to have
taken place to explore
role of NICE guidance
and QS in the work of
SCNs and how SCNs
can contribute to the
work of NICE.
Success Criteria
2014 - 15
80% of ATs have
implemented the NCB
policy for wor king with
NICE. 10 examples of ATs
specifying NICE QS in
secondary and specialist
contracts obtained.
80% of CCGs have a policy
for working with NICE. 40
examples of CCGs
spe cifying NICE QS in
contracts identified .
80% of SCNs have a policy
for incorporating NICE
guidance and QS into their
local activities .
2015 -16
80% of ATs using NICE QS
in primary care contracts
and 10 examples of
improved patient outcomes
derived from NICE QS
identified.
80% of CCGs using NICE
QS in contracts and 20
examples of improved
patient outcom es derived
from NICE QS identified .
80% of SCNs contributing to
the work of NICE (e.g.
membership on QSACs,
GDGs and co - ordination of
consultation responses). 10
examples of improved
patient outcomes derived
from SCNs using NICE QS
identified .
Lessons for guideline developers and
implementers
• Evaluating the impact of implementation activities and
teams is difficult but important and achievable
• Feedback on our activities from our “clients” has been
helpful in influencing the way the field team plans and
delivers its services in the future
• Focusing on immediate and intermediate implementation
outcomes over longer timescales, and developing
success criteria for implementation and engagement
activities is valuable