Reviewer Training
xxx zone
Welcome and introductions
Aims of Reviewer Training
• To promote understanding of the purpose and
implementation of the National Cancer Peer
Review Programme
• To provide peer reviewers with the skills and
information to enable them to fully contribute
and participate as equal team members in the
National Cancer Peer Review Programme
Learning Outcomes
Hopefully, by the end of the day you will:
• Understand the principles of the National Cancer Peer Review
Programme and your responsibilities as a reviewer
• Understand your role and its relationship with other team members
and the Zonal Team
• Understand what you need to do at each stage of the peer review
visit
• Have practiced the skills you will need when reviewing
• Be familiar with and recognise the importance of the CQuINS
electronic communications system
Session 1
The National Cancer Peer
Review Programme
What is Cancer Peer Review?
A process undertaken to assess the quality of cancer
services against IOG for NHS patients in England, enabling
quality improvement
An integral part of the cancer reform strategy(2007)
Supported by a set of detailed measures
It is not a statutory function, but is mandatory i.e.
commissioned by the SHAs through the DH
Purpose of Cancer Peer Review
To ensure services are as safe as possible
To improve the quality and effectiveness of care
To improve the patient and carer experience
To undertake independent, fair reviews of services
To provide development and learning for all involved
To encourage the dissemination of good practice
Organisation of the Programme
National Clinical
Outcomes
Group
Central
North
Zonal Advisory
Group
South
London
Zonal Advisory
Group
Zonal Advisory
Group
Zonal Advisory
Group
Who’s Who?
Acting
National
Coordinator
Ruth
Bridgeman –
Acting Deputy
National
Coordinator
Julia Hill
Quality
Director
Clinical Leads
XXX Zone Team
National Team
Quality Manager
xxx
Admin Support
xxx
Assistant Quality Manager
xxx
Quality Mangers
xxx
xxx
Quality Director
xxx
The Peer Review
Programme
Peer Review
Visits
Targeted
Externally Verified SelfAssessments
Sampled
Validated Self-Assessments
(annual)
All Teams
The Measures
• Evidence based using NICE evidence linked to “Improving
Outcomes Guidance” and agreed best practice based on national
consensus
• Development of measures for each topic is undertaken by an
expert group
• Consultation on new measures
• Measures coded as per year, topic and measure number
e.g. 08 2J 102
Handbook for the National
Cancer Peer Review Programme
Contains details of the process from start to finish
including:
1.
2.
3.
4.
5.
The Peer Review Programme
Annual Self Assessment
Internal Validation
External Verification
National Schedule for Peer Review Visits &
Conducting the Review
6. Outcomes of the Peer Review Process
7. Identification of Concerns
8. CQuINS
Session 2
The Cancer Peer Review Visit
What is a Peer Review
Visit?
Purpose:
• Provide an opportunity to meet with
members of a service to determine
compliance with the quality measures
• Identify any broader issues relating to the
delivery of a quality and safe service
• Provide a further external check on
internal quality assurance processes
What does a Peer Review
Visit entail for Reviewers?
Preparation
Visiting a locality/network
Working as a team to review evidence
Identifying questions for clarification.
Meeting with the team
Drawing conclusions
Report writing
Checking draft report
Who are Reviewers?
Multidisciplinary teams of
• Service Users, Clinicians, Managers, Commissioners
“Peers are people who have been trained
and working in the same discipline as the
people they are reviewing”
Reviewers will not normally review their
own network
Peer Review Teams
May Include:
User/Carer
MDT Lead
Clinician
Clinical Nurse
Specialist
Radiologist
Pathologist
Oncologist
Medical
Physicist
Therapy
Radiographer
Oncology
Pharmacist
Chemotherapy
Nurse
Palliative Care
Consultant
Trust Lead
Clinician, Nurse
or Manager
Network Lead
Clinician, Nurse
or Manager
PCT Cancer
Lead
Cancer
Commissioner
Dietician
Selection Criteria for
a Peer Review Visit
New Teams
Milestones not met for implementation of an IOG as agreed with CAT
Immediate Risks identified at previous peer review visits that have not yet been
resolved
Requests from organisations i.e. SHAs, Local and Specialist commissioners, PCTs,
Networks, Acute Trusts
% compliance with measures within lowest performance grouping
Concerns regarding rigor of Internal Validation
Stratified random sample based on % compliance (if available capacity)
The Peer Review
Visit Plan
December
Notification by
31st December
to teams to be
peer reviewed
during May March
Preparation for
review
- 6 WEEKS
Deadline for
submission of
evidence for
all teams to be
visited
- 4 Weeks
Pre-visit
meeting for
NEW TEAMS
with the Zonal
Team
or
Zonal Team
Preassessment
circulated
+ 8 WEEKS
Visits
MAY-MARCH
Each Network is
allocated one
month. Can
take from 1 to 4
weeks to
complete a
Network –
normally 1 day
per Locality
Report
published 8
weeks after
last review
day
Information for review
team
• Practical arrangements
• Emergency contact details
• Membership of visiting
team
• Access to key evidence
documents
• Compliance against the
measures
•
•
•
•
•
•
Timetable
Previous reports
Interpretation guidance
Cancer measures
Expenses claim form
Contextual information
Typical Timetable
Time
AM Reviews
08.30
Arrive & Introductions
09.00
Preparation
10.30
Meeting with the Service - up to 3
concurrent sessions
11.30
12.30
1.00
Arrive & Introductions and then
preparation
Lunch and report writing
Conclusions and report writing
13.30
3.00
3.30
5.00
PM Reviews
Lunch
Preparation cont.
Meeting with the Service - up to 3
concurrent sessions
Depart
Conclusions and report writing
Zonal Feedback to Trust Lead Team and visiting team depart
Session 3
Review of Evidence
MDT Key Questions
Can You:
Demonstrate that you have a properly constituted and functioning
MDT?
Demonstrate that you have effective systems for providing
coordinated care to individual patients?
Demonstrate that your team has adequate information to help it
improve service delivery?
Demonstrate how you are continuously improving your service
(including both clinical effectiveness and the patient experience)?
MDT
Evidence Documents
Operational
Policy
Annual Report
Team function
Achievements & challenges
Patient Pathway
Use of data to assess service
provision
Polices and procedures
MDT Workload & Activity Data
Clinical guidelines and
treatment protocols
National Audits
Local Audits
Patient Feedback
Trial Recruitment
Work Programme Update
Work
Programme
Actions to address
any issues
Plans for service
improvement &
development
Relationship of Measures
to Key Documents
• Information required to comply with the measures
should be contained in the key documents (Evidence
Guides were developed to support teams in the
preparation of evidence)
• When compiling the report you will need to refer to the
actual measures, not abbreviated versions or what you
think the measure is!
• Look at the compliance section of the measure
• Decisions against the measures are made as a team
• If unsure then check with the Zonal Team
Compliance with the
Measures
• Yes / No / N/A – at the time of the visit
– All aspects of the measure have to be met
• Comments
– No and …
– No but …
– Yes and …
– Yes but …
• Content of policies, procedures etc.
– Existence usually required by measure
– Content determines yes/no only so far as is
stipulated in the measure
Demonstrating
Agreement
Where agreement to guidelines and policies is required
this should be stated clearly on the cover sheet of the
relevant evidence document
Evidence Guides will indicate the groups and individuals
that need to be documented as agreeing the key evidence
documents
See Examples within Evidence Guides
Pilot for Breast and Lung
Service 2010/2011
Details of Clinical Lines of Enquiry:
• Identification of clinical indicators
• Data
• Clinical lines of enquiry
(See the Delivery Specification Guide on CQuINS
and evidence guides).
Identification of Clinical
Indicators
Discussion with SSCRG Leads, NCIN and Medical
Directors
Outcome of the work has been to develop clinical
indicators for Breast and Lung
It is the intention to feedback and review the
Clinical Indicators at the NCIN Site Specific Clinical
Reference Groups on an annual basis
Data
• Where National data is available, this will be
provided to both the review teams and MDTs
• A proforma will be provided to the Trusts for the collation
of local data. It should be returned to the Zonal Teams
and the National Cancer Peer Review Senior Information
Analyst by the end of the self-assessment period
• The proforma for collation of data for breast cancer is
available in the Delivery Specification Guide on CQuINS
Clinical Lines of Enquiry
• A briefing sheet on the relevance of the clinical
indicators
• Structure and ensure consistency of the
discussions about the data on a peer review visit
• A commentary on the clinical lines of enquiry
should be included in the PR and IV reports.
Clinical Lines of Enquiry
in Reports
• Question 3 for an NSSG: ‘Can you
demonstrate that you have effective
processes in place for evaluating services
across the network and identifying
priorities for improvement?’
• Question 4 for MDTs: ‘Can you
demonstrate how you are continuously
improving your service?’
Reviewing Evidence
Key
Questions
Evidence
Documents
Quality Measures
Reviewer Responsibilities
Reviewer
Responsibilities
Responsibilities
include:
• Working on behalf of the health community
within the zone
• Preparation before the visit – information
pack
• Reviewing the evidence and ascertaining
compliance
• Identifying issues relating to the Measures
and to the quality of service for patients
• Allocating questions for each reviewer
• Keeping to timetable
• Confidentiality...sensitivity
• Obtaining consensus re compliance/report
content
• Writing the report
Approach
• Open, fair, consistent and comparable
• Encourage development and learning
• All members of the team have a valuable
contribution to make
– “No question is a silly question.”
• Try to understand and follow the patient’s
journey
The Evidence –
Preparation on the Day
• Welcome and introductions
• Review of evidence documents to
– Establish a ‘feel’ for the service
– Initial consensus on compliance
– Review case notes if applicable – NHS
professionals only
– Identify and formulate questions
• Agree structure and process for the meeting
CQuINS
Using CQuINS V4
Available via the web site at: www.cquins.nhs.uk
• Secure web based database supporting each stage of
the cancer peer review process
• Records assessments, compliance with the measures
and reports
• Provides information for national analysis and reporting
CQuINS
In preparation for your visits you will need a
CQuINS password in order to gain access
to the following evidence:
• The teams self- assessments
• The teams externally verified reports
• User reviewers will be sent copies of the
key documents
CQuINS Homepage
Navigate to CQuINS
homepage
www.cquins.nhs.uk
Enter your registration
details
Viewing Key Evidence
Documents (1)
% compliance at self
assessment stage and
internal validation
stage (if applicable)
will be detailed in this
section.
OP – Operational
Policy
AR – Annual Report
WP – Work
Programme
APP – Additional
appendices
View or download
each evidence
document.
Any internal validation
reports will be
available to view or
download in this
section
Viewing Key Evidence
Documents (2)
If you select view the
document will be
displayed directly onto
the screen if it is in a PDF
format (recommended
format for all documents
uploaded to CQuINS).
If it is a word or excel
document a new page
will be opened so that
you can view it.
Group Work 1 – A Fictional
Lung MDT- Preparing to Meet
the Team
• You are reviewing the evidence for a Lung MDT and
need to make an initial assessment. Using the
spreadsheet, review the evidence for those measures
highlighted
• Consider whether you have sufficient information for
initial compliance and identify any questions you want to
ask the team
• Complete recording proforma
Key Points
from Group Work
• What conclusions did you reach after
looking at the evidence?
• What questions do you have that you
need to ask the team?
Session 4
Meeting with the Team
Question Types
Open
Encourage people
to talk
What, where, when,
why, how?
Closed
Checking facts
How many? do
you?
Hypothetical
Test contingencies
What if?
Probing
Greater depth
In what way? Tell
me more..
Leading
Pacifying
Use with care
You do, don’t you?
Multiple
Avoid
Who sends it, what
do you do and how?
Funneling Questions
• Open - to explore
• Probing - to clarify
• Hypothetical - to
test contingencies
• Closed - to confirm
Listening
Listen before deciding response
– Their story not yours!
– Be active
• summarise regularly
• ask confirmatory questions
– Encourage contributions from everyone
– What if they won’t stop talking?
Body Language
• Look for reaction and adjust style to suit
• Watch for defensive behaviour
•
•
•
•
sudden arm-crossing
agitation
face touching
avoidance of eye-contact
• Use empathy
Which Team Members
Should You Expect to Meet?
MDT
Review
Locality
Network
Group
Review
• Lead Clinician and CNS
• with other core members e.g. Surgeon,
Oncologist, Radiologist, Pathologist,
Palliative Care
• Chair of Locality Group,
• with Lead Cancer Team, Commissioner,
PCT and User Reps
• Chair of Network Group
• Small group of other key group
members inc. User Reps
Possible Challenges
The need to deal with ‘politically’ or
‘organisationally’ sensitive issues
The need to deal with immediate risks
The need to deal with a hostile organisation
The need to manage appropriate engagement
from the review team
Discussion Topics
• Recent improvements
• Tracking the patient pathway, referral, clinics,
diagnostics, MDT treatment options, follow up,
support (survivorship), discharge / death, links to
palliative care
• General team issues, audit, service
improvement, governance, clinical trial entry,
data collection
• Relationship with NSSG / referring teams
• Current clinical service development issues
Meeting the Team
•
•
•
•
•
•
•
Start on time and keep to time
Make sure everyone is introduced
Put at ease – and get comfortable
Develop rapport
Explain purpose and structure
Note-taking
Open questions to start
Closing the meeting
• Ideally, summarise the main issues
• Give the opportunity for further comment:
– “Is there anything we’ve missed?”
– “Is there anything else you’d like to tell us about?”
– “Which aspect of the service are you most proud of?”
• Thanks
• If serious / sensitive issues – talk to Team
Leader or Zonal Team ASAP
Summary
• Be prepared, know what you need to ask, why,
how and by whom
• Clear introduction
• Open question to start
• Listen actively
• Take good notes
• Summarise
• Give chance for further comments
• Thanks
Group Work 2 – Part 1
A Fictional Lung MDT
• Divide into groups
• Formulate your questions arising from the
previous group work
• Consider how they link to the 4 key MDT
questions
• Think about whether there are any areas
of concern
Group Work 2 – Part 2
• Read the brief for your character
• Reviewers need to decide who is asking
what, why and how
• MDT members need to decide on who will
respond to any issues
•
•
•
•
10 mins. – Question planning
15 mins. – Role preparation
25 mins. – Meeting with the team (20 mins. + 5 mins. debrief)
Return for feedback
Then in groups we will cover completing report
forms
Feedback from review meeting
Session 5
Completing Report Forms
Drawing Conclusions (1)
• Work together as team to produce an accurate,
objective report – one scribe
– All team members have a responsibility to contribute
– Consensus view
– Use the 4 key questions to support report writing
• Check each measure:
– Yes / No / N/A
– If no: Give reason why
– If very good: Say so!
• Agree issues:
– Always include MDT membership, attendance, cover
– Quantify issues if possible
– Write sentences …. or as near as possible
• Progress from last visit
Drawing Conclusions (2)
• Evaluate and describe the situation – avoid
solution/recommendation
• Legible and evidenced
• General conclusions
• Clarify/explain the point being raised.
– Waiting times are good/too long – The waiting times
are currently three weeks for ….
– Patient information is good/poor – There is no
provision of any local patient information…
Sample Compliance
Sheet
Compliant? Self-Assessment
Code
Measure
Internal Comments
SelfSA IV EV PR OP AR WP APP Assessment
08-2B-101 Single named lead clinician
Y
- - p7
08-2B-102 Named core team members
Y
- - p9 p7
08-2B-103 Team attendance at NSSG meetings
If separate pre-diagnostic MDT
08-2B-104 membership named
Meet weekly and record core
08-2B-105 attendance and protocols for referral
MDT agreed cover arrangements for
08-2B-106 core member
Y
- - p17 p7
Y
- - p9
Y
- - p12 p7
Y
- - p9
Internal
Validation
Zonal Team Comments
External
Peer
Verification Review
Categorising Review
Findings
Immediate
Risk
• An issue that is likely to result in harm and requires
immediate action
Serious
Concern
• An issue that could compromise the quality or
outcome of patient care
Concern
• An issue that affects the delivery or quality of the
service
Good
Practice
• Relates to the service and can be either innovative
or common practice undertaken very well
Group Work 3
• Exercise in your groups
• Identify which category each of
the issues identified would fall
into
Feedback on Concerns
Group Work 4
(30 mins.)
• Quickly check through the measures
• Any changes as a result of meeting with the team
• Any queries
• From the information you have gathered
work together to categorise issues and
document on the report proforma
Feedback from Group Work
PLEASE…..
Do not take the final report away
with you!
What Happens Next?
Feedback
Draft report circulated to Review Team and Trust /
Network for comment on factual accuracy
Final report published on CQuINS within 8 weeks.
Reports will be publically available
Session 6
Evaluation and Close
Aims of Reviewer
Training
To promote understanding of the purpose and
implementation of the National Cancer Peer
Review Programme
To provide peer reviewers with the skills and
information to enable them to fully contribute
and participate as equal team members in the
National Cancer Peer Review Programme
Learning Outcomes
• Understand the principles of the National Cancer Peer Review
Programme and your responsibilities as a reviewer
• Understand your role and its relationship with other team members
and the Zonal Team
• Understand what you need to do at each stage of the peer review
visit
• Have practiced the skills you will need when reviewing
• Be familiar with and recognise the importance of the CQuINS
electronic communications system
Resources Available
www.cquins.nhs.uk
NCPR Handbook
Evidence guides for each topic area
CQuINS help section
Slides and packs from today
Zonal Teams are able to offer support
Any Questions?