5th March 2015
THE ROLE AND EFFECTIVENESS OF
QUALITY CONTROL AUDIT: IMPROVING
THE ARCP PROCESS ACROSS
FOUNDATION SCHOOLS.
Dr B T Langham
Director of Foundation Training
for the East Midlands.
The tale of two Foundation Schools
TRENT
Kettering
www.hee.nhs.uk
East Midlands Local Education and Training Board
LNR
The Change
Prior to 2013
• Two Schools
• Two different processes
for sign off
• Equitable outcomes?
• Potential for challenge
ARCPs 2013
• Two Schools
• Same process
• Same sign off standards
issued to all trainees
and trainers
• All FTPDs trained for
role as ARCP chairs.
www.hee.nhs.uk
East Midlands Local Education and Training Board
The Challenge
Two Questions:
• Would outcomes across 2 schools and 11 centres be
equitable i.e. did the panel adhere to the evidence
required for a satisfactory outcome?
• How would we test this?
www.hee.nhs.uk
East Midlands Local Education and Training Board
Test through Audit
Audit of 10% of ARCP outcomes:
• Randomly selected
• 3 auditors not involved in local ARCPs
• Accessed e-portfolios
• Checked available evidence at time of ARCP
against ARCP checklist (used by all panels)
• Assigned an outcome + reasons
www.hee.nhs.uk
East Midlands Local Education and Training Board
Results of quality control audit
for 2013
100%
80%
60%
FY1
40%
FY2
20%
0%
LNR
Trent
Note: the quality control assessors agreed with all final outcomes.
www.hee.nhs.uk
East Midlands Local Education and Training Board
Results for 2013:
30%
25%
20%
FY1s
15%
FY2s
10%
5%
0%
LNR
Trent
Percentage of trainees awarded outcome 5s
www.hee.nhs.uk
East Midlands Local Education and Training Board
Outcome 5 awarded in Trent School
80%
70%
60%
50%
40%
30%
20%
10%
0%
FY1
FY2
A
B
C
D
E
F
www.hee.nhs.uk
East Midlands Local Education and Training Board
G
H
Next steps:
1. Faculty development day when results were
presented to FTPDs from both schools.
2. Recommendations:
– FPCs and FTPDs to monitor e-portfolios of all trainees.
– Trainees identified as currently having e-portfolio issues must be interviewed by FTPDs
before end of March.
– The ARCP outcome will be determined by the level of evidence required (see
LNR/Trent sign off requirements) – if evidence is missing an Outcome 5 must be
issued.
– Further audit of 2014 ARCP outcomes.
www.hee.nhs.uk
East Midlands Local Education and Training Board
Results over two years:
number of outcome 5s
35%
2013
30%
25%
20%
2014
15%
10%
5%
0%
LNR FY1s
LNR FY2s
Trent FY1s Trent FY2s
www.hee.nhs.uk
East Midlands Local Education and Training Board
Quality control assessor and
panel agreement at 1st ARCP
100%
80%
60%
2013
40%
2014
20%
0%
LNR
Trent
Foundation Year 1 trainees ARCP results
www.hee.nhs.uk
East Midlands Local Education and Training Board
Quality control assessor and
panel agreement at 1st ARCP
100%
80%
60%
2013
40%
2014
20%
0%
LNR
Trent
Foundation Year 2 trainees ARCP results
www.hee.nhs.uk
East Midlands Local Education and Training Board
Next Steps:
• Individual issues with panels/locations identified and
feedback delivered to panels.
• Further sharing of best practice across sites.
• Further audit of 2015 ARCP outcomes.
www.hee.nhs.uk
East Midlands Local Education and Training Board
www.hee.nhs.uk
East Midlands Local Education and Training Board
Summary
• Quality control audit revealed discrepancies in ARCP
outcomes (related to outcome 5)
• Sharing of quality control audit and faculty development
improved consistency.
• Individual panel feedback delivered to improve
consistency further.
• 3rd audit planned for August 2015.