Trainees` Guide to Workplace Based Assessment

Trainees’ Guide to Workplace Based
Assessment
Introduction
The College curriculum for specialist training in Psychiatry is supported by
an extensive assessment programme comprising both workplace based
assessments (WPBA) and formal MRCPsych examinations. Full details of
the
MRCPsych examinations appear on the
College
website
(www.rcpsych.ac.uk).
The guide describes WPBAs, outlines their purposes, tells you how many
you need to get done and tells you how to plan and organise them. You
should read it in conjunction with the guidance about the specific tools
that
is
available
on
the
HcAT
/
RCPsych
website
(www.hcat.nhs.uk/rcpsych).
The College Curriculum is the basis of the Specialty Training Programme
in Psychiatry. It has been approved by PMETB and comprises the Core and
General module, 6 specialty modules, and 3 sub-specialty modules. It
uses the same WPBA methods throughout. The entire curriculum is
mapped to Good Medical Practice. The Core and General module indicates
the phase of training through the use of colour: red = ST1; gold = ST2&3;
violet + ST4&5; and green = ST6. The colour codes are used in all
documents concerned with a specific phase of training, including the
WPBA forms. Documents that are not specific to a particular phase of
training (such as the Core and General module and some of the WPBA
forms) are colour coded blue. You should use the curriculum as an
electronic hyperlinked document.
There is also a list of competencies for each training level. Within each list
there are links back to the curriculum. This list of competencies is a
key document. You must familiarise yourselves with this in order
to manage your training with the help of your educational
supervisor.
Underlying Principles
Workplace based assessments must comply with the following:



They must focus on performance (ie what you actually do in the
workplace)
They must be evidence based
Evidence must be triangulated whenever possible (ie it must be
provided by different assessors, on different occasions, and if

possible using different methods)
Records must be permanent – assessment forms must not be lost
or destroyed (this is achieved by online storage).
Purpose
The purpose of the workplace based assessments is threefold.
Firstly, they have a formative function as the basis for feedback and
educational planning. This includes helping identify any specially focused
teaching and learning that might be necessary to fill gaps or resolve
weaknesses. You should have WPBAs completed regularly throughout your
placement to demonstrate effective review and appraisal. Therefore, do
not leave them until the end of each stage of training or clinical
placement.
Secondly, although individually they have no ‘pass’ or ‘fail’ score taken
together WPBAs contribute to the Annual Review of Competence
Progression (ARCP), which has a summative function providing evidence
of the attainment of competencies.
Thirdly, participating in a specific number of WPBAs is mandatory for
eligibility for the MRCPsych examinations (see later).
Assessments
There are 8 workplace based assessments available. These are described
in detail in the trainee and assessor guidance for each WPBA tool and are
merely outlined below.
o Assessment of Clinical Expertise (ACE)
The assessor observes a whole new patient encounter in order to
be able to assess your ability to take a full history and mental
state examination and arrive at a diagnosis and management
plan.
o Mini-Assessed Clinical Encounter (mini-ACE)
The assessor observes part of a patient interaction, for example
history taking or negotiating a treatment plan, and rates your
performance.
o Case-based Discussion (CbD)
You select two sets of notes of patients you have recently seen
and the assessor picks one to discuss. The discussion will allow
demonstration of clinical decision-making and the application of
clinical knowledge.
o Case Presentation (CP)
This tool can be used when you give clinical presentations and
involves assessment of domains such as presentation skills and
interpretation of evidence.
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
2
o Journal Club Presentation (JCP)
This can be used when you present a journal article and covers
domains such as analysis and critique and answering questions.
o Directly Observed Procedural Skills (DOPS)
This has more limited use in psychiatry compared to other areas
of medicine but can used in situations such as administering
ECT.
o Mini-Peer Assessment Tool (mini-PAT)
It allows co-workers to assess your attitudes and behaviours and
ability to work well with colleagues.
o Assessment of Teaching (AoT)
This is a new tool that has been developed after feedback from
the pilot programme. It allows an assessment to be made of
teaching skills and may relate to a lecture, tutorial or small
group teaching session that you lead.
Requirements
The requirements for the number of WPBAs to be completed in ST 1-3
are:
WPBA
Minimum number required per
year
ACE
2 in ST 1, 3 in ST 2 and
3 in ST 3
mini-ACE
4
CbD
2
mini-PAT
2
WPBA
Recommended number per
year
CP
1
JCP
1
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
3
DOPS
As the opportunity arises
AoT
As the opportunity arises
The guidelines in the above tables should be regarded as the minimum
number of WPBAs: many trainees will need to do more, for example to
provide additional evidence of achievement in a particular area of
professional practice. It is important to again note that there is no ‘pass’
or ‘fail’ score for WPBA. Therefore for exam eligibility purposes you are
only required to demonstrate that you have undertaken the required
WPBA, not that you have achieved a specific ‘score’.
WPBA and Examination eligibility
The requirements for each examination are summarised in the table
below. These requirements may be subject to change and it is
essential that you consult the College website for the most up to
date information. Trainees who enter ST 1 must fulfil the requirements
below to be eligible to sit the MRCPsych examinations. There are
transitional arrangements for eligibility for the MRCPsych examinations
which can be found in full on the College website. All trainees who enter
run through training at ST 2 or ST 3 and have passed Part 1 may take
Paper 3 and the OSCE under the previous eligibility criteria (duration of
training completed not WPBA). All trainees who enter run through training
at ST 2 or ST 3 who have not passed Part 1 may take Paper 1 and Paper 2
under the previous eligibility criteria but must complete the new criteria
for the OSCE to take Paper 3 and the OSCE.
Exam
WPBA requirement
Paper 1
4 mini-ACE at ST 1 level
2 CbD at ST 1 level
Paper 2
4 mini-ACE at ST 2 / 3 level
2 CbD at ST 2 / 3 level
Paper 3
Cumulative criteria of Paper One
and Paper Two
(8 mini-ACE and 4 CbD)
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
4
OSCE
8 ACE across a range of specialties
(2 in ST 1, 3 in ST 2, 3 in ST 3)
For trainees at the level of ST4 and above tools are currently being
developed and adapted to assess higher level performance and
competencies such as leadership skills. The existing WPBA tools should be
used to guide educational development and as evidence for your ARCP.
You should complete two rounds of mini-PAT a year and in addition aim to
undertake one WPBA a month.
You must note that, since they form the basis for feedback and
educational planning, you must arrange WPBAs at appropriate intervals
throughout your training rather than leaving them until the end of each
stage. In particular, we strongly suggest that within the early stages of
each phase of training or clinical placement (ie within the first month) you
have a mini-ACE or ACE and a CbD. This is because these tools reflect
closely the expected day-to-day work of a junior doctor in Psychiatry both
in terms of the assessment of cases and clinical reasoning and judgement.
The purpose of these early assessments is to identify what you are doing
well and in which areas you need to develop. They contribute valuable
information to the educational planning of your placement. Therefore it is
most unlikely that assessments made early will produce ratings consistent
with performance at the standard expected for the completion of the stage
of training.
Scale Ratings
All the WPBA forms use a 6-point Likert-type rating scale. The standard
for completion of each stage of training (ST1, 3, 5 and 6) corresponds to a
rating of 4. Higher ratings indicate performance above the minimum
standard for completion of that stage; lower ratings indicate performance
below the required completion standard. Since when you are in the early
part of any phase of training you will probably not be performing at the
standard required for completion of that phase, (and will, therefore, often
receive ratings below 4), most assessment forms have an additional item
where the assessor can indicate your global performance relative to your
stage of training. On the other hand, there might be some occasions when
your performance exceeds that indicated for your current stage of
training. In such circumstances you can choose to be assessed using
criteria (and rating forms) for a higher stage of training and, if you
achieve ratings of 4 or more, can record this in the appropriate portfolio.
Thus, for example, an exceptional ST1 Trainee might choose to be
assessed on their consultation skills at ST2/3 level and, if rated at 4 or
better, would record this in their ST2/3 portfolio as well as in their ST1
portfolio.
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
5
Electronic list of competencies (formerly, ‘Portfolio of Progress
and Attainment’)
Each phase of training (ST1; 2/3; 4/5; and 6) has a designated List of
Competencies: Log of Progress and Attainment. You must keep the
Curricula and associated documents in a single folder with their names
unaltered and you must enable macros for the hyperlinks to work. The
name that you give the folder is irrelevant. Although any of the
documents can be printed, they are large (especially the core curriculum)
and incomprehensible unless viewed in colour.
You can use the boxes in the List of Competencies to describe the
evidence you wish to use to demonstrate your achievement of that
competence. The evidence may consist of one of the WPBA tools but may
also include other evidence such as reference to supervision notes or to
reflective notes. You can print the electronic list of competencies annually
and filled it in by hand, but it is better to complete it electronically and
print it out annually. Then you can add it to your paper portfolio prior to
your ARCP. This means that for stages of training that span two years
(e.g. ST2-3) you should have a half completed list of competencies
printed out at the end of ST2 and a completed one at the end of ST3.
Paper portfolio
This replaces the previous College logbook. It allows you to collate other
evidence in addition to your WPBA. It will be available to download free
from the College website or a commercially printed copy can be purchased
(further details on the College website).
It has several sections including:




self-appraisal
professional development plan
reflective learning
record of publications, audit, research, teaching etc.
Competence Attainment
All trainees will have an Annual Review of Competence Progression (ARCP)
organised by their deanery. This is described in the Gold Guide and
replaces the RITA process. The key elements of this review are:
 appraisal
o Educational
o NHS
 assessment of performance
o workplace based assessments
o experience demonstrated e.g. from portfolio including audit,
research and other aspects of non-clinical competence
 planning.
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
6
Achievement of competencies will be considered in the Educational
Supervisor’s Report and the educational supervisor will consider WPBA
and other evidence such as examinations passed and experiential
outcomes recorded in the portfolio. The competencies will be considered
under 11 domains:
 providing a good standard of practice and care
 decisions about access to care
 treatment in emergencies
 maintaining good medical practice
 maintaining performance
 teaching and training, appraising and assessing
 relationships with patients
 dealing with problems in professional practice
 working with colleagues
 maintaining probity
 ensuring that health problems do not put patients at risk
The Educational Supervisor’s Report and the College Tutor’s Report will be
submitted for consideration by the ARCP. It is the ARCP that validates the
achievement of competence after considering the presented portfolio
information. It is important to note that as the ARCP considers a range of
evidence, it is possible for you to have performed well in your WPBAs but
be considered by the ARCP panel to require further development. The
outcomes for ARCP are described in the Guide to Postgraduate Specialty
Training
in
the
UK
(Gold
Guide),
www.mmc.nhs.uk/download/Gold_Guide_140607.pdf.
Carrying out workplace based assessments
You are responsible for organising your assessments. You should discuss
the areas in which you need to be assessed with your Educational
Supervisor and plan your WPBAs accordingly. Remember that you will
need to be assessed by a range of different assessors in different
situations to ensure triangulation of evidence. Don’t rely on your
Educational Supervisor to perform all the assessments! Also remember
that you must ask for the patient’s agreement when you perform a WPBA
that involves a patient.
There is an online system for the completion and storage of WPBAs. You
can register for this through the College website. After you register you
will be able to access the forms relevant to the mandatory WPBAs (i.e.
ACE, mini-ACE, CbD and mini-PAT). Your assessors must register with the
system and complete these assessments online. Your assessors can
continue to use the paper forms of the other WPBA tools, which may be
downloaded from the College website. You may already have completed a
number of the mandatory WPBAs on paper; you must now ask your
assessors to enter these assessments online, so they may be stored in the
system. You can keep other assessments that have been completed as
paper forms and can store them in your portfolio.
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
7