Information Pack - Specialty Selection Test (SST) Pilot
Specialty Selection Test (SST) Pilot
INFORMATION PACK
Page 1 of 10
Information Pack - Specialty Selection Test (SST) Pilot
Contents
Section 1: Introduction ......................................................................................................................................... 2
Section 2: One-page overview ........................................................................................................................... 2
Section 3: FAQs for Applicants .......................................................................................................................... 3
Section 2: FAQs for LETBs/Deaneries/Specialties ......................................................................................... 9
Section 1: Introduction
As part of the Medical and Dental Recruitment and Selection (MDRS) Programme, Health Education
England (HEE) is working with the devolved nations, the Work Psychology Group and academics in
the University College of London Medical School to pilot a Specialty Selection Test (SST). The SST
will be piloted across a number of different medical specialties, across the United Kingdom, and
includes a Clinical Problem Solving Test (CPS) and a Situational Judgement Test (SJT). The
purpose of this pilot is to evaluate whether these tests enhance the current methods of selection used
for ST1/CT1 level recruitment.
This information pack has been developed to assist with raising awareness about the SST pilot; the
objectives, the approach, and how you will be involved. The pack contains a one-page overview
document and Frequently Asked Questions (FAQs).
Your feedback is important to us, so please don’t hesitate to contact the team via email at:
[email protected] if you have further questions.
Section 2: One-page overview
The one-page overview document (appendix 1) is a high-level brief that covers the implementation
approach, who’s involved and why we are doing this. This document was distributed during August
2013.
Page 2 of 10
Information Pack - Specialty Selection Test (SST) Pilot
Section 3: FAQs for Applicants
1. What is the SST pilot and why are we doing this?
As part of the Medical and Dental Recruitment and Selection (MDRS) Programme, Health
Education England (HEE) is working with the devolved nations, the Work Psychology
Group and academics in the University College of London Medical School to pilot a
Specialty Selection Test (SST). The SST will be piloted across a number of different
medical specialties, across the United Kingdom, and includes a Clinical Problem Solving
Test (CPS) and a Situational Judgement Test (SJT).
The purpose of this pilot is to
evaluate whether these tests enhance the current methods of selection used for ST1/CT1
level recruitment.
We are seeking doctors’ help to participate and provide feedback about the pilot test. The
results from this pilot will enable a group of leading selection researchers to consider
whether there are any advantages with introducing this test and how the results correlate
with current selection methods. We will also continue to follow applicant progression for a
number of years, to ensure a long-term robust evaluation. Ultimately, we are assessing
whether the SST is a good predictor of a doctor’s performance throughout their training and
to ensure the selection process is reliable, valid, fair and cost-effective.
2. How will the SST pilot work?
We plan to pilot the SST between 13-25 January 2014 for applicants to specialty training at
ST1/CT1 level. On 11 December 2013, all applicants for ST1/CT1 will receive an email
invitation to take part in the Specialty Selection Test pilot. The test will be delivered online
through Pearson VUE test centres in the same way the General Practice (GP) test is run.
You will be asked to choose a time from a range of time-slots and a venue from one of
many Pearson VUE driving test centres across the United Kingdom (UK). These venues
are located within close proximity to hospitals across the UK.
The test itself will consist of a two hour and fifteen minute online test. It will be completely
computer based and the questions will be developed by leading specialists according to
person specifications and foundation programme curriculum. If you are already doing the
Page 3 of 10
Information Pack - Specialty Selection Test (SST) Pilot
GP test as part of your assessment for GP or Broad Based Training (BBT) you will not be
asked to do the additional test. If you are applying to a number of different specialities you
will only be required to sit the test once.
3. Who is eligible to sit the SST pilot?
Any doctor applying for ANY specialty training post at ST1/CT1 level is eligible to sit the
test for the Specialty Selection Test pilot.
All applicants will be asked to take part -
especially those in smaller specialties with fewer applicants - as we need to ensure that
enough trainees sit the pilot to provide enough data for a thorough assessment.
4. What will the SST pilot evaluate?
The pilot will evaluate the correlations of performance in the selection test to the selection
decision, and where applicable, to scores from currently used assessment tests as well. It
is important to note that the test results will not form part of the actual selection
process.
We aim to introduce a testing method that is fairer, more consistent, offers more
transparency and is less labour intensive, and so we will be assessing these components
too. By taking part in the pilot, your results and feedback will enable the team to evaluate
the selection process and introduce improvements to the way in which doctors are selected
for specialty training posts.
Comparisons to achievements in on-job performance and in professional exams will also
be made at a later date as ultimately, we want to assess whether the multispecialty test
results can predict how you perform during specialty training.
Page 4 of 10
Information Pack - Specialty Selection Test (SST) Pilot
5. Will the results contribute to recruitment scores or be passed on to regulating bodies?
Your results will not form part of the recruitment process nor will your results impact on your
fitness to practise. This is a pilot evaluation study only and as such, individual results will not
be shared with regulatory bodies.
Your results will only be seen by the research team who will evaluate the correlations of
performance in the selection test to the selection decision, and where applicable, to scores
from currently used assessment tests as well. It is important to note that the test results
will not form part of the actual selection process.
6. Which specialties are involved in the SST pilot?
A detailed list will be made available shortly. Discussions have taken place with Medical
Royal Colleges to be involved in and support the SST pilot, and most have confirmed their
support and willingness for their applicants to be involved in this pilot evaluation.
7. Why are we piloting a Specialty Selection Test – hasn’t this been piloted already?
Similar tests and pilots have been used before in many specialties, but not across
specialties – currently 7,000 out of the 12,000 doctors applying for specialty training take a
Clinical Problem Solving (CPS) and a Situational Judgement Test (SJT). The test we are
piloting will be similar to that used in General Practice (GP) and Public Health and similar
to the specialty specific tests that have been piloted in surgery, acute specialties, medicine
and other specialties. The test in GP was successfully implemented in 2007 and used to
date as part of the selction process, constituting 30% of a trainee doctor’s total recruitment
score. In comparison with other methods of selection, these tests have been shown to be
the best predictor of performance in the work place and likelihood of successful completion
of specialty training. Applicants consistently rate these tests as fair and appropriate to use
as sub-components of speciality selection.
Each of the devolved nations is focused on the National Health Services (NHS) objective to
recruit staff into the NHS with appropriate values and behaviours, as well as knowledge
and skills. The Specialty Selection Test will support the need to incorporate values-based
selection methods and is a possible way of meeting these requirements.
Page 5 of 10
Information Pack - Specialty Selection Test (SST) Pilot
This pilot is an evaluation of selection processes for the benefit of our patients and you, as
an NHS doctor.
8. How will I benefit from taking part in the SST pilot – what’s in it for me?
This pilot aims to improve the way doctors are selected in the future and this is your
chance to give your views on this selection process and make your voice heard!
By
participating in the pilot, you will also receive feedback on how you performed and go into
the draw to win one of five prizes, which includes a 32GB iPad and Kindle Fire HD.
As an applicant, your feedback will be viewed as very important and you will be able to
have your say on how appropriate you found the selection test to be in relation to selection
into specialty training, and whether you think it allowed you to demonstrate your aptitude
for specialty training. This information will then be used to inform the Specialties and other
stakeholders on whether these selection methods add additional value to the selection
processes currently used.
9. Why will the pilot test be across multi-specialties and not specific to the specialty that
I'm applying for?
The pilot of the SST is designed to be broad-based due to the nature of the overlap of
person specifications required by all specialties. A good example of this is the fact that all
specialties require trained doctors not just to be clinically competent, but also empathic and
sensitive with patients and colleagues, behaving with professional integrity, with the ability
to prioritise and cope well under pressure and work effectively in a team. All doctors who
have undertaken Foundation will be familiar with the Foundation Curriculum by which these
skills, values and behaviours are embedded.
The Specialty Selection Test does not seek to replace individual specialty specifications.
Each speciality will still have its own speciality specific elements within the selection
process.
Page 6 of 10
Information Pack - Specialty Selection Test (SST) Pilot
10. Have CPS and SJT tests been used before and if so, by whom?
Yes. The CPS and SJT test is successfully being used currently for applicants to General
Practice (GP) (since 2007) and Public Health (since 2009). Approximately 7000 out of 12,000
applicants to ST1/CT1 training already sit the test each year; though it is currently specific to
the specialty they are applying for.
A multispecialty pilot (of 800 applicants) was undertaken in 2010 across several ST1/CT1
specialties. The pilot test performed well, with the results correlating well with the selection
scores, but there wasn’t enough involvement across all specialties to determine whether the
test would be beneficial for all specialties. This is one of the reasons why we are undertaking
a Specialty Selection Test pilot across a wider range of specialties.
11. Is it mandatory to take part in the SST pilot?
It is not mandatory to take part in the SST pilot.
If you are applying for GP specialty training or Broad Based Training (BBT) then you will be
required to sit the GP selection test as it is part of the assessment centre for entry into GP
training. Questions from the SST pilot will be embedded in the GP test, but will not count
towards your score in the selection process. This will allow us to study how well the
multispecialty questions perform compared to items in the GP test.
12. Will travel expenses be reimbursed?
Yes, standard class travel expenses will be reimbursed for the SST pilot project up to the
value of £10.00. There is a large selection of Pearson VUE test centres, which are located
within close proximity to hospitals across the UK and should be easily accessible.
Page 7 of 10
Information Pack - Specialty Selection Test (SST) Pilot
13. Do I need to do the test if I’m applying for General Practice (GP) or any other specialty?
As indicated above, those applying to General Practice (GP) training and Broad Based
Training (BBT) who already undertake an online test will not need to sit an additional test for
the SST pilot. If you are applying to specialties other than General practice, we are seeking
your support to undertake the test and we will request your participation.
As such, your
involvement in the pilot would be greatly appreciated.
14. Will the mark I achieve be known to anyone involved in selection into my chosen
specialties?
Absolutely not! Your individual results will only be seen by the research team who will be
evaluating results to determine whether the SST pilot test adds value to the specialty
recruitment selection process.
15. How will the security of the data about me and my results be ensured?
The results and information obtained from this pilot will be anonymised and secured using the
same measures that are currently in place to protect selection results for General Practice
(GP). We will use secure file transfer protocols to manage your information, ensuring that
your data is kept completely secure.
16. Will the SST be used in the selection process for shortlisting during 2014 specialty
training recruitment?
Your performance in the piloting of the SST will not in any way affect your selection scores for
entry into specialty training in 2014 at ST1/CT1 level. It will not be used for shortlisting or at
any other stage of the specialty training process.
A poor performance in the pilot will not have any impact on your application to a specialty
training post at ST1/CT1 level.
Page 8 of 10
Information Pack - Specialty Selection Test (SST) Pilot
17. Where can I find out more information?
You can contact us at [email protected] for more information.
The Specialty
Training website will be updated regularly and the applicant guide is another useful source
where you can find more information about the pilot and what is involved.
Page 9 of 10
Information Pack - Specialty Selection Test (SST) Pilot
Appendix 1
A Specialty Selection Test is being piloted for entry into Specialty Training
As part of the Medical and Dental Recruitment and Selection (MDRS) Programme, Health Education
England (HEE) is working with the devolved nations, the Work Psychology Group and academics in the
University College of London Medical School to pilot a selection test that includes a Clinical Problem
Solving Test (CPS) and a Situational Judgement Test (SJT), for entry into Specialty Training.
Known as the Specialty Selection Test (SST) pilot, this will be piloted between 13-25 January 2014 for
applicants to specialty training at ST1/CT1 level. The results from this pilot will enable a group of leading
selection researchers to consider any advantages of introducing this test and how the results correlate with
the current selection methods used. We will share these results widely with stakeholders and continue to
follow applicant progression for a number of years for a long-term evaluation. Ultimately, we are assessing
whether the SST is a good predictor of a doctors' performance throughout their training and to ensure the
selection process is reliable, valid, fair and cost-effective.
Similar tests and pilots have been used before - currently 7,000 out of the 12,000 doctors applying for
specialty training take a CPS and a SJT1. Compared to other selection methods, these tests have been
shown to be the best predictor of performance in the work place and likelihood of successful completion of
specialty training.
The National Health Service (NHS) needs to recruit staff into the NHS with appropriate values and
behaviours, as well as knowledge and skills. This principle has been incorporated into NHS mandates and
organisation objectives. The Specialty Selection Test is a values-based selection method and is a possible
way of meeting these requirements. Applicants will be asked for their views after completing the test on
how appropriate they find the selection test in relation to selection into specialty training.
Discussions have taken place with Medical Royal Colleges and most have confirmed they wish their
applicants to be involved in this SST pilot. Applicants taking part in the pilot will sit the test only once,
irrespective of the number of specialties they apply to. All identifying information will be kept confidential.
The results will not form part of the selection process and will not be made available to selectors - it is a
pilot only.
Input from applicants is crucial and results will inform us about whether these selection methods add
additional value to the selection processes currently used. On 11 December, all eligible applicants will
receive an email invitation, once they have applied to a specialty, and will be able to choose a test centre
that is most convenient to them. This will be managed separately to the online application process for
recruitment into ST1/CT1 level.
As a member of a Local Education Training Board (LETB), Deanery or Royal College recruiting
organisation, you will receive further, specific, information regarding how this process will be managed.
HEE will offer support during the SST pilot from a staff member centrally within the Directorate of
Education and Quality (DEQ) who will respond to applicant queries specifically related to the pilot.
Frequently Asked Questions will also be available and the applicant guide will include relevant information
about the pilot.
For more information, please contact the team via email at: [email protected].
1
General Practice (GP) and Public Health use a CPS and a SJT test for specialty selection. Specialty specific tests have also been piloted
in surgery, acute specialties, medicine and other specialties. The test in GP was successfully implemented in 2007 and used to date as part
of the selection process; constituting 30% of a trainee doctor's total recruitment score.
Page 10 of 10