ASSESSING SURGICAL TRAINEES A MANUAL FOR SUPERVISORS AND TRAINEES Guidelines for assessing Surgical Trainees under clinical assessment The College particularly acknowledges the work done by the Royal College of Surgeons of England (RCSE), and the Royal College of Physicians and Surgeons of Canada (RCPSC), as well as Supervisors and Specialty Boards in developing, trialling and validating the assessment and management tools included in this manual. The assessment tools and the instructions for their use have been adapted from those sources, however copyright of their material remains with them. © Royal Australasian College of Surgeons 2007 TABLE OF CONTENTS SUPERVISION AND ASSESSMENT — AN OVERVIEW................................................................... 2 1.1 Aims of the College ............................................................................................................... 2 1.2 Aims of Assessment of Competence .................................................................................... 2 1.3 Supervision and Assessment of Trainees ............................................................................. 2 1.4 Comprehensive Assessment ................................................................................................ 2 In-training assessment ........................................................................................................................ 5 2.1 Needs Assessment ............................................................................................................... 5 2.2 Feedback............................................................................................................................... 5 2.3 Deficiencies ........................................................................................................................... 5 2.4 Developing a Plan ................................................................................................................. 5 2.5 The Written Warning ............................................................................................................. 6 2.6 Timing.................................................................................................................................... 6 2.7 Sign and Date ....................................................................................................................... 6 2.8 Regular Review ..................................................................................................................... 6 2.9 In-Training Assessment Form ............................................................................................... 6 Log Book............................................................................................................................. 12 ASSESSMENT TOOLS ..................................................................................................................... 13 1.5 Mini-Clinical Evaluation Exercise (Mini-CEX) ..................................................................... 13 1.6 RACS - Mini-Clinical Evaluation – SAMPLE Assessment Form ......................................... 14 1.7 Surgical DOPS (Directly Observed Procedural Skills) ........................................................ 15 1.8 RACS – Direct Observation of Surgical Skills (SURGICAL DOPS) SAMPLE Assessment Form ............................................................................................................................................ 19 1.9 Case-Based Discussion (CBD) ........................................................................................... 20 1.10 RACS – Case-Based Discussion (CBD) SAMPLE Assessment Form ........................... 21 1.11 360 Degree Survey or MINI-PAT (Peer Assessment Tool) ............................................. 23 4.8 RACS – 360-Degree SAMPLE Survey Form ...................................................................... 24 UNSATISFACTORY PERFORMANCE ............................................................................................. 28 1.12 The process ..................................................................................................................... 28 1.13 Chain of Responsibility .................................................................................................... 28 1.14 Managing Underperforming Trainees.............................................................................. 29 POLICIES .......................................................................................................................................... 33 1.15 Dismissal from Surgical Training ..................................................................................... 33 1.16 Grounds for appeal .......................................................................................................... 33 Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 1 SUPERVISION AND ASSESSMENT — AN OVERVIEW 1.1 Aims of the College As a fellowship based organisation, the Royal Australasian College of Surgeons commits to ensuring the highest standard of safe and comprehensive surgical care for the community we serve through excellence in surgical education, training, professional development and support. 1.1.1 Surgical training encompasses nine competences: • • • • • • • • • Technical Expertise Medical Expertise Judgment – Clinical Decision Making Communication Collaboration Management and Leadership Health Advocacy Scholar and Teacher Professionalism and Ethics Trainees are required to demonstrate competence across all nine areas. 1.2 Aims of Assessment of Competence Performance of a surgical trainee or international medical graduate undergoing clinical assessment is judged against predetermined, publicised standards. In cases where performance falls below this standard the aim of surgical training is to clearly identify the areas of unsatisfactory performance and to provide support, supervision and additional training to allow the trainee ∗ to meet the predetermined standards. 1.3 Supervision and Assessment of Trainees Supervision and assessment of Trainees by Surgical Supervisors is necessary to ensure quality of training, general progress, suitability to continue training, suitability to sit the Fellowship Examination, and the completeness of training. During training each Trainee will be the subject of in-training assessment reports and have a formal report at the end of each rotation. These assessments will be the responsibility of the Supervisor. 1.4 Comprehensive Assessment A comprehensive assessment of a Trainee can be provided by a combination of: • Mid-term and End-of-term assessment of Trainees using In-training Assessment forms • The Log Book • Workplace Assessment of Competence • Examinations The Assessment Plan (Table 1) indicates the variety of tools and the competencies for which they are most relevant. 1.4.1 In-training Assessment and Reports To assist with in-training assessment, assessment forms for Surgical Trainees can be accessed from the website. The form will not be considered valid unless signed by both the Trainee and the Supervisor. Most specialties require that reports are received within one month of the end of each rotation. Failure to do so could mean the rotation will be assessed as unsatisfactory. 1.4.2 Log Book Statistics Log Book statistics are compulsory for all Surgical Trainees and IMGs. ∗ The term ‘trainees’ in this booklet applies equally to ‘International Medical Graduates’ Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 2 Table 1 Assessment Plan – Aligning Assessment and Competencies Competencies Medical Expertise access and apply relevant knowledge to clinical practice maintain currency of knowledge apply scientific knowledge in practice recognise and solve real-life problems Assessment Tools Primary Other FE CBD CEX/CBD ITA FE FE DOPS/PBA LB DOPS/PBA LB DOPS/PBA DOPS/PBA DOPS/PBA DOPS/PBA 3600 CBD (AUDIT) LB LB/ITA LB CEX ITA/FE CEX/CBD ITA/FE CEX/CBD ITA/FE CEX CBD ITA/FE ITA/FE CBD ITA/FE CBD ITA/FE CBD ITA/FE CBD ITA/FE CBD ITA/FE CBD ITA/FE PBA CBD CBD PBA/CBD CBD ITA/FE ITA/FE ITA/FE ITA/FE ITA/FE CBD/ITA FE CBD/ITA FE CBD/ITA FE CBD/ITA FE FE Research Technical Expertise safely and effectively perform appropriate open surgical procedures consistently demonstrate sound surgical skills demonstrate procedural knowledge and technical skill at a level appropriate to their level of experience demonstrate manual dexterity required to carry out procedures adapt their skills in the context of each patient — each procedure maintain skills and learn new skills approach and carry out procedures with due attention to safety of patient, self, and others analyse their own clinical performance for continuous improvement LB ITA Judgement – Clinical Decision Making design and carry out effective management plans recognise the symptoms of, accurately diagnose, and manage common problems take a history, perform an examination and arrive at a well reasoned diagnosis efficiently and effectively examine the patient formulate a differential diagnosis based on investigative findings manage patients in ways that demonstrate sensitivity to their physical, social, cultural, and psychological needs recognise the most common disorders and differentiate those amenable to operative and non-operative treatment effectively manage the care of patients with trauma including multiple system trauma effectively manage complications of operative procedures and the underlying disease process accurately identify the risks, benefits, and mechanisms of action of currently used drugs indicate alternatives in the process of interpreting investigations and in decision making manage complexity and uncertainty with sound judgement consider all issues relevant to the patient advocate patient health identify and manages risk plan, and where necessary implement, a risk management plan organise diagnostic testing, imaging and consultation as needed select medically appropriate investigative tools and monitoring techniques in a cost-effective, and useful manner appraise and interpret results of investigations against patients’ needs in the planning of treatment critically evaluate the advantages and disadvantages of different investigative modalities evaluate the significance of data Communication Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 3 communicate effectively communicate information to patients (and their family) about procedures, potentialities, and risks associated with surgery in ways that encourage their participation in informed decision making communicate with the patient (and their family) the treatment options, potentials, complications, and risks associated with all treatment modalities communicate with and co-ordinate surgical teams to achieve an optimal surgical environment initiate the resolution of misunderstandings or disputes appropriately adjust the way they communicate with patients to accommodate cultural and linguistic differences and emotional status recognise what constitutes ‘bad news’ for patients (and their family) and communicate accordingly CEX ITA/FE CEX ITA/FE 3600 ITA 3600 ITA CEX ITA/FE CEX ITA/FE 3600 ITA 3600 ITA 3600 ITA 3600 ITA 3600 ITA/FE CBD ITA/FE CBD CBD ITA/FE ITA/FE 3600 ITA 3600 ITA CBD ITA/LB 3600 3600 3600 ITA ITA ITA CBD ITA/Research CBD ITA/FE Collaboration work in collaboration with members of an interdisciplinary team where appropriate develop a care plan for a patient in collaboration with members of an interdisciplinary team collaborate with other professionals in the selection and use of various treatment modalities assessing the effectiveness of each management option employ a consultative approach with colleagues and other professionals recognise the need to refer patients to other professionals Management and Leadership balanced decision making – see also Judgement – clinical decision making promote patient advocacy – see also Health Advocacy effectively use of resources to balance patient care and systemic demands identify and differentiate between resources of the health care delivery system and individual patient needs apply a wide range of information to prioritise needs and demands effectively assess and manage systemic risk factors manage and lead clinical teams – see also Collaboration is respectful of the different kinds of knowledge and expertise which contribute to the effective functioning of a clinical team direct and supervise junior medical staff effectively maintain accurate records contemporaneously maintain accurate and complete clinical records Health Advocacy promote health maintenance of patients promote health maintenance of colleagues look after their own health Scholar and Teacher recognise the value of knowledge and research and its application to clinical practice assume responsibility for own on-going learning draw on different kinds of knowledge in order to weigh up patient’s problems in terms of context, issues, needs, and consequences Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 4 critically appraise new trends facilitate the learning of others Research 3600 FE ITA 3600 3600 ITA ITA ITA ?? CBD ITA/FE CBD 3600 3600 3600 3600 ITA ITA ITA ITA ITA/LB Professionalism Appreciate the ethical issues consistently apply ethical principles regularly participates in audit identify ethical expectations that impinge on the most common medico-legal issues is accountable for their decisions and actions acknowledge their own limitations acknowledge and learns from mistakes act responsibly employ a critically reflective approach Key: DOPS Direct Observation of Procedures Recommended minimum 6 times per year, early in training (multiple raters are required) PBA Procedure Based Assessment Recommended minimum 4 times per year, later in training CBD Case Base Discussion Recommended minimum 4 times per year, (multiple raters are required) CEX Clinical Evaluation Recommended minimum 6 times per year, early in training 360 Degree Survey From people other than supervisors / surgeons 360 o ITA In-Training Assessment Form FE Fellowship Examination LB Log Book IN-TRAINING ASSESSMENT 2.1 Needs Assessment It is recommended that, at the commencement of each surgical rotation or period of oversight, the Surgical Supervisor of Surgical Training and the trainee undertake a needs assessment and set objectives for the trainee's forthcoming rotation. The trainee is required tomaintain a portfolio of his/her in-training assessments, which can be used at the time of the needs assessments. 2.2 Feedback Verbal feedback should be provided continuously throughout the rotation. 2.3 Deficiencies Deficiencies should be reported verbally to the trainee as soon as they are recognised. If the deficiency re-occurs, verbal reporting should be followed by a written report to the trainee outlining his/her deficiencies and a request for a meeting to discuss a plan for addressing the deficiencies. 2.4 Developing a Plan Together the Supervisor and trainee plan specifying specific goals to be achieved in remedying the deficiencies, remedial actions and a suitable timeframe. In order to ensure that appropriate records of the management of deficiencies, the use of the proforma for managing underperforming trainees is strongly recommended (see 5.2). Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 5 2.5 The Written Warning The written warning and plan for remedying poor performance should be signed and dated by both the supervisor and the trainee. This report should be appended to the formal six monthly in-training assessments. 2.6 Timing Most specialties required that formal in-training assessments are completed half way through and towards the end of each 6 month rotation. Reports are to be completed by the end of the rotation. 2.7 Sign and Date The in-training assessment must be signed and dated by both the trainee and the supervisor. 2.8 Regular Review The content of the in-training assessment report is subject to regular review. For most this is an educative exercise but if a performance is unsatisfactory then the Trainee is advised of his/her deficiencies orally during the previous training period and this is confirmed in the in-training assessment report. The Trainee is advised how to overcome deficiencies being experienced and is expected to correct these – see Section 5. If correction does not occur following due process, dismiss may occur. A decision to dismiss should be fully documented and in line with College policy. 2.9 In-Training Assessment Form Each specialty has developed an in-training assessment form (see following example) in which the key performance indicators and standards of competence are defined. Trainees are required to meet all of those standards, at every mid and end of term assessment. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 6 Royal Australasian College of Surgeons – An Example of End-of-Term In-Training Assessment PERIOD FROM: _____\____\_____ to _____\____\_____ NAME OF TRAINEE:________________________________________________________ No. of Days absent: __ Reason for absence (holiday/exam leave) _________________ NAME OF SURGICAL SUPERVISOR: _______________________________________ HOSPITAL: ______________________________________ (code) __ __ /__ __ / __ SURGICAL UNIT: ________________________________No. of surgeons on unit: EVALUATOR (completing this form): ________________________________________ Notes to evaluators on completing Evaluation Forms The competencies listed in the ‘Competent’ column are those which have been identified as being required of all trainees prior to graduation. Supervisors are to assess each trainee’s performance in each specified competence, using the four descriptors: Unsatisfactory Below the required standard for level of training Borderline Requires additional time, experience and/or additional training to reach the expected standard Competent Correctly demonstrates required competence – meets expected standard Excellent Consistently demonstrates an unusually high level of performance It is expected that the majority of trainees will fall in the ‘Competent’ category for most competencies. Supervisors are asked to write in the right hand column the letter U, B, C, E that best reflects the trainee’s performance during the training period for each specified competency Notes on the responsibilities of Surgical Supervisors in managing Trainees Surgical Supervisors play a crucial role in the continuing formative assessment of trainees. It is important that care and attention be given to Trainee’s performance of the identified competencies throughout their training. If a Supervisor is concerned about a trainee they are advised to record these concerns at an early stage and to ensure that both major and minor incidents are contemporaneously recorded so that any emerging pattern may be identified Surgical Supervisors are obliged to inform a trainee at an early stage of any concerns they might have. Supervisors should discuss their concerns with the trainee in a matter-of-fact and confidential manner, and recording the outcome of any discussions or interviews they might conduct. The outcome of such discussions or interviews should be a written plan of action to remedy the identified area(s) of concern, signed by both the Supervisor and Trainee All consultants in a Unit are to complete a form for both Mid and Final Term Assessments. After completing their reports Supervisors should discuss their reports and where possible come to a consensus. If this is not possible please either indicate on the submitted form and in the 'Comments' section where there are differences, or submit an individual report. If the Trainee does not participate in any discussion/interview/plan of action in a timely fashion the Supervisor must convey their concerns in writing to the Trainee and to the Chairman of the Regional Board in their State/Country. Trainee Responsibilities The trainee is required to ensure that separate assessment forms are filled in by each Consultant on the unit. The College must receive completed assessment forms and log book summary data no later than one month from the end of the term. Unless there are extenuating circumstances late lodgement of these forms will result in the 6 month term not being approved as a satisfactory training experience. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 7 MEDICAL EXPERTISE – able to access and apply relevant knowledge to clinical practice Unsatisfactory Borderline Competent Excellent (U) (B) (C) (E) Poor knowledge base Needs direction to Maintains currency of Outstanding knowledge Significant study knowledge Knows common areas in deficiencies or poor Struggles to Applies scientific knowledge to depth perspective Aware of the unusual correctly/ accurately patient care Allows deficiencies to Excellent application of apply scientific Reads appropriately, asks for persist knowledge in clinical knowledge to information and follows-up patient care Recognises and solves real-life situation problems Mid Term End Term TECHNICAL EXPERTISE – able to safely and effectively perform appropriate surgical procedures Fails to acquire Is inconsistent in Consistently demonstrates Excellent and advanced appropriate skills retaining procedural acquisition, practice and abilities in procedures and despite repeated knowledge/ skills retention of sound procedural techniques instruction/ practice. knowledge, surgical skills and Lacks attention to Excellent pre-operative Too hasty or too slow. detail. techniques for level of training preparation Rough with tissue. Hesitant. Poor manipulative Slow in learning Demonstrates manual Outstanding technician skills new skills dexterity required to carry out Fluent and always in procedures control Poor hand/eye Lapses in dexterity coordination Good hand/eye coordination Meticulous Unable to adapt skills Ongoing Adapts their skills in the Extremely good at and techniques weaknesses context of each patient—each adapting skills for varying procedure operative situations Struggles to adapt skills to different Excellent surgical contexts judgement Maintains skills Seeks opportunities to Lacks enthusiasm Fails to improve Effective in learning new skills learn new skills. and/or initiative to skills and/or learn participate and/or from experience learn Lacks care and Requires close Approaches and carries out Outstanding clinician diligence in approach supervision procedures with due attention Constantly aware and to safety of patient, self, and responds to patient, self ‘Near enough is good enough’ others and team members As surgical assistant Has lapses of Follows the operation with Anticipates the needs of fails to follow concentration guidance from the operator the operator & responds operation accordingly Ignores/fails to follow Occasionally Consistently analyses their Accurate in self-appraisal, up problematic acknowledges/ own clinical performance for excellent insight performance follows up on continuous improvement Seeks and accepts problematic criticism & responds Little recognition of Learns from feedback from performance appropriately deficiencies in skills others or techniques Ignores feedback Aware of own skill limitations JUDGEMENT – appropriate clinical decision making, ordering of investigations, consultation with other health professionals and patient management Incomplete or Hesitant or Takes a history, performs an Precise, thorough and inaccurate inconsiderate of examination, and arrives at a perceptive patient well-reasoned diagnosis Poor basic skills Lacks attention to Efficiently and effectively detail. examines the patient Incomplete/inaccurate Poor presentation/ Recognises symptoms, Accurate and efficient recognition of discussion of clinical accurately diagnose, and Considers a wide range of significant symptoms cases manages common disorders symptoms and factors Insightful perspective in Significant errors/ Occasional Differentiates those conditions case discussions omissions in inaccuracies in amenable to operative and diagnosis diagnosis non-operative treatment Frequent Sometimes Concise and correct on clinical inaccuracies history, confuses priorities details signs or diagnosis Arrives at appropriate conclusions in case presentations Inadequate or Unable to Selects appropriate Always selects optimal Inappropriate, poor appropriately justify investigative tools and investigations selection and/or use of selected monitoring techniques costExcellent interpretation interpretation investigations effectively Safe, efficient and cost effective approach to use Disregards patient’s Occasional errors in Appraises and interprets of investigations needs or interpretation that results of investigations circumstances could lead to patient against patient’s needs in the problems planning of treatment Disregards system Critically evaluates the needs advantages and disadvantages of different investigative modalities Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 8 Unable to make a decision Unable to suggest alternative interpretations Some suggested alternatives are inappropriate Ignores data that does not fit interpretation Presentation unclear, disorganised Poor record keeping – incomplete, disorganised, irrelevant, illegible – not up-to date Records difficult for others to follow Disinterested or indifferent approach to patients Fails to grasp significance or respond accordingly Copes poorly in situations of stress and/or complexity Under or over reacts Culturally incompetent Ignores/overlooks some patient’s needs Inadequate planning Inadequate involvement in pre & post-operative care Fails to grasp significance of symptoms or respond accordingly Slow to anticipate/ manage complications Can show signs of stress when managing trauma patients Slow to call for assistance Under estimates complexity and/or risk factors Formulates a differential diagnosis based on investigative findings Evaluates the significance of data Indicates appropriate alternatives in the process of interpreting investigations and in decision making Precise, well organised, thorough, systematic, focused • Presentation of findings • Indicates relevant alternatives • Decisions based on data Clear & concise presentation of findings Contemporaneously maintains Perceptive of relevant accurate and complete clinical information / data for records documentation Precise and focused Records very easily accessible Complies with required organisational structure Manages patients in ways that Excellent and highly demonstrate sensitivity to their developed ability to physical, social, cultural, and manage & interact with psychological needs patients and to anticipate and/or respond to their Considers all issues relevant to the patient needs Effectively manages the care Anticipates possible risks of patients with trauma and/or complications including multiple system In stressful situations always maintains orderly trauma approach and Maintains controlled approach demonstrates sound & demonstrates sound judgment judgement during times of stress/complexity Plans, and where necessary Outstanding clinician who implements a risk • anticipates possible management plan. risks/complicationsidenti Conscientious and reliable fies problems early follow-up • follows-up meticulously Effectively manage • coordinates and uses complications—operative other personnel procedures & underlying effectively disease process Identifies and manages risk Manages complexity and uncertainty COMMUNICATION – able to communicate effectively with patients, peers and other health professionals Trusted by patients. Listens Limited discussion Disliked by patients Possesses excellent well. with patients around because of poor interpersonal skills. Communicates with patients issues of informed interpersonal skills. Develops excellent (and family) about procedures, consent and/or Bad listener rapport with patients & potentialities, and risks treatment options. Poor communicator team members. associated with surgery in ways Inspires confidence Increases patient that encourage their anxieties. Patients delighted to be participation in informed Patients remain looked after by this decision making. confused or unclear trainee. Communicates with patients and/or unable to Demonstrates empathy (and family) the treatment follow instructions. appropriately. options, potentials, complications, and risks associated with all treatment modalities. Recognises ‘bad news’ for patients and relatives & modifies communicates. Limited perception of Unaware of patient’s Appropriately adjusts the way Always interacts patient’s perspective needs they communicate with patients effectively with patients or communication Unable to & relatives to accommodate according to their social needs communicate under cultural and linguistic & health needs varying differences and emotional conditions/situations status COLLABORATION – able to collaborate effectively with members of an interdisciplinary team where appropriate Refuses to facilitate Poor relationship with Good rapport with nursing and Always willing to help team function peers and other other medical staff. Willing to even if personally professionals help inconvenient Does not acknowledge the Excellent working Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 9 contributions of others May undermine team members or function Causes disruption/problems Fails to recognise own disruptive behaviour Reluctant/unable to work as a multidiscipline team member Self-focused Unreliable Fails to seek assistance with issues of patient care Ignores or is unaware of their own limitations Reluctant to offer assistance to other team members Ignores or fails to acknowledge misunderstandings Lacks understanding of contributions of other professionals to patient care Works effectively with some team members but not others Slow in referring patients to other professionals Employs a consultative approach with colleagues and other professionals relationship with other professionals Always supports colleagues and junior staff Communicates effectively with and co-ordinate surgical teams to achieve an optimal surgical environment Initiates the resolution of Effectively diffuses any misunderstandings or disputes problems in the surgical with peers, colleagues, and team others Respectful of & appreciates different kinds of knowledge and expertise which contribute to effective functioning of a clinical team Develops a patient care plan in collaboration with members of an interdisciplinary team Excellent team member Extremely knowledgeable about the contribution of different fields of care Aware of and seeks the contribution of different fields and refers patients in a timely and appropriate manner Collaborates with other professionals in the selection/ use of various treatments assessing the effectiveness of options Recognises and facilitates referral of patients to other professionals MANAGEMENT AND LEADERSHIP – able to effectively use health resources to balance patient care and system demands Unaware of Lacks insight into the Identifies and differentiates Willing to contribute to management impact of system between resources of the health services constraints and/or demands health care delivery system and management expectations individual patient needs. Poor interaction with Uses resources very Effectively assesses and and/or supervision effectively for patient Reluctant to take on manages systemic risk factors and management of care balanced with any management junior medical staff patient need responsibility Applies a wide range of Wasteful of information to prioritise needs Excellent role model for resources and demands junior medical staff, all ways offers support for Directs and supervises junior medical staff effectively junior medical staff HEALTH ADVOCACY- acts as an advocate for the patient to achieve optimal health outcomes Ignores/jeopardises Poor care of own Promotes health maintenance Maintains high level of own or colleagues health of colleagues fitness and encourages health or well-being Looks after own health others Advocates patient health Takes little interest Limited knowledge of Very knowledgeable and Discusses causal health issues in patient health causal issues relating active in advocating with patient beyond surgery to patient health patient health including preventative measures SCHOLAR AND TEACHER – recognises the value of knowledge and research and its application to clinical practice and teaches others effectively Assumes responsibility for own Reading of research Little evidence of Always keen to discover learning /texts is undirected reading texts or new knowledge Draws on different kinds of Has difficulty applying journals Takes extra courses & knowledge in order to weigh up knowledge to practice Needs direction to learning opportunities patient’s problems- context, study issues, needs & consequences Critically appraises new trends in General Surgery Avoids teaching if Ineffective as a Facilitates the learning of Enthusiastic/inspiring possible. teacher others teacher Logical and clear Poorly prepared, Competent and well prepared poorly delivered in teaching others Excellent teaching skills PROFESSIONALISM – displays the professional and ethical behaviours expected of a surgeon Behaviour Little knowledge / Consistently applies ethical Highly conscientious inconsistent with interest in ethical or principles Anticipates areas where ethical ideals medico-legal issues medico-legal issues may Identifies ethical expectations that impinge on common medico- arise legal issues Late, idle, Occasionally difficult Acts responsibly Applies self beyond the Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 10 unreliable, forgetful Off-loads work onto others to contact or leaves tasks incomplete Dependable, conscientious Always completes tasks ‘call of duty’ Copes poorly under stress ‘Disappears’ when problems arise Pays little regard to clinical audit Anticipates and remains efficient “when the going gets tough” Seems to thrive on pressure Has problems acknowledging/ recognising mistakes Unable to accept criticism Has inaccurate view of own performance Only accepts criticism from some Regularly participates in clinical audit Willing to undergo close scrutiny Responds appropriately to stress Acknowledges & learns from mistakes Accountable for own decisions/actions Recognises & acknowledges their limits Employs a critically reflective approach Over confident Prompt response to criticism marked improvement and positive change Has great insight into their level of performance Was a mid-Term Assessment carried out Was remedial activity required with written plan of action? Has there been significant improvement in relevant areas of performance? YES / NO YES / NO YES / NO Has this trainee been rated as less than competent in any areas If Yes it must correlate with ratings given above Have each of those areas been discussed with the trainee? YES / NO YES / NO Have those areas been identified and worked on during the term? YES / NO Provide further information on areas rated less than competent (If insufficient space attach separate document) Note: Details of area(s) of less than competent performance must be fully documented and attached to this assessment form OVERALL RATING (please circle appropriate box) Unsatisfactory Borderline Competent Excellent Borderline Competent Excellent RATING of LOG BOOK STATISTICS Unsatisfactory Recommendations regarding Future Training (Circle appropriate number/s) 1. Trainee should continue in a Training Position 2. Due to less than satisfactory performance the Trainee requires remediation. recommendations are appended Written 3. Due to continuing unsatisfactory performance that has not been rectified, the Trainee requires further counselling. A written documentation of this is appended. UNIT SURGEON ____ (print name) _______________________ (signature) ______(date) SUPERVISOR _______________________ (print name) _______________________ (signature) ______(date) SUPERVISOR _______________________ (print name) _______________________ (signature) ______(date) Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 11 SUPERVISOR _______________________ (print name) _______________________ (signature) ______(date) TRAINEE I AGREE WITH THIS ASSESSMENT: (signature)_______ (date) YES / NO Important Note: Trainees should ensure that this Trainee Evaluation Form, together with a copy of the log book summary are distributed as follows: 1. Original assessment and log book summary forms should be sent to the Royal Australasian College of Surgeons, Spring Street, Melbourne, Vic 3000 2. Copies of the above should be made and retained by the trainee for their portfolio records 3. Copies of the above should be made and retained by the assessing surgeon 4. One copy of the above should be made and sent to Hospital Supervisor of Advanced General Surgical Training 5. One copy of the above should be made and sent to Chairman, Regional Board in your State/Country. 6. A score less than Competent(C) in any category will be discussed by the Regional Board. LOG BOOK A Surgical Trainee is required to keep a record of the procedures they have undertaken in an official Log Book. The Log Book has been designed for the purpose of recording experience and in permitting an audit of the performance of the trainee and the unit in which they work. It will also assist in evaluation of the training post. The format of the operative Log Book is specific to each specialty. Log books may be obtained from the College’s Department of Specialty Training and Societies, or the relevant College Specialty website, as required. For Specialist Surgical Training in Neurosurgery, Otolaryngology Head and Neck Surgery, Orthopaedic Surgery, and Urology, log books may also be obtained from the relevant Society/Association. To assist in compiling the log, the trainee is advised to keep a note book to record the management of each patient in which the trainee plays a role, and entries are to be made concurrently with hospital management requirements. The note book information can then be used to compile the Log Book. In most specialties the Supervisor is required to review the Log Book every three months. This will be done at the same time as the in-training assessment forms are completed so that the log book rating can be accurately recorded. The Log Book remains the property of the Trainee (IMG). Trainees should comply with relevant National Privacy Principles (Commonwealth of Australia or New Zealand) regulating the collection, storage, access to, use, disclosure and de-identification of personal information. Summaries of Operative Experience are to be submitted by trainees together with the Intraining Assessment form.. Copies of the Summaries of Operative Experience should be retained by the Trainee and the Trainee’s Supervisor. On application to present for the Fellowship Examination, the Chairman of the relevant Surgical Board will review a summary of a candidate’s training based on progressive reports and an inspection of the Log Book, together with any other material pertinent to the candidate. Surgical Trainees and IMGs in second and subsequent years may not be registered unless they have completed and submitted Summaries of Operative Experience for the preceding year of training. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 12 ASSESSMENT TOOLS 1.5 Mini-Clinical Evaluation Exercise (Mini-CEX) Overview 1.5.1 Number of assessments The number of assessments is decided by each Specialty Training Board. Most specialties have nominated Mini-CEX assessment as a requirement of all their SET1, or SET1&2, trainees (see specific specialty regulations) The number of assessments conducted in each rotation and the trainee performance should be refected in the in-training assessment report. 1.5.2 What is mini-CEX? The mini-CEX is designed to assess skills essential to the provision of good clinical care and to facilitate feedback in order to drive learning. The assessment involves an assessor observing the trainee interact with a patient in a normal clinical encounter. The assessor’s evaluation is recorded on a structured checklist which enables the assessor to provide verbal development feedback to the trainee immediately after the encounter. • o o o o • o o A. Surgical trainees can use it to: Assess themselves against important criteria as they learn and perform practical tasks Build on assessor feedback Chart their own progress Produce evidence of competence for final review The method also serves the purposes of: Developing the dialogue between the trainee and their Supervisor Forming a portfolio of formative assessments at the completion of the rotation as to the level of performance achieved. How does it work? The process is trainee led; the trainee chooses the timing, the problem under the guidance of the Supervisor through the learning agreement. It is the trainee’s responsibility to ensure completion of the required number and type of assessments by the end of the rotation. However, a Supervisor may instigate an assessment if there are any concerns. The assessor observes the trainee undertaking the clinical encounter, doing what they would normally do in that situation. After completing the observation and evaluation the assessor provides immediate feedback to the trainee. Feedback generally takes about 5 minutes. o o Summary Observed clinical encounter evaluated against good clinical practice Evaluation of a trainee’s ability to communicate, examine, reason and organise when encountering clinical problems. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 13 1.6 RACS - Mini-Clinical Evaluation – SAMPLE Assessment Form Surname ................................................. First name……………………………………………….. Assessment date…………………… iMIS ID number................................................. Specialty: Cardio General Neuro Ortho OHNS Paed P&RS Urol Vasc Hospital ................................................................................................................................................... Clinical setting: ICU Emergency Department Type of case: New case Other ………………………………………….. Follow-up Focus of clinical encounter: History Diagnosis Management Complexity of case: Low Average High Explanation Assessor’s position: Consultant: ................................. Other health care professional: ................................................................................................................................................... Please assess and mark the following areas in relation to what you expect, given the trainee's stage of training: 1. History taking 2. Physical Examination 3. 4. 5. 6. Unsatisfactory or potentially dangerous Borderline, marginal, or needs attention Satisfactory Demonstrating a significantly higher level of skills than would be expected Not observed / not applicable Communicates to patients (and their family) about procedures, potentialities, and risks to encourage their participation in informed decision making Adjusts the way they communicate with patients for cultural and linguistic differences and emotional status Recognises what constitutes ‘bad news’ for patients (and their family) and communicates accordingly Recognises the symptoms of, and underlying significance of findings for common problems 7. 8. Organisation / Efficiency 9. Overall Clinical Care Suggestions for development ………………………………………………………………………………………........................................ …………………………………………………………………………………………………………….……… ………………………………………………………………………………………………………………… Other comments ………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………… Agreed action………………………………………………………………………………………………… Assessor’s signature: ………………….............. Assessor’s name……………………........................... Signature of person being assessed ………………….......................................................................... Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 14 1.7 Surgical DOPS (Directly Observed Procedural Skills) Overview 1.7.1 Number of Assessments The number of assessments is decided by each specialty training board. Most specialties have nominated DOPS assessment as a requirement of all their SET1, or SET1&2, trainees (see specific specialty regulations) The number of assessments conducted in each rotation and the trainee performance should be refected in the intraining assessment report. 1.7.2 What Is Surgical DOPS? • Direct Observation of Procedural Skills in surgery (Surgical DOPS) is a method of assessing competence in performing diagnostic and interventionist procedures during routine surgical practice. It also facilitates feedback in order to drive learning. • The assessment involves an assessor observing the trainee perform a practical procedure within the work place. The assessor’s evaluation is recorded on a structured checklist which enables the assessor to provide verbal developmental feedback to the trainee. • Surgical trainees can use the method to: o Assess themselves against important criteria as they learn and perform practical tasks o Build on feedback from a previous assessment o Chart their own progress o Produce evidence of competence for final review • The method also serves the purposes of: o Informing the ITA process via the portfolio of formative assessments at the completion of the rotation as to the level of performance achieved. • The surgical Supervisor will be trained in the use of surgical DOPS. Individual DOPS will be scored only for the purposes of providing feedback to the trainee. The overall rating on any one assessment can only be undertaken if the entire procedure is observed. A judgement will be made mid-term and at completion of the rotation as to the level of performance achieved by means of the ITA. • It isemphasised that the most important use of this system is “formative” i.e. to provide feedback to the trainee and assessor, and that the surgical DOPS form can be used routinely any time the Supervisor supervises a trainee carrying out a procedure. The aim isto make the tool part of routine training practice. Q. How does it work? A. The process is trainee led; the trainee chooses the timing, the procedure and assessor under the guidance of the assigned Supervisor through the learning agreement. It is the trainee’s responsibility to ensure the required number and type of assessments are completed by the end of the rotation. However, a Supervisor may instigate an assessment if these areas are a concern. The person carrying out the assessment should observe the trainee undertaking the procedure and doing what they would normally do in that situation. After completing the observation and evaluation the assessor provides immediate feedback to the trainee. Feedback takes about 5 minutes. Summary o Two or three observed procedures in each rotation depending on the period and the requirements of the Specialty o Trainee chooses timing, procedure and observer according to their learning agreement Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 15 DOPS – Notes for Assessors A. Can I beasked to be an assessor? You need not have prior knowledge of the trainee. Ideally you should have been approved as an assessor for the rotation and \ have had training in the use of surgical DOPS and expertise in the chosen procedure. Q. How doesit work? A. The process is trainee led. The trainee has chosen you to assess them. You observe the trainee undertaking the procedure and record your observations on the structure checklist. The trainee should do what they would normally do in the situation. After completing the assessment form, you provide feedback to the trainee, which will take about 5 minutes. o Applicable procedures Please ensure that the patient is aware that surgical DOPS is being carried out. o The encounter should be representative of the trainee’s workload. Trainees should only be observed undertaking procedures normally expected of them and in their usual work environment. This will be one of a comprehensive list of procedures relevant to the specialty and placement. • Specific points regarding form completion: o Number of times procedure previously performed by trainee. Please ask the trainee for their estimation/log book. o Difficulty of procedure Please score the difficulty of the procedure for the level of a trainee completing the core surgical curriculum program. o Definition of Easier than usual: uneventful procedure without any “usual” problems. o Definition of More difficult than usual: unexpected problems, unrelated to the expertise of the trainee. o Assessor Training Please read the entire form and guidance notes for both the trainee and assessor. Indicate that you have done this on the form together with any type of training you have had. o Satisfaction with DOPS This is about your satisfaction with the process not with how the trainee has performed on this occasion. o Using the scale Please use the full range of the rating scale. Comparison should be made with a doctor who is ready to complete that level. It is expected that some ratings below “meets expectations” will be in keeping with some trainee’s level of experience. Do not complete the overall rating unless you have observed the entire procedure. Feedback In order to maximise the educational impact of using Surgical DOPS, you and the trainee need to identify agreed strengths and areas for development. This needs tobe done sensitively and in a suitable environment. Feedback is best given immediately after the assessment. After the assessment and feedback You must sign and date the assessment form. Do not make copies of it. o Your responsibility for assessment will have ended at this point unless a discrepancy arises with a rating, which you may be asked to verify. o Trainee evaluations will be collated so that they build into an overall profile that can be added to the trainee’s portfolio. The overall profile will inform the trainee’s final review. It will also be part of the feedback for ongoing development. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 16 o At the end of the placement it is the trainee's responsibility to have acquired the required number and type of Surgical DOPS specific to their learning needs. DOPS – Notes for Trainees Q. Who can you ask to assess you? A. . This depends on the number of Surgical DOPS you are required to do in each rotation in your specialty, and the number of people suitable to make the assessment. Ideally you choose a different observer for each procedure. Ideally, each assessor will be approved as a trained assessor and have expertise in the procedure. Try to ensure that one of your observers during each rotation is your assigned Supervisor. You will also find that it helps you learn to assess yourself. The structured checklist offers you the criteria against which you will be assessed. You need toreflect upon the meaning of each item in order to understand what is required of you. Self-assessment can help you determine any gaps in your understanding or ability which you can bring to discussion with your assigned Supervisor and other senior colleagues. You can record self-assessments in the reflective practice section of your portfolio to monitor your progress (if you prefer, these can be removed before your annual review). Q. A. What is the purpose of being assessed? DOPS is designed to provide feedback that will help you improve your work performance. Therefore you should be assessed undertaking procedures normally expected of you in your usual working environment where you would normally do the procedure. It is important that you choose different procedures which cover the curriculum competences. Aim to provide evidence of competence in at least three different procedures within each year (or a representative number within your rotation). A comprehensive list of procedures relevant to your rotation should be available. You can arrange for assessment of any one procedure to be repeated until your skill reaches an acceptable standard or above. Details of all assessments must be recorded in your portfolio. Depending on the requirements of your specialty, by the completion of each year of training you should have been assessed in a range of procedures by a range of assessors. Q. A. Which procedures will be applicable? In the specialty regulations, modules and logbooks there are a comprehensive range of procedures relevant to your specialty and rotation. At your initial meeting with your Supervisor at the beginning of a rotation you can discuss the procedures that you need to cover within your rotation to demonstrate the competence required. Q. A. When can you use DOPS? Surgical DOPS can be used every time you carry out a practical procedure. It can be used at any time of day or night. You could, for example, ask your Supervisor to come with you to complete a procedure. It is your responsibility to ensure that an adequate number of assessments have been completed during each rotation. You can request more assessments than the required minimum as this provides you with more feedback to work with. Your assigned Supervisor may instigate more than the required minimum assessments if there are areas of concern, especially later on in a rotation. Feedback In order to maximise the educational impact of using Surgical DOPS, you and your Supervisor need to identify agreed strengths and areas for development. It is essential that you reflect on this feedback and try to overcome any weaknesses through further Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 17 dedicated learning, observation and practice. You can discuss methods of doing so with your Supervisor or other trainers. After the Assessment and Feedback You sign to state your satisfaction with the process at the bottom of the form. This is about your satisfaction with the process not with how you have done on the occasion of your assessment. You will be responsible for ensuring that all of the assessments aresubmitted to your Supervisor so that the information can be included in your end-of rotation report. Ratings will be collated so that by the end of your placement you will be given an overall assessment profile. You are required to maintain a file all feedback in your portfolio, which will be assessed at the end of rotation review. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 18 1.8 RACS – Direct Observation of Surgical Skills (SURGICAL DOPS) SAMPLE Assessment Form Surname ..................................................... First name……………………………………………. Assessment date…………………… iMIS ID number............................................. Specialty: Cardio General Neuro Ortho OHNS Paed P&RS Urol Vasc Hospital................................................................................................................................................. Clinical setting: Theatre ICU Emergency Department Other ...................................... Name of procedure: ……………………………………………………………..……….............................. Difficulty of procedure: Easier than usual Average More difficult than usual Number of times this procedure has been performed by this trainee prior to this occasion ………… Assessor’s position: Consultant ......................... Please assess and mark the following areas in relation to what you expect, given the trainee's stage of training: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Unsatisfactory or potentially dangerous Other health care professional……………… Borderline, marginal, or needs attention Satisfactory Demonstrating a significantly higher level of skill than would be expected Not observed / not applicable Ensure that it is the correct patient and that they have provided informed consent Prepares for procedure according to an agreed protocol Demonstrates good asepsis technique, and safe use of instruments/ sharps Performs technical aspects competently. Shows respect for tissue, identifies correct tissue planes Demonstrates manual dexterity required to carry out procedure Adapts procedure to accommodate patient and/or unexpected events Is aware of own limitations and seeks assistancewhen appropriate. Has insight. Completes required documentation (written or dictated) Analyses their own clinical performance for continuous improvement. Self-critical Overall ability to perform whole procedure Suggestions for development ………………………………………………………………………………………........................................ …………………………………………………………………………………………………………….……… ………………………………………………………………………………………………………………… Other comments …………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………… Agreed action: ……………………………………………………………………………………………………………. Assessor’s signature: …………………....................... Assessor’s name…………………….................. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 19 Signature of person being assessed ………......................................................................................... 1.9 Case-Based Discussion (CBD) Overview What is CBD? Case-based discussion (CBD) is designed to assess clinical judgement, decision making and the application of medical knowledge in relation to patient care for which the trainee has been directly responsible. It facilitates feedback in order to drive learning. CBD is not focused on the ability to make a diagnosis. It uses patient records as the basis for dialogue between a trainee and Supervisor (or clinical group) for systematic assessment and structured feedback about clinical cases that offer a challenge to the trainee (rather than routine cases). It enables the trainee to explain the complexities involved and reasoning behind choices they made. It also enables the discussion of the ethical and legal framework of practice. As the actual record is the focus for the discussion, the assessor can also evaluate the quality of record keeping and the presentation of cases. CBD is not a viva style assessment. Q. A. • Surgical trainees can use the method to: o Assess themselves against important criteria (especially that of clinical reasoning and decision-making) as they learn and perform practical tasks o Build on assessment feedback o Chart their own progress o Produce evidence of competence for final review • The method also serves the purposes of: o Developing the dialogue between the trainee and Supervisor o Forming a portfolio of formative assessments at the completion of the rotation as to the level of performance achieved How doesit work? The process is a structured discussion between the trainee and Supervisor (or clinical group) about clinical cases and how they are being managed by the trainee. The process is trainee led which means that trainees should ensure that their Supervisor is aware of issues within their cases that offer learning opportunities for discussion and that the required number of assessments are completed by the end of the rotation. Because learning is guided by the Supervisor through the learning agreement, cases may also be initiated by the Supervisor. The Supervisor can discuss the case in depth with the trainee, talking through what occurred, considerations and reasons for actions. Most assessments willtake no longer than 15-20 minutes. After completing the discussion and filling in the assessment form, the Supervisor provides immediate feedback to the trainee. Feedback will take about 5 minutes. o o Summary: Structured discussion of the challenging clinical cases which have been managed by the trainee Allows trainee’s decision-making and clinical reasoning to be explored in detail. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 20 1.10 RACS – Case-Based Discussion (CBD) SAMPLE Assessment Form Surname ..................................................... First name……………………………………………. Assessment date…………………… iMIS ID number................................................. Specialty: Cardio General Neuro Ortho OHNS Paed P&RS Urol Vasc Hospital..................................................................................................................................................... Clinical setting described: ……………………………………………………………………………………… Type of case /problem: ……………………………………………………………………………..………....... Complexity of case: Low Assessor’s position: Consultant Average Other health care professional…………………………………….. Please assess and mark the following areas in relation to what you expect, given the trainee's stage of training: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 14. 15. 16. High Unsatisfactory or potentially dangerous Borderline, marginal, or needs attention Satisfactory Demonstrating a significantly higher level of skill than would be expected Not observed / not applicable Diagnostic skills Communication with patients demonstrates sensitivity to their physical, social, cultural, and psychological needs Recognises the most common disorders and differentiates between those amenable to operative and non-operative treatment Identifies and manages complications of operative procedures and the underlying disease process Identifies and manages risks including planning for risk management Indicates alternatives in the process of interpreting investigations and in decision making Considers all issues relevant to the patient and differentiates between health care delivery resource and individual patient needs to prioritise needs and demands Evaluates and selects appropriate investigative tools and monitoring techniques in a cost-effective, and useful manner Appraises and interprets results of investigations against patients’ needs Maintains accurate and complete clinical records Identifies ethical expectations that impinge on medico-legal issues Assumes responsibility for own on-going learning Is accountable for own decisions and actions Overall clinical judgement Suggestions for development ………………………………………………………………………………………............................. …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… ……………………………………………………………………………………….............................. …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 21 …………………………………………………………………………………………………………… Other comments …………………………………………………………………………………………………………….. …………………………………………………………………………………………………………….. …………………………………………………………………………………………………………….. …………………………………………………………………………………………………………….. …………………………………………………………………………………………………………….. Agreed action: ……………………………………………………………………………………………………………..…….… …………………………………………………………………………………………………….………….…… …………………………………………………………………………………………….……………….……… …………………………………………………………………………………….…………………….………… ……………………………………………………………………………. Assessor’s signature: …………………………… Assessor’s name……………………………………… Signature of person being assessed .............................................................………………………… Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 22 1.11 360 Degree Survey or MINI-PAT (Peer Assessment Tool) Overview 1.11.1 Minimum number of assessments • One by the required number of assessors, usually towards the end of a rotation. 1.11.2 Number of assessors required: • At least five 1.11.3 What is the mini-PAT? The mini-PAT is a method of assessing competence within the remit of a team. It also facilitates feedback in order to drive learning. As part of a multi-professional team surgical trainees work with other people who have complementary skills. They are expected to understand the range of roles and expertise of team members in order to communicate effectively to achieve an excellent service for the patient. At times they will be required to refer upwards and at other times assume leadership appropriate to the situation. • Mini-PAT comprises a self assessment and the collated views from a range of co-workers o of a trainee’s performance (also described as 360 assessment and multi-source feedback or MSF). • Surgical trainees can use the method to: o Assess themselves against important criteria of team-working o Compare their self-assessment with their peer assessment and against the performance of others at their level in the same speciality o Address the gap between ratings, build on feedback and chart their progress o Produce evidence of competence for final review • The method also serves the purposes of: o Forming a portfolio of formative assessments at the completion of the rotation as to the level of performance achieved Q. A. Who initiates the process? The process is lead by either the Supervisor or the trainee. If the use of this process is required by the specialty, it is the trainee’s responsibility to ensure the required number of assessments have been completed by the end of the rotation or training year. Q. A. How doesit work? This is an anonymous assessment which is based on group, rather than individual responses. The only person who is required to identify themselves is the person being assessed. The forms are distributed with an envelope addressed for their return to the Supervisor or nominated person who will collate all of the responses. Q. A. Who could be asked to fill in the form? The following people have been identified as appropriate as potential assessors: Theatre nurse Radiologist Anaesthetist ICU staff Hospital administration (HR), plus The trainee (or IMG), who does a self-assessment Q. A. How is it scored? The responses are collated and the tallies are recorded onto a table. It is this table which is used for discussion between the Supervisor and trainee (see the example of collated scores) Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 23 4.8 RACS – 360-Degree SAMPLE Survey Form Surname ..................................................... First name……………………………………………. Assessment date…………………… iMIS ID number.............................................. Specialty: Cardio General Neuro Ortho OHNS Paed P&RS Urol Vasc Hospital................................................................................................................................................. Clinical setting/hospital post .............................................................................................................................................................. Instructions Please rate this doctor in comparison to other doctors with whom you have worked. Circle one number per item where 1 is the lowest rating and 5 is the highest rating. If you have insufficient contact with the doctor to evaluate him/her on a particular characteristic, circle UE (Unable to Evaluate) 1 – Below expectations for this competence – Unsatisfactory 2 – Performance in this competence is Borderline 3 – this doctor demonstrates competence in this characteristic – satisfactory performance 4 – Above expectations for this competence – Proficient performance 5 – Performance in this competence is Outstanding UE – unable to evaluate this characteristic Technical Expertise Technical Skills 1 2 Requires development of technical skills 3 4 5 UE Proficient technical skills Communication Communication with patients 1 2 Communication skills require development 3 4 5 UE Communicates very well with patients 1 2 Communication skills with Peers requires development 3 4 5 UE Communicates very well with Peers Able to resolve misunderstandings or disagreements 1 2 3 4 5 UE Never Always Collaboration Working in a multidisciplinary team 1 2 Tends to work alone, rarely collaborates 3 4 5 UE Always collaborates as appropriate Consults with other disciplines and appropriately refers 1 2 3 Consultation with colleagues or other professionals infrequent 4 5 UE Always consults and refers as appropriate Leadership 1 2 Manual for Supervisors and Trainees: Version 3 3 4 © Royal Australasian College of Surgeons 2009 5 UE 24 Rarely provides leadership Manual for Supervisors and Trainees: Version 3 Outstanding team leader. Leads by example. © Royal Australasian College of Surgeons 2009 25 Management and Leadership Respectful of expertise of others 1 2 Shows less than expected respect for peers 3 4 5 UE Always respectful of others Directs and supervises other team members including medical students appropriately for level of expertise 1 2 3 Rarely manages or supervises more junior team members 4 5 UE Manages junior team members very effectively Health Advocacy Commitment to improve health outcomes for patients 1 2 Rarely 3 4 Consistently 5 UE Scholar and Teacher Recognises value of learning and research and it’s application to clinical practice 1 2 Rarely 3 4 Consistently 5 UE 3 4 5 UE Always ethical Professionalism Consistently applies ethical principles 1 Inconsistent 2 Integrity and reliability 1 2 Does not meet commitments, may be late, not always reliable 3 4 5 UE Always honest and trustworthy Always reliable 3 4 5 UE Always acknowledges limitations Always learns from mistakes Acknowledge own limitations 1 2 Rarely acknowledges own limitations Does not admit mistakes Responsibility 1 2 3 Rarely accepts responsibility for own actions and decisions 4 5 UE Fully accepts responsibility for own actions and decisions. Never blames others. Critically reflective of own knowledge and skills 1 2 Reflective approach needs development Manual for Supervisors and Trainees: Version 3 3 4 5 UE Appropriately aware and self critical © Royal Australasian College of Surgeons 2009 26 Collation of 360 reports Name (of Trainee/Surgeon being appraised) …………………………………………………………………… Prior to the meeting between the trainee and their Supervisor (or the nominated person collating the responses) the responses from the Survey are collated and compared with the individual’s own appraisal. Indicators of potential issues arise if there is more than one respondent who rates the individual lower than 4 on an item, or where there is 2 or more points difference between the individual’s own rating and that of the majority of the responses. The following example of a tally sheet shows that the trainee consistently rates themselves higher than the other assessors and that there are several areas of concern. Example showing sections of a Summary Sheet 10 respondents plus the individual (T) 1 2 3 4 Technical Expertise 6 4 T Communication with patients 2 3 3 Collaboration – working in a multidisciplinary team 5 5 Management and Leadership 4 3 3 5 4 Professionalism – Integrity and reliability 5 UE T 2 T T 1/T Suggestions for development …………………………………………………………………………………………………………………… ................................................................................................................................................... ................................................................................................................................................... .......................................................................................................................................... …………………………………………………………………………………………………………………… ................................................................................................................................................... ............................................................................................................................................. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 27 UNSATISFACTORY PERFORMANCE 1.12 The process 1.12.1 Any deficiencies in performance should be reported verbally to the trainee as soon as they are recognised. 1.12.2 Verbal reporting should be followed by a written report to the trainee outlining his/her deficiencies, specifying specific goals to be achieved in remedying the deficiencies and indicating a suitable timeframe. 1.12.3 The written warning should be signed by and dated by both the Supervisor and the trainee. In order to ensure that appropriate records of the management of deficiencies, the use of the proforma for managing underperforming trainees is strongly recommended (see 5.2). The Supervisor should record the specific areas for improvement, specific mechanisms for improving performance, the plan for future monitoring and assessment, the benchmarks against which assessment will be made and a suitable timeframe. 1.12.4 This report should be appended to the formal six monthly in-training assessment. 1.13 Chain of Responsibility In the event of a trainee receiving a written warning or in the event of serious misconduct of a trainee, the Supervisor of Surgical Training must provide a written report to the Specialty Board or Regional Sub-Committee of Training. 1.13.1 Specialty Board (smaller specialties) or Regional Sub-Committee Responsibilities The Specialty Board or Regional Sub Committee is required to: • Notify the trainee, in writing, of the adverse report outlining all adverse information available to the Committee. • Provide the trainee with two week’s written notice to prepare a written submission for the relevant Committee and to appear before that Committee. The trainee may bring a support person to the meeting. • Review all the information and make recommendations for dismissal or probation to the Surgical Board with advice to the trainee. Reasons for the recommendations must be documented. The onus is on the Committee to substantiate its decision. The relevant detail of the meeting must be clearly minuted. 1.13.2 Surgical Training Board Responsibilities The Surgical Training Board is required to: • Make a decision re dismissal or deferment. • Provide minutes of the meeting and reasons for the decision • Provide the Censor-in-Chief with documentary evidence of the in-training assessments, written warnings, recommendations of the Regional Sub Committee and the Board, together with Minutes of the meetings held. 1.13.3 • • Censor-in-Chief Responsibilities Must be satisfied that due process has been followed before the decision is ratified. Surgical Board informs trainee, in writing of the decision. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 28 1.14 Managing Underperforming Trainees PERFORMANCE MANAGEMENT OF SURGICAL TRAINEE ON PROBATIONARY TRAINING Timely remediation of deficiencies in performance is more likely to result in improved training outcomes for the Trainee. The purpose of this “Performance Management” Form is to assist the Trainee in improving his/her performance, where one or more significant deficiencies in performance have been identified. This form may be regarded as a performance contract. Details entered into the form must be discussed between the Trainee and his/her Surgical Supervisor and the performance management process should be mutually agreed to by the signatories. There must be regular evaluation of the Trainee’s performance, referenced to this form, thereby allowing a measured assessment of performance by the Trainee. The performance management process and its outcome must remain strictly confidential. The contents of the form will be considered by the Specialty Board and will inform the decision of the Board regarding continuation in the training program and future training arrangements. 1.14.1 GUIDING NOTES Objectives • • • List the specific performance objective that the Trainee is required to meet. Do not combine objectives – list them separately if required. Objectives must be consistent with the training requirements and expectations of the Specialist Surgical Training Program, and must be referenced accordingly. Strategies to Meet Objectives • • • • • It is useful to list as many strategies as is appropriate to guide the Trainee in meeting an objective. Describe all appropriate methods by which the objective can be met. Specifically, describe how the Trainee is expected to behave / perform. Specifically, describe what reasonable supports will be provided to assist the Trainee. Ensure strategies are appropriate to the objective and to the work and training conditions. Performance Indicators • • • • • An indicator can beconsidered as an outcome that measures whether a performance objective is being met. Indicators must therefore be consistent with the specified objective and defined strategies. List as many indicators as is appropriate to guide the Trainee in meeting an objective. Performance indicators should include reasonable time frames. Performance indicators may be used to verify to what extent a Trainee has met an objective. Outcomes • • • • • A performance management period covered by this Form must be specified and should be sufficient to allow performance objectives to be met. The overall performance of a Trainee will be formally assessed at the completion of the review period – this date is to be set prior to commencing the performance management process. The overall performance of a Trainee for each performance objective will be rated according to the criteria tabled below. It is imperative that both the Trainee and the Surgical Supervisor are familiar with the criteria. Taking into account the outcome of performance management, continuation of a remediation process will be determined by the Board in … Surgery. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 29 Outcome Rating MET (M) NOT MET (N) EXCEEDED (E) Criteria • Independently achieves objective and meets all performance indicators. • Minor or no omissions or errors in achieving objective or in meeting performance indicators. • Requires repeated prompting or guidance to achieve objective or meet performance indicators. • Significant omissions or errors in achieving objective or in meeting performance indicators. • Minor or no omissions or errors in achieving objective or in meeting performance indicators. • No omissions or errors in achieving objective or in meeting performance indicators. The example below addresses a significant problem of time management (specifically about appropriate task organization and prioritisation). The strategies describe various approaches to ensure the Trainee participates in those activities central to the Trainee’s role as a junior registrar, and ensuring that adequate support is provided. The performance indicators reflect whether the Trainee is meeting the primary performance objective by defining specific activities that assist in time management. This may be prescriptive if required. If the Trainee is able to achieve all or most of the performance indicators with minimal prompting, guidance, omissions or errors, the Trainee may be assessed as having “Met” the performance objective at the completion of the review period. OBJECTIVE Effective time management (task organisation & prioritisation) STRATEGIES TO MEET OBJECTIVE • Be familiar with unit weekly timetable and schedule of outpatients, elective operating lists, ward rounds, unit meetings. • Punctual arrivals at morning ward rounds. • Attend entire ward round with resident staff. • Liaise with senior registrar in determining daily patient management plan – prioritise where required. • Delegates ward administrative tasks to resident and clerical staff. • Divide attendance at operating lists with senior registrar. • Maintain elective operation booking diary – liaise with booking clerk to review up to date waiting list. • Punctual attendance at operating Manual for Supervisors and Trainees: Version 3 PERFORMANCE INDICATORS • • • • • • • OUTCOME (Met, Not Met, Exceeded) Obtain, read and be familiar with surgical unit policy manual by 22-07-05. Meet senior registrar at 0730 for daily ward round in HDU. Confirm priority of patient care tasks with senior registrar after each ward round. Communicate task requirements to resident staff after each ward round. Assign elective operating sessions to either junior or senior registrar every Monday (must note in operating diary). Meet with booking clerk each Wednesday to schedule elective operations 1 month in advance. Discuss elective operation bookings at Monday morning unit meeting. Meet with consultant surgeon 10 minutes prior to commencement of each operating session – discuss patient progress. Complete all audit database entries within 24 hours of each operation (unit head to verify at weekly unit meeting). © Royal Australasian College of Surgeons 2009 M N E M M E N 30 • • • sessions. Liaise with consultant about patient progress. Maintain surgical unit audit database. Share medical student tutorial sessions with senior registrar – coincide tutoring with when not allocated to attend elective operating session • • • Submit and discuss 6-month unit surgical audit at unit meeting on 12-09-05. Present 6-month audit at divisional meeting on 19-0905. Conduct medical student tutorials every fortnight – submit tutorial topic to unit head 1 week in advance. E M M Notes to Trainees and Surgical Supervisors • • • • • • • • • • • The performance objective(s), related strategies and performance indicators in this Form must be discussed between the Trainee and the Surgical Supervisor. The performance objective(s), related strategies and performance indicators reflect the expected and required responsibilities, competencies and performance of the Trainee, consistent with the Specialist Surgical Training Program. The performance objectives must be achievable under the conditions of the work and training environment. The Trainee and Surgical Supervisor will regularly meet to review the Trainee’s progress in meeting the performance objectives. The Trainee and Surgical Supervisor are required at all times to openly and actively engage in the performance management process. At the completion of the performance management period, the performance of the Trainee will be rated according to the defined outcome criteria. All performance indicators for each performance objective must be completed in order to meet the performance objective. While on Probationary Training, the Trainee’s performance will be assessed on a monthly basis using this Performance Management Form as well as the standard In-training Assessment Form. NOTE: This Performance Management Form should be used to inform the overall assessment and does not replace the In-Training Assessment Form. Ifany In-Training Assessment Form be deemed Unsatisfactory, the trainee must be sent a written warning from the Surgical Supervisor, detailing the specific deficiencies, specifying the goals to be achieved in remedying the deficiencies, and noting that the specified deficiencies must be remedied within the first four months of the current rotation. If after 4 assessments there is no improvement in the documented deficiencies, the current rotation will be declared Unsatisfactory and will not be accredited. At this point, the Specialty Board may commence dismissal proceedings (refer to the College Dismissal Regulations which can be accessed via the College website at www.surgeons.org). The Trainee will receive at least two written warnings before dismissal is considered. The Performance Management Form must be signed by the Surgical Supervisor and the Trainee. Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 31 Performance management start date Performance management end date Frequency of review meetings MONTHLY Next formal performance review date Name of Trainee Signature of Trainee Date Name of Surgical Supervisor Signature of Surgical Supervisor Date Outcome Key: Met =M. Not Met = N. Exceeded =E NOTE: USE ONE SHEET FOR EACH OBJECTIVE OBJECTIVE STRATEGIES TO MEET OBJECTIVE PERFORMANCE INDICATORS OUTCOME (Met, Not Met, Exceeded) Month1 Month2 Month3 Month4 Comments: Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 32 POLICIES 1.15 Dismissal from Surgical Training 1.16 Grounds for appeal In summary an appeal may be made on the following grounds: 1. That there was an error in law or due process 2. Relevant or significant information was not used or was not available at the time of the original decision. 3. That the decision of the dismissing body was clearly contrary to the weight of evidence. 4. That irrelevant information was used. 5. The College did not follow its own policies or procedures 6. The decision was made without due consideration to the merits of the particular case 7. The decision was made for an improper purpose Manual for Supervisors and Trainees: Version 3 © Royal Australasian College of Surgeons 2009 33
© Copyright 2024 Paperzz