Resident as Teacher: Teaching Residents Direct Observation
and Effective Feedback in the Clinical Setting
Ian Mazzetti and Parveen Wasi
Friday, September 27, 2013
We do not have an affiliation (financial or otherwise)
with a pharmaceutical, medical device or
communications organization.
Je n’ai aucune affiliation (financière ou autre)
avec une entreprise pharmaceutique, un fabricant
d’appareils médicaux ou un cabinet de communication.
Ian Mazzetti and Parveen Wasi
Friday, September 27, 2013
We do not have an affiliation (financial or otherwise)
with a pharmaceutical, medical device or
communications organization.
Je n’ai aucune affiliation (financière ou autre)
avec une entreprise pharmaceutique, un fabricant
d’appareils médicaux ou un cabinet de communication.
Ian Mazzetti and Parveen Wasi
Friday, September 27, 2013
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Session Objectives
• Identify direct observation (DO) with feedback as an
effective teaching tool
• Identify opportunities within residency training for direct
observation and feedback by residents
• Use feedback-on-feedback techniques to improve skills
• Incorporate use of standardized forms to provide structure
for direct observation and feedback
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
4
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Acknowledgement
Eric S. Holmboe, MD
• Chief Medical Officer & Senior Vice President,
American Board of Internal Medicine and the ABIM
Foundation
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
5
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Residents as Teachers
• Survey data stresses importance of residents as teachers
» Residents report up to 25% of time spent in teaching
» Students report up to 1/3 of teaching from residents
• Differences between resident and faculty teachers
» Didactic lectures
» Bedside teaching
» Evidence-based teaching
» Problem-solving skills
» Asking questions
» Feedback
The International Conference on Residency Education
[Morrison et al Med Ed 2005]
[Busari et al Med Teacher 2002]
|
La conférence internationale sur la formation des résidents
6
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Residents as Teachers
• Well- recognized as integral part of residency training
» 2005 CanMEDS framework
• Resident as Scholar
» Accreditation Council of Graduate Medical Education
(ACGME)
• Increasing number of residency programs with Resident as
Teacher (RaT) courses
• Benefits to developing skills for residents as teachers
» Near-peer mentors
» Opportunities; increasing workload of faculty
» Future faculty
» Patient education
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
7
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Resident as Teacher
Many different Resident as Teacher (RaT) programs with
common workshops:
» Adult learning theory, bedside teaching, feedback –
can often be presented as evaluative/summative
» Structured teaching tools:
• SNAPPS
• One-minute preceptor
• Direct observation not emphasised as teaching
tool
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
8
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Resident as Teacher Programs
Future of Medical Education in Canada: Postgraduate
Education: White paper series
• Identified four key/evolving trends in effective Resident as
Teacher programs:
• Learner-centred approaches
• Longitudinal Program- activities based on year of training
• Integrated into regular work of the resident
• Discipline-specific: context important
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
9
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Direct Observation (DO)
• Important in the development of clinical skills
• Direct observation and specific feedback may be the most
effective method of improving performance
» ‘Teaching on the Fly’
• Teaching and assessment of CanMEDS intrinsic roles
• Extensive literature on Direct Observation by faculty and
use in evaluation of competency
• Not often emphasised as teaching tool in RaT programs
» Often used to facilitate the feedback component
[Holmboe E Academic Medicine 2004]
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
10
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
11
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Direct Observation
• Advantages for focus in RaT programs
» Ideally suited for clinical context
» Residents have most face time with junior learners
» On-call activities
» Near-peer mentorship
• May have better idea of level of training and expectations
• Continuity
» Can be tailored to level of training
» Maybe more receptive to DO for own training
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
12
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Advantages to DO training in Residency
» Time-efficient
» Complementary to staff teaching
» Teaching style may not be ingrained
» Evaluation of junior learners
» Improve own skills
• Mandated as part of teaching expectations for residents
• Future Faculty Development
• Part of the culture change
• Competency-based medical education
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
13
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
14
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Discussion
• Experiences with faculty DO
• Challenges with DO
• Challenges with training residents for DO
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
15
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Direct Observation: Challenges
• Often framed as evaluation challenges
» Frames of reference
» Inference when observing
• Lack of role-modeling behaviour by faculty
• Time on busy clinical services
• Deficiencies in Clinical skills
• Need faculty to observe and provide feedback
» Faculty champions
» Senior residents (PGY4-5)
[Kogan et al Medical Education 2011]
[Kogan et al Academic Med 2010]
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
16
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Direct Observation
Teaching
• Clinical Skills
• Intrinsic CanMEDS roles
Feedback
• Improves specificity
• May improve receptivity
Assessment
• Competencies
• Intrinsic CanMEDS roles
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
17
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Requirements for Direct Observation
» Training similar to faculty development
» Embed into clinical practice
» Clinical skills of residents
• Direct observation by faculty
• Confidence
» Practice, practice, practice
• know what one is looking at/for
» Knowledge of basic tenets
• Counselling
» Knowledge of expectations of level of training
• Clerkship objectives; PGY objectives
» Coupled with feedback skills
The International Conference on Residency Education
|
[Holmboe et al Annals 2004]
La conférence internationale sur la formation des résidents
18
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Opportunities: Inpatient
• Hospital ward
– Communication and PExam on daily rounds
– Discharge instructions to a patient
– Informed consent discussions
– Part of admission work-up
• Emergency Room
– Direct observation of certain part of the
encounter
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
19
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
The Patient Encounter
• Sampling “parts” of the encounter:
PHYSICAL
EXAM
INTERVIEW
COUNSELING
Key Message: EMBED the
Observation as part of clinical
care and supervision
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
20
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Know what you are looking for and at
Video: History taking
• Take 3-5 minutes on defining elements necessary for
history taking at a junior resident level
Scenario:
This lady has been admitted lower back pain. She has a
background of chronic pain and has developed acute pain
which is new in character. The junior is fatigued after a
long call shift.
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
21
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Group Exercise
• Discussion of strengths/weaknesses
• View the feedback by senior resident
• Feedback on feedback
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
22
Teaching Residents Effective
Feedback
Part II
Ian Mazzetti and Parveen Wasi
Friday, September 27, 2013
23
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Feedback techniques
 Creating a respectful, friendly, openminded, unthreatening
climate
 Eliciting thoughts and feelings before giving feedback
 Being nonjudgmental
 Focusing on behaviours
 Basing feedback on observed facts and specifics
 Giving right amount of feedback
 Suggesting ideas for improvement
 Basing feedback on well-defined, negotiated goals
• Direct observation is key
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
24
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Residents giving feedback
• Formative, not evaluative
– Integral part of the 25% of time we spend teaching
• Need to tailor to level of learner
• Bidirectionality of encounter leads to improved selfassessment
– Key for the adult learner
– Already part of the “hidden curriculum”
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
25
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Difficulties
• Giving and receiving feedback complex
• Difficulties with staff giving feedback
» Relationship with learner
» Desire to preserve relationship
» Impact on own evaluations
» Comfort with own knowledge/skills
» Perception of residents receptivity
• For residents:
» Establish credibility
» Understand goals of learners
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
26
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Group Exercise
Scenario:
Elderly man at the end of a complex admission for
congestive heart failure. Discharge instructions required
for diuretic dosing, salt intake, new medications, and
follow-up appointments. Please go and discuss the
discharge plan with the patient.
Before watching the video.
Please spend 5 minutes thinking about key features on
communication with a patient about discharge planning.
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
27
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Group Exercise
• Discharge planning video
– 4 minutes: PGY1 communicates with patient
– 4 minutes: PGY3 gives feedback
• Tasks
– Evaluate the PGY1 performance
• Based on your discussion, counselling and decision making form
(handout)
– Evaluate the feedback given by the PGY3
• “Feedback on PGY3” form
– Discuss!
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
28
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Getting started.................
• Define the context: What are they doing now? At what
level?
• Define the opportunities for direct observation and
feedback by residents
» Inpatient wards
» ER
» OR
» Ambulatory clinics
» Specific rotations (chief, senior)
• Education and instruction
• Feedback on feedback
» Faculty development and champions
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
29
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Opportunities within a residency program
• Longitudinal aspect is key
– Not just a “flash in the pan”
– Not a 6 month program; rather, interspersed through 3 years
– Different opportunities with more seniority, teaching
feedback skills accordingly
• Limitations:
– Time
– Learner buy-in
– Faculty modelling, availability
• Need to make it mandatory!
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
30
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Longitudinal Program: IM Example
PGY1
• Clinical skills medical students
• Workshop in PGY1 year September
• Mini-CEX ( recipient) by faculty and chief residents
PGY2
• Senior resident on CTU; on-call supervision
• Workshop on feedback/ DO July
• Mini-CEX (recipient) by faculty
PGY3
• Chief Resident on CTU
• Mini-CEX workshop; Feedback OSCE
• Participation in mid-term and end-rotation feedback
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
31
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Tools we’re using
• Feedback OSCE
• Feedback grid
• Feedback evaluation form (based on mini-CEX)
• Mini-CEX
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
32
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Tools: Feedback OSCE
•
2008: dual purpose to provide formal feedback on senior
resident (PGY3) skills in direct observation and feedback,
formative exam for PGY1s
• 10 min + 4 min feedback
•
PGY1 = examinee, rotates
•
PGY3 = examiner, feedback-giver
• “Refresher” workshop beforehand (longitudinal)
•
Faculty = observes interactions, circulates
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
33
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Tools: Feedback OSCE
PGY1
PGY3
Faculty
1
2
3
Flatland schematic
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
34
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Tools we’re using
• Feedback grid
• Going beyond “sandwich model”
– “Ask, tell, ask” model
• Action plan develops from FB encounter
• Feedback notes lead to more specific feedback for trainees
• Combine with mini-CEX
– Rating form: giving effective feedback
[Schum et al Amb Ped 2003]
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
35
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Feedback grid
Continue…
Start, or do more…
Comment on aspects of
performance that were
effective. Be specific and
describe impact. Highlight
things you would like to
see be done in the future
Identify behaviour the colleague
knows how to do, and could do,
or do more often
Consider…
Stop, or do less…
Highlight a point of growth for
the learner, a “doable” challenge
for future interactions
Point out actions that were not
helpful or could be harmful. Be
specific, and indicate potential
impact
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
36
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Tools we’re using
• Feedback grid
• Going beyond “sandwich model”
– “Ask, tell, ask” model
• Action plan develops from FB encounter
• Feedback notes lead to more specific feedback for trainees
• Combine with mini-CEX
– Rating form: giving effective feedback
[Schum et al Amb Ped 2003]
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
37
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Feedback rating form
Situational:
Objective of meeting identified
Creating an appropriate climate (respectful, friendly, openminded, nonthreatening)
Eliciting thoughts and feelings before giving feedback
Communication skills:
Descriptions of behaviour on a spectrum, not dichotomous
Invites comments/concerns
Focus on sharing ideas and information, not giving advice
Structure of discussion allows for self-assessment
Feedback:
Focusing on changeable behaviours
Being nonjudgmental
Basing feedback on observed facts and specifics
Giving right amount of feedback
Basing feedback on well-defined, negotiated goals
Meets the needs of the recipient, focusing on areas to be reinforced/improved
Action Plan:
Suggesting ideas for improvement
Alternatives are explored together
Follow-up plans are made
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
38
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Group exercise
• Mid-term feedback
• Rate the PGY3 on the mini-CEX form
• Feedback on feedback
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
39
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Learning objectives revisited
• Direct Observation with feedback is an effective teaching
tool
• Many opportunities for direct observation and feedback by
residents
– Need to make it mandatory!
• Feedback-on-feedback techniques may be used to improve
skills
• Standardized forms can provide structure for direct
observation and feedback
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
40
Resident as teacher: Teaching Residents Direct Observation and Feedback in the Clinical Setting | Mazzetti & Wasi
Acknowledgements
• Dr. Andrew Mulloy
• Dr. Lacey Pitre
• Dr. Sharon Marr
• Mr. Jamieson Child
• Ms. Mary-Beth Ribble
• Mr. Dave Geroux
The International Conference on Residency Education
|
La conférence internationale sur la formation des résidents
41
Help us improve.
Aidons-nous à nous améliorer.
Your input matters.
Votre opinion compte.
• Download the ICRE App,
• Téléchargez l’application de la CIFR
• Visit the evaluation area in the
North Building, Upper Level, near
registration, or
• Visitez la zone d’évaluation, au
bâtiment nord, niveau supérieur,
près du kiosque d’inscription, ou
• Go to: http://www.royalcollege.ca/
icreevaluations to complete the
session evaluation.
• Visitez le http://www/collegeroyal
.ca/evaluationscifr afin de remplir
une évaluation de la séance.
You could be entered
to win 1 of 3
$100 gift cards.
Vous courrez la chance de gagner
l’un des trois chèques-cadeaux
d’une valeur de 100 $.