10/8/2015
Agenda
Getting On Course-Staying on Course
 Feedback Overview
FEEDBACK IN VETERINARY
MEDICAL EDUCATION
Competence Ladder
•
Leadership Styles
•
Formative Assessment
•
•
Summative Assessment
Nonverbal
Communication
•
Impact of Feedback
Feedback
 Understand the importance of feedback in medical
•
•
Feed Up
•
Feed Back
•
Feed Forward
•
 Putting it All
Together
 Components of
Learning Objectives
 The “HOW”
•
Small and Large Group
Exercises
Descriptive vs Evaluative
Key Premise
Trust is essential to effective learning relationships
education
 Identify key elements of feedback
• Do our learners trust us?
 Participate in small and large group sessions
• How do we know?
applying key principles of effective feedback
 Propose ways of enhancing the feedback culture
within your institution
• What do we do to earn their trust?
• What have we done to compromise trust?
• What do we do to acknowledge broken trust?
Feedback Defined
Feedback Overview
Specific information about the comparison between an
observed performance and a standard, given with the
intent to improve the performance.
Med Education 2008 Feb; 42(2):189-97
1
10/8/2015
Competence Ladder
Attention: This is Important!
Do not assume you know the motives for
someone’s behavior. You don’t.
Assumptions you make about others’ motives
say more about you than about the other.
Create alternative hypotheses. “What would
cause a reasonable person to behave this way?”
Formative vs Summative Assessment
Formative Assessment (Continued)
Formative assessment
 The goal of formative assessment is to monitor
student learning to provide ongoing feedback that
can be used by instructors to calibrate their teaching
and by students to calibrate their learning. More
specifically, formative assessments:
Formative assessments are generally low stakes, which
means that they have low or no point value. Examples
of formative assessments include asking students to:


help students identify their strengths and weaknesses and
target areas that need work
help faculty recognize where students are struggling and
address problems immediately
https://www.cmu.edu/teaching/assessment/basics/formative-summative.html
Summative Assessments
The goal of summative assessment is to evaluate
student learning at the end of an instructional unit by
comparing it against a standard or benchmark.
 Describe 2 locations to monitor the pulse in an
anesthetized patient
 Use the stethoscope to auscultate the heart and to
identify a cardiac murmur
 Recognize the need for a behavioral consultation in
order to make a low stress handling plan for a
patient - Canine or Feline
https://www.cmu.edu/teaching/assessment/basics/formative-summative.html
Summative Assessments (Continued)
Summative assessments are often high stakes, which
means that they have a high point value. Examples of
summative assessments include:
 a midterm/final exam
 a final project
 a complete physical examination of the
cardiovascular system (including auscultation,
assessment of venous pulses, examination of arterial
pulses, and palpation of the precordium)
https://www.cmu.edu/teaching/assessment/basics/formative-summative.html
https://www.cmu.edu/teaching/assessment/basics/formative-summative.html
2
10/8/2015
Areas for Feedback in
Medical Education
Discussion
 Clinical competence
Physical examinations, history taking, diagnostic and
therapeutic plans, notes or discharge summaries
On a scale of 1-10, how important is giving
effective feedback in medical education?
 Communication competence
Patient/Client interactions; case presentation, team
interactions
 Professional competence
1
Not Important
10
Essential
Punctuality, time management, coping with
responsibility, organizational skills, showing interest in
learning
Impact
With Feedback
•  Confidence and
motivation
•  Interpersonal skills
Consequences to No/Poor Feedback
Without Feedback
• Receive a false “positive”
impression/overestimated
abilities
•  Learner satisfaction
• Underestimated abilities
•  Clinical performance
• Decreased confidence
•  Accuracy of self-
• Lack reinforcement of
assessment
•  Patient/client
satisfaction
When learners receive overly critical
appraisals of their work, it can have a
negative impact on learning.
effective performance
• Fail to correct poor
performance
Crommelineck & Anseel, 2013; Malloy & Boud 2013; Thomas & Arnold 2011; Davis et al 2006; Clynes & Raftery, 2008; Ende 1995; Laidlaw 2004; Waitzkin 1985; Spickard, 2008; Cantillon, 2008; Kluger & DeNisi, 1996; Hattie & Timperley, 2007
Bottom Line: It Matters. It’s ESSENTIAL.
The ability of instructors to accurately observe
trainees performing tasks and provide
effective feedback is…one of the most
important aspects of coaching and
medical training.
Kluger, A. N. & DeNisis, 1996; Hattie & Timperley, 2007
Houston…We Have an…OPPORTUNITY
Lack of effective feedback is one of
the most serious deficiencies in
medical education practice.
“Giving feedback is evidence of your interest in
the person and of your intention to help them
grow and develop.”
Norcini & Burch (2007), Seitchik; Driscoll, M.
Davis et al 2006; Ende 1995; Laidlaw 2004, Waitzkin 1985; Spickard, 2008; Cantillon, 2008
3
10/8/2015
Understanding the Gap: Facts and Stories
Subjective
Objective
 What are the
If It’s Worth Doing (and it is) Then
It’s Essential We Do it Well
 What are the
FACTS of your
experiences with
giving effective
feedback?
STORIES you have
told yourself about
giving effective
feedback?
 What are the systemic
barriers to giving
effective feedback?
 What are the personal
barriers to giving
effective feedback?
Discussion
(Self-Assessment/Peer Assessment)
On a scale of 1-10, how effective are you at giving
feedback in medical education?
1
Not Effective
10
Mastery
On a scale of 1-10, how effective are medical
educators in your setting (as a whole) at giving
feedback in medical education?
1
10
Not Effective
Mastery
Effective feedback answers three questions:
Where am I going? (goals)
How am I going?
Where to next?
Task level
Process
level
How to tie the
knot
Decision
making in
choice of suture
pattern
Feed up
Feed back
Feed forward
Selfregulation
level
Making a
decision from
the acceptable
choices
Self level
Nice job!
Hattie & Timperley, 2007; Dr. Karen Cornell
Essential Components of Feedback
1. Information on the goal of the performance
2. Information on the executed performance
3. Strategies to address the gap between task
goal and task performance
Discussion: Component 1/Feed Up
Information on the goal of the
performance
On a scale of 1-10 how would you rank your
effectiveness in communicating the goal/desired
outcome/objectives of the learner’s
performance?
1
Not Effective
10
Mastery
Sadler DR. 1989, Instructional Science 18:119-144
4
10/8/2015
Are Our Expectations Clear?
Not Clear
Clear
 Causes of cardiovascular
(CV) diseases in dogs,
cats, horses, cattle, and
camelids
 Know about
cardiovascular disease
 Pathophysiology of CV
diseases
 Heart failure
 Arrhythmias
 Hypertension
Superior
Outstanding knowledge base demonstrated by:
 Recall and understanding of key facts and concepts pertinent to
CV disease.
 Timeliness when responding to questions.
 Integration of knowledge with respect to clinical
pathophysiology, mechanisms of disease, and pharmacotherapy.
 Practical application of knowledge applied to the diagnosis of CV
disease.
 Practical application of knowledge to the therapy of CV disease.
 Ability to find highly-relevant clinical information in textbooks
and reliable on-line databases including PubMed.gov and
veterinary medical databases.
 Ability to ask instructors critical and nuanced questions related
to clinical issues and to respond in kind.
 Preparation for rounds and discussions.
Satisfactory
Satisfactory knowledge base demonstrated by:
Marginal
Marginal knowledge base characterized by:
 Recall of key facts and concepts needed for good quality, entrylevel management of CV diseases.

Clear gaps in the student’s knowledge base.

Inability to consistently recall key concepts and facts in a timely manner.
 Timeliness when responding to questions.

General inability to integrate pathophysiology, clinical disease, and medical
therapies
 Understanding of the relationships between pathophysiology,
clinical disease, and pharmacotherapy.

Relying on rote memory to answer questions instead of a fundamental
comprehension of those principles underpinning diagnosis and
management of CV disease.

Incomplete or disorganized application of knowledge within the clinical
setting.

Inability to find appropriate or credible answers to clinical questions in
textbooks or on-line resources.

Unfocused questions related to clinical issues.

Incomplete or partially incorrect answers to questions posed by instructors
during case management and patient rounds.

Unprepared or disorganized presentations during rounds.

A clear need for additional reading, self-study, and experience to improve
the student’s knowledgebase and ability to integrate principles of medicine
within practical situations.

A potential need for remediation in specific topics with specific oversight of
faculty members.
 Practical application of knowledge to the diagnosis of CV disease.
 Practical application of knowledge to the therapy of CV disease.
 Ability to find relevant clinical information in textbooks and
reliable on-line databases including PubMed.gov.
 Ability to ask instructors relevant questions related to clinical
issues and to respond in kind.
 Preparation for rounds and discussions.
Unsatisfactory
Unsatisfactory knowledge base characterized by:
 A general inability to recall, in a timely manner, key facts and
concepts about CV function in health and disease.
 Major gaps in the student’s knowledge base.
 Inferior understanding of relevant pathophysiology.
 Little to no comprehension of the principles underlying the
diagnosis and management of CV disease.
 Disorganized knowledge base, misinformation, or potentiallydangerous application of facts within the clinical setting.
 Inferior fact-finding skills when trying to answer clinical questions
in textbooks or on-line resources.
 Irrelevant or unfocused questions relevant to clinical issues.
 Incorrect or substandard responses to questions posed by
instructors during case management and patient rounds.
 Inability to present or discuss a case in the rounds setting.
 The need for a comprehensive consideration of factors affecting
performance, additional reading assignments, directed self-study,
tutoring, and administrative and faculty intervention organized at
the level of the Associate Dean for Student Affairs.
Let’s Check Back In: Component 1/Feed Up
Information on the goal of the
performance
On a scale of 1-10 how would you rank your
effectiveness in communicating the goal/desired
outcome/objectives of the learner’s
performance?
1
Not Effective
10
Mastery
5
10/8/2015
Discussion: Component 2/Feed Back
Information on the executed performance
Feedback Fundamentals
 Descriptive NOT
evaluative
On a scale of 1-10 how would you rank your
effectiveness in communicating information
about your learner’s executed performance?
1
Not Effective
10
Mastery
Suggested Template: WWW AND EBY
 Directed toward
changeable behaviors
 Specific NOT general
 Anticipated
 Behavior NOT person
 Well-timed
focused
 Given in small doses
 Solution focused
 Well thought out
 Considerate of the
needs of both giver
and receiver
Describe What You See
What Worked Well?
AND
What Would Make It Even Better Yet?
Bonvicini, K.
List as many brief observations as you can about the scenes above.
Descriptive Feedback
• Specific aspects of a
person’s behavior/
actions that impacted
on you.
• What you directly saw
and heard.
• Use “I” statements
• Provide concrete,
specific examples.
 I saw that you completed
your SOAPS by 8:00 a.m. and
administered all medications
at 8:00 a.m. and 6:00 p.m.
 I observed you correctly
identify differentials for
Simon including TCC of the
bladder and a UTI.
 I did not see you call Taco’s
owners as requested and we
had to remind you twice to
contact Simon’s owners.
Evaluative Feedback
 Subjective, often
perceived as negative or
destructive in nature
 Stimulates
defensiveness
 Limited data
 Ok when combined with
descriptive feedback
 Pleasure working with
you!
 You seemed disengaged
and disinterested.
 Your medical knowledge
is inadequate for this
point in your training.
 You were rude.
6
10/8/2015
Descriptive or Evaluative?
Try harder next time.
You aren’t at the level you
should be at this stage in the
learning process.
You maintained eye contact
and enunciated clearly
throughout your interaction
with Mrs. Smith.
You can’t act so shy all the
time. You just need to get in
there and take a chance.
Your attitude is unacceptable.
I thought you were rude in
rounds and always
disrespectful of your
classmates.
You improved consistently
throughout the rotation.
You demonstrated interest in
the subject matter by actively
participating in rounds,
asking questions, and
completing reading materials
as assigned.
Descriptive Feedback Starter Stems
WWW
EBY
Improvement Strategies
This is quality work
because…
Your thinking shows…
Your next steps might be…
Your thinking shows…
One thing to improve is...
You might try…
Two things you did really
well include…
You need more…
You might have better
results if you…
I noticed that…
You need less…
Your response would be
more effective if it
included…
Your writing tells me…
When explaining your
topic you…
Your writing tells me…
Describe What You See-Part 2
HOW You Say is as Important
as WHAT You Say
LIKELY EVEN MORE SO!
List as many brief observations as you can about the scenes above.
Six Supervision/Leadership Styles
 Drill Sergeant
 control, little nurturance

abusive role model/creates dependency
 Mother Hen
 warm, nurturing

difficulty with confrontation and boundaries/creates
dependency
 Inconsistent
 micro-managing to absence based

creates confusion, pushing of limits, replication with others
Taibbi, 2012
Six Supervision/Leadership Styles
 Crisis Oriented
 responsive to biggest/latest crisis; good emergency skills

difficulty with ongoing relationships; gets bored; cuts contact
suddenly
 Neglectful
 rationalizes that supervisee doesn’t need supervision

overstressed or difficulty with relationships; encourages peer
support
 Balanced
 nurturing and able to confront, set limits, and establish
healthy boundaries; adaptive/calibrated style based on needs
Taibbi, 2012
7
10/8/2015
Nonverbal Communication
 Kinesics
 Proxemics
 Paralanguage
Metacommunication Competence
People who are able to adjust their non-verbal
language to the needs of various situations are
generally better at diffusing troublesome
interactions such as arguments. This type of
communication skill is known as metacommunication
competence.
 Autonomic Changes
http://www.wisegeek.com/what-is-paralanguage.htm
Paralanguage
Clues are contained in even small fragments of speech,
and other people can "read" our voices with remarkable
accuracy. When we speak, we "encode" important
information about ourselves; when we listen to
others, we "decode" important information
about them.
http://www.berkeleymedia.com/catalog/berkeleymedia/films/communication/language_linguistics/the_human_voice_explo
ring_vocal_paralanguage_
3 Elements of Paralanguage
 Speech Pace
 Pitch
 Volume
http://www.arinanikitina.com/paralanguage-what-does-your-voice-say-about-you.html
Speech Pace
 Most important part of
paralanguage
 Largely influenced by our
emotional state
 Fast paced speech may
suggest that the speaker:
 Moderate paced speech
may suggest that the
speaker is:
•
calm
•
composed
•
confident
 Slow paced speech:
•
is agitated
•
loves to chat
•
seems monotonous
•
lacks seriousness
•
•
is easily manipulated
leaves the listener too
much time for thoughts to
wander
http://www.arinanikitina.com/paralanguage-what-does-your-voice-say-about-you.html
Pitch
 Pitch ranges from high to
low:
 High-pitched speakers
are interpreted as:
•
Men 120 Hz
•
lacking confidence
•
Women 220 Hz
•
insecure
•
weak
•
nervous
•
less truthful
 Low-pitched talkers are
associated with:
•
authority
•
credibility
•
strength
•
self-confidence
http://www.arinanikitina.com/paralanguage-what-does-your-voice-say-about-you.html
8
10/8/2015
Volume
Let’s Check Back In: Feed Back
Information on the executed performance
 Soft spoken voices can be
perceived as:
•
shy
•
insecure
On a scale of 1-10 how would your rank your
effectiveness in communicating information
about your learner’s executed performance?
 Too much volume can be
interpreted as:
1
Not Effective
someone who loses
temper easily
• unsafe
•
10
Mastery
http://www.arinanikitina.com/paralanguage-what-does-your-voice-say-about-you.html
Discussion: Component 3/Feed Forward
Common Examples of Feed Forward
Strategies to address the gap between task
goal and task performance
 Read more
On a scale of 1-10 how would you rank your
effectiveness in communicating/providing
strategies for improving the gap between your
learner’s task goal and performance goal?
 Engage more
1
Not Effective
Conduct
 Be more prepared
10
Mastery
Practice: Suggestions for Feed Forward?
Component
 Enhance your medical knowledge
Let’s Check Back In: Feed Forward
Unacceptable
Shows a lack of respect for members of the group and the discussion
process. Often dominates the discussion or disengages from the
process. When contributing, can be argumentative or dismissive of
others’ ideas, or resorts to personal attacks.
Ownership/ Does not play an active role in maintaining the flow of discussion or
Leadership undermines the efforts of others who are trying to facilitate
discussion.
Reasoning
Comments are frequently so illogical or without substantiation that
others are unable to critique or even follow them. Rather than
critique the text, he or she may resort to personal attacks on the
author instead.
Listening
Behavior frequently reflects a failure to listen or attend to the
discussion as indicated by repetition of comments and questions,
non sequiturs, off-task activities.
Strategies to address the gap between task
goal and task performance
On a scale of 1-10 how would you rank your
effectiveness in communicating/providing
strategies for improving the gap between your
learner’s task goal and performance goal?
1
Not Effective
10
Mastery
9
10/8/2015
A Good Rule of Thumb
If you wouldn’t want to sign your
name to the feedback you have given,
or have it videotaped and broadcast to
constituents as an example of who you
are as a person, sit back, take a deep
breath, and reconsider your words and
your approach.
Putting it All Together
1. In small groups, identify some specific, (challenging!)
feedback scenarios from your work life experiences
(please de-identify specific names/organizations)
2. As a group, agree upon at least one scenario to
address.
3. Individually, write out the actual feedback you would
give to the person in the scenario.
4. Share your individual statements and develop a
consensus statement
Enhancing Your Feedback Culture
1. Working in small groups identify specific
strategies for enhancing the feed up, feed
back, and feed forward process in your
organizations.
2. Please be prepared to share your strategies
with the large group.
10