ASSESSING TEACHING
PERFORMANCE (FOR LEARNING):
Summative & Formative
Deborah Simpson, PhD
Yale University School of Medicine - April 11, 2013
Perspec'ves Apply Implica+ons #1: Skeff: 7 Teaching Components
 Enhanced Learning
1. 
2. 
Establish Positive Learning Climate
o  Enthusiasm [content, teaching], respectful, admit limitations
Control of Teaching Session
o  Manage: time, pace, # key points ≅ learner(s)
3. 
Communication of Goals
o  Establish, Express Observable behaviors KSAs, Negotiate
4. 
Enhancing Understanding & Retention
o  Organized/Clear, Emphasis, Methods ≅ active learning, deliberate practice
5. 
6. 
Evaluation
o  Observation, Assessment of Goals: Formative & Summative
Feedback
o  Inform, reinforce, praise (or constructively criticize)
7. 
[Promoting] Self-Directed Learning [Teacher Knowledge]
o  Motivate - Identify and act on learning needs
o  Role Model (PBL&I)
2
Perspec'ves #2: Glassick Criteria – Scholarship
[Boyer, Scholarship Reconsidered]
Clear Goals Reflec+ve Cri+que Adequate Prepara+on Effec+ve Presenta+on Apppropriate Methods Significant Results 3
Perspec'ves GLASSICK’S CRITERIA
Clear Goals
BROAD INDICATORS (FOR DECISION MAKERS)
Learning objec+ves for the teaching session(s) are: •  Stated clearly •  Specific & At appropriate level DETAILED INDICATORS (FOR EXPERT REVIEWERS AND CONSULTANTS TO DECISION MAKERS)
Learning objec+ves are: •  Based on documented needs of learners •  Specific, measurable, achievable, realis+c, and +mely (SMART) •  In mul+ple domains (e.g., knowledge, skills, aPtudes, and/or behaviors) A d e q u a t e •  ≅ ins+tu+onal/program goals and integra+on with other components Prepara'on
of curriculum •  Subject maUer at depth and breadth matched to learners’ needs and +me available •  Material is up-­‐to-­‐date and evidence based •  Material is comprehensive and is logically integrated with other curricular components •  “Best prac+ces” …. •  Resources needed for teaching are specified and available •  Resource planning •  Adequate prepara+on for use of technology
•  Employs suitable range and variety of teaching strategies supported by learning theory…. Appropriate •  Teaching methods ≅ objec+ves •  Methods: feasible, prac+cal, ethical •  Uses interac+ve approaches and promotes self-­‐directed learning; promote cri+cal thinking Methods
•  Innova+ve teaching methods used Teaching methods include ways to monitor learners’ progress to achieve objec+ves If technology is used, it: •  Aids learning of the content; Is easy to navigate •  Is interac+ve (e.g., teacher to learner, peer to peer, learner to content, learner to technology)
•  Sa'sfac'on/reac'on Sa+sfac+on/Reac+on Learning: Measures of knowledge, •  Ra+ng of teaching by learners, peers or experts S i g n i fi c a n t •  skills, aPtudes, and/or behaviors •  Comparison of learner ra+ngs to ra+ngs of other teachers (internal, external) Results •  Applica'on: Desired performance Learning demonstrated in other sePngs •  Evidence of learning based on measurable changes in knowledge, skills, aPtudes, behaviors [Kirkpatrick]
•  Impact: On educa+onal programs •  Comparison of learner performance to established benchmarks and/or to other learners’ and processes within and/or outside performance in previous years ins+tu+on
Applica+on •  Demonstra+on of skills/behaviors learned from teaching ac+vity in subsequent sePngs or curricular components Impact •  Posi+ve evalua+on by knowledgeable peers, educa+onal leaders, curriculum commiUee •  Recogni+on by internal/external awards or incen+ves
Recognized a
s v
aluable b
y o
thers •  Peer reviewed publica+ons/presenta+ons of teaching strategies or instruc+onal materials (internally or externally) through: •  Invita+ons to provide faculty development, conduct workshops, or do presenta+ons about E ff e c ' v e •  Peer review teaching locally, at other ins+tu+ons, or in other regional, na+onal, interna+onal venues Presenta'on •  Dissemina+on = Presenta+ons , •  Invita+on to peer review other teachers locally or at other ins+tu+ons pubs and/or Use by others •  Breadth of dissemina+on and adop+on of teaching methods/materials: local, regional, na+onal, interna+onal
• 
Reflec+on a
nd r
esults o
f • 
Cri+cal analysis of teaching ac+vity using all informa+on from others and from self-­‐assessment R e fl e c ' v e 4
evalua+ons used for Cont Improve •  Evidence of ongoing improvement of teaching ac+vity based on cri+cal analysis and reflec+on
Cri'que
Perspec'ves #3: Teaching as a Competency:
Competencies for Medical Educators [Srinivasan et al inc Skeff 2011]
Core Competency Content Area Descrip'on: Competent Medical Educators… Core Teaching Competencies (Adapted AGGME) Medical Knowledge • Teach content & assess each learner’s abili+es… • Challenge & facilitate learners prac+ce high quality care Learner Centered • Demo commitment to learner’s success & well being + help learners grow into prof role(s) • Demo respect (explicitly value each learner’s • Flexibly tailor teaching & communica+on learning • Com expecta+ons, goals.. s+mulate learning • Provide specific feedback, direct observ Interpersonal & Com Skills contrib to T/L E) • Create a learning climate Professionalism & Role • Demo + role model best educ & • Inspire learners, model prof prac+ce content-­‐related prac+ces standards Modeling • Keep up to date educ + field Prac+ce-­‐Based Reflec'on & Improve • Demo cont self-­‐assess & • Rou+nely reflect on educ prac+ces learning to improve as educators • Accountable for ac+ons, +mely FU Systems Based Learning • U+lize resources within + > system of med to advocate & op+mize teaching & learning • Use shared curricula • Nego+ate resources  teach success • An+cipate trends w/in field edu plan 5
Perspec'ves Perspectives à Accountabilities
Society + Health Care Systems (Team)
EFFECTIVE TEACHER EPAS PERFORMANCE ITEMS SKEFF GLASSICK SRINI-­‐
HCS VASAN TEAMS Clear Goals •  Establish, Express, Nego+ate (Roles)     Climate: •  Enthusiasm, Respeceul (Safe) •  Accountable •  Admit Limits          Control Session •  Organized, Clear •  Time, Pace, # Key Points   Methods •  Best Prac+ces ≅ Goals (Triple Aim) •  Methods ≅ Ac+vate Higher Learning       Feedback & Evalua'on •  Obtain as Teacher/Provide Learner o  Forma+ve – Immediate (CUS) o  Summa+ve [High Stakes](CLER) •  PBL&I: Expect/Demo Self-­‐Directed Learning – CI (Self Assessment)       6
Perspec'ves Utility Formula
•  Apply van der Vleuten’s “Utility” Formula
for Designing “Assessments”
Validity Reliability Educa+onal impact Acceptable + Cost effec+veness Utility
R Perspec'ves Apply to a Tool? Implica+ons 7
Selected References
& Resources
• 
Gusic M, Amiel J, Baldwin C, Chandran L, Fincher R, Mavis B, O'Sullivan P, Padmore J, Rose S,
Simpson D, Strobel H, Timm C, Viggiano T. Using the AAMC Toolbox for Evaluating Educators: You be
the Judge!. MedEdPORTAL; 2013. Available from: www.mededportal.org/publication/9313
• 
Litzelman, DK, Stratos GA, et al. Factorial Validation of a Widely Disseminated Educational Framework
for Evaluating Clinical Teachers. Acad Med. 1998;73(6):699-695.
• 
Olle ten Cate. Nuts and Bolts of Entrustable Professional Activities. Journal of Graduate Medical
Education. 2013; 5(1): 157-158.
• 
Olle ten Cate. Competency-Based Education, Entrustable Professional Activities, and the Power of
Language. Journal of Graduate Medical Education. 2013;5(1): 6-7.
• 
Simpson, D, Fincher, RM, Hafler, JP, Irby, D et al. Advancing educators and education by defining the
components and evidence associated with educational scholarship. Med Educ. 2007;41:1002-1009.
• 
Skeff KM. Enhancing Teaching Effectiveness & Vitality in the Ambulatory Setting JGIM 1988;3(Mar/
Apr):S26-S33.
• 
Skeff, KM. The Chromosomal Analysis of Teaching: The Search for Promoter Genes. Transactions of
the American Clinical and Climatological Association 2007;118:123-132.
• 
Srinivasan M, Li S-TT, et al. “Teaching as a Competency”: Competencies for Medical Educators Acad
Med 2011;86(10):1211-1220.
• 
Van der Vleuten CPM. The assessment of professional competence: Developments, research and
practical implications. Adv Health Sci Educ 1996;1(1):41-67.
8