Competence by
Design
Kenneth A. Harris, MD FRCSC
Deputy CEO & Executive Director,
Office of Specialty Education
February 3, 2016
The best health for all.
The best care for all.
Discussion Topics
•
•
•
•
•
•
•
•
Overview of CanMEDS 2015 and CBD
Change fundamentals
The road to CBD
Assessment & ePortfolio
Minimum Data Set (MDS)
Ongoing collaboration
How to reach us
Available resources
2/6/2017
3
Key Concepts
Define Milestones & EPAs
2/6/2017
5
Milestones within an EPA
2/6/2017
6
Milestones and EPAs within Four
Stages of Residency
2/6/2017
7
2/6/2017
8
Assessment
• The nature of assessment will change, but the method of
service delivery will not
• CBD is a hybrid approach to competency-based/outcomes
focused training
• Time will be utilized as a resource to structure training
experiences and maintain a high standard of service
delivery
2/6/2017
9
Specific Themes
2/6/2017
10
Specialties Implementing CBD July
2017
• Anesthesiology
• Otolaryngology – Head and Neck Surgery
2/6/2017
11
Readiness
• Readiness is multifaceted and involves a variety of
partners beyond the Royal College
• Activities are underway with PGDs & CBME Leads to reach
shared goal of implementation by July 1st 2017
2/6/2017
12
A Collaborative Approach with
Local Champions & Change Agents
Deans, PG Deans
• Align visits for outreach with schools, collaboratively with Clinical Chairs, Hospitals & Ministries
Program Directors
• Co-create the design for their discipline in SC CBD workshops & bring it home as local champions
Faculty
• Faculty development resources and toolkit
Residents
• Regular engagement sessions with RDoCs and FMRQ
CBME National Faculty Leads
• Close collaboration with College
PG Managers, Program Administrators
• ICRE & advisory groups
2/6/2017
13
Change
Fundamentals
Rhonda St. Croix
Change Advisor
The best health for all.
The best care for all.
How people change
2/6/2017
15
Essentials for successful change
•Shared purpose
•Lead by example & build ownership/commitment
• Build distributed network for the change
•Leverage relationships & connections
• Change spreads via peer-to-peer influence starting with early innovators
& champions
•Change behaviours, get culture
• Small set of inter-connected behaviours creates high
impact
•Gather feedback to improve
• A deep cycle of learning & innovation
2/6/2017
16
Top strategies that work
Create & share your CBD story – compelling why
• Link CBD to needs & issues people care about
• What’s the payoff
Build your local network for CBD – create the waves
• Local leadership/sponsorship & change agents
• Give people time to understand, ask questions, address concerns/doubts & make CBD their own
• Liaison roles, retreats, meetings, committees, communiques
Make progress visible
• early disciplines implementing July 1
Translate CBD into simple new behaviours by role
• EPAs, assessments, competence cte, real time feedback
Start now (try/test) & evolve
Support – locally and Royal College
2/6/2017
17
The Road to CBD
Dr. Jolanta Karpinski
Associate Director,
Specialties Unit
The best health for all.
The best care for all.
Trying things on
Working the workshops
Cohort rollout
2/6/2017
19
What Does CBD Rollout Mean?
Creation of new version of specialty specific documents in CBD format
Royal College approval of new version of specialty standards
Training programs begin applying new standards to a group of residents
Residents trained under new standards
Program faculty teach and assess
interact with Royal College
using CBD standards
Residents trained in CBD system enter CPD
programs apply
procedures
PGME offices and
CBD policies and
2/6/2017
20
Cohort Implementation Plan
2/6/2017
21
2/6/2017
22
CBD is in
our future
36 specialties
and
subspecialties
Activation
Workshops
and design
Implementation
planning
Cardiology
Clin Imm and Allergy
General Surgery
Geriatric Medicine
Neo-Perinatal Med
Anatomical Path
Cardiac Surgery
Critical Care Med
Emergency Med
Gastroenterology
Anesthesiology
Forensic Path
Medical Oncology
Otolaryngology – Head
and Neck Surgery
Nuclear Medicine
Obs and Gyn
PMR (Rehabilitation)
Plastic Surgery
Psychiatry
Respirology
Rheumatology
General Int Med
General Path
Internal Medicine
Nephrology
Neurosurgery
Pediatrics
Radiation Onc
Urology
Surgical Foundations
CBD is
our
reality!
2/6/2017
23
Working the workshops
2/6/2017
24
CBD workshops
Specialty committee supported to attend
• All members and up to 3 guests
Work supported by Royal College CBD team
• Clinician educator, writer
Workshops entail
• Reinventing discipline in CBD framework
- Developing EPAs and work-based program assessment for the discipline
- Creating new document suite with CanMEDS 2015 and CBD content and
format
• Planning for implementation
2/6/2017
25
Core Components of CBME
• Competencies are clearly articulated.
• Competencies are arranged progressively.
• Learning experiences facilitate the progressive development of
competencies.
• Teaching practices promote the progressive development of
competencies.
• Assessment practices support and document the progressive
development of competencies.
2/6/2017
26
2/6/2017
27
2/6/2017
28
Examples of EPAs
In the real world
Driving on G1
In medicine
Running codes
2/6/2017
29
Progression in Internal Medicine
• Assessing, diagnosing, and initiating management for
patients with common acute medical presentations in
acute care settings
• Assessing, diagnosing and managing patients with
complex or atypical acute medical presentations
• Assessing and managing patients in whom there is
uncertainty in diagnosis and/or treatment
2/6/2017
30
Oversimplification of current
system
Rotations
• Rotation specific objectives
In-training evaluation reports
• Global assessment
• Timely?
• ?linked to clinical activities
Progression through training
• Onus to demonstrate lack of progression
2/6/2017
31
Oversimplification of CBD
• Training is sequenced in stages
• Supervisors and residents focus on EPAs designated for that stage
• Clinical and other training experiences are the basis to acquire the skills to
achieve those tasks
• Supervisors document observations of resident’s performance on
EPAs
• Coaching model: what can I do better?
• Competence Committees review progress
• Residents progress through stages
2/6/2017
32
Assessment in CBD
• Assessment for learning
• Re-emphasis on assessment in the workplace
• Assessment focused on discrete clinical tasks
• Assessment for purposes of learning
• Requirement for demonstration of competence
• Assessment as collection of evidence of progression
• Distinct promotion from one stage to another
- Competence Committees
• Tracking and coaching along the pathway
2/6/2017
33
Assessment for learning
• Role of coaching
• Guidance for learning
• Frequent
• Link to deliberate practice
2/6/2017
34
Assessment for progression
2/6/2017
35
Assessment of EPA Achievement
• EPA Assessment Encounters
• Point in time
• Single rater
• Specific context
2/6/2017
36
Putting it together: Wisdom of
Crowds
2/6/2017
37
Assessment of EPA Achievement
• EPA Assessment Encounters
• Point in time
• Single rater
• Specific context
EPA Achievement
- Multiple times
- Multiple raters
- Multiple contexts
2/6/2017
38
Getting ready for implementation
2/6/2017
39
Implementation ideas: plan
2/6/2017
40
Implementation ideas
2/6/2017
41
Sample of some CBD Resources –
• Introduction to competency-based medical education (CBME) and
Competence by Design (CBD): Video (7:33)
• Understanding Entrustable Professional Activities (EPAs): Video (5:11)
• CBD Cheat Sheet – brief summary of all key CBD concepts
• EPA Fast Facts – provides a brief overview of EPAs and how will be used
within CBD
• Competence by Design (CBD): An Overview (March 2016 PPT)
• Answering your CBD Questions: Part 1 - The Rationale
• Answering your CBD Questions: Part 2 – CBD Implementation
• Answering your CBD Questions: Part 3 – Glossary of CBD and CBME Terms
2/6/2017
42
Implementation ideas: do
Reinforce current strengths
• Value of clinical experiences
• Supervisor interaction with resident
New behaviours
• Focused, explicit clinical tasks
• More frequent “small” assessments
• Resident is “the last patient of the day”
• New language: “let’s go learn!”
New structures
• New clinical experiences
• Competence Committees
2/6/2017
43
Assessment
Essential Assessment Elements
Minimum data set (MDS)
ePortfolio
Jane Fulford, CPA, CGA
Chief Information Officer
The best health for all.
The best care for all.
Assessment Essential Elements
Key Points
• Agreement with the ‘Six Essential Elements of the CBD
Assessment Strategy’
• Use of the assessment forms is not mandatory, provided
faculties are in compliance with both the required use of
the Six Essential Elements and the transfer of data related
to the minimum data set
2/6/2017
45
Six Essential Elements of the
Assessment Strategy
1.
Assessment requirements, as defined by the Specialty committee, and
inclusive of:
• EPAs, as defined by the Specialty Committee
• CanMEDS-based milestones, as defined by the Specialty Committee
2.
Increased emphasis on direct and indirect observation
3.
Many low-stakes observations of focused clinical tasks
4.
Narrative, actionable, timely, concrete recorded feedback
5.
Curation, collation, and group decision-making by a Competence Committee
6.
Stages and progression of increasing entrustment, facilitated by group
entrustment decisions at the Competence Committee level
2/6/2017
46
Minimum Data Set (MDS) |
Key points
• An agreed upon sample of assessment data will be
transferred from PGME offices to the Royal College via
a third party vendor as a method to:
• Maintain oversight of the CBD system
• Enable Royal College operational functions (e.g., exam
planning and credentialing)
• Ensure continuous quality improvement
2/6/2017
47
Minimum Data Set |
Key Points
• Agreed to the transfer of a specific data set to the Royal College
• Domain I – Progression, credentialing, examinations and planning
• Learner identified to RC at point of entry into PGME
• RC Status Categories for learners identified by Faculty/Program, as decided by
Competence Committee:
• Resident promoted to exam-eligible and,
• Resident is certification-eligible
• Domain II and IV – Learning Analytics, Benchmarking and CQI of overall
system and CBD redesign
• Aggregate, de-identified learner assessment and system-tracked data shared with
the Royal College will include:
• EPA Achieved status, time-stamped
• Learner Action (promote to next stage), time stamped
2/6/2017
48
Minimum Data Set |
Key Points
• Domain III – Progressive Accreditation System
• Detailed discussion deferred while further development
work continues on the new standards and accreditation
process
2/6/2017
49
Minimum Data Set |
Key Points
Data Stewardship Committee
• Ensure oversight of types of assessment data collected and
transferred to the Royal College
• Facilitate discussion and transparency among key stakeholders
in the new CBD system and safeguard privacy concerns
• Continuous review of data points shared within MDS in order to
ensure appropriateness and ability of the MDS to answer key
questions, ensure rigorous standards, and measure outcomes
of the CBD system
2/6/2017
50
2/6/2017
51
Your Input Matters
Share your input with the Royal College via:
•
[email protected]
•
www.facebook.com/TheRoyalCollege
•
https://twitter.com/Royal_College
•
www.linkedin.com
For more information, visit:
•
www.royalcollege.ca/cbd
•
www.royalcollege.ca/cbd/resources
2/6/2017
52