The Standardized Patient and
Standardized Interdisciplinary
Team Meeting (SP-SIDTM):
Validation of a new performance-based
assessment tool
M. Yuasa, M. H. Nagoshi, C. M. Oshiro Wong,
M. M. Tin, A. B. Wen, K. H. Masaki.
Department of Geriatric Medicine and Office of Medical
Education, University of Hawaii
May 3, 2013
Financial Disclosure
This product was supported by the
following grants:
•The Donald W. Reynolds Foundation Next
Steps educational grant
•The John A. Hartford Foundation Center
of Excellence in Geriatrics, University of
Hawaii
•The Pacific Islands Geriatric Education
Center (HRSA grant)
Curriculum for IDT Geriatrics
at the University of Hawaii
• Create curriculum in geriatric
interdisciplinary team (IDT) care
for ALL levels of trainees at the
medical school
Curriculum for 4th Year
Medical Students
• Required Geriatrics and Palliative Care
Rotation for 4 weeks:
1. Clinical site experiences
2. Didactic sessions
• Evaluated by:
1. 6 Station Geriatric Medicine Standardized
Patient Examination (GSPX)
2. Final written examination
1 station revised and 1 station
added for IDT evaluation
New Evaluation Tool
SP-SIDTM
2 consecutive stations:
1. Standardized Patient (SP) – elderly
patient hospitalized after a fall
2. Standardized Interdisciplinary Team
Meeting (SIDTM) – participate in IDT
meeting for discharge planning with
team members:
•
•
Standardized Nurse (SRN)
Standardized Social Worker (SMSW)
The Patient: Ted Lee
75 year old male with a fall
• Students evaluate
patient first
• Patient must be too
sick to go home but
competent enough
create a decision
point
• Students write H&P
The Interdisciplinary
Team (IDT) Meeting
• IDT consists of RN
and MSW, with
student playing the
role of physician
• Discussion would
center on discharge
planning
Interstation Exercise
• Students fill in the Geriatric
Interdisciplinary Care Summary (GICS):
A systematic approach to the management
of Geriatric Syndromes
• Graded by faculty based on predetermined
criteria
GERIATRIC INTERDISCIPLINARY CARE SUMMARY
Domain
Problem(s)
Disciplines to
address problem(s)
Treatment
Plan(s)
Rehabilitation 1. Deconditioning
Unsteady gait
2. ADL dependent/Need
assistance with ADLs
PT
OT
RN
1. Continue rehab
2. Equipment needs
3. Fall precautions/
ADL assistance
Cognitive
MD (Geriatrician/
Psychiatrist)
Evaluation
F/U with PCP
No active issues
Concern about capacity
Methods: SP-SIDTM
• After initial validation and revision, given to:
All 4th year medical students (MS4)
- part of 4th year Geriatrics and Palliative
Care Required Rotation Assessment
Geriatric Medicine Fellows (GMF)
- part of ACGME Core Competencies
Assessment
• 52 MS4 and 6 Geri Fellows in 2011-2012
academic year
1. Checklist
%
 Fellows scored significantly higher than medical
students in SIDTM and SP-SIDTM
2. Communication Skills
%
 No difference between fellows and students
3. Patient Note/GICS
%
 Fellows scored significantly higher than medical
students in SIDTM and SP-SIDTM
4. Total Score
%
Total score = weighted average of component scores
(50% Checklist + 30% Patient Note/GICS + 20% Communication Skills)
 Fellows scored significantly higher than medical
students in SIDTM and SP-SIDTM
“Rate the overall difficulty for
your level of training”
%
 Both rated that difficulty was “moderate”
“Was the time allotted for
this meeting...”
%
 Both rated the time was “about right”
“How believable was the
case scenario?”
%
Both rated that the case was “highly believable”
“Before today, how often had you
participated in this type of meeting
regarding an actual patient?”
%
 Fellows had more experiences than students
Reliability
Cronbach’s α
Overall
0.871
SP
0.780
SIDTM
0.866
GICS
0.732
Cronbach’s α > 0.7  Good reliability
Inter-Observer Correlation
Pearson’s r
P value
Checklist
0.823
< 0.0001
Communication
Skills
Total
0.453
0.003
0.693
< 0.0001
r =1: Scores are exactly the same between two
observers (SRN and SMSW)
p < 0.05: Significant difference between MS4
and GMF
Conclusions
• The SP-SIDTM was positively rated by
trainees and generated scores with high
reliability
• Observations recorded on checklists
showed excellent inter-observer correlation
• Geriatric Medicine Fellows scored
significantly higher than 4th year Medical
Students
These observations support the validity
of this novel assessment tool.
Strengths
• We are the first to report on the
validation of a performance-based
assessment of IDT skills using
standardized team members
• SP-SIDTM will be available on POGOe
Limitations
• Numbers of trainees were small
• Results may not be generalizable to
other institutions
Why do we need interdisciplinary
team (IDT) education?
• Aging population has steadily increased
65 yo: 35 million in 2000  72 million by 2030
= 20% of population
• Providing care for elderly is a constant challenge
– Combination of multiple medical problems
(acute and chronic)
– Polypharmacy
Best
– Frailty
addressed
– Physical/Cognitive abilities
by IDT!
– Psychosocial problems
Task
1. Create a situation that most students
would have encountered in training
2. Situation must feel real to the student
3. Students evaluate pt discussed in IDT
4. IDT member would interact naturally
5. Keep the “team” small to decrease
administration costs (SP training &
stipends)
6. Must include as part of the 4th year
required rotation Clinical Skills evaluation
(15 min SP station and 10 min writing)
GICS Swiss-Cheese Model of Geriatrics
A Systematic Approach to the Management of Geriatric Syndromes
Developed by the University of Florida College of Medicine
2
1
5
6
3
7
4
8
Geriatric Syndrome
(e.g. Falls, Dementia)
Frailty/Loss of Independence
Eight main domains:
1. Physical Medicine &
Rehabilitation
2. Cognitive
3. Emotional
4. Medical/Surgical
5. Nutritional
6. Environmental
7. Social/Caregiver
8. Economic
Reference: JAGS 200;57:1685-1691
GERIATRIC INTERDISCIPLINARY CARE SUMMARY
Domain
Medical/
Surgical
Problem(s)
Disciplines
to Address
Problem(s)
Pain management RN
Humeral fracture MD-Ortho
PCP
Rehabilitation Arm and hip pain
restrictive:
needs assist for
dressing, bathing,
feeding, walking
transferring
Cognitive
No mood
problems
27
RN
PT
OT
MD
RN
Treatment
Plan(s)
-Continue
hydrocodone,
acetaminophen
-Continue senna
-Continue
assistance
-3 more days
intensive inpt
PT/OT with
transition to SNF
vs. home PT for
3 weeks
Continue to
monitor
Future Directions
• The same format of SP-SIDTM can
be used for other Geriatric
syndromes, e.g. dementia,
depression
• Validating checklists to measure
facilitative skills among trainees
• Validating the score in a real patient
setting
Evaluation by Trainees
Characteristics
Fourth Year
Medical Students
Mean (SD)
Geriatric
Medicine Fellows
Mean (SD)
p
Case Difficulty
4.7 (1.0)
3.8 (0.8)
0.057
Time Allotted
4.5 (0.8)
4.0 (0)
0.127
Believability
5.6 (1.4)
5.5 (1.4)
0.919
Previous similar “real”
experience
3.0 (1.2)
4.2 (0.4)
0.027
• Case difficulty rated 1=Minimal to 7=Extreme (4=moderate)
• Time allotted rate 1= too short to 7=Too long (4=about right)
• Believability rated 1=Not at all to 7=Extremely (4=Moderately)
• Previous similar “real” experience rated 1=Never, 2=Once,
3=Twice, 4=3-5times, 5=6 or more