A FLIPPIN’ UPDATE
Second Year of a PubMed Flipped Classroom
Lynn Kysh, MLIS
Clinical & Research Librarian
Not the first time this has been
presented…
■ The Flipping Librarian: Flipped Classroom for Year 1 Medical Student
PubMed Instruction, Innovations in Medical Education Conference,
February 2015
■ A Flip or A Flop?: PubMed Instruction for Health Sciences Students,
Medical Library Association Annual Meeting, May 2015
BUT NOW WITH MORE DATA!
Components of Flipped Classroom
Part I: Video Tutorials
Part II: In Person Session
■ 6 “Talking Head” Tutorials
■ 24 students paired with
librarian instructor
■ Online components with
Zaption
■ 5 – 15 minutes each
■ 60 minutes to complete
series
■ Group work applying skills
learned in video tutorials
■ Time to address student
questions and knowledge
gaps
Timeline
Introduced
module to
students
Aug
10
Inperson
session
Aug
10 -18
Time for students
to complete
module
Aug
18
Sept
11
Unit Multiple
Choice Exam
Added interactive features with Zaption
• Uploaded video content
• Added interactive elements
• Open response
• Multiple choice
• Pauses
• Basic analytics
Created LibGuide
• Links:
• Video tutorials
• PowerPoint slides
• Sites mentioned
• Sites for more info
Zaption Tutorial Screenshot
Zaption Tutorial Screenshot with Question
Zaption Tutorial Screenshot: Wrong Answer
Changes
■ Updated screenshots for new website
■ Decreased number of MeSH multiple choice
questions
■ Easter Eggs = Cat Pictures
■ In person session included comparison of PubMed
and Google Scholar
Tutorial Survey Respondents
147
53
2014
2015
In Person Session Survey Respondents
177
27
2014
2015
The secret to improved feedback?
Paper surveys at in-person session!
(And finishing 10 minutes early)
Tutorial Data
Percentage of Students Who Viewed Tutorials
(n=186)
80%
78%
Introduction Basic Search
81%
Advanced
Search
78%
Results &
Filters
71%
74%
Citation
Information
Access Full
Text
Percentage of Viewers Who Responded to Questions
(n=151)
77%
79%
78%
Advanced
Search
Results &
Filters
85%
80%
N/A
Introduction Basic Search
Citation
Information
Access Full
Text
Tutorials: Looking up MeSH Terms (n=114)
Tylenol
Medical School
Service Animals
96%
4%
88%
12%
78%
22%
Correct
Series2
Series1
Incorrect
Tutorials: Identifying Citation Information
(n=114)
Library Full Text Access
MeSH Terms for Citation
Find Similar Citations
99%
1%
83%
7%
77%
23%
Correct
Incorrect
Tutorial Evaluation
Learning Objectives (n=133)
68%
68%
60%
35%
29%
26%
1% 1% 4%
Use Filters to Narrow
Results
Strong Disagree
2% 0% 3%
2% 0% 1%
Navigate between search
results and citations
Disagree
Neutral
Agree
Access full text through
NML
Strong Agree
Tutorial Evaluation (n=133)
Time
Content
96%
80%
20%
2%
0%
Too Short
Just Right
Too Long
Too Basic
2%
Just Right
Too Advanced
The Good
■ Lynn, you are a fantastic teacher and your little added touches of
personal style were fantastic!
■ Lynn made a potentially really boring topic magically enjoyable/easy to
watch with her enthusiasm and cat photos!
■ The ability to watch at 2x speed is amazing. I really learned a lot about
searching PubMed.
■ I love how friendly and accessible it was!
■ Very straightforward!
■ 
■ So helpful. Wish I could have seen these while doing my undergraduate
thesis. Thank you!
The Bad
■ Less “interactive portions” to cut time, maybe 1 per
video
■ I would prefer they just be posted on Youtube
rathern than using Zaption & having to create an
account
■ It would be good to have an online pdf or
instructions instead or in addition to refer to
■ Too long, make them shorter
The Techy
■ Zaption seems a bit glitchy; often times it
would freeze or skip around.
■ It took me to a Zaption video that was not part
of the tut
■ I couldn’t access the videos for a while
In Person Session Data
In-Person Evaluation (n=173)
65%
62%
54%
43%
35%
32%
1% 0% 2%
Build a search strategy using
keywords
Strong Disagree
1% 0% 2%
1% 1% 1%
Build a search strategy using
MeSH
Disagree
Neutral
Assess which strategy is more
effective for my search
Agree
Strong Agree
In-Person Evaluation (n=173)
I am better prepared to use PubMed to support my
academic and clinical work in medical school
52%
40%
1%
0%
Strong Disagree
Disagree
3%
Neutral
Agree
Strong Agree
The Good
■ It was a good length. Instructors were knowledgeable
■ I feel much more confident using MeSH than I was
before.
■ Helpful. Learned new things on top of Zaption tutorials.
■ Useful, didn’t think I would get much out of it but I did!
■ Watching other classmates conduct searches was the
most helpful
■ I learned best from other classmates
The Bad
■ More structured presentatin on how to fully utilize
MeSH effectively
■ Did not find this super helpful because I did all of
this in the online tutorial
■ Maybe make it optional in case we have extra
questions & make the tutorials mandatory. It’s a lot
of time for a small amount of info.
What Students Wanted to Learn More About
■ MeSH subheadings
■ More practice
■ More advanced and detailed ways to search
■ Improve integration of three resources
■ Other citation managers. Other search sourches
■ Library’s other resources
■ Web of Science
■ Cochrane Reviews
Lessons Learned
• Timing is everything
• Without timing, incentives or accountability required
• Keep self-paced learning modular, engaging and upbeat (if possible)
• Most students will complete the online curriculum the day/evening
before the in-person session
• Video tutorials will need to be updated every year
• Consider online tools other than Zaption
Lessons Learned
• Don’t waste in-person session time repeating what was covered in
modules
• Technology is great, but facilitating group work is the most important
part of the module
• Side Effect: increased the presence and positive reputation of the
library and its services
• Examine feedback as a whole; don’t focus on the negative
Questions?
Lynn Kysh, MLIS
[email protected]
Thanks!
Lynn Kysh, MLIS
[email protected]
Eek, world's cutest cat
Keck School of Meowdicine 2018