Research in Experiential
Education: Focus on Qualitative
Studies
Moderator:
Gina Baugh, PharmD
School of Pharmacy
West Virginia University
Learning Objectives
• Distinguish terminology and key aspects of qualitative research.
• Formulate questions about experiential education best
answered through qualitative research.
• Describe elements of rigor to look for when reading qualitative
research reports.
• Analyze experiences in various research projects to identify
lessons learned and questions remaining.
• Generate an initial list of qualitative resources and training ideas
for EE section members.
Webinar Speakers
• Sheila Chauvin, PhD, MEd, Professor, School of
Medicine and School of Public Health, Louisiana
State University
• Jessica Johnson, PharmD, BCPS, Associate
Professor, College of Pharmacy, Xavier University of
Louisiana
• Teresa O’Sullivan, PharmD, BCPS, Acting Assistant
Professor, School of Pharmacy, University of
Washington
Which statement best describes your familiarity
with qualitative research?
A. I know very little about qualitative
research.
B. I read and understand qualitative
research papers but haven’t tried
such research.
C. I have participated in qualitative
research to a limited extent.
D. I have been an author on more
than one paper involving
qualitative research methods.
What is qualitative research?
What is qualitative research?
Qualitative research
•Encompasses a variety of non-quantitative approaches
or methods
•Examines, describes, interprets, and translates naturallyoccurring phenomena in specific contexts
•May include behaviors, perceptions and perspectives,
interactions, and how people in those contexts construct
meaning from or make sense of such phenomena,
experiences, and contexts.
What is qualitative research?
Qualitative research
•Encompasses a variety of non-quantitative approaches
or methods
•Examines, describes, interprets, and translates naturallyoccurring phenomena in specific contexts
•May include behaviors, perceptions and perspectives,
interactions, and how people in those contexts construct
meaning from or make sense of such phenomena,
experiences, and contexts.
Qualitative Data Manuscripts:
Communicating Rigor
•
•
•
•
•
•
Introduction
General study design
Sampling rationale and recruitment
Data collection and processing
Results: themes and quotes
Discussion
Qualitative Research Terms
•
•
•
•
•
•
•
•
•
Epistemology (study of knowledge acquisition)
Positivism (quantitative research)
Mixed methods (uses both qual and quant methods)
Phenomenology (study of shared experience)
Ethnography (shared cultural beliefs, behaviors)
Narrative analysis (exploration of written stories)
Document analysis (analysis of written documents)
Content analysis (analysis of communicated material)
Grounded theory (includes theory generation)
Key Qualitative Research Terms
• Emergent methodology
Quantitative vs Qualitative Research Flow
Data
collection
Hypothesis
Study design
Study
design
Data
analysis
Sample size calc
Data collection
Data saturation
Sample size
Data analysis
Hypothesis
Key Qualitative Research Terms
•
•
•
•
•
Emergent methodology
Inductive process
Thematic analysis
Coding
Data saturation
Qualitative Analysis Methods
• Step 1: Develop research question and study
design
• Step 2: Analyze text to identify emergent
themes
• Step 3: Code the data
– periodic group review/discussion to resolve discrepancies
• Step 4: Identify data saturation
• Step 5: Develop from themes a hypothesis
and corresponding use for results.
Step 1. Develop a research question
How do we (pharmacy faculty) define,
teach, and assess professionalism?
• Deductive approach researcher-centered
How do students develop their
professional identity?
• Qualitative analysis of reflective essays to
examine professional identity formation in
APPE students
Background: 3-part process
• Part 1: Reading
– ACCP’s Tenets of Professionalism for Pharmacy Students
– ASHP’s Statement on Pharmaceutical Care
– ACCP’s The Definition of Clinical Pharmacy
• Part 2: Small group discussion and 6-week internal
medicine APPE
• Part 3: Reflective Writing
– “reflect on your current professional identity and attempt to
determine what your beliefs about professionalism are, where
they come from, and how you can grow”
Step 2: Identify emergent themes
using constant comparative methods
• Read data independently.
• Keep a running list of emerging themes.
• Continue to modify themes as new data
are added.
• Compare lists with co-code developer.
• Lumpers vs. splitters issue.
Step 2: Identify emergent themes
• Goal is to create themes
– specific enough to be distinct
– expansive enough to capture most of the
data
• What is the “right” number of themes?
– too few: results superficial
– too many: results unmanageable
– develop theme tiers and select the tier most useful
and manageable
Step 3: Coding
• Revisit data to apply identified themes
for formal coding by original coders.
How would you code the following excerpt?
“I believe that I have also gained confidence by interacting with other
medical professionals. Doing research and expressing my concerns to
physicians and medical students regarding patient care allowed me to
make a difference in someone’s treatment while also gaining
confidence.”
How would you code the following excerpt?
“I believe that I have also gained confidence by interacting with other
medical professionals. Doing research and expressing my concerns to
physicians and medical students regarding patient care allowed me to
make a difference in someone’s treatment while also gaining
confidence.”
A. The pharmacist and patient-centered care relationships
How would you code the following excerpt?
“I believe that I have also gained confidence by interacting with other
medical professionals. Doing research and expressing my concerns to
physicians and medical students regarding patient care allowed me to
make a difference in someone’s treatment while also gaining
confidence.”
A. The pharmacist and patient-centered care relationships
B. Role of the team member and influence of teamwork
How would you code the following excerpt?
“I believe that I have also gained confidence by interacting with other
medical professionals. Doing research and expressing my concerns to
physicians and medical students regarding patient care allowed me to
make a difference in someone’s treatment while also gaining
confidence.”
A. The pharmacist and patient-centered care relationships
B. Role of the team member and influence of teamwork
C. Developing/learning professionalism from situations
How would you code the following excerpt?
“I believe that I have also gained confidence by interacting with other
medical professionals. Doing research and expressing my concerns to
physicians and medical students regarding patient care allowed me to
make a difference in someone’s treatment while also gaining
confidence.”
A. The pharmacist and patient-centered care relationships
B. Role of the team member and influence of teamwork
C. Developing/learning professionalism from situations
D. Developing pharmaceutical knowledge and skills
How would you code the following excerpt?
“I believe that I have also gained confidence by interacting with other
medical professionals. Doing research and expressing my concerns to
physicians and medical students regarding patient care allowed me to
make a difference in someone’s treatment while also gaining
confidence.”
A. The pharmacist and patient-centered care relationships
B. Role of the team member and influence of teamwork
C. Developing/learning professionalism from situations
D. Developing pharmaceutical knowledge and skills
Tips for Coding
• Don’t get too literal, don’t take data out
of context
• Review data with coauthor for rigor
(peer review or audit)
Step 4: Recognize Data Saturation
• Important: Data collection and analysis occur
simultaneously
– Analysis of one focus group session might
influence questions asked at a second
• Data saturation:
– No new themes or theme modification
generated from new data
Step 5: Developing Results
• Develop from themes a hypothesis and
corresponding use for results
– Conceptual framework/explanation for
themes that answers the research question
Core APPE Interpretation
• Surveyed EE directors in the US in 2010
about core APPE interpretation
• Lessons learned: 3 is a magic number in
qualitative research
Characteristics of Excellent
Preceptors
• Desire for better measures of preceptor behaviors than
those advocated by ACPE
• Thematic analysis of student answers to prompt:
“Comments about this preceptor” for 21 excellent
preceptors
• Cohen’s kappa measures categorical data association
• Themes were tested against 2 other preceptor groups
• Identified 15 precepting behaviors: 5 role modeling, 6
coaching-teaching, 4 facilitating
• EE databases filled with good qualitative research data
Medical Literature Evaluation Education
Survey
• Surveyed US MLE instructors about
curriculum, teaching, and assessment.
• Close ended-questions don’t always
contain an option that matches
respondent’s best answer.
• Open-ended questions can help inform
close-ended questions and vice-versa.
• Mixed methods research useful for
survey data.
Current Projects
• Student experience in new practice model:
use of in-depth interviews
• Measuring what APPE readiness look like:
use of focus groups
• Additional lessons:
– Thin line between qualitative research and
program assessment.
– Quotes are “proof” in qualitative research
Qualitative Data Reports and
Standards of Rigor
• Introduction
• Methods
– Member checking
– Verification
• Results: themes and quotes
– Agreement testing
– Triangulation
• Discussion
– Audit trail
Poll: Qualitative Research Interest
• I can’t think of a project where I
could use qualitative research.
• I have thought of at least one
project where qualitative methods
could be useful.
• I have started a qualitative
research study and I need help!
• I feel comfortable with qualitative
research and would be able to
mentor new investigators.
Recommended Resources for
Qualitative Research
• Merriam S. Qualitative analysis: a guide to design and
Implementation. San Francisco (CA): Jossey-Bass, 2009.
• Anderson C. Presenting and evaluating qualitative research.
Am J Pharm Educ 2010;74(8): Article 141.
• Guest G, Name EE, Mitchell ML. Collecting qualitative data.
Thousand Oaks (CA): Sage Publications, 2013.
• Guest G, MacQueen KM, Namey EE. Applied thematic analysis.
Thousand Oaks (CA): Sage Publications, 2012.
• Krippendorff K. Content analysis. 3rd ed.
Thousand Oaks (CA): Sage Publications, 2013.
• Saldana J. The coding manual for qualitative
researchers. 2nd ed. Thousand Oaks (CA):
Sage Publications, 2013.
Recommended Resources for
Qualitative Research
• Identify a qualitative research mentor
– Social/administrative pharmacy faculty
– Medical education
– Social sciences departments
– Other institution
• Lynne Robins, PhD; Jennifer Danielson, PharmD,
MBA, CDE; Stanley Weber, PharmD, BCPP
Questions?
Wrap-up
• Thank you for attending!
• Archives in AACP EE Section web site.
• We appreciate your feedback on this
session.
• Next webinar: Quantitative and Mixed
Methods Research, April 2015.