We thrive in information-thick worlds because of our marvelous
and everyday capacities to select, edit, single out, structure,
highlight, group, pair, merge, harmonize, synthesize, focus,
organize, condense, reduce, boil down, choose, categorize,
catalog, classify, list, abstract, scan, look into, idealize, isolate,
discriminate, distinguish, screen, pigeonhole, pick over, sort,
integrate, blend, inspect, filter, lump, skip, smooth, chunk,
average, approximate, cluster, aggregate, outline, summarize,
itemize, review, dip into, flip through, browse, glance into, leaf
through, skim, refine, enumerate, glean, synopsize, winnow the
wheat from the chaff, and separate the sheep from the herd.
Edward Tufte
Envisioning Information
WELCOME!
Effectively using qualitative data
Evaluation Workshop Series
March 4, 2011
Presenters:
Brian Pittman and Ellen Shelton
Goals for the session
 Define qualitative data and compare it to
quantitative data
 Learn the basics of qualitative data methods:
─ Organization and processing
─ Analysis and interpretation
─ Reporting
 Apply learning
wilderresearch.org
Establishing our framework.
DEFINING QUALITATIVE DATA
wilderresearch.org
Working with qualitative data is NOT
Qualitative research is not a single set of
theoretical principles, a single research
strategy or a single method.
The SAGE Dictionary of Social Research Methods
wilderresearch.org
Working with qualitative data IS
The analysis of qualitative data is a creative
process. There are no formulas, as in statistics.
It is a process demanding intellectual rigor and
a great deal of hard, thoughtful work.
Michael Q. Patton, How to Use Qualitative Methods
wilderresearch.org
Qualitative data – examples
 Open-ended question(naire)s
 Interviews or focus groups
 Notes and other observational data
 Documents, forms, or reports
 Photos and video
 Maps
 Music and sounds
wilderresearch.org
Quantitative & Qualitative – working together
 Commonly called MIXED METHODS
 QUANT-itative data often describes WHAT
happened
 QUAL-itative data can often contribute the HOW
or WHY something may have happened
wilderresearch.org
Qualitative methods – WHEN?
 Different research questions call for different
methods
 Examples:
─ Interest in processes, implementation, or formative
evaluation
─ Require attention to nuance or detail
─ Focus on emergent, unknown, or un-established
outcomes
─ Lack of established quantitative measures
wilderresearch.org
So you have decided to use qualitative
methods. First, you must get organized!
ORGANIZING QUALITATIVE DATA
wilderresearch.org
Handling qualitative data
 Physical or electronic documents
(source documents)
 Spreadsheet software (tables)
 Dedicated Qualitative Data Analysis (QDA)
software
wilderresearch.org
Process – HIGHLIGHT and NOTE
wilderresearch.org
Process – SEGMENT and SORT
wilderresearch.org
Process – CODE
wilderresearch.org
Coding – SCHEMA
 Develop a code book / list of codes
 Revisit your research questions
 Method is important
(Open  Hybrid  Closed)
 Hierarchy and scope are important
wilderresearch.org
Coding – QUALITY MANAGEMENT
 DOCUMENTATION!
 Code definitions
 Decision rules
 Check coding
wilderresearch.org
Organized? CHECK!
Now what do these data say?
ANALYZING QUALITATIVE DATA
wilderresearch.org
Analysis – REDUCTION
 We thrive in information-thick worlds because of our marvelous and
everyday capacities to select, edit, single out, structure, highlight,
group, pair, merge, harmonize, synthesize, focus, organize,
condense, reduce, boil down, choose, categorize, catalog, classify,
list, abstract, scan, look into, idealize, isolate, discriminate,
distinguish, screen, pigeonhole, pick over, sort, integrate, blend,
inspect, filter, lump, skip, smooth, chunk, average, approximate,
cluster, aggregate, outline, summarize, itemize, review, dip into, flip
through, browse, glance into, leaf through, skim, refine, enumerate,
glean, synopsize, winnow the wheat from the chaff, and separate
the sheep from the herd.
 Our ability to reduce complexity allows us to
flourish.
wilderresearch.org
Analysis – QUANTIFY
 Code occurrences or frequencies
 Mindful of what the numbers mean
 CAREFUL!
─ Can build into more sophisticated methods
─ It can also lead you astray!
wilderresearch.org
Analysis – look for PATTERNS
 Identify content areas:
─ Highest importance
─ Greatest prevalence
 Are there differences
─ Based on client characteristics
─ Based on “dosage” levels
wilderresearch.org
Analysis – MODEL
 Relating themes, categories, or patterns to each
other
─ Process / outcomes matrix
─ Logic model
 Build typologies or taxonomies
 Use to test hypotheses and theories
wilderresearch.org
Analysis – VALIDATING
 No tests like quantitative analysis
 User-defined process:
─ Confidence in a data point (strength)
─ Consistency within a source (internal)
─ Confirmed across sources (inter-source)
─ Verify through other knowledge and methods (external)
 Transparency is the new objectivity
wilderresearch.org
Validate – STRENGTH of data point
 Provides detail and context
 Unambiguous and explicit
 Appropriate and fitting
 Lacks depth or description
 Bias, partial, or unknown source
 Issue with data collection
wilderresearch.org
Validate – INTERNAL verification
wilderresearch.org
Validate – INTER-SOURCE verification
wilderresearch.org
Validate – EXTERNAL verification
wilderresearch.org
Share your understanding.
REPORTING QUALITATIVE DATA
wilderresearch.org
Revisit your research questions
 Answer them!
 Are there new questions that need new more
data to answer?
 Did your data answer any additional questions?
wilderresearch.org
Present the essentials
 Focus on the most useful findings
 Meet the primary interests/information needs of
the stakeholders
 Present multiple perspectives
wilderresearch.org
Balance analysis and description
 Provide interpretation and conclusions
 Illustrate with examples
 Document how you got there
wilderresearch.org
Time to apply what we have learned.
SMALL GROUP ACTIVITY
wilderresearch.org
Using Excel to group and sort by theme
1.
2.
3.
4.
5.
6.
7.
8.
9.
My view of our family and my impression of how the children act have been enlightened.
new
understanding of
kids actions
Better understanding of other's feelings. Trying to understand the situation will never be the better
same as a normal family.
understanding of
feelings
I have better communication with my partner and his children.
better
communication
Seeing my step son & his comments & actions in a different light & trying to be more
new
patient & responsive.
understanding of
stepson
I am stepping back from some of the immediate parental duties when it comes to my stepchildren. In the beginning of our 8 year marriage we both thought I should take an active
part in the role. We have both found out slowly this doesn't work with the eldest of his 3. I
decided now to step back more with the other 2 and not seem so "in their face" when it
step back; be less
comes to the parenting piece.
assertive
I am using more awareness of the various topics we discussed, and not get as frustrated
with the situation.
more aware
More aware of issues
more aware
I have been a step-parent for 21 years. After that seminar, I came to understand that I will
never be looked at as a co-mother. Therefore, I was not as uptight on Mother's Day as I
new perspective
have been in past years, and I had a wonderful day with my adult step-children.
on relationship
Taking into consideration the step-parent and the challenges he has in our new family unit. ?? (knowledge or
awareness)
wilderresearch.org
more patience;
respond
differently
more patience,
less frustration
interpret situation
differently
Concept (or theme) mapping
wilderresearch.org
Highlight on (copy of) original document
wilderresearch.org