Getting Your Stories Straight:
Using Examples and Anecdotes
as Outcome Measures
Montgomery County
September 17, 2015
Barry Jay Seltser
Today’s Goals
• Share ideas on using stories and examples as
outcome indicators
• Improve your ability to be systematic and
thorough in using stories
• Provide handouts and suggestions for future
use
Qualitative Outcomes
2
Why Tell Stories?
• Discussion:
– Advantages
– Disadvantages
Qualitative Outcomes
3
Drawing Verbal Pictures
Qualitative Outcomes
4
What Makes a Story Work?
• Discussion
Qualitative Outcomes
5
Elements of a Story
Qualitative Outcomes
6
Elements of a Story
• Suggested elements to focus on
– What was the situation?
– What did we do?
– Why did we do this?
– What happened as a result of what we did?
– Why does it matter?
– Handout 1
Qualitative Outcomes
7
Attribute Change!
• Key goal: To attribute any change or
improvement to what you did
• Stories are powerful when they link your
actions to the client’s outcome
• Ask questions
– What did we do that made a difference?
– Why did our actions “work”?
Qualitative Outcomes
8
Types of Success Stories
• Examples of stories you have heard about
your agency (group discussion)
• Strengths and weaknesses of using these
types of stories (group discussion)
Qualitative Outcomes
9
Who, where, what, when?
The stories are there--How do you unlock them from
wherever they are hiding?
Qualitative Outcomes
10
Overall Point: Be Systematic!
• Don’t be passive
• Solicit stories on a regular basis, from a broad
set of informants
• Collect, document, and store your narratives
uniformly
• Handout 2
Qualitative Outcomes
11
Who do you ask?
• Who are your best informants? (May not be
the client—consider family, teachers, friends,
your staff members, etc.)
• Who knows about your program’s ability to
help achieve better outcomes?
Qualitative Outcomes
12
Where do you ask?
• Where would they be most comfortable
talking about their experiences?
• Who would they be most comfortable talking
to?
• Are they most likely to give you examples
– In person
– On the phone
– In an e-mail or letter
Qualitative Outcomes
13
What do you ask for?
• Consider how you want to use and present
the information (Handouts 3 and 4)
• Identify your key audiences and their needs
and interests
Qualitative Outcomes
14
When do you ask?
• Try to ask for stories routinely when
informants are providing other information
– Exit interviews
– Regular reporting cycles
– Training sessions
Qualitative Outcomes
15
Some Possible Approaches
•
•
•
•
•
Individual interviews
Focus groups/small group discussions
Open-ended survey questions
Unsolicited ad hoc letters, phone calls, etc.
Questions at end of meetings, meals, etc.
Qualitative Outcomes
16
Logistical Issues
• Professional facilitator, objective staff member
collecting information
• Recording technology (tape/digital recorders,
flip charts or computers for capturing
examples)
Qualitative Outcomes
17
Your Words or Their Words?
• Why are direct quotations valuable?
• What are some problems with direct
quotations?
• What do you want your audience to hear?
Qualitative Outcomes
18
Qualitative and Quantitative
• Combine two approaches where possible
• On surveys or interviews, try to ask both
closed-ended and open-ended questions
Qualitative Outcomes
19
Presenting/Disseminating
• Handout 5 provides some suggestions
• Two results to avoid:
Qualitative Outcomes
20
Aggregating Stories
• When feasible, identify multiple stories for a
common theme
• Try to estimate frequency of impacts
• Identify a range of themes/categories for
grouping and reporting stories
Qualitative Outcomes
21
Summary
•
•
•
•
Be systematic
Figure out what you need, and ask for it
Link your intervention with the outcomes
Link stories with quantitative outcome
information whenever possible
• Start where you are, and slowly improve
Qualitative Outcomes
22
Resources
• CDC, “Impact and Value: Telling Your
Program’s Story” (2007)
www.cdc.gov/oralhealth/publications/library/success_stories
_wkbk.htm
• Richard Krueger (expert on focus groups):
http://www.tc.umn.edu/~rkrueger/about.html or his book
Focus Groups (1988)
• Organizations such as ESC, local colleges,
other independent experts, your colleagues in
other agencies
Qualitative Outcomes
23
Thank you!
• Please complete the brief evaluation
• Thank you for your participation!
• If you need additional support or advice from
Barry Seltser, please contact the Pro Bono
Consultant Program – 240-777-2605 or
[email protected]
Qualitative Outcomes
24