What is a Library ‘Outcome’ and
How Do You Measure One?
Andy Stewart
Library Director
C. L. Wilson Library
Missouri S&T
A “Reality” Check…
Session Audience – 10/7/10
Outcomes-Based Evaluation Expertise
100%
80%
60%
40%
20%
0%
"Outcomes-based
WHAT?"
"Moderately aware
"Well-read and
"Conducted 3 or
of the concept"
chomping at the bit more OBE projects"
to try one"
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
"Evaluation based
on WHAT?"
"Moderately aware
"Well-read and
"Conducted 3 or
of the concept"
chomping at the bit more OBE projects"
to try one"
Outline:
Introduction
Definitions
Logic Models
Examples
Related Resources
Looking Beyond
Questions
Introduction
Definitions:
“What is outcome evaluation?
IMLS defines outcomes as benefits to people: specifically,
achievements or changes in skill, knowledge, attitude, behavior, condition, or
life status for program participants.”
A little more from IMLS:
“Note that for IMLS, while academic knowledge and skills are
foundations, learning is broader. It includes the cognitive, social, and
physical; the practical, entertaining, and personal; the formal and informal;
and many, many more domains and settings. Learning represents knowledge,
attitudes, skills, and behaviors that support individual success in our complex
world—in short, outcomes. Outcomes allow us to know something about the
extent to which we have (or haven’t) reached our audiences. Information
about outcomes allows us to strengthen our services. Equally
important, it communicates the value of museums and libraries
to the broadest spectrum of those to whom we account. Without
data, it’s been said, “you’re just another guy with an opinion.”
From IMLS FAQ: Do I have to do this?
“IMLS does not currently require its grantees to
conduct outcome-based evaluation, but it supports and
encourages it as a valuable management tool. At the same
time, IMLS is required to report to Congress in outcome-based
terms; we cannot do that without input from you. We
consider the consistent use of outcome-based evaluation to be
an effective and efficient way for all programs to capture
critically important information and to tell their story
persuasively. IMLS is gradually strengthening information
about outcome-based evaluation in guidelines for its
discretionary grant programs and its program for State
Library Agencies, and is considering the benefit
of making outcome-based evaluation for funded
programs a requirement at some future time.”
Logic Models
Andy’s definition:
A systematic way to plan programs and activities,
applicable to a wide array of interests and disciplines, which :
* aids in identifying and surfacing assumptions
* includes the interests of disparate stake-holders
* clarifies the link between beginning, intermediate,
and final program/project steps
* helps insure there is a logical connection between
the activity plans and the anticipated goal, and
* facilitates the evaluation process.
“Basically, a logic model is a systematic and visual
way to present and share your understanding of the
relationships among the resources you have to operate
your program, the activities you plan, and the changes or
results you hope to achieve.”
Resources
/ Inputs
Activities
Your planned work
Outputs
Outcomes
Impact
Your intended results
Page source: WKKF LMDG, p. 1, (modified Figure 1)
Certain
resources
are needed
to operate
your
program
(Library).
Resources/
Inputs
IF you have
access to
them, THEN
you can use
them to
accomplish
your
planned
activities.
IF you
accomplish
your planned
activities,
THEN you
will hopefully
(sic) deliver
the amount
of product
and/or
service that
you intended.
IF you
accomplish
your planned
activities to
the extent
you
intended,
THEN your
participants
will benefit
in certain
ways.
IF these
benefits to
participants
are achieved,
THEN certain
changes in
organizations,
communities,
or systems
might be
expected to
occur.
Activities
Outputs
Outcomes
Impact
Your planned work
Your intended results
Page source: WKKF LMDG, p. 3, (modified Figure 2)
Real-life example (or two)
UIUC – 2003
OSEDA - p. 39
Bibliography
-- Session handout
1. In-person or Online NNLM classes; right here in Missouri.
Find out more:
http://nnlm.gov/training/schedule/index.html?_prefs=&go=Y
&areg=nnlm&keyword=&region[]=OER#results
2. Sample NNLM Outreach Class (One day):
Measuring Your Impact: Using Evaluation for Library
Advocacy
http://nnlm.gov/evaluation/workshops/
measuring_your_impact/ [accessed 9-7-2010]
Looking Beyond
Action Step 1
Read / Download:
Perspectives on Outcome Based Evaluation for
Libraries and Museums
http://www.imls.gov/pdf/pubobe.pdf
Looking Beyond
Action Step 2
Read / Download:
Association of College and Research Libraries ,The
Value of Academic Libraries, Researched by Oakleaf,
Megan for (ACRL, 2010).
www.acrl.ala.org/value
Looking Beyond
Action Step 3
Self-Guided Course, Shaping Outcomes
Available here:
http://www.shapingoutcomes.org/course/index.htm
[accessed 8-31-10]
Questions?
What is a Library ‘Outcome’ and
How Do You Measure One?
Andy Stewart
Library Director
C. L. Wilson Library
Missouri S&T
[email protected]