One Shot to Teach
One Shot to Assess
Outcome Assessment
For One Shot Library
Instruction Sessions
What do we mean by
assessment?
from the University of Oregon Teaching Effectiveness Program:
“Assessment is the process of gathering and discussing information
from multiple and diverse sources in order to develop a deep
understanding of what students know, understand, and can do with
their knowledge as a result of their educational experiences; the
process culminates when assessment results are used to improve
subsequent learning.”
http://tep.uoregon.edu/workshops/teachertraining/learnercentered/assessing/definition.html
First Things First: Surveys
(this is what we usually do)
• Surveys typically gauge user satisfaction
• Surveys are sometimes useful for self-evaluation
• Surveys of students help us know what they like, not
what they’ve learned
• Surveys of faculty help us know what they like and may
help us to evaluate what we can do better for their
classes
• Surveys are frequently used by libraries to satisfy
accrediting organizations and campus administrators
and, as such, may not really tell us what we need to
know to improve our teaching
Surveys are not
Assessments
…not really
• Surveys do not really help us estimate what learning has
taken place
• Surveys do not tell us what else we need to teach
• Surveys do not tell us what our students don’t know
• Surveys do not tell us if we have really helped our
students find the information that they need to complete
their research projects
So, why do we need
assessment?
• Internal Review – Anecdotal evidence is common, but
how do we really know if we’re succeeding in our
instructional programs?
• Feedback to Instructional Faculty – What did we teach
their students in the short time that we had them?
• Program Improvement – Part of internal review. What
can we do better to make our instructional program firstclass?
• Accreditation – Accrediting agencies now focus on
instructional outcomes. What have students gained from
instruction? Have they learned?
The most important reason
SACS aside, all other accrediting agencies aside, the
most important reason to assess our one-shot sessions
is to find out what our students are learning and what we
can do to improve their learning experiences. Why try to
teach, if we can’t determine that we are really teaching
something?
The goal… back to Oregon
“…the process culminates when assessment results are used to improve
subsequent learning.”
The whole goal of teaching is to enrich the lives of those
who are the recipients of teaching. Assessment is circular
and should lead us back to our primary goal for teaching:
ensuring learning.
How can we assess the
50 minute sessions?
There are many ways to assess learning.
If we establish clear learning objectives and teach
to the objectives, we should be able to assess
student learning.
Right?
If assessment is so
straightforward, why doesn’t
everyone do it?
• Time – Most frequently cited reason for not assessing
one-shot sessions
• Nature of the one-shot instruction session – we see
these students once; follow-up is difficult
• One size does not fit all – library sessions will be as
diverse as the curriculum: English, anthropology,
computer science, biology, nursing, etc., etc.
…and further
• Student cooperation/participation – we can’t force them
to complete an assessment
• Faculty buy-in – ties in with student cooperation
• Difficulty in fitting the library into the syllabus
• Timing – when to schedule the assessment
So, what to do?
A few ideas
• Some libraries use the “one-minute paper” or “half sheet
response”
• Brief, but effective
• Focuses on testing responses to a handful of questions
that center on learning objectives for the session
Sample one-minute paper
questions
1. What is the difference between an indexing/abstracting database like
Basic Biosis (or other database) and a full-text database like EbscoHost?
2. Which kind of database was more useful to your research and why?
3. How can you tell if an article is from a scholarly source or from a popular
magazine?
4. How do you know if a web site is suitable for academic work?
Source: Choinski, E. and Emanuel, M. (2006) The one-minute paper and the one-hour class. Reference Services Review, 34(1): 148-155.
Another idea: minute
paper/muddiest point
Reflective exercise that asks two questions:
What was one new thing that you learned today?
What questions do you still have about things
covered in class today?
Source: Radcliff, C.J., Jensen, M.L., Salem, J.A., Burhanna, K.J., & Gedeon, J.A. (2007) A practical guide to information
literacy assessment for academic librarians. Westport, CT: Libraries Unlimited.
Minute paper/muddiest point
Advantages (according to Radcliff, et al.):
Requires little time and money
Can be administered in the classroom
Little or no need for faculty collaboration
Measures affective, behavioral, and cognitive domains
Minute paper/muddiest
point
Disadvantages (Alderman)
Reviewing/evaluating answers will be time-consuming
Probably not practical for every class
Student participation voluntary
Really should be coordinated before-hand to be most
effective
Another idea:
one sentence summary
Asks the question: Who does what to whom, when, where,
how, and why?
Originally conceived by T.A. Angelo and P. Cross in their
book Classroom Assessment Techniques (1993). (cited by
Radcliff, et al.)
One sentence
summary example
WHO OR WHAT?The Boolean operator “AND”
DOES WHAT?
limits or narrows
TO WHAT OR WHOM?
search results
WHEN?
in a keyword search
WHERE?
in the online catalog
HOW?
by joining two different concepts
WHY?
in order to provide a list of books that are most
appropriate for my topic
Resulting sentence: The Boolean operator “AND” limits or narrows search
results in a keyword search in the online catalog by joining two different
concepts in order to provide a list of books that are most appropriate for my
topic.
Source: Radcliff, et al.
Advantages/disadvantages
Pretty much the same as for the one minute paper:
Easy to administer
Doesn’t take long to complete
Doesn’t require coordination with faculty
BUT
Lots of paperwork to read and assess
Student participation is voluntary
May not be practical for every class
Another idea:
defining feature matrix
Example:
Source: Radcliff, et al.
And there are other assessment
tools
• Multiple choice instruments could be designed for
specific classes to test stated objectives
• Survey and assessment could be used with the same
class to gauge satisfaction/instructor effectiveness and
to assess learning outcomes
and so on…
Ideally…
• Teaching/instructional librarians would have no other job
assignments but teaching and assessment
• Library instruction would be integrated into ALL research
classes
• Teaching/instructional librarians would be able to follow
their trainees throughout the semester and really be able
to assess what they learned
• Academic libraries would have adequate staff to do all of
the above
Realistically…
We’ll find a method that “satisfices.”
This may mean randomly selecting classes for evaluation
This may mean having each teaching/instructional librarian run
evaluation on one class each semester
This may mean identifying willing subject faculty who can include a
formal library component in their research courses
What we plan to do
at the Carpenter Library
• Design a brief web-based questionnaire
• Provide library users who have attended instruction an
opportunity to participate
• Maybe even run a prize drawing to encourage
participation
• Use this as a first step in learning what our students are
learning
• Investigate formalizing a library component within some
of the core research classes at UNF
“Had we but world enough,
and time…” Andrew Marvel
The above was lifted from an article by Donald Barclay in RQ.
More pointedly, Barclay questions
“How can the average teaching librarian conduct
meaningful evaluation?”
His response
“Set your sights lower and do the best evaluation you can
with what you have.”
Source: Barclay, Donald. (1993). Evaluating library instruction: Doing the best you can with what you have. RQ, 33(2): 195-204.
References
Barclay, D. A. (1993). Evaluating library instruction: Doing the best you can with what you have. RQ, 33, 195-202.
Carter, E. W. (2002). "Doing the best you can with what you have:" lessons learned from outcomes assessment. The
Journal of Academic Librarianship, 28(1/2), 36-41.
Choinski, E., & Emanuel, M. (2006). The one-minute paper and the one-hour class: Outcomes assessment for one-shot
library instruction. Reference Services Review, 34(1), 148-155.
Colborn, N. A. W., & Cordell, R. M. (1998). Moving from subjective to objective assessments of your instruction program.
Reference Services Review, 26(3-4), 125-137.
Lindstrom, J., & Shonrock, D. D. (2006). Faculty-librarian collaboration to achieve integration of information literacy.
Reference & User Services Quarterly, 46(1), 18-23.
Mackey, T. P., & Johnson, T. E. (2007). Developing an integrated strategy for information literacy assessment in general
education. JGE: The Journal of General Education, 56(2), 93-104.
Neyer, L. (Fall 2003). Library instruction assessment in upper-level courses. Academic Exchange, 7(3), 85-89.
Rabine, J. L., & Cardwell, C. (2000). Start making sense: Practical approaches to outcomes assessment for libraries.
Research Strategies, 17(4), 319-335.
Radcliff, C. J. (2007). A practical guide to information literacy assessment for academic librarians. Libraries Unlimited.
Saunders, L. (2007). Regional accreditation organizations' treatment of information literacy: Definitions, collaboration,
and assessment. The Journal of Academic Librarianship, 33(3), 317-326.
Wong, G., Chan, D., & Chu, S. (2006). Assessing the enduring impact of library instruction programs. The Journal of
Academic Librarianship, 32(4), 384-395.