STUDENT
LEARNING
OUTCOMES
The Holy Grail of Academic Library
Assessment
Lisa Norberg, Barnard College
ENYACRL Spring Conference 2012
Understanding Library Impacts Framework
Learning activities crosswalk
Student use of
the library
- Resources
- Services
- Facilities
During
learning
activities in
the academic
major
Supports
attaining
learning
outcomes
defined &
assessed by
faculty
Communicated
to stakeholders
using
recognized
frameworks
(Tuning &
VALUE)
Critical incident
questionnaire
ULI Framework © Derek Rodriguez, 2012
The Understanding Library Impacts Project
Crosswalk for History at Barnard
Learning activities
Learning expectations
-
-
Getting started
Choosing a topic
Developing a thesis
Gathering evidence
Finding secondary sources
Citing
Writing
Preparing a presentation
Constructed through content
analysis of History rubrics
& program-level expectations.
Subjected to inter-coder
agreement testing
Discipline-specific skills
Thesis & argument
Evidence & analysis
Writing and citing
Barnard
History
department’s
capstone
expectations
The Understanding Library Impacts Project
External
Frameworks
-
VALUE rubrics
Tuning outcomes
for History
Critical incident questionnaire
Library use
- Electronic
resources
- Traditional
resources
- Services
- Facilities
Learning
activities
Helps and
problems
Open ended
&
local questions
Anxiety and
confidence
Developed and refined in
qualitative studies (2006, 2007)
Delivered using the Qualtrics
web-based survey application
The Understanding Library Impacts Project
Demographics
(+ non-traditional
student scale)
Academic
challenge (NSSE)
Most common uses during project at Barnard, %
of respondents, n=24
books
library catalog
e-journals
study space
printers
interlibrary loan
internet search engines
reference question
library instruction
databases
non-library web-site
e-books
e-primary sources
computers
archival materials
reference books
research consultation
0%
20%
40%
60%
The Understanding Library Impacts Project
80%
100%
‘Most important’ library uses by learning activity
and % of respondents at Barnard, n=24
100%
92%
80%
58%
60%
40%
E-Resource
Traditional
Service
Facility/Equipment
20%
0%
The Understanding Library Impacts Project
‘Most important’ uses while ‘gathering evidence’
at Barnard by % of respondents, n=24
books
interlibrary loan
e-primary sources
archival materials
study space
databases
library catalog
e-journals
research consultation
printers
computers
library instruction
reference question
chat reference
e-books
67%
54%
0%
10%
20%
30%
40%
The Understanding Library Impacts Project
50%
60%
70%
Linking use to expectations for learning
92% of respondents used their ‘most important’ electronic resource when
gathering evidence to support a thesis
A time when students demonstrate abilities of …
Locating primary sources
Distinguishing between primary and secondary
sources
Evaluating and interpreting primary sources
Advancing an argument using evidence from
primary sources
The Understanding Library Impacts Project
Assessed by
faculty when
grading
student work
‘Most important’ uses while ‘writing’ at Barnard
by % of respondents, n=24
books
study space
e-primary sources
archival materials
e-journals
printers
consortium-ILL (Borrow-Direct)
databases
research consultation
interlibrary loan
library catalog
0%
10%
20%
30%
The Understanding Library Impacts Project
40%
50%
60%
Linking use to expectations for learning
58% of respondents used their ‘most important’ library facility/equipment
when writing
A time when students demonstrate abilities of …
Advancing an argument using evidence from
primary sources
Communicating the argument in a coherent, well
organized paper
Following discipline specific style standards
The Understanding Library Impacts Project
Assessed by
faculty when
grading
student work
Proportion of respondents reporting helps and
problems at Barnard, by theme, n=24
Access to information
Convenience
Availability of space
Ease of use
Helps
Problems
Help finding information
Anxiety
Affect of staff
Access to tools
0%
20% 40% 60% 80% 100% 120%
The Understanding Library Impacts Project
5%
Challenges by type, n=22
36%
Assignment related
Evaluating information
Finding information
Time management
45%
14%
“Developing a thesis statement to differentiate project from
other scholars.”
“I had way too much information”
“I was worried about finding primary source material”
“I had trouble finding newspaper articles from multiple
newspapers.”
The Understanding Library Impacts Project
Challenges faced by type and activity
at Barnard, n=22
16
14
12
10
8
6
4
2
0
Time management
Assignment related
Evaluating information
Finding information
The Understanding Library Impacts Project
One student’s experience
Student B-16 is aged 18-22, a history major,
and a senior attending college full-time. She
works a job 10 – 20 hours per week.
She used 18 types of library resources,
services, and facilities when working on her
project including reference and a research
consultation
Learning objectives
… to learn to develop a feasible and
original research topic, to develop better
research skills … to learn to engage
meaningfully with my peers' academic work
… engage at a more intense level with the
primary and secondary sources relevant to
my study “
Expected abilities
- select a topic
- develop a thesis
- evaluate and interpret
primary sources
- advance argument using
evidence
Image courtesy of finaid.org
The Understanding Library Impacts Project
Challenge
At a somewhat early point in the project, I realized that the scope of my
project was much too wide … and I had an extremely difficult time
narrowing down my topic appropriately.
Student B-16's most important library uses & challenge by learning activity
getting choosing a developing a
oriented
topic
thesis
gathering
evidence
finding
secondary
sources
citing
writing
e-journals
books
books I checked out from the library
Borrow-Direct
study space
challenge
The Understanding Library Impacts Project
Affect
She was anxious before (4 “agree”) and during the
project (4 “agree”), but reported improved
confidence after completing the project. (3  5
“strongly agree”)
How she overcame her challenge
To narrow my topic, I consulted with professors and my thesis adviser. I also
actually widened my research in an attempt to step back and try to determine
the significant questions regarding my general field of study. My finalized topic
came about in large part due to the suggestions of a professor.
Image courtesy of Hollins University
The Understanding Library Impacts Project
Future work for ULI
id
institutional
datasets
Use
Academic
work
tasks
“These resources were very
easy to use, but without
meeting with library staff, I
would have never known
they were there …”
id
Learning
Outcomes
rubric
scores
grades
attainment
2012-2013
- Repeat projects in History
- Extend to Social Sciences
- Extend to alternative information
services, such as the ERL
- Add assessment results
For more information about ULI
please visit http://bit.ly/rqXk13
The Understanding Library Impacts Project