Gathering eresource
usage data at King’s
College London
CRIStin Summer Meeting, Oslo, 8 June 2016
Anna Franca, Head of Collection Development, King’s
College London, on behalf of UKSG
Overview
 About UKSG and King’s College London
 Why we need statistics
 COUNTER reports
 Collecting the data
 Analysing the data and JUSP
 Challenges
 Final thoughts
Mission
“ UKSG exists to connect the knowledge
community and encourage the exchange
of ideas on scholarly communication. It is
the only organisation spanning the wide
range of interests and activities across the
scholarly information community of
librarians, publishers, intermediaries and
technology vendors”
UKSG Usage Statistics – Practical Skills for
Librarians
King’s College London
• Russell Group research-led
university est. 1828
• c. 33,000 staff and students
• King’s Health Partners, an
AHSC with three London
NHS Trusts
• Period of major growth
 New Faculty of Business &
Management
 Increasing numbers of
international students
 Development of King’s Online
and distance learning
programmes
Library Services
• Six libraries in
central London
• 5.1 million pounds
spent on print and
electronic
resources in
2014-15
Eresources: statistical snapshot
2014-2015:
 50,000 ejournal titles
395,000 ebook titles
690 databases made available
 5.9 million ejournal full text downloads
1.9 million ebook requests
5.9 million database searches
Ongoing investment in collections means more
usage data to collect and manage
Why collect usage statistics?
 To support cancellation or renewal decisions
 To show return on investment
 For management information or reporting
purposes, e.g. SCONUL annual return
 To demonstrate…
The eresources life-cycle
Evaluation as crucial
element of the
eresources lifecycle
 Usage statistics are
required to support
this process
 Helps to determine
if a subscription has
‘value’
Acquire
Renew/
cancel
Review
Make
accessible
Evaluate
“The purpose of the COUNTER Code of Practice is to
facilitate the recording, exchange and interpretation
of online usage data by establishing open,
international standards and protocols for the
provision of vendor-generated usage statistics that
are consistent, credible and compatible”
COUNTER Code of Practice Release 4, p. 3
(http://www.projectcounter.org/r4/COPR4.pdf )
 Launched in March 2002
 Counting Online Usage of Networked
Electronic Resources
 International initiative that sets standards that
facilitate the recording and reporting of online
usage statistics
 Supported by publisher, intermediary and
librarian communities
 First Code of Practice covering journals and
databases published in 2003
 Books and reference works in 2006
 Release 4 is now a single integrated Code
of Practice covering journals, databases,
books and multimedia
 Now considered the gold standard for
measuring usage
The three Cs!
CONSISTENT
CREDIBLE
COMPARABLE
Why is this important?
 Enables like for like comparison of usage statistics
from different publishers
 Reports are easy to understand – no expertise
required
 Data can be manipulated, analysed and imported
into other systems
 By having a standard it makes it easier to
determine the value and impact of our eresources
Standard reports
Report
Description
Journal Report 1
Number of successful full-text article requests by month and Journal
Journal Report 1 GOA
Journal Report 1a *
(Optional report)
Journal Report 2
Journal Report 5
Number of successful gold open access full-text article requests by month and Journal
Number of successful full-text article requests from an archive by month and journal
Database Report 1
Total searches, result clicks and record views by month and database
Database Report 2
Platform Report 1 (formerly
Database Report 3)
Access denied by month, database and category
Total searches, result clicks and record views by month and platform
Book Report 1
Number of successful title requests by month and title
Book Report 2
Number of successful section requests by month and title
Book Report 3
Access denied to content items by month, title and category
Book Report 4
Access denied to content items by month, platform and category
Book Report 5
Total searches by month and title
Multimedia Report 1
Number of Successful Full Multimedia Content Unit Requests by Month and Collection
Access denied to full-text articles by month, journal and category
Number of successful full-text article requests by year-of-publication (YOP) and journal
At King’s we mainly collect and
use…
 Journal Report 1 (JR1)
 Journal Report 1A (JR1A)
 Journal Report 1 GOA (JR1 GOA)
 Book Report 1 (BR1)
 Book Report 2 (BR2)
 Database Report 1 (DB1)
Journal Report 1 (JR1) – Number of Successful Full-Text Article Requests by
Month and Journal
About the
report
Yearly totals
What about Archival and GOA content?
 JR1A provides the total number of successful fulltext article requests from an archive by month and by
journal
 JR5 provides the total number of successful fulltext article requests by year of publication (YOP) and
journal
 JR1 GOA Number of successful gold open access
full text article requests by month and journal
These reports can be used in conjunction with the
JR1 to help institutions determine which usage has
come from a backfile or from older years.
Database report 1 (DB1) – Total Searches, Result Clicks and Record Views by Month
and Database
Database Report 1 (R4)
Institution X
Period covered by Report:
2015-01-01 to 2015-12-31
Date run:
12/01/2016
Database
Natural Product Updates
Natural Product Updates
Natural Product Updates
Natural Product Updates
Synthetic Reaction Updates
Synthetic Reaction Updates
Synthetic Reaction Updates
Synthetic Reaction Updates
The Merck Index Online
The Merck Index Online
The Merck Index Online
The Merck Index Online
Total Searches, Result Clicks and Record Views by Month and Database
Type of activity
Total Feb-15
Jan-15
Reporting Period
Publisher Platform User Activity
0
0
1
Regular Searches
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Searches-federated and automated
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Result Clicks
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Record Views
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Regular Searches
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Searches-federated and automated
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Result Clicks
of Chemistry
pubs.rsc.org
Royal Society
0
0
0
Record Views
of Chemistry
pubs.rsc.org
Royal Society
0
4
56
Regular Searches
of Chemistry
rsc.org
Royal Society
0
0
0
Searches-federated and automated
of Chemistry
rsc.org
Royal Society
0
0
54
Result Clicks
of Chemistry
rsc.org
Royal Society
0
0
54
Record Views
of Chemistry
rsc.org
Royal Society
Subscribed database
Mar-15
0
0
0
0
0
0
0
0
2
0
1
1
Apr-15
0
0
0
0
0
0
0
0
4
0
2
2
Book report 1 (BR1) – Number of Successful Title Requests by Month and title
COUNTER Release 4
Book Report 1 (R4)
Institution X
Reporting Period Total
Number of Successful Title Requests by Month and Title
Period covered by Report:
2015-01-01 to 2015-12-31
Date run:
2016-03-03
Publisher
Platform
Book DOI
Total for all Titles
ABC of Antenatal Care
BMJ Books
EBSCOhost
ABC of Asthma (ABC series) BMJ Books
EBSCOhost
ABC of Clinical Genetics
BMJ Books
EBSCOhost
ABC of Clinical Haematology
EBSCOhost
ABC of Complementary Medicine
EBSCOhost
ABC of Diabetes
BMJ PublishingEBSCOhost
Group
ABC of Eyes
BMJ Books
EBSCOhost
ABC of Heart Failure
EBSCOhost
ABC of Hypertension
Wiley
EBSCOhost
ABC of Nutrition
BMJ PublishingEBSCOhost
Group
ABC of Palliative Care (ABC series)
BMJ Books
EBSCOhost
ABC of Resuscitation
BMJ PublishingEBSCOhost
Group
ABC of Sexual Health
EBSCOhost
Accident and Emergency Nursing
Whurr
(Accident
Publishers
EBSCOhost
& Emergency Nursing)
Action Research in Health Care
Blackwell Publishing
EBSCOhost
Adult Cardiac Surgery: Nursing
Whurr
Care Publishers
and Management
EBSCOhost
Proprietary Identifier
ISBN
ISSN ReportingJan-2015
Period Total
Feb-2015 Mar-2015Apr-2015 May-2015
5055
608
432
547
496
419
320867
978-1-280-19808-3
3
0
0
0
0
0
1046771
978-1-4051-7131-1
2
0
0
1
0
0
320368
978-1-280-28528-8
8
0
0
2
4
2
1006833
978-1-280-19805-2
1
0
0
1
0
0
993766
978-0-585-38461-0
3
0
0
0
0
0
1008571
978-1-280-19809-0
2
0
1
0
0
0
1035140
978-1-4051-4481-0
24
0
0
6
1
4
1046773
978-1-4051-7134-2
6
0
0
2
0
0
1046810
978-1-4051-7135-9
12
1
1
1
0
0
1012719
978-1-4443-1422-9
3
0
0
0
0
0
1046806
978-1-4051-7136-6
7
0
0
0
0
1
1026648
978-1-280-19804-5
3
0
0
1
1
0
993680
978-0-585-38631-7
1
0
0
0
1
0
1028252
978-0-470-03121-6
8
1
0
0
0
0
993153
978-0-632-06347-5
3
0
0
0
1
0
1025529
978-0-470-03129-2
3
1
0
1
0
1
Book report 2 (BR2) – Number of Successful Section Requests by Month and Title
Book Report 2 (R4)
Institution X
Number of Successful Section Requests by Month and Title
Section Type:
Chapter
Period covered by Report
2015-01-01 to 2015-12-31
Date Run:
2016-03-04
Type of section
Publisher
Platform
ISBN
ISSN
Total for all titles
Project MUSE
Low-Wage Work in the Wealthy World
Russell Sage Foundation
Project MUSE9781610446303 n/a
Cyberspace and National Security
Georgetown University
Project
Press
MUSE9781589019195 n/a
Women, Gender, and Terrorism
University of Georgia
Project
PressMUSE9780820341309 n/a
Biopolitics
NYU Press
Project MUSE9780814752999 n/a
Anthropology at the Front Lines of Gender-Based
Vanderbilt
Violence
University
Project
PressMUSE9780826517821 n/a
Beyond the Resource Curse
University of Pennsylvania
Project MUSE9780812206173
Press
n/a
Water Pollution Policies and the American States
State University of Project
New York
MUSE9781438435435
Press
n/a
Transitional Justice
Rutgers University Project
Press MUSE9780813550695 n/a
Europe and China
Hong Kong University
Project
Press,
MUSE9789882208940
HKU
n/a
Venezuela’s Bolivarian Democracy
Duke University Press
Project MUSE9780822394310 n/a
Global Governance and the UN
Indiana University Project
Press MUSE9780253004154 n/a
Child Rights
Purdue University Press
Project MUSE9781612492056 n/a
Cosmopolitanism in the Age of Globalization The University Press
Project
of Kentucky
MUSE9780813134666 n/a
Pax Sinica
Hong Kong University
Project
Press,
MUSE9789882208773
HKU
n/a
Reporting Jan-2015
Period Total
Feb-2015 Mar-2015 Apr-2015
3971
326
379
517
379
367
12
41
6
97
138
4
8
20
4
130
3
3
12
12
111
3
4
27
13
61
0
30
22
3
59
23
13
5
0
58
0
0
58
0
52
5
5
37
1
50
3
0
0
0
50
0
50
0
0
48
1
0
10
0
47
2
0
0
0
46
0
26
0
14
46
0
0
0
0
Access
denied
reports can
help you
evaluate
what your
users are
interested
in
accessing
that you
don’t
already
subscribe
to.
ACCESS DENIED
Journal Report 2 (R4) - Access Denied to Full-Text Articles by Month, Journal and Category
Institution X
Period covered by Report:
2016-01-01 to 2016-04-30
Date run:
05/04/2016
Journal
Publisher
Platform
Total for all journals
Future Medicine
Future Medicine
Total for all journals
Future Medicine
Future Medicine
Aging Health
Future Medicine
Future Medicine
Aging Health
Future Medicine
Future Medicine
Biomarkers in Medicine
Future Medicine
Future Medicine
Biomarkers in Medicine
Future Medicine
Future Medicine
Access Denied Category
Access denied:
concurrent/simultaneous user
license limit exceeded
Access denied: content item not
licensed
Access denied:
concurrent/simultaneous user
license limit exceeded
Access denied: content item not
licensed
Access denied:
concurrent/simultaneous user
license limit exceeded
Access denied: content item not
licensed
Reporting
Period
Jan-16
Total
Feb-16
Mar-16
0
0
0
0
646
207
199
233
0
0
0
0
17
5
4
8
0
0
0
0
40
6
10
22
Collecting the data the old way
 Manual downloads from multiple publisher
websites required to gather usage
statistics
 Administration of different usernames and
passwords to access individual sites
 Many spreadsheets needed to store and
track the data
 Time consuming and labour intensive!
Automation
 SUSHI (Standardized Usage
Harvesting Initiative)
 NISO standard
 Protocol for automating the collection of
usage statistics into local systems
 Replaces time-consuming user-mediated
collection
 Works with ERMs and usage aggregation
services e.g. 360 Counter, Ustat, JUSP
(Journal Usage Statistics Portal)
 Frees up time for evaluation and
analysis
 Removes repetitive and timeconsuming manual process
Cost per use analysis
 Journal C cost institution X a total of £2,500 in
subscription fees in 2015
 The total usage for that year was 6,032 full text
downloads
£2,500 divided by 6,032 = £0.41
This gives us a cost per use of £0.41 per full text
download
What do we do with the data
• Cost per use analysis
• Explore higher cost per use in more detail
• Evaluate core title usage where applicable
• Automatically renew all single ejournal
titles up to an agreed percentage increase
threshold
• Make greater use of JUSP for automated
data aggregation and reporting features
JUSP
 Journal Usage Statistics
Portal (JUSP)
 Single point access to
journal and ebook usage
data
 Designed by and for the
library community
 200 libraries in the UK
signed up and over 80
publishers and
intermediaries
What data is collected in
JUSP?
Journals
JR1 - Journal Report 1
JR1A - Journal Report 1A
JR1 Gold Open Access
Books
Book Report 1
Book Report 2
Book Report 3
Key benefits of JUSP
Evaluates the whole picture by comparing
different deals and including data from
intermediary services
Inform decisions around substitutions or
cancellations by analysing trends and
usage
Saves times by gathering and downloading
usage data directly into library management
systems and usage analysis tools
JR1 excluding backfile usage and GOA
Adding core/subscribed titles
• Core titles are marked up in the KB+
(knowledge Base Plus)
• Information transfers from KB+ to JUSP
• Low usage of subscribed titles might
indicate some titles could be
swapped/substituted
• Low usage of non-subscribed titles might
indicate deal is not good value
Identifying
core titles
Trends over
a six year
period for a
publisher
ejournal
package at
King’s –
shows
usage
increasing
steadily over
6 year
period
SCONUL reporting
Ustat
 Ex Libris service for SFX
customers for aggregating usage
and comparing across platforms
and publishers
 Data loaded in COUNTER format
manually or automatically by SUSHI
 Includes DB1 usage data not
included in JUSP
 Interoperability with JUSP
What else?
 JUSP doesn’t include everything so we
continue to download some reports manually
 Spreadsheets! Unfortunately, still necessary
 Other types of usage report e.g. NCC
statistics
• Other tools: 360 Counter (Proquest), EBSCO
Usage Consolidation
Challenges: factors to consider when
evaluating usage
• OA and archive content
• Title changes and
transfers
• Platform design
• Data mining/crawling
• Impact of discovery
systems
• Aggregator platforms
• Exchange rates
• Journal characteristics
e.g. article length,
quality
• Subject area
• Platform access and
membership fees
• User behaviour
Apples and oranges
 No obligation for vendors to
sign up to COUNTER
 Vendors may provide Non
COUNTER Compliant stats but
there is no standard governing
how they do this
 They can package and present
statistics as they choose
 Difficult to compare different
formats
Encourage NCC publishers to
become COUNTER members
Image taken from Wikipedia (https://en.wikipedia.org/wiki/Apples_and_oranges), 27 April 2016
COUNTER is only half the picture
How can we as Librarians
incorporate other methods of impact
assessment into our process?
Image taken from http://altmetrics.org/manifesto/ 9th May 2015
Final thoughts
• Usage statistics are ONE way the library can
assess the value of its eresource subscriptions
• Important to consider other factors at play…
o Niche/specialist resources
o Changes to teaching/research
o Wider environment
• View all data with caution and balance with
qualitative data
• Start with a question. This will define what you
need to collect and then what to do with the
data
Thank you
Email: [email protected]
Linked In: https://www.linkedin.com/in/anna-fran%C3%A7a-6844a616