Building on sand?
Using statistical measures to assess
the impact of electronic services
Dr Angela Conyers
evidence base
research & evaluation
UCE Library Services
UCE Birmingham
Key questions
• Why collect usage statistics?
• What statistics to collect?
• How to collect them?
• How to analyse them?
• How can libraries be assisted in
this task?
• E-measures project funded by HEFCE
working with 25 UK higher education
libraries
www.ebase.uce.ac.uk/emeasures/index.htm
• NESLi2 study of usage statistics funded
by JISC working with 17 UK higher
education libraries
www.ebase.uce.ac.uk/projects/NESLi2.htm
Why collect usage
statistics?
• Because they’re there
• Because SCONUL has asked for them
• To help with promotion and user
support
• For budgeting and decision-making
• To aid bench-marking
What usage
statistics to
collect?
1. Identify type of resource:
•
•
•
•
•
•
Serials
Databases
E-books
Digital documents
Virtual visits
Electronic enquiries
What usage
statistics to
collect?
2. Identify reliable statistics:
COUNTER compliance:
Serials: JR1 report – number of full-text article
requests – 40 publishers/suppliers
Databases: DB1 report – total searches and
sessions by month and database – 13
publishers/suppliers - DB2 report – total
searches and sessions by month and service
– 15 publishers/supplier
E-books: new draft COUNTER code of practice
How to collect?
•
•
•
•
Identify sources of statistics
Collect passwords
Download
Establish where gateway statistics to be
added
• Encourage all suppliers to be COUNTER
compliant
How to analyse?
1. Identify supplementary
information required:
– Cost of deal (subscribed titles and
e-access costs)
– Lists of subscribed titles
– Number of FTE users
– Total library serials budget
How to analyse?
2. Identify groupings:
–
–
–
–
Usage range
Price band
Subject category
Subscribed/unsubscribed titles
Usage range
Nil and low use (under 10 requests)
nil use
1-4 requests
5-9 requests
Medium use (10-99 requests)
10-49 requests
50-99 requests
High use (100 or more requests)
100-499
500-999
1000-9999
10,000 plus
Price band
Unpriced: those for which no price could be
found
Low price – under £200
Medium price - £200-399
High price - £400-999
Very high price - £1,000 and over
Other groupings
Subject category:
STM (science, technology & medicine)
HSS (humanities and social sciences)
Subscribed and
unsubscribed titles
Assessing value for
money
• Average cost per request – all requests
• Average cost per request – subscribed titles
• Average cost per request – unsubscribed
titles
• Cost per FTE user
• Cost per title
• Yield per £ for access to unsubscribed titles
How can libraries
be assisted with
the task?
NESLi2 study main recommendations to
the JISC:
–
–
–
–
Clearer, more transparent deals
Review of pricing and cancellations policies
Portal site for publisher usage statistics
More support to libraries in monitoring usagetoolkit, workshops etc
How can libraries
be assisted with
the task?
NESLi2 study main recommendations to
the higher education library community:
– Libraries to ensure they have sufficient internal
resources to manage and analyse usage data
– Libraries to be encouraged to take a more joined-up
approach to the management of print and e-journal
resources
The future?
•
•
•
•
Workshops
Tool-kits
Benchmarking groups
Outsourcing
Angela Conyers
[email protected]
www.ebase.uce.ac.uk
Some results –full-text article requests
Total usage (all libraries, all publishers)
2500000
2000000
1500000
1000000
500000
0
Jan-Jun 2003
Jul-Dec 2003
Jan-Jun 2004
Some results – costs
Cost ranges of titles
30%
25%
20%
15%
10%
5%
0%
no price
Low (under
£200)
Medium
(£200-399)
High (£400999)
Very high
(£1000 or
over)
Some Conclusions
• Number of full text article requests considerably higher in
large old universities in this study
• Overall increase in usage 2003-4
• Balance of usage and costs generally led to similar range
of costs across large and small universities
• Low average cost per request
• Small % of titles generated high usage
• Low use titles were mostly low price
• High use titles were generally high cost
• STM titles most heavily used.
• Data limitations