COUNTER:
making statistics useful
Peter
Shepherd
Director
COUNTER
January 2007
Background

Understanding usage



Usage statistics






Different approaches
Role of usage statistics
Should enlighten rather than obscure
Should be practical
Should be reliable
Are only part of the story
Should be used in context
How can usage statistics help us measure…



Success?
Value?
Behaviour?
COUNTER
Codes of Practice


Definitions of terms used
Specifications for Usage Reports








What they should include
What they should look like
How and when they should be delivered
Data processing guidelines
Auditing
Compliance
Maintenance and development of the Code of
Practice
Governance of COUNTER
COUNTER: current Codes of Practice

1) Journals and databases







Release 1 Code of Practice launched January 2003
Release 2 published April 2005 replacing Release 1 in January 2006
Now a widely adopted standard by publishers and librarians
60%+ of Science Citation Index articles now covered
Librarians use it in collection development decisions
Publishers use it in marketing to prove ‘value’
2) Books and reference works




Draft Code of Practice published in February 2005 for comment
Final version incorporating feedback was launched March 2006
Relevant usage metrics less clear than for journals
Different issues than for journals


Direct comparisons between books less relevant
Understanding how different categories of book are used is more
relevant
Journal and Database Code of Practice
Usage Reports

Journal Report 1


Journal Report 2


Total searches and sessions by month and database
Database Report 2


Turnaways by month and journal
Database Report 1


Full text article requests by month and journal
Turnaways by month and database
Database Report 3

Searches and sessions by month and service
Code of Practice for books

Book Report 1


Book Report 2


Turnaways by month and service
Book Report 5


Turnaways by month and title
Book Report 4


Number of successful section requests by month and title
Book Report 3


Number of successful requests by month and title
Total searches and sessions by month and title
Book Report 6

Total searches and sessions by month and service
Journal Report 1
Full text article requests by journal
Html and PDF totals
reported separately
COUNTER:
deriving metrics from Journal Report 1

Local metrics





For libraries and library consortia
At journal, collection and publisher level
To compare the cost-effectiveness of journal subscriptions
To assess the value of Big Deals
Global metrics



For authors, funding agencies, libraries and publishers
At article, journal, collection and publisher level
To compare quality and value
COUNTER: ‘local’ metrics

JISC (UK Joint Information Systems Committee)


Funded by UK higher education funding councils
Supports higher education in the use of information and
communications technologies






Access to information and communication resources
Advice on creation and preservation of digital archives
Implications of using ICT
Network services and support
Research to develop innovative solutions
National overview of online journal usage


Develop a reliable, widely applicable methodology
Use COUNTER Journal Report 1 ‘article full-text requests’
Local metrics: JISC Project

COUNTER data was analysed in relation to:




Metrics derived from this analysis









usage range
Price band
Subject category
Trend in number of full-text article requests
Full text article requests per title
Full text article requests per publisher package
Full text article requests per FTE user
Most requested titles
Usage of subscribed vs.. unsubscribed titles
Cost per full-text article request
Cost per FTE user
Summary report available at:
www.ebase.uce.ac.uk/projects/NESLi2.htm
Local metrics: JISC project

Growth in article downloads





Publisher
Publisher
Publisher
Publisher
A: 12%- 208%
B: 12%- 59%
C: 23%- 154%
D: 22%- 81%
Cost per download




Publisher
Publisher
Publisher
Publisher
A: £0.97- £5.26
B: £0.70 - £2.91
C: £0.80 - £3.29
D: £0.45 - £2.26
COUNTER: ‘global’ metrics

Impact Factor







Well-established, easily understood and accepted
Endorsed by funding agencies and researchers
Does not cover all fields of scholarship
Reflects value of journals to researchers
Over-emphasis on IF distorts the behaviour of authors
Over-used, mis-used and over-interpreted
Usage Factor




Usage-based alternative perspective
Would cover all online journals
Would reflect value of journals to all categories of user
Would be easy to understood
Global metrics: UKSG Project

Assess the feasibility of developing and implementing journal
Usage Factors


Level of support from author, librarian and publisher communities
Data from which UF would be derived




Factors in the calculation




COUNTER Journal Report 1?
Article numbers
Process for consolidation, calculation and reporting of UFs
Level of reporting
Total usage
Articles
Report in April 2007


Just completed set of 29 interviews with industry leaders
Wider online survey will take place in February 2007
UKSG Project: feedback





Are the COUNTER usage statistics sufficiently robust?
Frustration at lack of comparable, quantitative data on journals
Should items covered by restricted to articles?
Many journals still have significant usage in print
Diversity of views on the factors in the calculation






Specified usage period
Specified publication period
Usage data is more susceptible to manipulation
Will the journal be a meaningful concept in the future?
Two measures with different limitations are better than one, and UF
will be derived from a set of credible, understandable data
Usage data will be used as a measure of value, whether publishers like
it or not
Current issues

Interface and user behaviour effects on usage statistics



Reporting separately purchasable digital archive usage



Currently all usage for a journal is usually reported together
Separately purchasable archives mean we need separate reports
for archival content, or a year of publication breakdown of usage
Usage in Institutional Repositories




E.g. downloading HTML and PDF of the same article in one
session
COUNTER is testing data filter solutions, but what does the
duplicate downloading signify?
Growth in Institutional Repository (IR) content
Need for credible IR usage statistics
IR usage statistics already being collected, but no standards
What to count and what not to count:

How to deal with partial open access journals
How to deal with journals whose content becomes free after a fixed
time period
New Development - SUSHI

Standardized Usage Statistics Harvesting Initiative
(SUSHI)



No mechanism yet for automatically retrieving, combining,
and storing COUNTER usage data from different sources
NISO-sponsored XML-based SUSHI aims to provide a means
to do just this, via a standard model for machine to machine
automation of statistics harvesting.
COUNTER and NISO have signed an agreement to work
together on the development of SUSHI. More details of
SUSHI can be found at:-
http://www.niso.org/committees/SUSHI/SUSHI_comm.html
Other technologies that may
impact usage reporting
Metasearch






Also known as:
 Federated search
 Broadcast search
User presented with a single search interface
Searches multiple information sources at same time
Eliminates (or assists with) resource selection
Retrieve, consolidates and ranks results
A new challenge for statistics
Metasearch: a new challenge


What can be affected
 Session counts
 Search counts
Why?
 “Search all” option or automatic selection/search
of many resources
 Perform simultaneous activities
 Optimization techniques
Prefetching


Google, and other services are introducing
prefetching of links to improve performance
How it works…
 A user performs a search that results in a
number of results.
 Google will introduce <link> tags for the
first few to cause the browser to fetch in the
background
 The prefetched pages are loaded into
browser cache
Prefetching

If the results were links to publisher full text
then:
 Each prefetch is a full text request
 When the user does click the link, another
header request is made, that could be
considered yet another full text request
 Without some kind of control, this activity
could result in significant over-counting
Prefetching


The prefetch action is identified in the HTTP
header of the request
Vendors can change systems to recognize
prefetch and do one of:
 Simply return an error (don’t deliver full
text), the end user does not see this error, or
 Fulfill the request, but do not count the
transaction
What do usage statistics tell us about…



Success?
Value?
Behaviour?
Usage statistics: measuring value

JISC Project has identified some basic metrics derived
from COUNTER data








Trend in number of full-text article requests
Full text article requests per title
Full text article requests per publisher package
Full text article requests per FTE user
Usage of subscribed vs. unsubscribed titles
Cost per full-text article request
Differences between subject fields
But….


Limited to data from COUNTER-compliant vendors
Does not distinguish between different types of usage
Usage statistics: measuring success

Impact Factor


Widely used as a measure of ‘success’ for a journal
But…




Citation habits vary across different scientific fields
Citation patterns depend on journal type
Do not cover all journals
Usage Factor?



An alternative measure
Relevant in applied fields, where citation levels are lower
But….


Requires global usage statistics
Requires universal adoption of the same standards
Usage statistics: measuring behaviour

Usage of different components of the journal


Variations between fields


TOC, abstract, full-text, references
Physics, medicine
Variations between institutes

Academic



Teaching, research, etc
Between departments
Industrial
COUNTER Membership

Member Categories and Annual Fees
(2007)






Publishers/intermediaries: £530
Library Consortia: £355
Libraries: £265
Industry organization: £265
Library affiliate: £106 (non-voting member)
Benefits of full membership



Owner of COUNTER with voting rights at
annual general meeting, etc.
Regular bulletins on progress
Opportunity to receive advice on
implementation
http://www.projectcounter.org
Apply for COUNTER
membership
For more information……….
http://www.projectcounter.org
Thank you!
Peter Shepherd, COUNTER
[email protected]