Measuring the usage and impact of the Irish Research

An evolving project
John Cox
Deputy Librarian
National University of Ireland, Galway
IReL in Brief
 Established in 2004
 Government-funded
 Focused on research community
 Accessible at 7 Irish universities
 Covers most disciplines
 c. 90 “products”
 25000+ e-journals
 40000+ e-books
IReL Content: STM
ACM Digital Library
EI Village
IOP Journals
SIAM Journals
ACS Journals
Crossfire Beilstein
LWW Journals
SpringerLink
AMA Journals
Derwent Innovations
Index
MathSciNet
Synergy STM Jrls
AIP Journals
Embase
Nature Journals
Taylor & Francis STM
Jrls
APS Journals
Emerald Xtra
OUP Journals
World Scientific
Publishing
Annual Reviews
ESDU Engineering
RSC Journals
Wiley Current Procotols
Biomed Central
Euclid Prime
Science Online
Wiley Ref Works
Biosis Previews
Faculty of 1000
Science Direct
Wiley Interscience Jrls
BMJ Journals
GeoScienceWorld
Scientific American
Web of Knowledge
CAB Direct
Hindawi Journals
Scientific World Jrl
Zentralblatt Mathematik
Cell Press Journals
IEL/IEEE
SciFinder Scholar
110 Single Jrl Titles
IReL Content: AHSS
Blackwell Synergy HSS
Historical Abstracts
Oxford DNB
Sage Journals Online
Academic Search
Premier & Business
Source Premier
International Medieval
Bibliography
Oxford English
Dictionary
SourceOECD
CUP Journals
JSTOR
Oxford Reference
Online
Taylor & Francis HSS
Journals
Econlit
Justis
Philosopher’s Index
Westlaw IE
Factfinder
LexisNexis
Professional
Project Muse
Westlaw UK
Film Index International
Literature Online
PsycArticles
Wilson OmniFile
Global Market
Information Database
Making of Modern Law
PsycInfo
Women Writers Online
Hein Online
MLA International
Bibliography
Routledge
Encyclopedia of
Philosophy
World Development
Indicators
IReL Management
 Part-time, voluntary groups staffed by member libraries
 Steering Group
 Monitoring Group
 Licencing Group
 Web Group
 IT Group?
Monitoring Group: remit
 Collate and monitor performance statistics in relation to the value for
money of IReL titles.
 Collate and monitor downtime of IReL titles.
 Suggest retention or cancellation of IReL titles based on information
gathered.
 Provide summaries of changes of content in IReL major services i.e.
deletions of titles or addition of new titles.
 Note deficiencies of IReL information supply with regard to specific
areas of research.
 Suggest ways of continuing to promote the IReL service.
Monitoring Group: members
 Rosarii Buttimer, University College Cork
 John Cox (chair), National University of Ireland, Galway
 Aoife Geraghty, University of Limerick
 Arlene Healy, Trinity College Dublin
 Jack Hyland, Dublin City University
 Fiona McGoldrick, IRIS
 Niall McSweeney, National University of Ireland, Galway
 Claire Moran, University College Dublin
 Val Payne, National University of Ireland, Maynooth
Activities
 Downtime register
 Usage statistics
 User survey
 Ongoing interaction with:
 Steering Group
 Users
 Vendors
Downtime (known)
Usage Statistics
 Excel templates for e-journals, databases
 Basic quantitative indicators of uptake and value
 Number of downloads/searches
 Cost per download/search
 Top 10 journals per resource according to downloads
 Number and % of journals per download “band”
 Turnaways
 Annual frequency, with trend reports
E-Journal Example
Database Example
Annual Report of IReL Usage
 Mix of figures and commentary
 Summary table of download volumes and costs
 Most downloaded journals overall
 Analysis by download band
 Usage by type of resource
 Trends by discipline
 Comparison with earlier year(s)
Value for Money
Consolidation of Usage
Trends of interest (STM only)
 Strong uptake in nursing, chemistry
 Journal of Advanced Nursing has most downloads
 Cost per download compares very favourably to ILL
 Usage tends to increase over time
 Significant % of journals with <50 downloads
 Lower usage, higher costs for non-journal resources
Compilation Difficulties
 Labour-intensive
 Mix of COUNTER/non-COUNTER data
 Costs – need to factor in:
 IReL /local payments initially
 “maintained spend”
 VAT
 Total consortium figures
 Some vendors slow to respond
 Timing, eg synchronisation with subscription decisions
 Unanswered questions, eg impact, quality, satisfaction?
2007 IReL Impact Survey




Given priority over 2006 usage stats compilation
Essential complement to statistical data
Pre-consultation with researchers and funders
Focused distinctively on:
 Value to researchers
 Purpose of use
 Impact on work
 Satisfaction with coverage
 (Recognised) use of IReL resources
 Role of print
 Access
Who Participated?
 2266 researchers in all disciplines
 Staff
 Research only, eg centres
 Research and teaching
 PhDs
 Research Masters
 7 institutions
Findings of Note
 IReL includes 75% of researchers’ “top 5” journals
 But… gaps include journal backfiles, newspaper archives
 Significant access (eg off-campus) and discovery issues
 Lack of association with IReL
 55% don’t need print copies of IReL journals
Use of IReL
How IReL Benefits Research
 Speed
 Ease of online access
 Coverage, including multidisciplinary
 Currency
 Stronger competitiveness
 Easier collaboration
 New areas of research now possible
How IReL Benefits Teaching
 Faster transfer of ideas to lecture hall
 Integration of online journals in Blackboard
 Easier access to course readings
 Wider choice of sources
 Updated teaching materials
IReL is a Luxury, not a Necessity
IRel is a luxury, not a necessity
1,296 Disagree Strongly
622 Disagree
62 Agree
88 Agree Strongly
157 Don't Know
7.1 %
Don't Know
4.0 %
Agree Strongly
2.8 %
Agree
28.0 %
Disagree
58.2 %
Disagree Strongly
0
100
200
300
400
500
600
700
800
900
1,000 1,100 1,200 1,300 1,400
Discontinuation = “disaster”, “Dark Ages”, “would leave”
Blackwell Synergy: AHSS users
Survey experience
 Labour-intensive
 Seemed to engage senior stakeholders more than stats
 Good on impact, quality of experience
 Influential in likely continuation of IReL funding
 Helpful in identifying specific gaps in coverage
 But important to correlate findings with stats
Different Messages?
Usage Data: areas for development
 Zero use titles?
 Correlation with impact factors?
 Cost per student/staff FTE?
 E-book data
 Outputs, eg researcher publications
In Conclusion
 Usage data and user survey complementary
 Stats
 valuable indicators of activity
 identifiers of uptake and value for money
 guidance on subscription decisions
 what and when?
 Survey:
 actual user experience
 satisfaction levels
 impact, return on investment
 who, how and why?