Journal level impact assessment
a diversity of new metrics
Sarah Huggett
Publishing Information Manager, Scientometrics & Market Analysis, Research & Academic Relations
25th September 2012
A multi-faceted journal evaluation landscape
2
Journal level metrics: why and how?
Growth in output
Appropriate use
3
Results of a quick poll…
4
Thousands
Growth of scholarly communications
~3% per year
30
20
10
0
<1900
00s
10s
20s
30s
40s
50s
60s
70s
80s
90s
00s
5
Use of journal indicators
“1) For journal editors and publishing houses the impact
factors give market research information. Editors are able
to approximate “their” journals’ standing in comparison
to other thematically closely related periodicals.”
“2) For libraries impact factors are useful for collection
development, especially in combination with further
indicators, primarily journal prices. Combined indicators,
for example, Euros per unit of impact factor or a
combination of this indicator with cost-per-use, may be
effective for serial selection.”
3) Authors get hints on journals in which they can
publish their research results (in case there are any
authors in academia who do not know “their” appropriate
periodicals).”
“In no case is it possible to use
a journal impact factor on the
article level to evaluate the
influence of an article, an
author or an institution.”
6
The skewness of citation distribution in journals
% citations
100
80
60
Cumulative contribution of
articles with different citation
rates to total journal impact.
40
20
0
0
20
40
60
80
% articles
100
7
Citations rates vary across fields
0
0
1
1
2
2
3
3
4
4
5
5
6
6
Multidisciplinary
Biochemistry, Genetics & Molecular Biology
Neuroscience
Immunology & Microbiology
Chemistry
Pharmacology, Toxicology & Pharmaceutics
Medicine
Chemical Engineering
Environmental Science
Agricultural & Biological Sciences
Psychology
Earth & Planetary Science
Materials Science
Physics & Astronomy
Nursing
Health Professions
Energy
Computer Science
Veterinary
Engineering
Mathematics
Economics, Econometrics & Finance
Social Sciences
Business, Management & Accounting
Arts & Humanities
8
Citations rates vary across article types
Citations
Reviews
Short communications
Articles
0
2
4
6
8
10
12
14
16
18
20
Years after publication
9
Popularity metrics
Impact Factor
Source Normalized Impact per Paper
10
Popularity metrics
Journal Citation Reports
11
Source Normalized Impact per Paper
Available on Scopus
Similar to Impact Factor, but considers 3 years
Measures contextual citation impact
Citations weighted by the likelihood of citation in the subject field
of source
Improvements
in progress
Year -3
Year -2
Year -1
Citing year
12
Prestige metrics
EigenFactor
Scimago Journal Rank
13
Prestige metrics
Journal Citation Reports
14
SCImago Journal rank
Freely available at scimagojr.com; on Scopus
Similar to Impact Factor, but considers 3 years
Self-citations limited
Citations weighted by the SJR of the citing journal
Improvements
in progress
Year -3
Year -2
Year -1
Citing year
15
Usage
MESUR
COUNTER
16
Usage
17
Citation Ethics
Why do we cite?
How should we cite?
18
Citations: “a private process with a public face”
1. paying homage to pioneers;
2. giving credit for related work;
“Reward or persuasion?
The battle to define the meaning of a citation”
3. identifying methodology, equipment, etc.;
4. providing background reading;
5. correcting one’s own work;
6. correcting the work of others;
 Coercive citations
 Citation cartels
7. criticizing previous work;
8. substantiating claims;
9. alerting researchers to forthcoming work;
10. providing leads to poorly disseminated, poorly indexed, or uncited work;
11. authenticating data and classes of fact-physical constants, etc.;
12. identifying original publications in which an idea or concept was discussed;
13. identifying the original publication describing an eponymic concept or term, e.g. Hodgkin’s disease;
14. disclaiming work or ideas of others;
15. disputing priority claims of others.
19
The metrics of the future?
20
A multi-dimensional picture
EF
h-index
ImInd
IF
SJR
21
A multi-faceted journal evaluation landscape
Thank you for your attention
Any questions or comments?
22