Paying Referees?
Stuart Ward
Director Resources
EPSRC
World-leading research requires excellent peer review
•
Good refereeing is vital;
•
EPSRC operates a peer review system that is fair, flexible,
open, easy to understand, and efficient to operate;
•
The College is at the heart of the EPSRC peer review system:
EPSRC Peer Review College
•
•
•
•
•
•
The EPSRC college contains over 4100 members
Wide range of disciplines and backgrounds
15% of members are UK non-academics
11% of members are from outside the UK and are
both academic and non-academic
Members are appointed for 4 years
All members are offered training
EPSRC Peer Review College
College
Grant
Investigators
Female
12%
11%
Non-White
9%
10%
Age: Under 40
19%
35%
40-50
41%
35%
50-60
29%
21%
60+
11%
9%
Peer review process - video
Why do referees do it?
• Sense of community/duty
• Altruism/it’s how science works
• Quality control
• Knowing what’s going on
• Power and influence
But not
• Money
Problems with refereeing
For the Research Council
• Quality
• Timeliness
• Referees do not fully understand the
refereeing process
For the Referee
• Lack of time/Too many other things to do
• Lack of recognition
• Referees do not understand how EPSRC
uses referee reports
Solution?
For the Research Council
• Faster, better refereeing
• Raise prestige
• Train referees
For the Referee
• More time
• More recognition
• Get training
Payments for Refereeing – “Peer Miles”
• Introduced in 2001 to raise prestige of refereeing
and improve response rate
• One “Peer Mile” for a referee’s report
• Two “Peer Miles” for a report returned on time
• At year end “Miles” gained by department added up
• EPSRC distributes £750k (€1.1M) to Departments
for use for approved purposes – conferences,
students support etc
• Each “Mile” worth about £30 (€45)
Payments for Refereeing – “Peer Miles”
Benefits
• Gesture of appreciation - recognition for refereeing in department and by individual, but not a major
motivator
• Payments go to approved purposes - no direct
payments to individuals (and thus no taxation)
• Administration simple – with light touch, audit process
• 95% of heads of departments and 90% of referees
favoured the scheme (2003 survey)
Use of Payments for Refereeing
• visits to conferences
• publication costs
• staff or student development
• teaching seminars – paid for travel to these
events
• travel budgets
• expenses for invited speakers
• equipment for PhD students, small pieces of lab
equipment, bits of software, computer equipment
Value of a “Peer Mile”
£36
30000
27500
£34
£32
22500
£30
20000
Points Earned
Actual Point Value
£28
17500
£26
15000
01/02
02/03
03/04
04/05
Scheme Year (1 Sep to 31 Aug)
05/06
Points Earned
Value of 1 Point
25000
Payments made to academic institutions for
scheme year 2005-2006
£45,000
£40,000
Annual Payment
£35,000
£30,000
3
Payments made to academic
institutions for scheme year
2005-2006
5
£25,000
5
£20,000
7
£15,000
5
£10,000
17
£5,000
79
£0
Institutions
Referee Reports (Grants) Usable & Received On Time
55%
50%
45%
40%
35%
Start of Incentive Scheme
30%
25%
20%
2001
2002
2003
2004
2005
2006
2007
Referees Not Responding - 3 Month Rolling Averages
20%
Start of Scheme
18%
16%
14%
12%
10%
8%
Feb 01
Aug 01
Feb 02
Aug 02
Feb 03
Aug 03
Feb 04
Aug 04
Feb 05
Aug 05
Feb 06
Aug 06
Issues for the future
• Payment to other types of referees? How?
• Value of Fund – should it be increased?
• Introduce a quality factor?
• Provide closer direct feedback of results to referees
and Heads of departments
Thank You
Comments to Stuart Ward, EPSRC
[email protected]