Monitoring of the SEE-ERA.NET Pilot Joint
CALL (PJC) and the implementation of the
Lessons learnt in the SEE-ERA.NET PLUS
JOINT CALL
Marion Haberfellner
Centre for Social Innovation (ZSI), Vienna
Overview of Monitoring Activities:
•
Monitoring Plan established and external monitoring expert contracted (Jan 07)
•
Thematic conferences monitored: Belgrade conference attended, questionnaires analysed
and presentation of results at Bucharest Rep-SEE meeting (Feb-Mai 07)
•
PJC documents and data studied and preliminary analysis (Jun-Oct 07)
•
PJC survey: questionnaire elaborated and finalised in cooperation with SEE-ERA.NET
consortium and INTAS (Sep-Oct 07)
•
Sofia quality assurance meeting, Thessaloniki Steering Board (Oct 07): discussing PJC and
PJC survey
•
Interim monitoring report (Nov 07) – including PR monitoring, PJC monitoring and PJC survey
analysis
•
End monitoring report (August 2009)
2
Remarks on PJC results: country participation
Country
No teams in
eligible proposals
% of SEE eligible
teams
SEE inhabitants
in Mio
% of SEE
inhabitants
Ratio:
teams/Mio
inhabitants
Albania
61
4,30%
3,60
1,55%
16,94
Austria
59
4,15%
8,20
3,54%
7,20
Bosnia Herzegovina
83
5,85%
4,55
1,96%
18,24
Bulgaria
125
8,80%
7,32
3,16%
17,08
Croatia
142
10,00%
4,49
1,94%
31,63
France
44
3,10%
63,71
27,49%
0,69
FYROM
118
8,31%
2,05
0,88%
57,56
73
5,14%
82,04
35,40%
0,89
192
13,52%
10,71
4,62%
17,93
Hungary
62
4,37%
9,96
4,30%
6,22
Montenegro
50
3,52%
0,68
0,29%
73,53
Romania
83
5,85%
22,28
9,61%
3,73
Serbia
227
15,99%
10,15
4,38%
22,36
Slovenia
101
7,11%
2,01
0,87%
50,25
1420
100,00%
231,75
100,00%
Germany
Greece
SUM
3
Remarks on PJC results: country participation
•
In absolute numbers, most teams have participated from Serbia (227), Greece
(192) and Croatia (142)
•
Number of teams compared to number of inhabitants per SEE country:
Montenegro, Slovenia and FYROM show highest participation to population ratio
•
All WBC countries have a high participation rate. Due to:
- PJC targeted at these countries
- consortium condition: at least one WBC per application
- but shows also that ALL WBC took advantage of the PJC and that there is huge
interest for developing scientific cooperation in the region and with EU members
•
EU member states:
- Slovenia, Greece and Bulgaria show highest participation – closer to region,
established scientific networks
- other EU member states relatively lower participation, especially the bigger
countries FR, DE – low budget, consider awareness raising measures for RJC in
these countries
4
PJC Survey - Results: Comments on positive aspects
•
Established cooperation with colleagues in the region
•
Perspective of future projects with higher budgets
•
Knowledge and results sharing
5
RECOMMENDATIONS/Funding and Call Budget

Better steering of financial resources of the call: countries should foresee
reserves to prop up their share of call funding, if necessary. Such adding of
financial resources would allow funding more and the most excellent projects, and
prevent exhaustion of funds.

Another alternative to avoid exhaustion of funds would be a “real common pot”
scheme, where each country transfers its contribution to the call in a common pot.
In this scheme the joint funds would be used irrespective of national origin and be
distributed to the most excellent top ranking project proposals. The principle of
“juste retour”, where each country receives the funds back that it invests in such a
call, could not be guaranteed any more. But administrative procedures and project
selection procedures would be much simplified.

Increase funding per project and the call budget overall (feedback of
participants)
6
RECOMMENDATIONS/Procedures

Consider simplifications of the electronic submission for project
applications (too much information requested for size of projects,
duplication of requested information on the submission system)

Consider establishing one single multilateral contract per project, to
reduce administrative procedures for scientists.

Adapt the number of different instruments to the call budget; keep
the call simple and avoid offering a too broad range of instruments. Too
many instruments add complexity and increase administrative cost and
procedures.
7
RECOMMENDATIONS/Procedures

Make the rules of the game sufficiently clear and avoid
changing them during call implementation, project selection and
contracting (virtual common pot concept, eligibility and evaluation
criteria)

All of this needs to be made sufficiently clear and published with
the call document.

Evaluation criteria should be adapted to the funding instrument.

Transparency of evaluation should be increased overall.
8
Remarks on PJC results: „Rules of the game“
•
Not all important “rules of the game” for the PJC call were fully
made clear to the applicants.
•
Examples:
- which entities are eligible for receiving funding
- project selection: VCP procedure has not been made clear in
call guidelines
- Summer Schools: rule of 50% minimum participation from WBC
had not been outlined in call guidelines
9
Remarks on PJC results: procedural weaknesses in
the evaluation
•
Some assessment/evaluation procedures of the PJC were not
fully appropriate.
•
Examples:
- the same assessment (evaluation) criteria were applied for all
three different instruments (RP, NP, SS)
- different weights were applied for evaluation criteria
- discrepancies between two evaluations: when shall a third
evaluation be requested?
- technical: an evaluator mixed up the evaluation sheets of two
projects for SS
10
RECOMMENDATIONS/PROCEDURES:
EVALUATION

Consider establishing a scientific council (SC), composed of well
renowned scientists.

This council would support the project selection process by screening
those remote evaluations for appropriateness, which show high
discrepancies in scores and by checking and adjusting the ranking list of
proposals selected for funding.

Call: in relation to the size of the call in financial terms and in terms of
projects to be funded. (small scale call: few colleagues, might consist of
scientists not originating of countries involved in the consortium so as to
keep this institution neutral)
11
RECOMMENDATIONS/PR and Awareness raising
activities:

Promotional conferences for a call: useful tool for consortium
building and networking. Adapt the number and duration of
such conferences to the size of call; (a call with a modest
budget & a limited number of projects to be funded: promotional
measures accordingly be limited and well targeted at countries
with low participation in the pilot call)

Take additional promotional measures for the call in those
countries, where turnout of scientists was rather low, budget was
not exhausted and which participation is essential for networkbuilding for scientists from WBC (e.g. DE, FR).
12
RECOMMENDATIONS/Success rates:

Better steering of the success rate: consider implementing a
future call in a two step procedure. In a first step only short
outline proposals of few pages, in which the scientific idea and
approach is briefly described, shall be requested. These outline
proposals shall be evaluated and only best ranked project
consortia be invited to submit a full proposal. This procedure
helps steering the success rate, reduces administrative effort on
the side of scientists and avoids frustration.

Another option for steering the success rate would be to have a
more narrow topical focus of the call.
13
RECOMMENDATIONS/Success rates:

In general it seems sensible to make a more profound analysis of
possible topics of the call, by considering for example the
scientific strengths of the WBC. Results of ongoing networking
projects for the WBC region (e.g. INCO-NET WBC) should be
taken into account here.

Support of a call implementation agency (INTAS) proved
essential for the pilot call and will be sensible also for the success
of a larger scale joint call. Synergies with other regional
ERA.NETs funded under FP7 should be considered here.
14
Remarks on PJC results: Success rate
•
Success rate is rather low.
Certainly it can be justified with the argument of the Pilot phase (testing for
interest in such an activity); to a lesser extent with the limited effort for preparing a
proposal.
•
For the Real Joint Call the expectations will be a bit different. Target a better
success rate, e.g. by calling for still more focussed topics, by limiting the number
of different instruments, etc.
•
Team success rate:
- in absolute figures Greece, Serbia, FYROM most successful
- in relation to eligible teams France, Germany and Hungary;
- countries with high number of eligible teams pay to a certain extent price of this
success: teams cannot be funded, exhaustion of funds - VCP
15
Remarks on PJC results: Success rate per
instrument
100%
80%
60%
215
70
36
321
19
7
5
31
Research Projects
Networks
Summer Schools
Total
40%
20%
0%
Funded projects
Eligible projects
16
Remarks on PJC results: Success rate per country
Country
No teams in
funded proposals
Team success
rate funded
No teams in
eligible proposals
Team success
rate eligible
Albania
8
5%
61
13%
Austria
8
5%
59
14%
Bosnia & Herzegovina
8
5%
83
10%
Bulgaria
13
8%
125
10%
Croatia
14
9%
142
10%
France
8
5%
44
18%
Former Yugoslav Republic of Macedonia
15
10%
118
13%
Germany
11
7%
73
15%
Greece
18
12%
192
9%
Hungary
9
6%
62
15%
Montenegro
6
4%
50
12%
Romania
6
4%
83
7%
Serbia
18
12%
227
8%
Slovenia
13
8%
101
13%
1420
11%
Sum
155
17
SEE-ERA.NET contacts
Monitoring team
Manfred Spiesberger (external expert)
Email1: [email protected]
Email2: [email protected]
Centre for Social Innovation:
Marion Haberfellner (previously Peter B. Mayr)
CNRS:
Jean-Luc Teffo
Celine-Delacourt Gollain
National Authority for Scientific Research in Romania
Iulia Mihail, Anca Ghinescu, Alexandra Gurau
18
Lessons learnt for the SEE-
ERA.NET PLUS Joint Call
19
LESSONS LEARNT FROM THE PJC 1/1
FUNDING

Funding increased – for the complete call budget and for each individual
project
Not feasible:

Virtual common pot:

No national reserves for propping up of funds due to financial crisis
PROCEDURES

Only one instrument (research projects)

one single multilateral contract per project

Rules of the game clear
However: Different national eligibility criteria
20
LESSONS LEARNT FROM THE PJC 1/2
EVALUATION

Scientific Council established
PR/AWARENESS RAISING ACTIVITIES

PR increased for countries with low turnout (France, Germany)
SUCCESS RATES

Two-step procedure

Reduction to two topics

Selection of topics: results of consultations in the wbc-inco.net
project
21
SEE-ERA.NET PLUS contact details
Coordinator:
Centre for Social Innovation (ZSI)
Marion Haberfellner
[email protected]
[email protected]
Linke Wienzeile 246
A-1150 Vienna, Austria
Phone: + 43 1 49 50 442 - 67
Mobile: + 43 699 114 802 80
Fax:
+ 43 1 49 50 442 – 40
http://plus.see-era.net
http://www.see-era.net
22