FP7 Energy Mid-Term
Evaluation
Multiple framework services contract
with re-opened competition for impact
assessments and evaluations
TREN/A2/143-2007
Final Report
Client: DG ENERGY
ECORYS Nederland BV
Robert Williams
Koen Rademaekers
Sil Boeve
Matthew Smith
Rotterdam, July 2010
Final Report - Mid Term Evaluation of NNE FP7 Projects Supported by DG ENERGY
Table of contents
Executive Summary
0
1 Introduction
1.1 Purpose and Status of this Report
1.1 Overview of method
4
4
4
2 Relevance / Utility / Acceptability
2.1.1 Definition
2.1.2 Specific Questions and Summary Answers
2.2 Intervention Logic / Objective Tree
2.2.1 Fundamental Objective of the Programme
2.2.2 Specific Objectives
2.3 Consultation Views
2.3.1 Evolution of calls in FP7
2.3.2 Fewer Projects with Less Partners
2.3.3 Nature of Projects Requested and Supported
2.3.4 Picking Winners and Research vs. Demonstration
2.3.5 Balance Between Renewable Energy and Energy Efficiency
2.3.6 Quality of Supported Research
2.3.7 Increased support for International (non EU) partners
7
7
7
10
11
11
13
13
13
15
16
17
18
19
3 Efficiency
3.1.1 Definition
3.1.2 Specific Questions
3.2 Participant Survey
3.3 Consultation Views
3.3.1 Efficiency improvements that have occurred
3.3.2 Areas for improvement
3.3.3 Project management and external experts
3.3.4 Project Applications and Evaluation Procedures
3.3.5 Co-ordination with IEE
20
20
20
22
23
23
24
25
25
27
4 Effectiveness
4.1.1 Definition
4.1.2 Specific Questions
4.2 Indicators
4.3 Measuring progress
4.3.1 Commercial vs. Research Impacts and Benefits
28
28
28
29
30
32
FP7 Energy Mid-Term Evaluation
4.3.2 Indicators of Project Level Impact – Carbon Savings
4.3.3 Programme Level - Strategic Added Value
33
34
5 Consistency
5.1.1 Definition
5.1.2 Specific Questions
5.2 Project Survey Findings
5.3 Project Coordinator and Participant Interviews
36
36
36
37
37
6 Allocative / Distributional
6.1.1 Definition
6.1.2 Specific Questions
6.2 Programme Data Analysis
6.2.1 Total Allocations
6.2.2 Allocation by Activity
6.2.3 Specific Activity Area
6.2.4 Geographic Spread
6.2.5 Network Analysis
6.2.6 SME Participation
6.2.7 New Member State Participation
6.2.8 Gender Balance
6.3 New vs. Repeat Applicants
39
39
39
41
41
43
45
47
49
49
51
51
52
7 Sustainability
7.1.1 Definition
7.1.2 Specific Questions
7.2 Strategic Energy Technologies (SET) Plan
7.2.1 Consultation Views on the SET plan
7.2.2 Officer Views of the SET plan
7.2.3 Framework Programme Development
7.2.4 Longevity of Impacts and Improving Dissemination
53
53
53
54
55
57
58
59
8 Conclusions and Recommendations
8.1 Programme Relevance / Strategic Fit
8.2 Information to Applicants
8.3 Programme Management
8.4 Project and Programme Impact
8.5 Programme Future Direction
62
62
62
63
64
65
Annexes
66
1 Context
1.1.1 High Level Policy Drivers
1.1.2 Energy research under FP7
1.1.3 Division between DG Energy and DG RTD
1.2 Lessons from previous evaluations
1.3 Programme History and Evolution
67
67
69
70
71
76
Final Report - Mid Term Evaluation of NNE FP7 Projects Supported by DG ENERGY
2 Strategic Energy Technology (SET) Plan
2.1 Overview
2.2 Wind
2.3 Solar (PV)
2.4 Solar (CSP)
2.5 Electricity Grids
2.6 Carbon Capture Storage (CCS)
78
78
80
82
82
84
85
3 Evolution of calls
90
4 Participant Survey
4.1.1 Project Information
4.1.2 Relevance and Effectiveness
4.1.3 Efficiency
4.1.4 Results and Outputs
4.1.5 Project Impacts
4.1.6 Other comments
4.2 Past Beneficiary Survey
95
95
97
104
108
112
119
119
5 List of Consultees
123
6 Stakeholder Interviews
6.1 Evolution of calls in FP7
6.1.1 Fewer Projects with Less Partners
6.1.2 Large vs. small projects
6.1.3 Mix of technologies in approved projects and calls
6.1.4 Mix of Technologies and SET Plan Alignment
6.2 Quality of Supported Research
6.3 Increased support for International (non EU) partners
6.4 Measuring progress
125
125
125
126
127
127
130
131
131
7 Project Participant Interviews
7.1 Relevance / Utility
7.1.1 Global Competitiveness
7.1.2 Measuring the Impact
7.1.3 International Co-Operation
7.2 Efficiency
7.2.1 The Application Process
7.2.2 Project Size
7.3 Effectiveness
7.3.1 FP7 Objectives
7.3.2 R&D benefits Vs. Commercial benefits
7.4 Sustainability
7.4.1 Future focus for FP7
7.4.2 Strategic Future of FP
7.4.3 Longevity of Impacts
7.4.4 Improving Dissemination
7.4.5 SET Plan
134
134
134
135
135
137
137
139
141
141
143
144
144
146
147
147
149
FP7 Energy Mid-Term Evaluation
8 Network Analysis of FP6 and FP7
8.1 Energy FP6
8.2 Energy FP7
150
151
156
Final Report - Mid Term Evaluation of NNE FP7 Projects Supported by DG ENERGY
Glossary
CCS
CHP
CSP
CT
DG
DG ENER
DG ENTR
DG ENV
DG INFSO(C)
DG RTD
DG TREN
ECAS
EE
EERA
EII
EIT
EU ETS
EU-12
Carbon Capture and Storage
Combined Heat and Power
Concentrated Solar Power
Carbon Trust (UK)
Directorate General
Directorate General Energy
Directorate General Enterprise and Industry
Directorate General Environment
Directorate General Information Society and Media
Directorate General Research and Technological Development
Directorate General Transport & Energy
FP7 Energy project IT system
Energy Efficiency
European Energy Research Alliance
European Industrial Initiative
European Institute of Innovation and Technology
European Union Emission Trading Scheme
Bulgaria, Cyprus, Czech Republic, Estonia, Hungary, Latvia, Lithuania,
Malta, Poland, Romania, Slovakia and Slovenia.
EU-15
Austria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland,
Italy, Luxembourg, the Netherlands, Portugal, Spain, Sweden and the
UK.
EU-27
All 27 members of the European Union (see EU-15+EU-12)
FP (5, 6, 7, 8) Framework Programme
GHG
Greenhouse Gases
ICT
Information and Communication Technologies
IEE
Intelligent Energy Europe
IPR
Intellectual Property Rights
JRC
Joint Research Centre
JTI
Joint Technology Initiative
KIC
Knowledge and Innovation Community
KPI(s)
Key Performance Indicator(s)
KTP
Knowledge Transfer Partnership (UK)
MDG
Millennium Development Goal
MS(s)
Member State(s)
NEF
FP7 Energy project IT system
NMS(s)
New Member State(s) – see EU-12
R+D
Research and Development
RES
Renewable Energy Sources
SET
Strategic Energy Technology plan
SETIS
Strategic Energy Technology plan Information System
SME(s)
Small Medium Enterprise(s)
ToR
Terms of Reference
WP
Work Programme
FP7 Energy Mid-Term Evaluation
Executive Summary
This report presents the mid term evaluation of the non nuclear energy research and
demonstration projects supported by DG Energy 1 under the 7 th Framework Programme
(FP7).The approach followed has been:
•
•
•
•
Programme data an alysis: A review of the available data on applications and
approved projects.
Literature review: To illustrate the policy context and to help update / reconstruct the
objectives of the programme.
A participant survey : Covering all project partners involved in the target group of
FP7 projects plus another survey of recipients of FP6 funding who have not yet
applied for FP7 funds.
Consultations: A series of structured interviews with high level stakeholders, DG
officers and project coordinators and partners f rom 10 current projects.
The report presents our findings and conclusions against each of the main evaluation
questions and then presents and describes a series of recommendations structured via the
main stage s of the programme life cycle. The evaluation questions, our findings and
recommendations are summarised below.
Relevance / Utility / Acceptability
“To what extent are the objectives of a public intervention (project, programme or policy)
appropriate regarding the needs perceived and the problem t he intervention is meant to
solve?”
“The extent to which stakeholders accept the policies in general and the particular
instruments proposed or employed.”
•
•
•
•
•
•
1
Programme understanding among applicants is ok but could be better.
Projects see a weaker link to co mpetitiveness than to enhancing research .
Past applicants are still interested in the FP and do not feel excluded.
The SET roadmaps need to be explicitly linked to FP, this will become easier over
time.
RES appears to get more funding than EE, though other EU programmes improve the
ratio. An important reason is the lack of successful EE market mechanisms .
There is a need to retain funds for smaller and more research focussed projects – DG
RTD do this, but the ENER link to their activities should be retained .
To aid simplicity this report refers to DG ENERGY throughout although prior to 2010 it was actually DG Transport and Energy
(TREN).
Final Report - FP7 Energy Mid-Term Evaluation
0
•
•
•
•
•
•
There is a need to i mprove the clarity on the definition on ‘demonstration’ in the
application material.
The participants regard themselves as among the best in the world. This is hard to
prove with the only real proof becoming apparent ex-post. Finding this proof requires
increased evaluation.
The current amount of i nternational (non EU) co-operation is acceptable offering a
balance between access to expertise and market vs. leakage and non EU benefits.
Applicants should be required to estimate the CO2 saving potential from their bids ,
though guidance is needed, it does not need to be precise and it should not have an
increased role in ranking bids .
The strategic fit of the programme has improved since FP6 and during FP7 – this
reflects the larger and fewer projects and SET plan alignment.
Policy influence is positive but view on importance and impact is mixed.
Efficiency
“How economically have the various inputs been converted into outputs and results? Were
the (expected) effects obtained at a reasonabl e cost?”
•
•
•
•
•
•
•
SET alignment has brought benefits of reduced officer workload.
Time from approval to contract still too long, though the admin burden is ‘as
expected’ for the majority.
Previous FP experience doesn’t appear to reduce the time taken to administer
projects, though it does lead to a slight increase in application time.
The retention and recycling of unspent budget at the end of the programme should be
investigated.
Application and evaluation procedures – suggestions:
o Formalise pre submission meeting procedures and applicant interviews .
o Off site (remote) evaluations preferred over on site.
o A more inclusive end of evaluation meeting.
Programme management - suggestions:
o Project management KPIs .
o Increased ease of access to external expertise.
IT systems, some issues, but improving and outsourcing makes it hard to address.
Effectiveness
“What effects (impacts) have been obtained by the intervention and, in particular have these
effects contributed to the achievement of the objectives of the intervention? ”
•
•
•
•
CO2 savings are suggested as the best indicators despite some participant resistance.
Patents and citations are not the ideal indicators for a demonstration focussed
programme, though they are still of interest.
The SET plan KPIs will be of great assist ance in measuring project success.
Commercial participants are more interested in and optimistic on commercial
benefits, than research focused participants. This may seem obvious but it explains a
number of tensions within projects and with regard to defin ing acceptable measures
of success.
1
Final Report - FP7 Energy Mid-Term Evaluation
Consistency
“The extent to which positive / negative spill over into other social, economic or
environmental policy areas are evolving?”
•
•
Multiple socio- economic benefits have been identified, as spin off benefits from the
project core, these include::
o Employment, environment, economic, health, networking and skills.
On the issue of en ergy prices, the project objectives are to improve cost effectiveness
and efficiency, which should; reduce costs to the consumer but energy prices and
subsidies are complex a nd set at MS level.
Allocative / Distributional
“The extent to which disproportionate negative / positive distributional effects of policies are
evolving.”
•
•
•
•
Programme data analysis shows:
o 89% of the programme budget to date has b een spent in the first 4 calls.
o Major drop in applicant numbers from 231 in the first call to 45 in the fourth.
o Major reduction in areas open for bidding.
o Increase in chance of success, from 11% to 33% or 1 in 3.
o The average total project value has increased from 6.1 MEUR to 17.6 MEUR
(8.9 MEUR on average over the 4 calls) .
o The industrial match funding has increased over time.
SME participation
o 19-20% which is above average (13%) and target (15%) .
o Have much to offer but no desire for quotas .
NMS
o Can add value, but quotas generally resisted .
Clustering and networking
o Old MSs dominate - 95% of funding goes to EU 15 .
o Research organisations are still highly relevant to the programme but its
change in focus has lead to an increase in industrial partic ipation from 47%
to 64% in terms of EU contribution granted .
Sustainability
“Will the effects achieved last in the medium or long term?”
•
•
SET plan alignment is vital to the future nature of the DG ENER FP.
o Misunderstanding of RTD as well as demo scope .
o KPIs need to be referenced .
o SET can learn from the experience of FP.
o Debate over where to sit the FP in the SET.
Dissemination
o Academic routes adequate for their purposes .
o Some support for networking and joint promotion (EU facilitated) .
o Clarification of pos t project funding would be helpful.
Final Report - FP7 Energy Mid-Term Evaluation
2
Recommendations
The recommendations have been structured in line with the programme life cycle.
•
Information to applicants
o Clearly state the programme objectives to applicants.
o Clarify that SET includes research as wel l as demonstration.
o Clearer description of ‘demonstration’ .
•
Programme relevance / strategic fit
o Maintain communication with DG RTD on positioning / focus of calls etc.
o Clearly reference the SET plan in FP ENER calls .
o Clearly articulate energy efficiency a s a top objective and analyse the whole
of the FP for energy efficiency related projects.
•
Programme Management
o Trial remote evaluations of applications, with an on site briefing.
o Clarify the position on the availability of DG officers for pre submission
meetings with applicants.
o Request applicant interviews for the largest projects.
o Trial a longer and more inclusive, end of evaluation decision meeting.
o Develop and circulate project management KPI checklist.
o Clarify position on availability of expert advise rs to DG officers.
•
Project and Programme Impact
o Add a post project obligation and payment to report on project impact post
completion.
o Request an estimate of CO 2 potential savings in all applications.
o Trial collective presentation of FP projects at a high profile trade show /
conference.
o Clarify position, and circumstances, for additional funds at project end.
•
Programme Future
o Accepting that the SET plan is key to the future of the FP Energy projects, a
debate needs to be had on how, and if, the FP should retain its identity.
o The SET plan operation should learn for the experience and knowledge of
the FP.
o DG ENER need to continue to make their voice heard in the evolution and
development of the wide range of EU activity starting up in relation to low
carbon research, development and demonstration, in order to try to m aximise
the synergies with other EU spend .
3
Final Report - FP7 Energy Mid-Term Evaluation
1 Introduction
1.1
Purpose and Status of this Report
This is the final report for the Mid-Term Evaluation of the Seventh Framework Programme
(FP7) Energy p rojects supported by the Directorate General for Energy (DG ENER).
Our intention is to meet the requirements for the final report as defined in the Terms of
Reference (TOR) - which are:
"A final report that will take into account the commission's comments and requests. The final
report, containing all relevant information shall also provide a written summary aimed at a
non -specialised audience. As all impact studies shall be available to the public, no form of
confidential data shall be contained in the final report (they shall be included in a separate
Annex)".
Executive summary
intended for non
specialists, followed by a
summary of answers to
each evaluation question
then recommendations.
To this end, the report has an executive summary designed to be understandable by non
specialists and is structured with an introduction followed by the findings presented against
each of the primary evaluation questions. Each of the primary evaluation questions is defined
and then our reasoned and justified answers to the specific questions of relevance from the
Terms of Reference are given. The main findings from each of the research methods used
(literature review, data analysis, participant survey and consultations) are then presented. The
report concludes with a s ection which summarises our key findings and recommendations
structured against the programme life cycle.
The annexes to the report include the detailed results of the participant survey, a detailed
review of relevant policy and previous evaluations, a re view of programme evolution and the
SET plan, the detailed findings from the stakeholder and participant surveys and a list of
consultees.
1.1
Overview of method
Figure 1.1 outlines the methodological approach we have taken to this study.
Final Report - FP7 Energy Mid-Term Evaluation
4
Figure 1.1
Methodology
Literature review of
policy and previous
evaluations.
Our desk research has involved a review of the previous evaluations of relevance to this
work as well as the programme documentation and the fas t evolving policy analysis in this
field from information sourced by ourselves and signposted during consultations.
Consultations with
experts, DG officers and
participants.
Our consultations have involved a number of groups and methods. In consultation with the
Commission we contacted a number of high profile experts in the field of low carbon
research and development and asked them to act as an expert panel for our project. The
purpose of this panel was to help us ensure that our work is in line with th e most current
thoughts in the field. To this end the initial consultation with the panel members covered the
intervention logic for the programme, the survey questions we proposed, and the structure
and nature of the calls to date, and in the future. The next stage of interaction with the expert
panel concerned the draft conclusions and recommendations of the draft final report.
We carried out structured interviews with a broad cross section of Commission officers from
DG Energy and DG Research. These vi ews were very valuable in both testing the opinions of
5
Final Report - FP7 Energy Mid-Term Evaluation
others, testing the survey results and in collecting ideas and suggestions for improving the
programme.
We also interviewed a number of high level stakeholders. These individuals were identified
and selected with the assistance of the Commission as being able to give an excellent insight
into the thinking and decisions behind the programme we are evaluating as well as giving
high level opinions on the current thinking in the sectors of most relevance, i.e. renewable
energy and energy efficiency. We also carried out individual interviews with over 15 current
project participants, from an agreed sample of 10 current projects, who are by definition
important stakeholders in relation to the energy aspects of the FP.
Review and analysis of
programme data.
We have reviewed and analysed the data on the projects funded to date to seek insight in to
the distribution of funding by nature of applicant and subject matter. In addition to this
analysis we have reviewed the strength and nature of the networking between organisations
and countries that the programme has enabled.
An online survey of 2 major groups of Framework Programme participants was carried out:
On-line survey of current
and previous
participants.
1. Current participants (project co -ordinators and partners) in FP7 Energy
projects – this covered the 49 projects approved and funded under the 2007 and
2008 calls.
2. Past participants in FP6 but not FP7 – principal contractors or project co ordinators of FP6 Energy projects that as far as we could determine were not
receiving for FP7 Energy funding.
Table 2.2 Shows the response rates that were achieved. The survey for current FP7
participants achieved a remarkably high response rate of 46%, with over 220 res ponses. A
good response rate of 20% was also achieved among group 2. Response rates of this size will
give a highly representative indication of the views of FP7 energy participants and high
levels of confidence can be applied to the results of the current participant survey.
High response rate on
participant survey.
Table 2.2 Survey response rates
Contacts
emailed
Inactive /
Opted out
emails
Total actual
survey
audience
Responses
received
As % of
total
audience
1. Current FP7 Energy
participants
510
33
477
221
46%
2. FP6 (but not FP7)
participants
251
49
202
40
20%
Group
The survey questions and detailed replies are included as an annex to this report.
Final Report - FP7 Energy Mid-Term Evaluation
6
2 Relevance / Utility / Acceptability
2.1.1
Definition
Relevance and Utility “To what extent are the obje ctives of a public intervention (project,
programme or policy) appropriate regarding the needs perceived and the problem the
intervention is meant to solve?”
Acceptability “The extent to which stakeholders accept the policies in general and the
particular instruments proposed or employed .”
2.1.2
Specific Questions and Summary Answers
The TOR contains the following key issues and questions which we have classified under this
heading. Each question is followed by our opinions based on the data, analysis and opini ons
that we have collected.
What is the perception of the supported activities by external actors (consortium members and
coordinators, especially for the industry sector)? What could be the approach to better match their
expectations?
Participants perceive a
good match between the
programme and society’s
energy needs.
This issue was much discussed with all of those consulted. The participant survey showed
that they perceived a good, and improved, fit between the programme objectives and their
own and soc iety's wider energy needs.
Past participants still see
the programme as
relevant to them.
An interesting finding of the participant survey was the improvement that respondents saw in
terms of the strategic fit of the programme since previous FPs and also during FP7. This is a
vindication of the change in programme targeting and the alignment with the SET plan.
Past participants in the DG Energy part of the FP6, who had not yet applied for FP7 funds
were surveyed with regard to why they have not reapplied, in order to see if the programme
was somehow missing their needs. It appears that there was no strong perception of this and
that these past applicants would reapply if a suitable project became available.
The most popular reason given by FP6 participant s who have not received FP7 funds for not
yet applying is that a suitable project for them to participate in has not yet come along,
another reason quoted was the lack of a suitable call for their area of research. This indicates
that some researchers are being excluded, however the reduction in the breadth of the calls
was generally regarded as being the correct decision for othe r reasons – mainly efficiency.
7
Final Report - FP7 Energy Mid-Term Evaluation
Industrial participants
and stakeholders support
the move to larger
projects.
There was a clear consensus that the shift to larger projects was welcomed, particularly by
the industry sector. From a policy and replication perspective the large projects should have
the larger impact in comparison to smaller (but more numerous) projects that they are
expected to. The industrial participants found the larger budgets and increased chance of
success in terms of being awarded funding, to be positive.
The programme has a
positive influence on the
profile of low carbon
research and
technologies across the
EU.
The perception of the programme, and its influence on the activities of the energy industry
was raised by one strategic consultee. Their opinion was that the FP has had very positive
role in persuading large energy companies to invest more in low carbon r esearch, by both
funding research into areas that they were unwilling to pursue or by encouraging them to
pursue them with funding support. The example of the research funding put toward smart
networks was mentioned as a good example of this , as this is an area that electricity utilities
had not pursued to the extent which some felt they should , but FP funding has helped others
pursue the area and the utilities have now increased their own efforts. The consultee felt that
without FP it is quite possible tha t this increase would not have happened, though it is very
hard to know the counterfactual. The case could be made that the expertise funded by the FP
in this area would not exist otherwise , so we would be behind in terms of a starting point for
the SET plan. This was felt to be a reasonable assumption given the very low funding that
went to clean energy R+D in MS level energy research.
The policy influence of the FP was also mentioned as an important factor in pushing the issue
up the agenda of MSs, and h elping build the case for the policy targets in this area (the
20/20/20 package) which has been developed over the last 3 -4 years. However a number of
the project participants were less convinced of the importance of the FP in influencing the
agendas of companies and MS. We feel that the strategic influence of the programme is
positive, in terms of promoting low carbon energy research and technologies, but relatively
minor in comparison to wider market and policy signals.
To what extent do the FP7 activiti es mentioned in the programme "Cooperation" adequately match
the political objectives in the field of energy?
An objective tree
combining FP7 and
energy policy objectives
was constructed and
found generally
acceptable with many
complementarities.
A key way in which we attempted to answer this question was by reconstructing the objective
tree and testing this with stakeholders. This objective tree attempted to c ombine the
objectives in the fi eld of energy with those of the FP7 programme more widely. This pr ocess
illustrated that such a combination was possible but that there were some differences of
opinion on where the priorities lay. The majority felt that the objective of reducing
greenhouse gas emissions (and those objectives which link most closely to t his, such as
reduced demand, improved efficiency and increased use of renewable energy) was the most
important, while others felt that objectives relating to supporting and enhancing European
competitiveness and research abilities should be at least as imp ortant. In the opinion of the
evaluators the fact that an objective tree created from a combination of energy and FP7
objectives has a very large number of complementary and mutually beneficial objectives
(such as the positive contribution of energy effici ency to EU competitiveness) illustrates the
strong match between FP7 and energy policy objectives. There remains a need to keep
applicants aware of the breadth of objectives that the programme has and to help them
understand the ways in which their project s could and should contribute to these.
Final Report - FP7 Energy Mid-Term Evaluation
8
Applicant understanding
of programme objectives
is acceptable but could
be better.
The participant survey indicates that they have a reasonable, though not very strong,
understanding of the programme level objective s. While a very strong understanding would
be ideal, it could be argued that it is not vital in attracting good and relevant projects, though
it does indicate that perhaps more effort should be made to clearly articulate the objectives to
applicants. The b eneficiaries do regard their projects as having a clear link with the high level
objectives of the FP7, particularly reducing greenhouse gas emissions and improving energy
security. The objective that least projects feel a strong match with is the competit iveness of
the energy industry. It is also interesting to note the low risk rating that most project partners
assign their own projects.
The programme appears
to support more RES
than EE projects. Across
the whole FP the balance
may be closer but market
conditions are more
favourable in terms of
large scale replication of
RES technologies.
As is discussed under allocation later in this report there are more renewable energy projects
supported by ENER than energy efficiency projects. With regard to the bala nce between
renewable energy and energy efficiency focused projects. Most of those consulted agreed that
the balance was in favour of renewable energy, though it was pointed out that there are other
parts of the FP which support energy efficiency related p rojects (e.g. resource efficiency, ICT
projects) and joint calls related to buildings, which go some way to address this balance. The
most significant reason given for more renewable energy than energy efficiency
demonstration projects coming forward and b eing supported relates to the presence of much
better market conditions, mechanisms and incentives to support renewable energy in
comparison to the relative failure for such market incentives to materialise for energy
efficiency.
DG ENER focus on
demonstration projects
while DG RTD focus on
further from market
research. This split and
coverage need to be
made clear to applicants.
With regard to the demonstration focus of the calls the case was made that DG ENER should
look to retain a portion of funds to s upport smaller, more research focused projects. The clear
counter argument to this is that the DG RTD supported elements of the programme will
support projects of this nature. However it is clearly important for DG ENER to maintain an
input to the focusing of DG RTD calls. It was suggested that the call for proposals needs to
make it very clear to applicants what is meant by a demonstration project in order to avoid
applicants attempting to present research focused projects as demonstration.
Individual s ectoral strategies or sectoral road maps are aimed at further improving the
implementation of FP7 and at ensuring a better achievement of the expected deliverables. Should
they be developed?
The SET industrial
initiative road maps are a
clear and sensible way to
meet the needs for an
FP7 energy roadmap,
and as such should be
referenced in the
programme documents.
The key issue for this question relates to the existence of the Industrial Initiative plans and
their associated road maps and Key Performance In dicators (KPIs) that have been recently
developed under the Strategic Energy Technology (SET) Plan. For most relevant energy
technologies these plans have a very strong relationship with the objectives of the FP,
particularly relating to demonstration and replication. For those technologies there was a
clear, and in the opinion of the evaluator logical, consensus that the DG ENER FP work
programme should make explicit reference to the SET plan and the Industrial Initiative
roadmaps. Some areas, particularly energy efficiency, are so far less well covered by the SET
plan Industrial Initiatives but this should improve over time. The detail of the plans for more
research focussed areas is also less clear, though this too should become clearer as the SET
plan develops
9
Final Report - FP7 Energy Mid-Term Evaluation
What would be the added value of international cooperation if implemented in the supported
activities?
The current approach to
funding non EU partners
is accepted as a good
compromise.
This question was interpreted as an interest in investigati ng an easing of the restrictions /
increasing the funding available for non EU partners. There was support for such an increase
/ easing in some technologies where either the non EU country involved was exceptionally
strong (e.g. Brazil in biofuels) or the technology was so far from market that a global
approach to solving the problems was seen as beneficial (e.g. wave power). There are other
parts of the FP, and MS level programmes available which work on this basis. However with
regards to the DG ENER par t of the FP the existing situation was felt to be a good
compromise between the benefits of accessing non EU expertise and demonstrating in these
markets against the risks of loosing intellectual property from the EU and supporting non EU
industries.
Does FP7 play an adequate role in positioning Europe on the global map of science and technology
for energy?
The project participants
consider their status as
among the world leaders,
but this is hard to prove.
Virtually all of the project participants consu lted ranked themselves and their projects as
among the world leaders in their fields. A number of the strategic consultees had a slightly
lower opinion of the quality of many of the participants. The most quoted international
competitor was the USA who wer e characterised as adopting a more focussed and intense
approach than the FP with large sums being directed to specific areas, which gives them a
good chance of getting ahead in these areas. The link with MS programmes and industrially
funded research is a n important one here as both of these sources of funding are larger than
the FP. The SET plan, with its intention of leveraging private and MS funds, is a potentially
vital element here.
An effective way of
testing impact and
ranking projects is to
formalise (and fund) post
projects reporting of
impacts.
There was little progress made in identifying measures which would help prove this. It was
generally agreed that although useful and relevant, patents and citation indexes, were too
open to manipulation an d not ideally suited to the demonstration, replication and
commercialised energy saving goals of this part of the FP. All of the indicators suggested and
discussed, including many of the SET plan KPIs would require an increased ex -post
evaluation of projects. To this end it was suggested that some form of additional, formalised
post project reporting should be required. The authors of this report suggest that some form
of relatively low cost payment could be offered to project coordinators as an incentive t o
provide short update reports at fixed periods after project completion.
2.2
Intervention Logic / Objective Tree
Our first step in considering the questions of relevance was to formalise what the programme
objectives are by drafting an updated objective t ree based on programme documentation and
our own knowledge, which we then tested via consultations with strategic stakeholders, DG
officers, project participants and our expert panel. The m ost significant update to the
objectives is the creation of the SET plan and the decision to align the FP7 projects supported
by DG Energy with it.
Final Report - FP7 Energy Mid-Term Evaluation
10
2.2.1
Fundamental Objective of the Programme
Some feel FP funds
should supplement MS
funds and avoid MS
specific strengths.
One consultee stated that in terms of the fundamental pu rpose and targeting of FP funds it
should be made clear that ideally the funds are intended to supplement Member State (MS)
level research and not provide a substitute or means of filling in the gaps. Each MS should
focus on those areas which it is best pl aced to exploit, for example where a country has a lot
of a particular resource, e.g. wave energy in the UK or Portugal, the MS should lead the
funding in this area, possibly with FP funding to supplement it. However, given that low
carbon research has no t been heavily supported in all member states, though this has rapidly
improved, the FP funds have provided two fundamental support functions:
FP funds can also help
justify MS activity and
help fill gaps for others.
1.
2.
Additional funds and policy justificat ion / validation for those MSs that are already
active and pursuing low carbon energy research and implementation.
Funds to support and retain capacity in those MSs where the research and policy
activity to support low carbon technologies has been less in tensive.
CO2 reductions are
generally accepted as
the primary objective of
the programme though
industrial
competitiveness and
security of supply are
high profile, and
generally
complementary.
Most of those consulted agreed that for this programme, at a g lobal level the most important
objective is reducing greenhouse gas emissions. The other high level energy related
objectives (security of supply and industrial competitiveness) were thought to be important,
and to be relevant, but were more 'side effects' from the reduced fossil energy demand that
improved energy efficiency and increased uptake of renewable energy. However some
thought that promoting the industrial capacity and innovative abilities of Europe was the
primary objective.
Social objectives are
present but were not felt
worthy of more intense
targeting.
An important high level objective which one consultee thought should be made more
apparent is that of the social aspect. This includes a number of issues or relevance, from
protecting the global population from the impacts of climate change, to protecting the
population from high energy costs. The energy cost issue is a complex one as this can be
interpreted in two ways in this area; e.g. more renewable energy can have a negative impact
on energy prices, particularly in the short term, as energy prices go up to fund the subsidies
provided to renewables. On the other hand much research is focussed on reducing the cost of
renewable energy which will result in lower cost to the consumers than would b e the case if
the research had not been funded.
2.2.2
Specific Objectives
Energy efficiency was
identified as an important
and specific objective.
Energy efficiency has been explicitly mentioned at a high level in the objective tree. The
justification for this is that energy efficiency is the most cost effective way of reducing GHG
emissions, and if 'closeness to market' is an important criteria for the programme it should
arguably be higher rated than renewable energy support.
There is multiple cross
over with other EU
funded programmes.
Although beyond the scope of an objective tree the relevant point was raised that there are
other EU funds with similar objectives, for example schemes such as the EU ETS. A number
of consultees raised the point that an object ive regarding the need to achieve system changes,
or enabling them would be a useful addition. For example smart grids (which is a multi
dimensional issue) and Carbon Capture and Storage (CCS) – which requires the optimisation
of multiple technologies and where the optimum choice is not yet clear.
11
Final Report - FP7 Energy Mid-Term Evaluation
Final Report - FP7 Energy Mid-Term Evaluation
12
2.3
2.3.1
Consultation Views
Evolution of calls in FP7
Our consultations have revealed a number of interesting points regarding the way in which
the calls under FP7 have evolved. The first call was described as being drafted with an FP6
mentality which led to a relatively high number of projects with a high number of participants
being supported.
There is no formal, specific evaluation of the projects supported by DG TREN under FP6 but
it is clear from the opinions gathered that there were a number of problems. These problems
can be summarised as follows:
The poor perception of
FP6 was an important
factor in the decision to
realign FP7.
•
•
The high number of projects and the high number of partners within these projects
resulted in a high level of administrative requirements for TREN, for example more
partners increased the chances of partner changes, which is an admin heavy task.
The high number of partners implied a low budget per partner which can increase the
likelihood of the project receiving little attention.
One external consultee raised the point that the decision (to reduce the number of areas open
and go for fewer, larger projects) could be justified solely for administrative efficiency
reasons. These related to the very poor perception among some beneficiaries and applicants
that developed regarding FP6. These could be summarised as stemming from lots of small
projects, with lots of partners, leading to diluted efforts which acted as a disincentive to many
applicants. The creation of this disincentive needed to be addressed and the approach adopted
has achieved this.
Project participants felt
that FP7 was a better
strategic fit than FP6.
When asked their opinion on whether the FP7 programme had become better able to help
achieve its objectives in comparison to FP6 and during its lifetime many of the on line survey
respondents replied that they did not know, but a very clear majority of those that did express
an opinion thought it had improved since FP6 with a smaller majority thinking it has
improved during its lifetime.
FP6 participants who
have not yet applied for
FP7 funds still consider
the programme
relevant to them.
Previous participants in DG ENER FP Energy funded projects also felt that the programme
remained relevant to them. The fact that no appropriate project had arisen was found to be the
most important factor in not applying to the FP7 Energy programme from the survey of
previous FP Energy applicants. Other lesser factors were also identified including; no FP7
funding in their research area, a focus on national (public) funding sources, insufficient
funding and issues around transnational partners and the required consortium approach.
There was no significant support for the view that FP Energy was no longer relevant or that
previous experience of FP acted against future participation in the programme. Overall the
views of the programme were positive from past participants that have not yet applied to FP7,
the majority feeling it likely that they would apply for FP7 funding in future if their research
area was called and a suitable project for their organisation arose.
2.3.2
Fewer Projects with Less Partners
The switch to fewer projects, with fewer partners, which began in the second call and has
been retained since, raised a number of points.
13
Final Report - FP7 Energy Mid-Term Evaluation
Fewer projects with less
partners has helped
enable efficiency
improvements.
A benefit which was mentioned by both Commission staff and external consultees was the
reduction in the administrative workload of commission staff. This brings a number of
positives, including:
•
•
•
The project officers have a more reasonable workload, so can do a better job.
The officers are able to focus more closely on the project, so learn about it, and from
it, more, for example each project should now get at least one visit from the project
officer.
Learning from the projects is useful as the project officers are involved in policy
development so this direct and practical involvement with research is of clear benefit.
The ideal situation, in terms of project workload on an officer, was described as a portfolio of
some large and high profile projects plus some smaller ones.
Larger projects are
accepted as higher
profile than smaller
projects.
The generally increased scale of projects (average project value up from €6m to €15m)
means that the projects should be of sufficient scale to achieve high impact and visibility on
the research landscape. Larger projects are also more likely to attract attention within the
Commission which helps give profile to the policy goals and activity of DG Energy and can
even help in terms of speeding up Commission administration.
Industrial participation
has increased.
Industrial participation in the DG ENER part of FP7 is high, currently reported as over 70%
which is much higher than the rates achieved under previous FPs. The high level of industrial
participation has led to large sums of private funds being attracted to match the commission
funds. This is believed to build in a positive cycle in that a large cash contribution from a
private firm will increase and improve the management and input from the company, which
makes for better projects with less management input required from the Commission.
The reduction in the
number of areas open
per call has helped
increase the budgets
per project and reduce
competition.
The major reduction in the number of topics open per call was also described as having the
following benefits. As the total budget remains the same, the amount per open topic has
increased. As the number of potential, credible, participants in a particular field is limited the
implication, and reality, of the situation has been larger budgets per project and larger
budgets per consortium member. This increase in budget is perceived as being attractive to
potential applicants. This issue is illustrated by the reduction in the number of bids and the
subsequent increase in the chances of a successful application.
One consultee, who is employed by a large company that has participated in FP7 projects,
agreed that fewer and smaller projects resulting in a higher success rate (for applicants) does
make the FP more attractive to large companies, as they see a better chance of winning and
the effort they expend is more likely to result in funding.
The increased chance
of success have made
the FP7 more
attractive to industy.
The improved chances of success when bidding for funds, narrowing from 1 in 10 previously,
to 1 in 5, or even lower, is more in line with the chances of success companies typically work
with when bidding for commercial contracts. This motivates better bids, particularly from
commercial organisations. Improved motivation also relates in part to the significant time
input required in preparing a bid of this nature, for example the need to identify
demonstration sites usually requires the agreement and buy in of a third party such as a local
authority.
Final Report - FP7 Energy Mid-Term Evaluation
14
Bigger projects have
led to the exclusion of
some participants and
is not ideal for all
subject areas.
With regard to the question of a general approach of many small vs. fewer and larger projects
there were a number of interesting points raised. While it was agreed that in general projects
with lots of partners are more difficult to manage, there are some cases and target markets
where this approach is beneficial. For example in some building related projects the ideal
consortium would have a representative from each part of the long (in comparison to
renewable energy) supply / value chain. Although this implies a high number of partners
many will only have a small, but important, involvement
Bigger projects makes
it hard for small
research organisations
to participate. This
relates to the size and
focus on
demonstration as
opposed to research.
However the change to fewer and larger projects is recognised as having a number of down
sides. These include:
• A perception of a focus on large companies, though in terms of SME participation
rates this does not appear to be true.
• Some past participants are unhappy that their opportunity to participate has reduced.
This is most often the case for smaller research institutes, which are more
concentrated in certain member states.
• There is a perception of decreased participation from new member states, partly due
to potential applicants there lacking the administrative capacity and experience to
submit bids.
• With regard to projects which are not focussed on demonstration, the overall
approach of less and bigger projects is less obviously beneficial. In these more
research based areas narrower calls will lose potential ideas and approaches. Bids of
this nature are often easier to put together than large demonstration type projects, for
example because there is no need to agree large sites, so applicants are more likely to
be willing to accept a lower chance of winning.
• It is recognised that, although improved, the flexibility in the financial arrangements
could still be better.
2.3.3
The calls are
intentionally copresented by DG
ENER and RTD.
Nature of Projects Requested and Supported
Those consulted raised the following points regarding the mix of technologies covered in the
calls and approved projects.
The fact that each call description is presented as a combined document between DG Energy
and Research was reported as intentional as it gives a picture of a coordinated and larger
approach than separating the focus areas between the two DGs.
Beneficiaries consider
their projects to be
riskier than would be
expected for a
demonstration
focussed programme
In terms of participants perception of the cost and risk profile of their projects around 80% of
respondents to the survey felt their project was of medium or high technical risk. This
indicates a quite high risk profile for a programme which is seeking projects relatively close
to market and one where 46% of the recipients consider that their project is of a
demonstration nature. It is possible that the beneficiaries were describing risks other than the
risk of the technology not being commercialised.
Participants
understand the
programme objectives
reasonably well but
not fully.
In terms of participant understanding of the aims and objectives of the programme, 76% of
the respondents felt they understood them very, or moderately well. A similarly high
percentage of respondents felt their project was a good or very good match with each of the
high level objectives, with contribution to energy competitiveness felt to be slightly less
strongly supported than reducing carbon emissions and improving security of supply. The
energy specific objectives of the programme were also felt to be well matched, with
15
Final Report - FP7 Energy Mid-Term Evaluation
developing technologies which could achieve short term market penetration slightly less
strongly supported than the other energy objectives.
2.3.4
Picking Winners and Research vs. Demonstration
The programme
continues to balance
picking winners
against supporting
multiple possible
approaches.
The question was raised of balance between the objectives, with the assumption that a
portfolio approach, where projects covering all potential areas of research are requested, will
allow comparison between the technologies and multiple approaches to the same technology.
This is a first step to 'picking winners' both within a technology area and between technology
areas. Such a portfolio approach also allows cross sectoral issues to be picked up.
This leads on to questions around the fundamental targeting of the programme, which
revealed a number of alternative approaches and opinions, some of which are energy specific
and some of which are more generic, though still relevant.
.. and between
addressing
weaknesses or
supporting strengths.
The consultee from the UK’s Carbon Trust (CT) provided an interesting comparison between
their approach and their perception of how the FP works. He described a high level split in
the approach, such that a programme could either focus on known areas of strength or
improving areas where the EU performance is currently weak, but improvements are
necessary / thought to be beneficial. The CT approach was described as being to offer a range
of schemes to fit the variety of R+D needs. This approach was felt to differ from that of the
FP which is perceived as starting from a final objective and working in a top down manner to
specify the research it wishes to support.
One consultee described the current situation in terms of technology choice in low carbon
research as still being at the ‘let all flowers bloom’ stage, i.e. the problem is so large and
important, and the potential solutions so diverse that it is unwise to try and focus on particular
technologies to the exclusion of others. Another reason not to remove any technologies and to
continue to support all is that the policy environment in terms of member state and industry
commitment to low carbon is still fragile. It is also the case that the technologies which
eventually emerge as the most important may not be those which appear best placed at the
moment, or those which are technically / theoretically the best, as this is how technology
development works.
There is a clear need
to support research as
well as demonstration.
With regard to the focus of the projects, one consultee stated that the focus on near to market
research was understandable, as industry is motivated by the prospect of profits and there is a
need to spur carbon savings as soon as possible. This needs to be balanced with a recognition
that innovation does not finish when a product comes to market. Ideally there should be some
funds available to support all stages of the research cycle, including product development.
This point is of relevance to reducing energy use in the buildings sector, as here research
needs to focus on the cost reduction of relatively mature technologies - i.e. more like product
development but still research. Industry needs to have confidence that they can look for
support to help them plan investments.
Some of the DG officers felt that ideally the programme should look to retain some balance,
within each project and across projects, between demonstration for commercial benefits and
scientific benefits, with the projects selected for the scientific benefits typically being more
cutting edge and riskier. This balance needs to start in the calls for proposals. On a similar
note one officer felt that it would be good to retain some of the budget to support smaller
projects. Another officer agreed that supporting some smaller projects reflects the industry
approach as they will carry out smaller scale testing prior to large demonstration.
Final Report - FP7 Energy Mid-Term Evaluation
16
Applicants need to be
clearly reminded what
demonstration as
opposed to research
means.
In order to help applicants clearly understand what ENER mean by ‘demonstration’ projects
it was suggested that the definition used in the call for proposals needs to be very clear. This
will avoid applicants submitting and presenting research projects as demonstration (not to say
that research projects shouldn't be funded). The attempts in previous calls to define
‘demonstration’ could be improved.
The DG RTD / DG
ENER split is clear to
the DGS.
From the DG RTD perspective it was felt that DG ENER now managed the demonstration
projects and DG RTD the research focused projects. It is important that research is continued
to be supported as it drives innovation. With regard to putting both research and
demonstration projects under DG Energy, this approach was raised by a limited number of
consultees but was not widely supported by the majority, including those from DG RTD. One
officer felt that the clear focus on demonstration had helped clarify the division between DG
Research and DG ENER – this split (in terms of distance from market) had not always been
obvious to external observers.
Putting the research
focussed activities
under the control of
DG ENER was not
widely supported.
2.3.5
Balance Between Renewable Energy and Energy Efficiency
It appears that RES
projects receive more
funds than EE but this
simple split misses the
whole story.
With regard to the split between RES and EE one DG officer made the comment that if this
split is done by project numbers it can be misleading as the large CONCERTO projects might
have been 3 to 4 separate projects in the past but the focus on larger projects has led to them
being combined. Analysing by funding can also be misleading as some of the efficiency
projects will have an impact on a lot of buildings despite being relatively low budget in
comparison to RES projects. The building specific calls have helped to keep the balance –
which is better than it appears.
Energy efficiency
brings quicker financial
returns and CO2
savings than RES so
some say it (EE)
should be the priority.
In general terms, heat related projects are less supported than electricity related projects, if
the programme wishes to reflect the use of energy, this is an under representation. This also
relates to the fact that heat is not a traded and transported commodity like electricity.
Improving the efficiency of its use will require approaches that are attractive to consumers,
which has been something of a market failure to date.
Other parts of the FP
fund EE related
projects.
Energy efficiency is more cross sectoral than RES in terms of the technologies being
developed. Much of the research / improvements available relate to controls and ICT and
these projects would typically not come under DG ENER. Energy use in buildings is complex
and many of the FP projects / calls are pursued in conjunction with other DGs, (2 calls of this
nature were mentioned). One officer raised the logical point that any relative (to renewables)
lack of energy efficiency demonstration projects, fundamentally relates to the lack of a
winning commercial business model for energy efficiency. The technology is mostly well
known and proven, but the economic models and incentives are still to be properly devised.
The market conditions
/ subsidies are better
developed for RES
than for EE, making
demonstration projects
more feasible in RES.
Many of those consulted felt that the current approach of supporting large, demonstration
focussed projects presents an easier opportunity for renewable energy focussed projects than
for energy efficiency focussed projects. This was believed to partly relate to the nature of the
projects, for example a large scale installation of a new wind turbine is a simpler prospect
(one site and one technology) than installing multiple small scale efficient appliances. The
Concerto approach (considering multiple low carbon solutions on a spatial basis) was
mentioned as a way in which this issue had been addressed in the past.
17
Final Report - FP7 Energy Mid-Term Evaluation
Demonstrating simple
energy efficiency
projects risks
becoming industrial
subsidy.
2.3.6
With regard to funding projects focussed on industrial energy efficiency, another problem
with these is that there is a perception that industry should be pursuing these opportunities
anyway, particularly as energy costs are so high, leading to a significant risk of a lack of
additionality. Despite this concern some projects are funded, about 20% of the project
portfolio. It was suggested that the ideal solution would be a cluster of projects to help get
scale and momentum.
Quality of Supported Research
Views on the quality of the research supported were sought from Commission Officers and
project participants.
Global ranking is
considered in
evaluations.
The global ranking of energy R+D in Europe was reported as an issue which is always
considered when assessing applications. The point was also raised that as there are now fewer
but larger) projects it becomes even more important, because the numbers of projects are
lower, so more is more pressure on each individual project to deliver something of global
significance.)
Many DG officers and
participants regard the
work in their fields as
world leading. Others
are less convinced but
without the FP they
agree the situation
would be worse.
With regard to biofuel research and demonstration, the opinion was raised by a DG officer
that FP support has helped put the EU at the front of global research. However, it is difficult
to compare EU research with the position in the US, as their approach is different. The US
approach involves more funding and more concentrated efforts on a technology once it has
been selected, e.g. ten projects funded at one time in biofuels in order to increase the chances
of success. This gives more intense coverage in some areas than the EU, but less in others.
Most of the coordinators interviewed felt that FP7 put Europe at or near the top of the world
rankings in terms of energy research, and that it was important, indeed critical, for EU
competitiveness that that was the case, though one project co-ordinator felt Europe lagged
behind global competitors somewhat. He did comment, though, that FP7 was helpful in
somewhat redressing the imbalance.
The US approach to
funding energy R+D
tends to be more
funds and more
focussed.
That said, few felt they had a sufficient grasp of the overall balance of energy R&D funding
globally on which to make comparisons, though in their particular topics and fields of
expertise, most felt the FP7 projects were cutting edge. A number commented that on
renewables development and deployment, Europe was world leading, and particular
applications such as laser treatment of photovoltaics was global state of the art. Another
consultee also felt that the EU was currently leading in CCS but the US were now dedicating
large resources to this area and could overtake the EU. The comparison to the US was made
by another consultee who felt that the FP7 has smaller budgets than the US Department of
Energy research programmes, both for research and demonstration focussed projects. One
interviewee pointed out that there was a global increase in the level of interest in, and
resources directed towards renewable energy sources, which he characterised as a global shift
away from IT R&D towards renewable energy.
Low carbon R+D
investment is
increasing globally due
to market and policy
pressures.
This opinion was endorsed by another consultee, a large commercial partner, who
commented that the absolute importance of EU or Member State funding of research had
reduced somewhat in recent years, as companies have a sufficiently strong commercial driver
to undertake research in house or with partners but without public funding, the benefits to
firms being greater control over the research, the ability to progress more swiftly (not having
to wait for a decision on funding award and subsequent contract negotiation), less
Final Report - FP7 Energy Mid-Term Evaluation
18
bureaucracy and, perhaps most importantly, the ability to keep highly commercially sensitive
work secret. The downside of purely commercial work is that results don't get shared. The
implication is that the terms for commercial partners to participate need to be made
sufficiently attractive such that strategically important work could be done within the context
of an EU public/private project.
2.3.7
Increased support for International (non EU) partners
Other sources of
funding, including
other parts of the FP,
are focussed on
growing international
cooperation with non
EU partners.
The question of whether the quality of projects would be improved if it was made easier for
non EU applicants to participate in projects drew a variety of comments. It was pointed out
that there are already a number of other parts of the FP, and other EU funding sources
specifically targeted on cooperation with partners from outside the EU. For example a joint
RTD/ENER topic, under the Specific International Cooperation Action, focused on EU
cooperation with India. There are also overseas aid programmes which can support energy
infrastructure development and it should be remembered that the FP has a very different
purpose to the aid budget.
More cooperation with
non EU partners is of
benefit where the
world lead is outside
the EU, or the market
is immature.
On international partners, the benefits of additional inclusion were felt to vary by sector.
Where there are clearly global leaders who are not EU based, (e.g. biofuels in Brazil) there is
a good case, but where the EU leads (e.g. wind), or where the project will result in
developments outside of Europe the case is much less strong. For new sectors (e.g. wave) all
developers have the same initial goal (i.e. a working large scale prototype) so it is possibly
easier and more sensible to collaborate.
.. or the market is
outside the EU.
Two interviewees in favour of international co-operation argued that "artificial geographic
limits" do not reflect the reality of teams and production capacity being placed anywhere
globally, even if the parent company is European. One argued that, given the global nature of
the GHG problem, it was important to have global co-operation on solutions. Other points
raised in favour of increased cooperation with non EU partners included the expertise in
particular technologies that exists outside the EU, the extra scale that more international
partners could bring, which is of particular relevance for major issues such as CCS and the
benefits of being able to site a demonstration project outside the EU where the largest
potential market for that technology is outside of the EU.
There are risks of
knowledge and
commercial benefit
leaking outside the
EU.
Where non EU cooperation is supported it should be done on an equal basis (in terms of
funding support) and care needs to be exercised with regard to leakage of technology and
expertise, especially to countries who are actually at the cutting edge themselves, e.g. current
coal generation plant installations in China are more efficient than in the USA and emit less
CO2 per kWh produced. Contractual complexity also increases with the inclusion of non EU
partners, which is another reason for limiting it.
In conclusion the most commonly held opinion was that international partners can improve
the quality of projects, but also bring risks and downsides so the current flexibility is a fair
compromise.
19
Final Report - FP7 Energy Mid-Term Evaluation
3 Efficiency
3.1.1
Definition
How economically have the various inputs been converted into outputs and results? Were
the (expected) effects obtained at a reasonable cost?
3.1.2
Specific Questions
The TOR contains the following key issues and questions which we have classified under this
heading. Each question is followed by our opinions based on the data, analysis and opinions
that we have collected.
Are the activities carried out efficiently, clearly, appropriately and cost effectively?
Participants find the
admin burden high but
as expected.
The participant survey indicates that the majority of participants do not consider the
administrative effort required of them to be excessive / more than they expected. However it
should be pointed out that a number of participants do still consider the administrative efforts
required to apply and participate as being high.
Time form approval to
contract is too long.
The aspect which drew the most consistently negative comment was the length of time taken
to move from project approval to contracting. This is an issue across the FP and an area
where improvements would be appreciated by both participants, stakeholders and DG ENER
staff.
Previous FP
experience increases
the time spent on
applications.
Analysis of the survey returns to identify if previous experience of the FP Energy programme
reduces the time required for applications and administration found that this has little or no
effect on time spent. In fact, contrary to expectation, the only slight effect detected was that
those with past experience tended to spend more time on applications than those without. For
SMEs there was variation from the mean on efficiency, on average they spent less time on
applications but more time on project administration than non-SMEs
SMEs spend less time
on applications but
more on operation.
Fewer and larger
projects has boosted
efficiency.
External expert
assistance is available
but the arrangements
are unclear.
It is clear from the DG Officer, stakeholder and participant views that the move to fewer and
larger projects has brought an improvement in the efficiency of programme management. The
idea of using an external agency to administer the ENER FP projects was not strongly
supported and a reasonable case was made that the added value that is gained from the DG
officers getting a first hand insight into research and development in their energy policy
fields outweighs any efficiency gains which an agency might bring. Although external expert
support is available to assist DG ENER officer in managing projects and reviewing the
interim and final reports the availability of this expertise appears to be on a somewhat ad-hoc
basis. The programme management could be improved if this situation was clarified.
Final Report - FP7 Energy Mid-Term Evaluation
20
FP management is
fragmented across the
DGs
One consultee felt that the fragmented way in which the FP is managed causes problems in
terms of management optimisation, as different parts use different approaches which makes it
hard to compare, and harder to analyse and group the expenditure.
Are the human and financial resources devoted to the supported activities appropriate to reach the
objectives stated?
Funding is generally
regarded as enough,
more could be used
but the economic
downturn and the
focus on attracting
private and MS match
runs against this.
The majority of those asked this direct question felt that the resources were appropriate,
though everyone would appreciate more funding. Given the increases in budget that the FP
has already had, the economic problems currently facing the EU and the SET plan approach
(looking to attract more private, MS and other funds for low carbon energy research and
demonstration) it does not appear appropriate to suggest seeking major additional funds at
this point in the FP7.
Unspent funds at the
programme end
should be retained in
this field.
With regard to the overall budget of the FP in this area, a technical point which was raised
was the benefits that would derive from a mechanism which enabled funds which were not
spent, as a result of project failure / early termination) to be retained within the programme,
rather than lost back into the general Commission funds.
Have the IT systems been sufficient to allow the efficient and effective management of the
programme?
IT has some problems
but not major,
Outsourcing slows the
speed of response.
The quality and ease of use of the IT systems involved in applying for and administering FP
funds was somewhat higher regarded by the project participants than was expected. Although
some participants noted problems with the IT system (e.g. the searchability of CORDIS could
be improved, problems with the participant identification code system) most felt that the
systems had improved over time. There is some frustration evident among DG ENER staff
regarding the IT systems, largely due to the slow rate of change and the difficulties of
improving systems which are operated by external contractors.
Has the application process been efficient in terms of timing and content of calls?
Positive suggestions
for improving the
administration are:
- Clarify DG officer
access
- Interviews for large
projects
- Remote evaluations
-More inclusive end of
evaluation meetings.
.- Project management
KPIs
- Clarify access to
external experts.
The successful applicants had no major concerns with the application process. The DG
ENER officers consulted made a number of helpful suggestions regarding the details of the
application and evaluation process. The ideas which appear strongest and best supported, and
therefore worthy of at least a trial, are:
• Pre-submission advice- Clarification of the procedures for accessing Commission
staff prior to the application. Although already covered in briefing days the
procedures should be clarified.
• Applicant interviews- Short interviews for the applicants behind large projects.
With the exact individuals – i.e. which partners the coordinators chooses to bring
along, being left to their discretion.
• Remote evaluations - An approach to evaluation where the experts are briefed in
Brussels, then evaluate the projects at their own premises. The on site procedure
involves the external experts attending an ‘evaluation week’ where the meetings and
panel reviews occur in Brussels with all the evaluators present, over a period of 5
days. In remote evaluations the external evaluators take the bid material home to
review following a Brussels based briefing. The majority of the officers consulted
felt that on balance remote evaluations were better than on-site, with the ability they
21
Final Report - FP7 Energy Mid-Term Evaluation
•
•
•
Less well supported
suggestions were:
- 2 stage applications.
- increased financial
monitoring by officers.
- page limits.
offer to attract better experts (due to increased flexibility) outweighing the concerns
over document security, we agree with this conclusion.
A more extensive end of evaluation meeting – involving the external experts and
extended discussions to arrive at the agreed approved project list. If the resources and
budgets are available to support this then it appears a sound idea that is worth trying.
A set of project management KPIs – to enable easier and more fomalised ongoing
assessment of project progress. A short summary sheet would be useful for project
officers, combining procedural and output / quality indicators.
Increased use of external advisors to assess interim and final reports – to bring
additional technical expertise and scrutiny to the reviews of project progress. It
appears that this advice is available to project officers on request, but the procedures
do not appear to be fully consistent. Therefore, on the assumption that the resources
are available to provide more support, the procedures should be clarified to enable
more consistent, and presumably greater, uptake of external advice.
3.2
Those where the balance of pros and cons are much closer are:
• Two stage application process. The positive sides of this are that bidders can submit
an outline proposal without dedicating the extensive efforts required to submit a full
bid, which could attract more bids, particularly from those for whom the time
required is less available (e.g. SMEs). However on the negative side the process adds
an extra step of administration and, partly due to the need to review the likely high
level of projects quickly, the step runs the risk of missing potentially strong bids
which have not fully developed. These negatives in combination with the relatively
small number of projects that are capable of being supported (due to their high cost)
and the attraction of the high success rate for industrial applicants, lead us to the
conclusion that this approach is not currently worth pursuing.
• Increased involvement of project officers in financial monitoring – to give the
officers a deeper understanding of the projects and to increase scrutiny as to the
appropriateness as well as accuracy of expenditure. This idea was not as well
supported as the other suggestions and we agree that on the assumption the financial
aspects of the project are properly reviewed at application stage and any budget
change requests are reviewed by project officers this should not be pursued.
• A page limit on applications and reports – to focus the efforts of applicants and
beneficiaries and avoid the inclusion of superfluous detail. Given the practical /
procedural and, potentially even legal difficulties of enforcing this we suggest a
continuation and re stressing of the current approach where applicants and projects
are reminded of the preference for concise applications and reports on the assumption
that it is possible to present good ideas, and good work, concisely and overly long
and unfocussed work will be at a disadvantage.
Participant Survey
The findings of the survey indicate that participants in the FP7 Energy programme typically
spend between 20-45 person days each in preparing their application and meeting the
management and administration requirements of the programme. For many the burden is
Final Report - FP7 Energy Mid-Term Evaluation
22
lower but for a small minority of participants (around 10%) the time required is over 100
person days.
Previous FP
experience does not
significantly reduce the
admin efforts made.
Further analysis of the survey returns, to identify if previous experience of the FP Energy
programme reduced the time required for applications and administration, found that this has
little or no effect on time spent. In fact, contrary to expectation, the only slight effect detected
was that those with past experience tended to spend more time on applications than those
without. For SMEs there was variation from the mean on efficiency, on average they spent
less time on applications but more time on project administration than non-SMEs.
Participants are
broadly satisfied with
the admin apart from
the length of time to
contract.
Overall the programme participants were broadly satisfied with the various elements of
programme implementation and particularly with the scientific and technological areas
covered, the strategic orientation of the programme and project payment arrangements. There
was one clear area where satisfaction was much lower and this was the time from project
approval to contracting. For those with prior experience with the Energy Framework
Programmes a majority believed that their experience of FP7 is better than in the past and
only a very small minority believe it is worse.
Admin arrangements
have improved FPs.
The view on efficiency from the survey responses is mixed, significant time commitments are
still required by applicants and participants, but it can be said with some confidence that
administrative and management processes have improved since previous FPs. One area for
potential improvement is in reducing the time taken from project approval to contracting.
3.3
3.3.1
Consultation Views
Efficiency improvements that have occurred
Fewer, larger projects
has improved
programme efficiency.
The DG officers consulted, suggested that the shift in focus to larger, fewer projects was
partly driven by a need to reduce ENER officer workload and it was generally accepted that it
has resulted in them being able to monitor projects more closely. DG RTD project officers
typically manage between 10 and 20 projects, which is regarded as a high workload.
Larger projects
improve participant
efforts / attention.
The shift in the focus of the programme – from smaller projects to larger demonstration type
projects has required a change in the attitude / focus of effort by the officers as the projects
require a different approach in terms of coordination and management (as a result of their
larger scale). The change is also regarded as rewarding as the projects increasingly involve
the largest, highest profile actors in the energy sector (for instance Vattenfall, Volkswagen,
EON, ENEL, Siemens etc.). This involvement is also genuine, e.g. often as a coordinator,
rather than in the past when in some projects the large companies were involved but their
involvement was perhaps less crucial to the project scope. This makes for projects with an
opportunity to give bigger impacts on the sector and on society as a whole.
Improved since
previous FPs.
Efficiency in terms of handling and length of procedures is improving. E.g. the online
submission and negotiation tool is better than the previous paper / manual approach.
The reduction in the number of projects per project officer has been a factor in the reduction
of the amount of time taken to process amendments, an issue that has been a long standing
complaint from FP participants.
23
Final Report - FP7 Energy Mid-Term Evaluation
A number of project participants suggested indicators that could be used to measure the
efficiency of the programme and projects, these included:
•
•
•
3.3.2
Time to contract is too
long. There are FP
wide efforts to address
this.
time to contract;
results of R&D obtained vs. costs, meaning the results are those of the various
projects funded in a framework programme compared with the costs of R&D to
obtain those results;
Number of projects funded; and comparison of the above results with other similar
programme (MS or European).
Areas for improvement
Time to contract is still accepted as being too long by the DG officers interviewed. It was
also recognised that the complexity of the documents and forms required makes it time
consuming for coordinators and partners to take part and it may particularly discourage
SMEs. Reducing this complexity is an issue around balancing flexibility and control. There
is though the intention to further simplify the procedures for taking part in EU-funded
research projects – this was announced end of April 2010 by the research Commissioner
following a consultation for ideas in this respect 2. The overall aim is to make participation
transparent, less bureaucratic and attractive to the best researchers and innovative companies
in Europe and beyond.
IT could be improved
but outsourcing makes
the responses slower.
IT systems were generally agreed to still show room for improvement though the point was
raised that it is difficult for DG ENER to effectively intervene in this issue when the majority
of the IT services are outsourced outside the DG.
Participants are
generally supportive of
the IT with some
suggestions as to how
to improve user
friendliness.
Three or four coordinator respondents felt that IT systems were adequate. One person rated
them as "seven out of ten". Others had issues. One commented that… "the IT system is
complicated and there are different applications, different contact points, passwords and so
on. In addition, the IT platform is considered to be not very user friendly. Integration of
submissions and a single contact point would be helpful."
Four people had used the NEF system – one said it didn't work at all; the other two
recognised it was still being piloted, but felt that once the bugs had been ironed out, it should
work satisfactorily. Another found the system ok but commented on the large amount of
unnecessary information on the screen which could be addressed by hiding functions off the
screen until needed. The ECAS user guide was also felt to be overly complex
Comments on Cordis were generally slightly negative, with it being described as "non user
friendly", "chaotic" and "difficult to extract information from". Potential improvements to
Cordis identified include automatically sending information on calls by email to those who
have registered details, classification of projects by theme and information on what has been
funded (at the moment this list of projects has to be obtained from the national representative
and then used as a basis for online searches to obtain further details).
2
http://europa.eu/rapid/pressReleasesAction.do?reference=SPEECH/10/194)
Final Report - FP7 Energy Mid-Term Evaluation
24
3.3.3
Project management and external experts
There were a number
of suggestions on how
to improve programme
management….
The suggestion was made that for projects with some initial research required prior to
demonstration project finance could be structured in a way to allow a 'break point', where the
project could either continue with full funds, or be stopped, depending on the success or
otherwise, of the initial research.
Programme managers
should get more
involved with financial
monitoring.
The suggestion was made that technical officers would gain a more rounded and better
informed view of the project if they had some involvement with the financial monitoring and
control of the project. At present this is done by the financial cell who are understood to be
more concerned with the accuracy and correctness of the figures, e.g. expenditure vs. budget,
rather than if they are realistic and appropriate. The project officers would need some training
in order to do this role. One officer could see no benefits to be gained from more officer
involvement in the financial management and reporting of projects, particularly as projects
now get more officer time as they have less projects to oversee.
Page limits on bids
and reports.
A page limit on the proposals and reports was suggested as a way of improving the efficiency
of the evaluation process, as often much of the detail, particularly for large demonstration
projects, is superfluous. A two step application and evaluation procedure was also suggested.
The first stage would be a short (e.g. 10 pages maximum) summary submission, which could
be reviewed relatively quickly and, if of sufficient quality, a full application could be
submitted and reviewed. This approach is used in DG RTD (with both stages being evaluated
remotely).
Two stage
applications.
Project management
KPI checklist.
It was suggested that the management / monitoring of projects would be made easier if there
were an agreed set of measures (Key Performance Indicators) of project progress. Such a list
was reported as existing in the past (for FP5) though this was regarded as too long.
Suggestions for a list of KPIs were that it should contain a mixture of administrative type
measures – e.g. have the WPs been achieved to schedule and more commercial / scientific
measures, e.g. progress towards a marketable product. This would enable progress to be
reviewed and the presence and quality of any contingency plans to be tested.
Concerns about high
staff rotation in DG
RTD.
Some concern was raised about the ability of DG RTD staff to properly manage some of the
complex projects they fund, as the staff do not stay in their roles very long (too much staff
rotation).
More use of / access
to external experts
would be appreciated
by some DG officers.
The increased use of external experts to help evaluation etc. was mentioned by one officer as
a way to help ease the administrative burden of this on DG ENER officers, such external
experts were suggested as also being useful to help project officers review interim and final
reports as they have high levels of technology knowledge. The use of external experts to
assist in ongoing programme management was reported as also being used by DG RTD on a
discretionary basis.
3.3.4
Project Applications and Evaluation Procedures
The method of evaluating proposals was discussed, with the two procedures used by various
parts of the FP – remote and on site, both seen as having pros and cons. The on site procedure
involves the external experts attending an ‘evaluation week’ in Brussels, where the meetings
and panel reviews occur over a period of 5 days with all the evaluators present. In remote
25
Final Report - FP7 Energy Mid-Term Evaluation
evaluations the external evaluators take the bid material home to review following a Brussels
based briefing.
The positive aspects of remote evaluations described by officers included the following:
Remote evaluations
offer more flexibility to
external staff and can
help independence.
•
•
Other DGs use this
approach.
But there are security
risks.
Consortium interviews
for the largest projects
would be helpful.
Improved flexibility and efficiency for evaluators in terms of how they manage their
time - which should help attract better experts.
Less chance of a team decision emerging, as although this is not the intention it is
very hard to avoid when the evaluators are all in the same room – this could also be
viewed as a negative in that the grouped approach helps to moderate / check
opinions.
In order for remote evaluations to be effective it was noted that online sharing of evaluations
would be helpful. Some joint FP calls, e.g. with DG INFSO, were described as having been
evaluated by this method. It was also pointed out that the experts need to be more
experienced and to understand the aim of the programme better, as there is less chance to
help / guide them. There is also more of a need to manage the reviewers – as they are remote
their timely progress needs to be reviewed and managed. There is also a risk of confidential
documents getting lost in transit to the reviewers. Concerns about the confidentiality of the
application material meant that some officers thought ‘on site’ evaluations were preferred
over the remote approach. DG RTD use both remote and on-site evaluations but reported that
there were no criteria for selecting which parts of the FP use which approach. The majority of
the officers consulted felt on balance that remote evaluations were better than on-site.
With regard to evaluating the large projects now being submitted it was suggested that a
meeting / interview / hearing with each consortium should be requested / offered. This would
enable ENER to clarify any points that were not clear in the bid. The point was also made
that the position on meetings prior to project submission should be formalised and clarified,
as currently it could be interpreted as giving some bids an unfair advantage (to both explain
their own projects and seek clarification) via their own contacts. The approach of a formal
pre-screening submission and meeting, as used by DG RTD, was suggested as a way of doing
this.
For the large demonstration projects it should be made clear that there is a preference for the
coordinator to be an industrial partner – as they should be best placed to commercialise the
results.
A more inclusive end
of evaluation meeting
would improve the
decision making.
It was suggested that the ranking of projects at the end of the evaluation process would
benefit from a more in-depth process, for example an extended discussion panel including
external experts and two technical officers as moderators during the evaluation of large
projects.
Participants generally
happy with the
application process.
The project participants generally felt that the application and evaluation process was
reasonable, if time consuming, with a number of projects employing external assistance. As
with the DG Officers there were a number of suggestions as to how this could be improved.
There was some support for a 2 stage application process, including from those who had used
this in other parts of the FP. There was some concern over the lack of availability of project
officers prior to submission, though the fact that this was not a consistently reported issue
suggests that the officer comments about a need for clarification of the procedures is genuine.
Final Report - FP7 Energy Mid-Term Evaluation
26
3.3.5
IEE coordination
needs to be
maintained.
Co-ordination with IEE
One officer felt that there is potential to better coordinate the FP activities of ENER with
those pursued under Intelligent Energy Europe3 (IEE). IEE is moving towards calls that have
a technology focus which suggests an opportunity to coordinate with the FP (which also has
technology focused calls) the softer issue focus of IEE should complement the FP. However,
another officer felt that coordination between the two programmes was already in place and
working well enough on a sectoral basis, e.g. by particular renewable technology.
3
Intelligent Energy Europe is part of the Competitiveness and Innovation Programme. Its strategic direction is set by DG ENER but it
is administered by an agency staffed by a combination of DG officers and external staff. The programme focuses on the non
technical barriers to the take up of energy efficiency and renewable energy, e.g. information failures, skills issues, transposition
of directives etc.
27
Final Report - FP7 Energy Mid-Term Evaluation
4 Effectiveness
4.1.1
Definition
What effects (impacts) have been obtained by the intervention and, in particular have
these effects contributed to the achievement of the objectives of the intervention?
4.1.2
Few impacts to date
from new projects so
measurement has
been the focus.
Specific Questions
The TOR contains the following key issues and questions which we have classified under this
heading. Each question is followed by our opinions based on the data, analysis and opinions
that we have collected. This area presents some difficulties for a mid term evaluation as it is
looking for impacts from projects which have only been up and running for a maximum of 2
years and therefore have not reached a point where impacts are yet apparent. Therefore
although we have sought to find evidence of impact we have also considered the issues of
how success could and should be measured.
What is the impact of the supported activities on the large-scale deployment of energy technologies
in the European Union? What is the evidence of structural change as a result of these activities?
How can this impact be improved?
Prospects for
commercial
deployment are
ranked more highly by
company project
partners.
Our survey and interviews with project participants illustrate that, unsurprisingly, research
focussed partners are more hopeful of, and are motivated by research / scientific benefits and
outcomes, while participating companies are more hopeful of, and motivated by commercial
benefits. Commercial impacts were felt by the participant survey respondents to be the most
likely at EU level, with less expectation of local, regional, national or global impact, this is
in-line with the level of the FP7 programme, though not necessarily industrial objectives.
Policy influence (on
MS and EU) is positive
but relatively minor.
The project participants were also asked how important the programme was with regard to
influencing the policies of the EU, member states and the energy industry, which would be a
strong indicator of a move towards structural change. The general view was that its influence
is positive but relatively minor. The positive aspects mentioned included generic issues such
as raising the profile of R&D. With regard to energy companies, the opinion was that they are
now increasing the R&D efforts in this area, but this is largely due to them seeing the
commercial necessity and benefits of doing so.
Identifying and listing the concrete results of the research funding and assessing positive and
negative outcomes.
29% of projects expect
patents to result from
their work.
When asked about the expected outcomes from their projects 29% of the participants
predicted that patents would be produced. This is higher than the 13% of patents that were
produced from FP5 projects. This could represent an over optimism on behalf of the projects.
Final Report - FP7 Energy Mid-Term Evaluation
28
Special attention should be given to assessing the impact on competitiveness of the European
economy and on security of supply, as well as the potential for reduction of CO2 emissions in the
energy supply chain.
70% of participants
were unable to
estimate the CO2
benefits of their
projects (if successful).
The CO2 question in
the application would
show the market
understanding of the
applicant as well as
the energy benefits.
An interesting finding from the participant survey related to the questions posed asking if the
beneficiaries could provide an estimate of the CO2 savings their project should enable if
successful. The vast majority said they could not do this and neither could they estimate the
scale of replication of their project. This reply appears to contradict the high number of
respondents who felt that their projects would reduce CO2 in another survey question. We did
suspect that the questions was not understood by many respondents however our interviews
with project participants suggest that this way of thinking about project impact is not one that
participants are happy with. In the consultants view this could relate to the participants
wanting to calculate the savings to too high of a degree of accuracy. Where this question in
asked in the UK Carbon Trust application forms the intention is to gain an insight into the
applicant’s optimism regarding their innovation and their knowledge of the market into
which it would be entering, i.e. its approximate scale and the key competitors.
Stakeholders support
the call for applicants
to estimate the CO2
benefits of their
projects.
Participants think the
CO2 question is very
hard to answer.
The majority of the strategic consultees felt that given the importance of CO2 savings, and the
clear link which should exist between demonstrating low carbon technologies and large scale
replication, project applicants should be very clearly instructed to estimate the scale and
nature of the CO2 savings that their projects could enable. The project participants were
concerned with the difficulty of making such assessments and a number were concerned that
this approach would favour larger scale nearer market technologies. In order to allay these
concerns any additional requirement for CO2 saving estimates should include simple
standardised methods and the question should receive no more emphasis in the evaluation
procedure than it already does.
Patents and citations
are interesting
indicators but should
not receive extra
scrutiny.
The other indicators which were most suggested and discussed were patents and citations.
The consensus, from both stakeholders and participants, was that although these measures
were useful and of some relevance to demonstrating innovation and high quality research
they should not be the subject of additional efforts to seek out and measure. The main reasons
for this are the ability to manipulate both indicators, the lack of a definite link between either
and large scale replication / take up of low carbon technologies and the fact that good
projects may well seek to avoid both in order to protect their intellectual property.
4.2
The objective tree
translates into
indicators of success.
Indicators
Given that the FP7 projects supported by DG ENER have only been running for a maximum
of 2 years it is not yet possible to investigate their actual outputs and results. Therefore much
of our efforts to answer these questions have centred on the consideration on how project and
programme success could and should be defined and measured. The standard approach in
evaluations is to translate the programme objectives (from the objective tree) into specific
indicators that can be used to assess and measure the success of the programme and the
individual projects within it.
As with the objective tree we investigated this issue by drafting an initial list of indicators
and then discussing this with our expert panel and strategic stakeholders. The findings of this
consultation and the resultant list of potential indicators are presented below.
29
Final Report - FP7 Energy Mid-Term Evaluation
Table 3.1 Objectives and associated indicators
Programme Level
Objective Group
Indicators
Policy Influence
Opinions of MS and EU level policy makers
Alignment of MS policies with EU policies
Management
efficiency and
simplification
Processes have been modified in the light of previous evaluations
Projects start on time and receive payments on time
Survey feedback on satisfaction with processes
Reduction in time from agreement to contracting
Measures achieve
objectives efficiently
Grants awarded to leading companies, researchers and institutions
More SMEs participating than in previous FPs
Participation of New Member States
Benchmarking with other parts of the FP and international comparators on:
Activities promoted
match the objectives
set
Work programmes match across to the "essential issues" identified by the
EU
The activities funded match objectives
Project portfolio supports wider policy goals (economic, societal)
Projects supported are balanced across the energy sub-themes
Adequate level of
funding
Level of applications is neither too high nor too low
Funding compares favourably to other RTD themes
Funding has increased commensurate with policy priorities
Balance of funding between sub-themes has changed in response to
emerging priorities.
EU leadership in
energy RTD
Consortia are led by and/or include leading global researchers and research
organisations
Benchmarking with international comparators
- total patents
- total actual and potential CO2 savings
- total actual and potential RES installations
- citations
- Industry match funding
Project Level
Objective Group
Indicators
Contribution to the
energy priorities
RES uptake – actual and potential if research succeeds
Carbon savings - actual and potential if research succeeds
Larger scale of projects - on the assumption that this addresses
fragmentation
Exploitation and commercialisation of results – patents (or patentable IPR)
High quality
Citations
Patents
Opinion of external experts
Increasing crosssectoral impacts of
the research
Analysis of projects shows strong cross-sectoral focus
Project consortia include industrial players and other end-users
4.3
Measuring progress
The DG officers that were consulted raised a number of interesting points concerning the
measurement of project, and programme achievement.
Final Report - FP7 Energy Mid-Term Evaluation
30
The following measures of success were suggested, with the proviso that they can only be
measured some time after the project funding period has passed.
Impact and success
are only apparent
some years after
funding.
•
•
•
•
•
The speed and scale of technology deployment.
The establishment of European companies as world leaders in their field (as has
happened with wind energy).
Achieving / contributing to the policy objectives, i.e. replication and the CO2 savings
each replication achieves.
Improving the cost effectiveness of the technology, which requires clarity on the cost
reductions achieved, even if only a forecast.
Progress towards market for the technologies supported, this is more complex to
measure and requires an in depth analysis by field.
Impact is project
specific.
Fundamentally each project should be evaluated on whether it achieved its goals. For
example, for a project demonstrating a new plant – did it get the plant up and running? Did it
achieve its targets (e.g. % efficiency)? Has it reduced the production cost? Has it led to
replicate plants in the EU? However even if some (or all) of these goals have not been
achieved the project can still produce useful results, although this would require an in-depth
and project specific review.
The programme does
not do enough to
measure impact.
It was generally recognised that these issues are not very well picked up in the project
monitoring process and that the applicants can be somewhat weak on specifying these
measures of success in their applications. There was some concern that the follow up of
projects was lacking and that the focus of attention is much more on launching and running
projects and programmes rather than looking for their longer term / ultimate impact. It was
suggested that metrics such as an index on related growth or 'build and run' related indexes,
would help address this and illustrate programme sustainability. It was suggested that ex-post
monitoring of impacts, could be a contractual obligation on the projects but this would imply
long contractual periods which would be difficult to enforce once payment (for the bulk of
the project) has been given. This is a strong point and one which is picked up in more detail
in the sustainability section of the report.
The use of patents and citations (resulting from FP funded projects) although interesting and
positive, was not felt to be a very strong or suitable indicator of success for the projects
funded. There were a number of logical reasons given for this opinion, as follows:
Patents and citations
are relevant but open
to manipulation and
miss many beenfits.
•
•
•
•
•
•
Spurious innovations can be patented if this is seen as an end in itself.
Citations can also be manipulated.
For the demonstration projects which are now the focus of FP ENER, it would be
expected that much of the innovation being demonstrated is already patented.
Sometimes the innovation in a project is the scale, which is not really a patenting
issue.
Some innovations achieved by programmes cannot be effectively or efficiently
patented.
Increasingly ideas are not patent protected anyway, due to the difficulty of
international patent protection.
There were very mixed views among the coordinators on the best metrics to use to measure
the impact of the programme. For every participant who suggested that one or other of the
proposed metrics (citation index, patents, overall R&D spend, and R&D spend per capita
31
Final Report - FP7 Energy Mid-Term Evaluation
GHG emissions) is a good indicator, three or four other participants cited drawbacks with
each one:
Participants agree that
impact is post funding.
Picking up the theme of the importance of results, 6 interviewees said that the true impact of
the programme can only be measured by the downstream impacts, be that sales volumes,
number of production plants constructed, increased turnover of participating companies,
number of new start up/spin off companies created, or GWh of energy saved or GHG
emission reduction.
Project bid evaluation
already considers CO2
savings.
The DG ENER officers confirmed that at the evaluation stage, they and the evaluators judge
how effective a project could be, by looking at what it says it will produce and how this could
ultimately address the issues of energy efficiency and CO2 savings. The proposals also have
to say how the results are going to be used. The 'impact' assessment criterion rates projects on
the potential impact through the development, dissemination and use of their results. This is
judged by looking at a combination of the quality and effectiveness of the S/T methodology,
the work plan and the appropriateness of measures they propose for disseminating and
exploiting their results, including how IPR issues are addressed.
Participants think their
projects match
programme objectives
and 30% expect to
patent something as a
result.
In respect of effectively delivering outputs that contribute to programme objectives, the
survey responses showed that a high proportion (>70%) of respondents expect to produce a
demonstrator, prototype or pilot as a result of their project, but that only 30% expect an
exploitable product (patent) to be among the project outputs. There are a variety of reasons
why this may be the case, which may be worth exploring further. It will be interesting expost, to compare these expectations of results with the actual results reported, as if 30% of the
products are able to lodge a patent this will be a significant improvement from the actual
results of FP5 where only 13% did this4.
4.3.1
Research focussed
project participants
expects research
benefits, companies
expect commercial
benefits.
Commercial vs. Research Impacts and Benefits
Reviewing the project participant interviews it is apparent that partners with a research
background were more optimistic of achieving, and are arguably more motivated by, the
scientific advances that they expected their project to make. As expected the converse of this
was that commercial partners are more optimistic of, and motivated by, the potential sales
and income that a successful demonstration project can help generate for them.
From the survey it also appeared that the programme may potentially be less effective in
delivering against the commercial objectives of the EU, than it is against the scientific.
Responses from the survey showed that respondents are more positive about achieving
significant scientific or technical impact rather than commercial. There was some variation
between partners and co-ordinators, with partners more positive in improving their
competitive positions and increasing profitability. A similar variation was found between
private companies and non-companies, with private firms more confident of significant
positive impacts against commercial objectives. SME status appeared to have little or no
effect on the respondent’s expectations for commercial gain. Commercial impacts were felt
by respondents to be the most likely at EU level, with less expectation of local, regional,
national or global impact, this is in-line with the level of the FP7 programme, though not
necessarily industrial objectives.
4
Ex Post of NNE Projects Funded by DG TREN under FP5. Ecotec Research and Consulting. 2006
Final Report - FP7 Energy Mid-Term Evaluation
32
4.3.2
Potential CO2 savings
are requested in
applications to a UK
programme.
These show market
awareness of the
applicant, which is key
to commercialising a
demonstration project.
Stakeholders agree
with the benefits of
asking for CO2
savings.
But don’t over rely on
the accuracy.
Indicators of Project Level Impact – Carbon Savings
A number of consultees pointed out that comparable national level energy research schemes
across the EU had more tightly defined output indicators for the projects they support. For
example the Carbon Trust in the UK requests estimates of potential carbon saving and market
replication from the very first application. These estimates are used to judge the theoretical
carbon benefits of the proposal – both in terms of the savings of an individual installation and
the replication scale. For example a very small improvement to every car has a massive
potential, but a major improvement to a very rare industrial process may still be viable but
has a much smaller potential. The question on replication potential also provides a clear
insight into the applicant’s knowledge of the market they expect their innovation to enter
into. If this knowledge is lacking it is a good indicator of an applicant not well suited to
commercialising a product. However, the Carbon Trust consultee stated that they could see
that some other FP objectives would be more difficult to measure, for example, if quality of
research is measured by citations in peer reviewed journals, this will favour a certain type of
project and not one which necessarily saves the most carbon.
With regard to asking projects to estimate the carbon savings their project should result in if
successful, there was general acceptance of this idea, though with a number of qualifications.
It was pointed out that research can be complex and the exploitation of the results varies
widely and usually occurs most intensely after a project has finished (in terms of FP funding).
So if the Commission asked for information on expected carbon savings post exploitation
they would need to be careful that they were not swayed by over-ambitious claims - that they
would not check up on anyway as they would occur after the FP funding period. The
majority of those questioned thought that the CO2 savings question is important as it
quantifies the impacts and clearly links to the priority target for ENER – 20/20/20. Even
though it is not necessarily an easy question to answer, applicants should be obliged to
attempt to do so, as long as the method to do this is clearly explained, a rational view taken
on any claims which appear extravagant, and it was not used as the only criteria for judging
applications.
Its hard to see in
advance the long term
large CO2 benefits
from come from some
projects.
On a more fundamental level one consultee pointed out that the ultimate carbon savings that a
project achieves are very difficult to isolate as they are driven by such a complex, and
member state specific set of drivers. There are many clear examples of this, such as the high
concentration of concentrated solar power (CSP) plants in Spain, where there are significant
subsidies available to support the technology, compared to Italy where the solar conditions
are very similar but no such subsidy exists and therefore no CSP plants have been installed.
Application guidance
asks about CO2 but
there is no specific
question on the form.
The guidance documents on completing the application form ask about CO 2 savings
(although there is no explicit question in the form) and this is an important factor in assessing
bids. Replication potential is also raised as relevant but it is recognised that this a more
complex issue than CO2 as it requires a commercial understanding of EU market. Some
officers also thought that estimating CO2 savings was also too difficult for applicants to
estimate. These markets vary a lot by technology, e.g. retrofit vs. new and by MS and even
region. For example with biomass the replication potential depends on resource availability –
this varies widely and at a very local level – and is therefore very hard to know for the whole
of the EU. Estimation of replication also requires some prediction of the future, which is
complex, though this could be improved, for example a view on future fossil fuel prices is
key to estimating replication potential for the next 10-15 years.
Complex MS market
variations make the
CO2 savings estimate
difficult.
33
Final Report - FP7 Energy Mid-Term Evaluation
CO2 savings are
closely linked to
markets and subsidies
etc..
4.3.3
FP has a positive
policy influence on EU,
MS and companies.
.Low carbon energy
r+d has increased
over the last 3-5 years.
It was also recognised that the ease and speed with which a low carbon technology becomes
commercial is strongly influenced by factors other than the technology, i.e. financial
incentives. This reflects the fact that at the present time most technologies targeted by the
programme are still not competitive with conventional (fossil) solutions, though some are
getting close (e.g. wind). The length of time that such financial incentives will be required is
very closely linked to the fossil fuel price. However the support provided by the FP will
improve the technologies and make them more competitive with fossil fuels. A number of the
officers described it as a balance to strike between supporting the ideas which are very close
to commercial and those which are higher risk. If projects are too close to market the support
becomes subsidising what the companies should do themselves, if too risky, the
implementation may be difficult to achieve and too few of the projects succeed.
Programme Level - Strategic Added Value
One consultee raised the point that an extremely valuable programme level benefit was the
positive strategic and policy influence. She voiced the opinion that even relatively recently in
2006/7 when FP7 started, the implied logic (of the overriding importance of low carbon
energy) was not generally accepted by the energy industry or at MS level. Some MSs had
adopted this policy position and were actively pursuing it while others (probably the
majority) were not pursuing it with as much interest. This position was evidenced by the very
low percentage of energy related R+D expenditure that was targeted on clean / low carbon
technologies, 3-5% of the total. This figure has rapidly increased over a very short period of
time to the point where it now the main focus of research expenditure. The consultee felt that
FP7 was an important stimulus in increasing this percentage, even though in comparative
terms the funding amounts involved in the FP are very low. The FP also had an important
influence in helping move clean energy up the political agenda, its objectives are now the
generally accepted approach, with all MSs sharing the same objectives. This reflects the rapid
policy development in this area as climate change has been widely accepted and firm policies
and goals have been put in place to start to address it. Therefore in considering the impact of
FP7 it is important to look at both direct investment in FP7 projects and the influence it has
had on the policies and spending of MSs and the energy industry.
The FP has had a role
in this increase but the
scale is hard to know.
The project participants were asked how important the programme was with regard to
influencing the policies of the EU, member states and the energy industry. The general view
was that its influence is positive but relatively minor. The positive aspects mentioned
included generic issues such as raising the profile of R&D. With regard to energy companies,
as has been mentioned previously, they are now increasing the R&D efforts in this area, but
this is largely due to them seeing the commercial necessity and benefits of doing so rather
than any influence from the FP.
Well presented FP
projects can be very
helpful in illustrating
policy goals.
The point was also raised that the presentation of the results was key to their ultimate
influence. Where results are presented as guidance there will often be a wide variance in how
MSs interpret this. One coordinator felt that “the policy influencing role is critical and that
the Commission is absolutely driven by it”. He could see the policy role and targets (2020) as
very important and in some ways as a 'weight on the shoulders' of commission people. This
could be a downside in creating too much pressure and pushing too fast. He also felt that the
policy influencing role on MSs was important as some states are lagging behind and that the
influence of FP7 is good in helping them to catch up.
Final Report - FP7 Energy Mid-Term Evaluation
34
Larger projects with
higher profile help in
terms of policy
influence.
One participant felt that the increased scale of projects was a positive factor in terms of their
ability to influence policies. A new MS participant, from an energy utility company, reported
that his company had been actively encouraged to take part in the FP by their government.
35
Final Report - FP7 Energy Mid-Term Evaluation
5 Consistency
5.1.1
Definition
The extent to which positive / negative spill over into other social, economic or
environmental policy areas are evolving.
5.1.2
Specific Questions
The TOR contains the following key issues and questions which we have classified under this
heading. Each question is followed by our opinions based on the data, analysis and opinions
that we have collected.
How can the impact and added value of collaborative research that cuts across scientific disciplines,
industrial sectors and policy fields be further enhanced with a view to better address large societal
challenges?
Aware of equality
issues but a low
contribution expected.
The survey highlighted that participants are aware of the benefits that they should receive
regarding boosting and protecting their research capacity but a low percentage considered
their project would bring benefits in terms of equality in science. Sexual equality was a low
priority for projects, though it was agreed that this issue was appropriate for consideration
and could be more systematically measured and recorded, for example measuring how many
women actually worked on a project when completed.
Energy costs should
reduce via project
success but its beyond
the scope of the FP to
influence these.
The issues of energy cost, and the potential increases that particularly renewable energy can
bring, were an issue that was picked up as missing from the objective tree and it does appear
that this issue is not actively considered in many projects. However it was pointed out that the
purpose of the funding is to improve the cost effectiveness of the technologies and that
energy prices and charging mechanisms are an issue beyond the control of the FP, and the
EU.
Positive spill over is
expected into skills,
health, env, etc, but
these should not be
subjected to additional
monitoring efforts
during project
operation.
The project participants were asked to comment on any spillovers from their projects into
wider socio-economic policy areas. In general this revealed a number of positive and
interesting benefits from projects, but as with the other benefits and impacts many of these
were felt to only become apparent some time after the projects have completed. While the
participants recognised these benefits they were generally felt to be tangential to the main
purpose of the projects and as such it was not felt that they should receive significant
additional monitoring. Many of the described benefits could be seen to clearly related to (i.e.
stem from) the standard outputs expected from a project. Some of those questioned agreed
that more could be done in this area by projects, for example educational benefits from
projects going into schools etc. However, such activity would divert the focus and resources
of the project team, and is not their core competence, and should therefore not be something
Final Report - FP7 Energy Mid-Term Evaluation
36
that project partners do. If such benefits are required it would be better pursued via another
programme
5.2
Surveyed project
participants expect
positive spill overs
from their work.
Project Survey Findings
In respect of contributing to wider EU policy objectives over 50% of those surveyed expected
their project to have a significant impact. This is particularly the case in respect of quality of
life and health and safety. Impacts on employment are lower but still over half expect
significant employment impacts.
Spillover into wider EU science and technology benefits was strongly expected by
respondents, with over 75% expecting impacts of medium or high significance on policy
objectives in this area from their project. The one exception to this was gender balance in
science with a much more mixed view of the impact of their projects.
The survey respondents also thought that their projects would have significant impacts on
wider environmental objectives, this was particularly felt to be the case in the areas of GHG
emissions reduction, energy efficiency and increasing the share of renewable energy.
5.3
Participant interviews
confirm that positive
spill overs are
expected.
Project Coordinator and Participant Interviews
The project participants were asked to comment on any spillovers from their projects into
wider socio-economic policy areas. In general, this revealed a number of positive and
interesting benefits from projects, but, as with the other benefits and impacts, many of these
were felt to only become apparent some time after the projects have completed. While the
participants recognised these benefits, they were generally felt to be tangential to the main
purpose of the projects and as such it was not felt that they should receive significant
additional monitoring.
Reflecting this opinion one coordinator stated that projects “need to guard against unrealistic
expectations of the impact on society as any social impact may take a long time to emerge”.
He also suggested that “In order to capture evidence of wider impacts perhaps the
Commission could produce a set of case studies which provide concrete examples”.
.. e.g. jobs, skills,
health, income and
environment.
Some of the positive socio-economic spill over described included the following:
•
•
•
•
•
•
•
•
Creating jobs
Improving skills – in research, manufacturing and operation and maintenance.
Improving health (via improved air quality from reduced use of fossil fuels.)
Influencing the pattern of economic development (e.g. away from crowded urban
areas and creating more activity in rural areas).
Economic benefits – captured by industrial revenue from downstream commercial
sales.
Environmental benefits – correlated to the CO2 savings.
Technology transfer to new countries.
Informal network building
37
Final Report - FP7 Energy Mid-Term Evaluation
These benefits could
be increased but this
risks tangential
activity.
As can be seen, many of these benefits are clearly related to (i.e. stem from) the standard
outputs expected from a project. Some of those questioned agreed that more could be done in
this area by projects, for example educational benefits from projects going into schools etc.
However such activity would divert the focus and resources of the project team, and is not
their core competence, and should therefore not be something that project partners do. If
such benefits are required it would be better pursued via another programme
Sexual equality is
recognised as relevant
but its hard to address
in a male dominated
sector
The issue of sexual equality in projects was raised and it was agreed that this issue was
appropriate for consideration and could be more systematically measured and recorded, for
example measuring how many women actually worked on a project when completed. Some
examples of good practice were offered by projects, including a high level of interest from
girls when carrying out science promotional work in schools and a growing renewable energy
company (ABNT) where over 60% of the workforce is female. Although one coordinator felt
it was not important to promote equality within the programme and other mechanisms such
as promotion of science at a young age, promotion of programme results as subject in the
secondary schools, etc. were more appropriate.
More guidance on best
practice would help.
Final Report - FP7 Energy Mid-Term Evaluation
38
6 Allocative / Distributional
6.1.1
Definition
The extent to which disproportionate negative / positive distributional effects of policies are
evolving.
6.1.2
Specific Questions
The TOR contains the following key issues and questions which we have classified under this
heading. Each question is followed by our opinions based on the data, analysis and opinions
that we have collected. Some of the issues of balance of funding are also discussed under
relevance.
Measuring the coverage of research subjects relevant to objectives of FP7 and identification of gaps.
The analysis of programme data has revealed some interesting issues, these include:
•
•
•
•
•
More RES than EE
projects are funded
but the split is
complex.
The vast majority (89%) of the programme budget to date has been spent in the first
4 calls. The 11% that has not been spent is mainly due to a reduction in funding
during negotiation to a number of projects. It is assumed that this can roll forwards
into the remaining calls.
The number of applicants to each call has dropped significantly over the 4 years of
operation, from 231 in the first call to 43 in the third and 45 in the fourth, This relates
to a major reduction in the number of areas open for bidding.
The chance of a bid being supported has increased from 11% to 33% or 1 in 3.
Over the 4 calls the average project value has increased from €6.1 million to €17.6
million.
The industrial match funding has increased over time.
The programme analysis indicates that there are more renewable energy focused projects than
energy efficiency. In part this is thought to relate to the better fit of renewable energy to large
demonstration projects than is the case for energy efficiency projects. It was also pointed out
that energy efficiency is picked up and supported by other parts of the FP. An important
reason for the relative lack of energy efficiency projects was also felt to be a failure for
market mechanisms that support deployment to develop in the same way as they have for
renewable energy.
As far as country participation is concerned, providing figures for each member state and associated
countries, identifying those which are less involved and factors to explain why, together with
recommendations to implement in order to eventually bring them aboard.
39
Final Report - FP7 Energy Mid-Term Evaluation
Over 1/3 of
participants are new to
the FP.
The survey illustrated that some 55% of the participants surveyed have taken part in FP
projects before, though some 38% were entirely new to it. This illustrates that the programme
is still attracting a healthy amount of new entrants, from what is a relatively small target
group, although the assumption that past participants have a higher success rate appears to
remain true.
Old MSs dominate
funding >95%.
Germany and Spain have received the largest amount of funding to date with the members of
the EU15 taking 95.1% of the funding to date.
Measuring the proportion of SMEs participation and their role, providing with findings on the
specificities of SMEs in the sector of energy technologies, including in terms of competitiveness and
employment
SME participation rate
is high.
The issues of SME and new Member State participation are much discussed in evaluations of
this nature. Although the smaller number of larger projects being supported means that the
number of SMEs participating is reduced, the rate is reported as 19-20% which is higher than
the FP target of 15% and higher than FP7 average level of 13%.
SMEs are useful but
there is no desire for a
quota.
The generic barriers of attracting SMEs to participate in the programme are well known and
many were repeated in the consultations. These include the conflict which often exists
between the desire to see quick and commercial results that SMEs have in comparison to the
longer time scales that are typical for FP projects and amongst research partners. However it
was pointed out that SMEs can often bring a high level of innovation to projects, particularly
in clean energy where many of the technologies are, by industrial standards, new.
There were a number of suggestions made by project coordinators on how to encourage
SMEs to participate in the FP. Over half of the ten coordinators interviewed though that some
enhanced level of funding support for SMEs would help attract them. The suggestions
included higher support rates or an increased share of the project budget. Two coordinators
thought this could be done by limiting the number of participants, perhaps via a specific
stream of funding. Such an SME orientated strand could also offer a streamlined approach,
with less onerous requirements and quicker, even pre, payments. It was suggested that a
reduction in the administrative efforts required in the application process would make the FP
more attractive to SMEs, but it was accepted that there would always need to be some
minimum threshold of information required.
In conclusion, the majority of project coordinators and participants interviewed agreed with
others stakeholders and felt that the focus must be on getting the best team in order to achieve
the highest standards of R&D, irrespective of quotas for SMEs/MSs/gender etc and that one
should not include a SME just to satisfy a quota requirement.
NMs have a low
representation, but this
is hard to address.
Similar points were made regarding new Member State participation, with the additional
point that their applicants have not had the opportunity to build up the capacity and
experience over numerous FPs that participants from the old MSs have.
Final Report - FP7 Energy Mid-Term Evaluation
40
6.2
Move to more
focussed calls in fewer
areas with larger
projects.
Programme Data Analysis
The FP7 Energy programme has already evolved since 2007, with the SET plan one of the
major factors in its evolution in this period. This evolution is evident through the number of
topics called each year as shown in the table below. In each year the number of topics called
has declined, from the 2007 call with 45 topic areas open to the 2010 call with only 8, as the
programme has moved towards having a tighter strategic research focus. This move also
appears to have contributed, by the 2010 call, to an increase in the proportion of called topics
with at least one project funded from 42% to full coverage of 100%, and a consequent
decrease and elimination in 2010 of the number of topics called where no projects were
funded.
Table 6.1 Analysis of topics called and funded
2007-TREN2
2008-TREN1
2009ENERGY-2*
2010
ENERGY-2*
Topic Areas Open
45
18
10
8
Project proposals
evaluated
231
130
43
45
449
Projects approved for
funding
26
22
10
15
73
No of call topics with at
least 1 project funded
19
8
7
8
42
42%
44%
70%
100%
51.9%
26
10
3
0
39
58%
56%
30%
0%
48.1%
2
2
0
0
4
Call >
As % of total called
No of call topics with no
projects funded
As % of total called
No of projects funded in
non-call topics
Total
* 2009 and 2010 figures are provisional as these projects remain at the negotiation stage
6.2.1
Total Allocations
Our analysis of the 73 projects already approved (or in negotiations) for funding from the
first 4 calls of FP7 Energy shows the following:
89% of the budget to
date has been spent.
Much of this relates to
projects not
proceeding post
negotiation.
Total EU Funding of €446.3 million has been allocated across 73 projects in the first 4 calls.
This represents an allocation of 89% of the €501.4 million budget. Within this there was
variation in funding allocation with the 2007 call allocating less than 70% of its budget while
the 2008 and 2009 calls allocated over 95%, and 2010 over 100% of their budgeted
allocations. The main reasons at this stage appear to be a reduction in funding allocated to 6
projects funded under the 2007 call, for each project these cuts represented at least 50% of
the original budget, 1 project (2ndVEGOIL) was also funded under the 2007 budget but is
now allocated to DG MOVE and therefore outside the scope of this evaluation. It is unclear if
this process will be repeated in future calls as projects progress, funding has so far been
reduced on one project (EMPYRO) from the 2008 call.
41
Final Report - FP7 Energy Mid-Term Evaluation
No. of applications has
gone down
significantly.
The number of applications submitted has gone down considerably from 231 in the first call
to 45 in the fourth. This is thought to mainly be a result of the significantly reduced number
of areas ‘open’ for bids and requests for larger demonstration projects in the second, third and
fourth calls. This reduction in the number of applicants has led to a clear increase in the
chances of a submitted bid being approved, with the success rate increasing from 11% to
33% or around 1 in 3.
Average project value
has almost tripled and
industry match has
increased.
The funding the EU has provided has been supported by around €378 million of investment
by the project partners, so that total investment across the 74 projects totals over €824
million. On average the EU investment has secured matching commitments from project
participants equal to around 85% of the EU funds invested. Alternatively it can be said that
EU funding represents around 54% of the total value of the funded projects. Finally the
average project value has increased through each project call as budgets remain of similar
size but fewer projects are funded, over the 4 calls the average project value has increased
from €6.1 million to €17.6 million. Similarly the total value of the largest funded project in
each call has grown significantly.
Some very large
projects are now being
supported.
Table 6.2 Analysis of project funding
2007-
2008-
2009-
2010
TREN-2
TREN-1
ENERGY-2*
ENERGY-2*
Budget (€ mio)
128
147
100
126.4
501.4
Projects approved for funding
26
22
10
15
73
11.3%
16.9%
25.6%
33.3%
16.5%
86.7
142.5
85.8
131.4
446.3
67.7%
96.9%
85.8%
104.0%
89.0%
72.1
107.3
66.3
132.0
377.7
158.8
249.8
152.1
263.4
824.0
Average project value (€m)
6.1
11.4
15.2
17.6
8.9
Highest Project Value
28.3
21.7
58.0
71.6
Call >
Success % (applications /
approved)
EU funding of approved
projects
% of budget allocated
Other funding of approved
projects
Total value of approved
projects
Total
*2009 and 2010 figures are provisional as these projects remain at the negotiation stage
DG ENER have
contributed some
€31m to other FP calls
in combination with
other DGs.
DG ENER also part financed joint FP calls with other DGs. To date these have been:
•
FP7-ENERGY-2009-BIOREFINERY 15 MEUR
•
FP7-2010-NMP-ENV-ENERGY-ICT-EeB 15 MEUR - Energy Efficient Buildings
•
FP7-OCEAN-2010 1MEUR - Ocean (horizontal action call falling under activity 10)
The projects partly supported by DG ENER under the joint calls have been excluded from the
other analysis to avoid fractions of projects and participants that would not help analysis.
Final Report - FP7 Energy Mid-Term Evaluation
42
6.2.2
Allocation by Activity
In the first 4 calls funding has been allocated across the 8 activity areas as listed in table 6.3.
Only 72 projects are listed here as 1 project, SET-Plan_se2009.eu, was an ad-hoc project that
funded the SET Plan and fell outside the activity categories.
Table 6.3
Activity
Activity areas funded in first 3 FP7 Energy calls
Projects
Total Project
% of Total
Funded
Value (€m)
Value
21
253.4
30.8%
12
144.3
17.5%
8
44.2
5.4%
0
0.0
0.0%
Cross cutting technologies 5 & 6
3
12.5
1.5%
6
Clean Coal Technologies
6
64.1
7.8%
7
Smart Energy Networks
9
185.1
22.5%
8
Energy Efficiency and Savings
12
118.2
14.3%
9
Knowledge for Policy making
1
2.2
0.3%
10
Horizontal Actions
0
0.0
0.0%
72
823.9
100.0%
Code
2
3
4
5
5+6
Description
Renewable Energy Generation
Renewable Fuel Production (excluding
contribution to bio refineries)
Renewables for Heating and Cooling
CO2 capture and storage technologies
for zero emission power generation
Total
Appears that RES gets
more support than EE.
Figure 4.1 shows the proportions of funding allocated to each activity area. This clearly
shows that activities 2, 3 and 4, that cover the renewable energy categories, account for over
half (~54%) of all projects funded, while work on cross cutting technologies and clean coal
accounts for around 9% of projects funded, and smart energy networks and energy efficiency
savings account for just over 1/3 of funding (~37%). This breakdown highlights a clear
commitment to renewable energy and energy efficiency technologies which is generally
consistent with the FP7 objectives and priority areas.
But the split is not
simple, e.g. smart
networks are relevant
to both.
Figure 4.2 shows the amount funded by activity area for each of the first 4 calls. There are a
few key changes evident between the calls. Funding for activity 2, renewable energy
generation has remained relatively stable through the calls but substantial funding for activity
3, renewable fuel production, in 2007 and 2008 was reduced to zero in 2009 and returned
again in 2010. Cross cutting activities were funded in 2007 and 2010 but not 2008 or 2009,
funding of clean coal activities was absent in 2007 but present all subsequent years. Funding
for smart energy networks and energy efficiency activities was a substantial part of each of
the calls but the balance between the 2 elements has swung with each call, from fairly even in
2007, to primarily energy efficiency in 2008, to primarily smart energy networks in 2009 and
to only smart energy networks in 2010.
43
Final Report - FP7 Energy Mid-Term Evaluation
Figure 6.1 Funding by Activity Area – Total of 2007, 2008 & 2009 calls
FP7 Energy Funding Allocated by Activity Type - Total First 4
calls
Energy
Knowledge for
Efficiency and Policy making
Savings
0.3%
14%
Renewable
Energy
Generation
31%
Smart Energy
Networks
22%
Clean Coal
Technologies
8%
Cross cutting
technologies
2%
Renewables for
Heating and
Cooling
5%
Renewable Fuel
Production
18%
Figure6.2 Funding by Activity Area – By Call (EU Contribution only)
160
Know ledge for
Policy making
140
Energy Efficiency
and Savings
120
Euros (Millions)
Smart Energy
Netw orks
100
Clean Coal
Technologies
80
Cross cutting
technologies
60
Renew ables for
Heating and
Cooling
40
Renew able Fuel
Production
20
Renew able
Energy
Generation
0
2007-TREN-2
2008-TREN-1
2009-ENERGY-2
Call (Year)
Final Report - FP7 Energy Mid-Term Evaluation
44
2010 ENERGY-2
6.2.3
Split within themes is
complex – with many
projects being cross
cutting.
Specific Activity Area
The 73 projects funded under the first 4 calls have focused on a number of the 38 specific
activity areas. Table 4.4 presents the number of projects and total value of projects at this
level. Within Renewable Energy Generation the largest share (30%) goes to cross-cutting
issues, this encompasses 2 projects approved in the 2010 call, one is a solar power –
desalination demonstrator and the other a solar CSP plant with CHP, mixed fuels and fresh
water applications. These cross-cutting projects aside Photovoltaics, Biomass and Ocean
Energy take the next largest shares, each over 15% of the total EU contribution to project
value, and also each representing around 5% of total FP7 Energy project value.
Within activity 3 Renewable Fuel Production, projects in Second generation biofuels from
biomass dominate the project value, with around 2/3 of all EU funded value in this activity.
This represents 6 projects worth €51 million, the second largest funding of a single area
across all FP7 Energy areas, representing over 11% of all DG Energy FP7 Energy project
funding.
Within activity 4, Renewables for Heating and Cooling, funding is split relatively evenly
across 4 of the activity areas.
Clean coal covers
carbon capture and
storage (CCS).
As noted previously, there is no funding for CO2 storage under activity 5 but there is some
funding in these areas under the cross-cutting activity 5+6. Within activity 6, Clean Coal
generation, the primary area of funding has been conversion technologies for zero emission
power generation, with €41 million allocated over 6 projects.
Under activity 7, Smart Energy Networks, projects are split across all 3 areas but funding is
heavily skewed to the Development of interactive distribution energy networks, this area
representing the single largest funded value of any activity area at €83.4m, representing
almost 19% of all FP7 funding to date.
Within activity 8, Energy Efficiency and Savings, there are a number of projects in areas 8.1,
8.2 and 8.4, with the largest funding, of over €37m, going to 8.4 to support the integration of
renewable energy and energy efficiency under the CONCERTO initiative.
Table 4.4
Activity
Code
2
Specific funding activity areas in first 3 FP7 Energy calls
Description
Renewable
Energy
Generation
Area
Code
Description
Projects
Funded
Total EU
% of total
Funding €
in activity
m
code
% of
whole
2.1
Photovoltaics
5
28.6
21.8%
6.4%
2.2
Biomass
4
22.6
17.2%
5.1%
2.3
Wind
5
14.1
10.7%
3.2%
2.4
Geothermal
0
0.0
0.0%
0.0%
2.5
Concentrated solar power
1
6.0
4.6%
1.3%
2.6
Ocean energy
4
20.7
15.7%
4.6%
2.7
Hydro-Electric
0
0.0
0.0%
0.0%
2.9
Cross-cutting issues
2
39.4
30.0%
8.8%
45
Final Report - FP7 Energy Mid-Term Evaluation
Activity
Code
3
4
5
5+6
6
7
8
Description
Renewable Fuel
Production
Renewables for
Heating and
Cooling
CO2 capture
and storage
technologies
for zero
emission power
generation
Cross cutting
technologies
Clean Coal
Technologies
Smart Energy
Networks
Energy
Efficiency and
Savings
Area
Code
Description
Projects
Funded
Total EU
% of total
Funding €
in activity
m
code
% of
whole
3.1
First generation biofuel from
biomass
1
5.0
6.5%
1.1%
3.2
Second generation biofuels
from biomass
6
51.0
66.8%
11.4%
3.3
Biorefineries (Part supported
by ENER)
0
0.0
0.0%
0.0%
3.4
New energy crops
3
19.2
25.1%
4.3%
3.5
Alternative Routes to
Renewable Fuel Production
0
0.0
0.0%
0.0%
3.6
biofuel use in transport
0
0.0
0.0%
0.0%
3.7
cross-cutting issues
2
1.2
1.5%
0.3%
4.1
Low/medium temperature solar
thermal energy
3
6.1
24.8%
1.4%
4.2
Biomass
2
7.0
28.5%
1.6%
4.3
Geothermal energy
1
4.3
17.5%
1.0%
4.4
Cross-cutting issues
0
0.0
0.0%
0.0%
4.5
Cross cutting issues
2
7.1
29.1%
1.6%
5.1
CO2 Capture
0
0.0
-
0.0%
5.2
CO2 Storage
0
0.0
-
0.0%
5+6.1
Power generation technologies
for integrated zero emission
solutions
0
0.0
0.0%
0.0%
5+6.2
Cross cutting and regulatory
issues
3
8.6
100.0%
1.9%
6.1
Conversion technologies for
zero emission power
generation
6
41.0
100.0%
9.2%
6.2
Coal-based polygeneration
0
0.0
0.0%
0.0%
7.1
Development of inter-active
distribution energy networks
4
83.4
84.8%
18.7%
7.2
Pan european energy networks
2
4.9
5.0%
1.1%
7.3
Cross cutting issues and
technologies
3
10.0
10.2%
2.2%
8.1
Efficient energy use in the
manufacturing industry
3
10.2
16.0%
2.3%
8.2
High efficiency polygeneration
4
14.7
23.1%
3.3%
8.4
Innovative integration of
renewable energy supply and
energy efficiency in large
communities: CONCERTO
4
37.8
59.4%
8.5%
8.5
Innovative strategies for clean
urban transport: Civitas-plus
0
0.0
0.0%
0.0%
8.6
socio economic research and
innovation
0
0.0
0.0%
0.0%
Final Report - FP7 Energy Mid-Term Evaluation
46
Activity
Code
9
10
Area
Code
Description
Knowledge for
Policy making
Horizontal
Actions
Projects
Funded
Description
% of total
Funding €
in activity
m
code
thematic promotion and
dissemination
1
1.0
1.6%
0.2%
9.1
Knowledge Tools for Energyrelated Policy Making
0
0.0
0.0%
0.0%
9.2
Scientific Support to Policy
1
2.2
100.0%
0.5%
10
Horizontal Actions
0
0.0
-
0.0%
72
446.2
100%
Geographic Spread
It was made clear in the inception phase of this evaluation that a balanced distribution of
project funding between individual EU member states was not now regarded as a primary
objective of the FP7 Energy programme. As discussed in more detail in section 2 the
objectives of the programme are to fund high quality research, in areas of strategic need
which have the potential to have significant short and / or long term impacts, regardless of
location. At the same time the organisation and spread of funds between member states,
associate countries and 3rd countries is relevant to FP7 objectives and commitments to
international science and technology co-operation. Encouraging participation from newer
member states is also a desirable outcome.
Vast majority of funds
go to the EU.
Analysis of funding provided in the first 4 calls shows that over 90% of funds are allocated to
beneficiaries in EU member states. Of the remainder around 7.7% is allocated to beneficiaries
in associated countries 5 and 1.9% is allocated to beneficiaries in 3rd countries.
95% of funds to the
EU15, matches GDP
mix but not population.
Of the funding allocated to beneficiaries in EU member states, approximately 95% has been
allocated to beneficiaries located in EU-15 countries, and only 4.9% to those in the new
member states (see table 3.5). This balance varied from 91% / 9% in the 2008 call to 98% /
2% in the 2010 call. This balance is broadly similar to the GDP split between the EU-15 and
non EU-15 member states which is around 93% / 7%, but not the population balance which is
closer to 80% / 20%. It should be noted that 2 of the non EU-15 member states (Latvia and
Malta) have not yet received any funding under the FP7 programme.
Table 4.5 Funding by Geographic Area – Total first 3 calls
Beneficiary group
FP7 funded
Value (€m)*
%
Member state
group
Funding to EU-
Funding to EU
Member States
Funding to FP7
Associate Countries
5
% of
whole
8.7
TOTAL
6.2.4
Total EU
408.3
15
90.4%
Funding to non
EU-15
34.8
FP7 Funded
Value (€m)*
%
388.4
95.1%
19.9
4.9%
7.7%
Associated countries are listed in ftp://ftp.cordis.europa.eu/pub/fp7/docs/third_country_agreements_en.pdf
47
Final Report - FP7 Energy Mid-Term Evaluation
Funding to 3rd
8.4
1.9%
Countries
*- based on participant grant request in 2010 call – it is likely that some participant requests will be reduced – this
is unlikely to materially affect these proportions.
Figure 4.4 shows the total EU funding by beneficiary country for the first 3 calls of FP7
Energy. This shows that by value, beneficiaries in Germany and Spain received the largest
funding, receiving over €65 million in total. Other major national concentrations of
beneficiaries include Denmark, Italy, the Netherlands, France, the UK and Belgium, with
beneficiaries receiving more than €25 million in each country over the 4 calls.
Figure 4.4 EU Funding by Country of Beneficiary – Total first 3 calls
Country of Beneficiary
.Germany and Spain
are the biggest
recipients – by a large
degree.
Germany
Spain
Denmark
Italy
Netherlands
France
UK
Belgium
Finland
Greece
Sw itzerland
Norw ay
Austria
Sw eden
Portugal
Hungary
Lithuania
Ireland
Poland
Egypt
USA
Czech republic
Turkey
Luxembourg
Croatia
Israel
Romania
Slovakia
Russia
Slovenia
China
Ukraine
Bulgaria
India
Vietnam
Serbia
Macedonia
Cyprus
Estonia
Tunisia
Algeria
Australia
South Africa
Thailand
Chile
Kazakhstan
Brazil
0
10
20
30
40
Euros (millions)
Final Report - FP7 Energy Mid-Term Evaluation
48
50
60
70
6.2.5
Old MSs are best
networked in FP
projects.
Network Analysis
Annexe 9.8 of this report presents a detailed network analysis of the project participants
(based on the first 3 calls of FP7) and their interlinkages for both FP7 and FP6. There are a
number of interesting findings from this analysis. As the diagram below indicates the EU 12
remain the most networked of the participating countries.
Figure: Collaboration visualization on country level (FP7, all links)
6.2.6
SMEs have positive
attributes to bring to
projects.
But quotas are seen
as counter productive.
SME Participation
The issue of SME participation, and associated programme level indicators raised a number
of interesting points. The objective of stimulating innovation capacity within energy SMEs
was well known and accepted as justified. It was also pointed out that SMEs often have some
of the most innovative ideas in what are, by industrial standards, very new technologies. As
such their inclusion within the programme is valuable. However it was also pointed out that it
is a difficult balance to strike between including SMEs and aiming to achieve world class
research. The same point was made with regard to including partners from new member
states. Although there are many exceptions to this apparently elitist view, i.e. world class
research projects are very much possible with SME and new member state partners, any
approach which forces projects to meet some ‘quota’ of SMEs or new member state
participation was felt to run the risk of imposing extra costs and reducing the effectiveness of
a project. The suggestion was made that the best way of solely supporting and expanding
SME and new member state research capacity is via a dedicated strand or separate
programme.
49
Final Report - FP7 Energy Mid-Term Evaluation
SME participation
rates are high.
SME participation rates in the DG ENER part of FP7 are actually higher than typical for the
FP as a whole and above the approximate target. Despite the perception that larger projects
would be off putting to SMEs, their participation rate is relatively high, at 19-20% against a
Commission target of 15% and a FP7 average level of 13%. Though the number of SME
participants will have gone down, as for all types of participants as the number of projects has
reduced. The route for SME inclusion is believed to typically be to provide a particular
component or expertise within a consortia led by a large company or other organisation. This
was contrasted with the position in DG Research where the perception is that SMEs are
crowded out by research providers. It was also reported by DG officers that the inclusion of
SMEs in a project application was viewed as a positive addition when assessing bids. It was
also pointed out that other SMEs take part as sub contracting third parties to non SME
participants.
SMEs are not best
suited to the
coordination role.
The point was raised that the demonstration projects now favoured by DG ENER need large
industrial partners, particularly in the lead. The smaller organisations still have an important
role – but typically not as the leader, e.g. SMEs to do specific components and RTD
organisations measure technical outputs. It is also hard for smaller organisations to bring the
scale of funds required to match fund the large demonstration projects now being pursued. A
number of officers felt it was difficult for SMEs to be a project coordinator as in comparison
to large organisations they will, by definition, have less administrative resources.
The issue of SME inclusion raised a number of comments from the project coordinators. One
UK coordinator felt that in the UK, SMEs are not coming forward to be involved in projects
and there is no encouragement for them to target EU funds. He felt that this was leading to
good ideas not being explored, with part of the problem being a perception among SMEs that
the process of obtaining funding is too complicated or the chances of being successful are too
slim.
Participants are happy
to include SMEs and
feel the opportunity is
these for those who
see k it out.
Additional funding and
networking might help
attract SMEs.
However the majority of the project coordinators thought that SME participation in projects
could bring benefits to the SMEs, the projects and the economy of the EU. One SME
participant felt that SME participation can help compensate for the domination by larger
firms in some technologies. He also strongly felt that the discipline required to receive FP7
funding was a major benefit to SMEs in getting them to improve their systems to 'big firm'
standards – this was particularly useful to document processes and to ensure that all useful
knowledge generated by the project was captured. He felt that the Commission provided
excellent support to SMEs to do this. Another SME participant felt that there were no
particular barriers to their participation in FP7 (apart from the frustration over extended
timescales). Another project coordinator stated that they had 2 SMEs in their project and that
he felt they were highly competent and it was beneficial to project to have them involved. A
participant from a research organisation felt that opportunities for SMEs to participate are
available if an SME wishes to take the initiative.
There were a number of suggestions made on how to encourage SMEs to participate in the
FP. Over half of the ten coordinators interviewed though that some enhanced level of funding
support for SMEs would help attract them. The suggestions included higher support rates or
an increased share of the project budget. Two coordinators thought this could be done by
limiting the number of participants, perhaps via a specific stream of funding. Such an SME
orientated strand could also offer a streamlined approach, with less onerous requirements and
quicker, even pre, payments.
Final Report - FP7 Energy Mid-Term Evaluation
50
Some SMEs put of by
the administrative
burden.
It was suggested that a reduction in the administrative efforts required in the application
process would make the FP more attractive to SMEs, but it was accepted that there would
always need to be some minimum threshold of information required. A two stage application
process was suggested as a good idea for SMEs as it gives them the opportunity to test an
application with less risk. Another idea for attracting SMEs was to stage networking events
where SMEs could meet FP7 partners. One participant felt that if the Commission asked for a
project to reduce its number of partners then SMEs are often the easiest partners to cut.
In conclusion the majority of project coordinators and participants interviewed felt that the
focus must be on getting the best team in order to achieve the highest standards of R&D,
irrespective of quotas for SMEs/MSs/gender etc and that one should not include a SME just
to satisfy a quota requirement.
6.2.7
NMS participation can
give benefits but they
lack the experience
and network of old MS
participants.
New Member State Participation
A programme level indicator which is well known and important for political reasons is the
participation rate by new member states (NMSs). The political importance of this indicator
was accepted and it was pointed out that there are some strong competencies in energy
research in the new member states which should be supported.
The general consensus was that NMSs do participate less than would be ideal. It was felt that
this could partly be explained by the relatively small size of their network of potential FP
collaborators, in comparison to the old MS applicants. Their (NMSs) lack of ability to
provide the required match funding was also thought to be an issue, as was their relative lack
of experience / capacity in the administrative requirements of bidding and running a project.
Setting any type of formal target or quota for NMSs was not regarded as a good idea as the
risk of their tokenistic inclusion was too large. Where they are included in projects they need
to be relevant and performing a task which needs their skills, though it was agreed that it can
be difficult to attract NMS partners without incentives such as a minimum budget. However,
it was pointed out that there are areas of strengths for the new MSs which should be
identified and exploited, and that there are also clear benefits to a consortium in terms of cost,
e.g. the same component/expertise can be sourced cheaper from a new than old MS. This is
also a benefit to EU competitiveness and consortium efficiency in up-skilling and including
these partners.
The project coordinators and participants were asked their opinion on the participation rates
from new member states (MS). As with SME participation, all of those questioned felt that
there was merit in including and encouraging partners from new MS. The lack of experience
that new MS partners have in preparing applications and forming consortia was recognised as
a negative factor for them. One coordinator with two new MS partners in his project reported
that they were very good contributors and had been brought in for specific expertise and
unique competencies to add to their project. A new MS participant stated that SMEs from
NMSs were particularly hard to attract.
6.2.8
Gender balance is
recognised as
important but hard to
address.
Gender Balance
Including women in projects was recognised as a legitimate policy goal. Officers felt that
applicants are aware of this and complete the relevant gender aspects section in the proposal
and related action plan. However it is recognised that the applications tend to include
51
Final Report - FP7 Energy Mid-Term Evaluation
standard responses and monitoring data, and therefore that there was some potential for
improvement in this aspect. The suggestions made by the officers included a check list of
gender inclusion issues and sign posting to best practice in this area in order to help
coordinators improve their responses and projects. However it was also pointed out that it is
difficult for projects to be strikingly original in terms of a 'gender aspects action plan' as what
is possible depends on the field. On this point the opinion was raised that the majority of
energy topics refer to a male dominated sector and imposing fixed tariffs, on a project tender
mix, would be counter productive. As with fixed targets for any aspect of the partner or team
mix it is more important to have the right (high quality) people involved.
6.3
New vs. Repeat Applicants
1/3 of participants are
new to the FP.
The survey responses showed that around 1/3 of participants are involved in an EU Energy
related funding programme for the first time, but also that over half have been involved in a
previous FP programme, while almost 75% had received funding nationally. The high
proportion of those with previous FP experience points to this being beneficial to making a
successful application, but at the same time the significant minority without any prior FP
experience, highlights that prior experience is not an essential requirement, and that the
programme remains accessible to new entrants. Analysis of the survey responses also showed
that it was rare for an organisation to have previously applied for EU research funding but not
national research funding, the majority had received both, and a sizeable minority (~20%)
had previously received neither.
Appears accessible to
new applicants.
Based on the survey evidence FP7 Energy can be said to remain accessible to participants,
with a healthy intake of new participants to the programme and retention of previous
participants. On this basis the programme has no disproportionate negative or positive
allocation or distribution effects.
Project coordinators were questioned regarding the importance of previous FP experience in
putting together a successful application. The coordinators agreed that such experience did
help but did not feel that this should necessarily be viewed as a problem as long as good
projects and organisations were supported. One coordinator felt that if previous projects had
been successful this was a positive indicator that should go in favour of an applicant and that
it could be the case that additional funding would be needed to help move projects out of
R&D towards commercialisation.
Final Report - FP7 Energy Mid-Term Evaluation
52
7 Sustainability
7.1.1
Definition
Will the effects achieved last in the medium or long term?
Also considered
programme future.
7.1.2
In addition to these issues of persistence we have presented the views on the future direction
of the programme under this heading. These issues could arguably be placed under relevance,
as they relate to maintaining relevance into the future, but we have placed them under this
heading to aid report balance.
Specific Questions
The TOR contains the following key issues and questions which we have classified under this
heading. Each question is followed by our opinions based on the data, analysis and opinions
that we have collected.
To what extent has the Commission actively collected and synthesised data on project and
programme outputs, results and impacts or contribution to objectives?
To what extent have the results of the actions been properly disseminated to stakeholders and the
public? What is their exploitable potential, and to what extent can one say that this potential has
been fully exploited?
Inter FP project
networking may work
but its risky.
Collective, EU
budgeted, promotion
of projects at high
profile conferences.
Clarification of project
funding extension
needed.
With regard to improving project dissemination, the project participants had mixed opinions
on the benefits of increased networking between related FP projects. Some were supportive
but others, some with experience of trying such an approach, felt that competition between
companies and the risk of a lack of synergies made the idea not worth pursuing, with existing
networking and conference arrangements being sufficient. There was more support for the
idea of some collective dissemination of projects, possibly funded and organised by the EU.
This idea would involve the presentation of related FP projects at high level sectoral trade
shows / conferences. The collective profile raising benefits of this suggestion appear sound
and as such it appears worth trialling.
A number of project participants felt that some additional funding should be made available
to project post completion to help with commercialisation issues. If this is difficult the
procedures and circumstances under which additional funds can be provided, at project end,
should be clarified.
What are the major challenges for energy demonstration and industry-led activities in the coming
year? How could FP7 address them in the best way?
53
Final Report - FP7 Energy Mid-Term Evaluation
Potential future
priorities included
follow on,
manufacturing
expertise and x cutting
issues.
There were a wide variety of suggestions made as to what areas should be targeted for future
support. A number of these related to follow on work in specific technologies (in which those
questioned were active). Some more general areas suggested included fuel cells, biofuels and
wave with support for manufacturing expertise and capacity highlighted in a number of areas.
A number of cross cutting areas were also highlighted including smart grids and the links
between biofuels, hydrogen production and electric vehicles. All of these technologies and
issues are the subject of the ongoing work by the technology platforms and the SET plan.
Given that the results of this evaluation will be directly put into practice for the elaboration of the
work programme 2012 and 2013 (which will benefit from an increased budget), what would be the
most suitable activities to include and why? What would be the most suitable funding options to
open reaching a balance between demonstration activities and policy research?
The SET plan is key to
the future focus of the
FP.
The SET plan was recognised by all those consulted as being key to the future focus and
nature of FP energy R&D. There appears to be something of a misunderstanding among
current FP participants that the SET plan will only support large scale demonstration projects
led by large companies and will do nothing to support more fundamental research and SMEs.
This misconception should be addressed.
KPIs in SET plan are
becoming clearer and
this should improve
over time.
The Implementation plans for the industrial initiative aspect of the SET plan contain a
number of KPIs that illustrate the development areas and goals for a number of technologies.
These plans and the KPIs should be clearly referenced in the future FP calls, with alignment
between FP Energy and SET expected to increase, as the plans become detailed and the level
of success of SET becomes more apparent.
DG ENER could only
direct FP funds via
SET but focus on
projects which ‘fit’ FP
criteria.
There were varying opinions on the degree to which DG ENER support under FP should be
solely directed via the SET plan. The most common views was that if the SET plan succeeds
this is likely to happen anyway, with the FP perhaps able to fund those projects that MSs
were less keen to support because of their EU wide nature, or that companies were less keen
to support due to their higher risk.
SET should learn from
the FP.
There is a clear and logical hope that the SET plan can learn from the experience of the FP in
terms of programme direction, procedures and operation.
7.2
The DG ENER FP7
programme is already
linked with the SET
plan.
Strategic Energy Technologies (SET) Plan
As has been made clear elsewhere in this evaluation, FP7 has evolved over its life with the
decision to make the calls more focussed and influenced by the priorities made in the
Strategic Energy Technology (SET) Plan. The annexe to this report contains a detailed
description of the SET plan and its recently released Implementation Plans.
Its is clear that for energy related research the remainder of FP7, and the potential FP8 will be
strongly influenced by the ongoing content, aspirations and success of the SET plan.
Therefore, although it is not the purpose of this report to evaluate the SET plan its current and
future importance has led us to seek views on it during our work.
Final Report - FP7 Energy Mid-Term Evaluation
54
7.2.1
Consultation Views on the SET plan
The concept of the
SET plan is very
welcome.
The SET plan was described as a step change in the approach to low carbon research and
development compared to the last 3 years (and before). The key aspect was felt to be the
industrial initiatives, which should enable activity on the scale required to make a real
impact, by pulling in and directing industry money and industrial ability to drive market
introduction. However it is not yet clear exactly how the industrial initiatives will be achieved
and some industries have expressed reservations due to the amount of money they are
expected to commit and the feeling that they have been presented with a fait accompli
regarding the choice of technologies.
The SET plan is
evolving quickly.
In terms of the scope of the technologies in the SET plan it is arguably right to keep the scope
as wide as possible and to keep the plan as a living document. This would enable the plan to
develop as it becomes clearer which technologies are the most appropriate, this is very
difficult to predict currently, given the market complexities involved.
SET plan links to MS,
EU and private funds
are not yet clear. .
It is recognised that the SET plan is duplicating MS activity to an extent, though the point
was raised by one consultee that the SET plan process was started in advance of large scale
activity and commitment by most MSs, and other international competitors such as the USA.
It is now the case that the US, Korea, Japan and China now all have equivalents to the SET
plan and their nation status means they are able to instigate their plans quicker than the EU.
The SET plan could ideally help MSs avoid duplication by being very clear on the areas it
intends to fund and the roadmaps to 2013 do show the areas that are the expected focus. MS
level implementation of clean energy research is however a complex and fast moving field
and it difficult to try and complement this activity. There is potential for it be used to help
coordinate and supplement MS level activity, for example the SET plan has called for 12
CCS schemes around the EU and this could take the form of industry / public cooperation in
a number of MSs. The boundary between EU and MS funded research programmes is an
interesting area and not one that has been much covered, one consultee, with a management
role in the UK’s Carbon Trust (who fund research into low carbon) felt that exploring and
explaining this boundary would allow better EU-MS links.
The SET plan
approach is being
used in other parts of
the world.
SET plan KPIs are
useful but vary in
detail.
The FP future in SET
is a challenge to
define.
This change is not
welcomed by all.
The key performance indicators (KPIs) in the SET plan were described as having been hard
to agree. They are intended to be steps on the way to ultimate goals (of reduced Carbon – via
more res and improved energy efficiency). Another consultee pointed out that there is a
marked variation between technologies with clearly specified targets for some and much
vaguer aspirations in others. This poses a risk that technology areas with vague aims will lead
to vague projects.
Future for FP8 and the SET plan
The future focus and nature of energy research under the FP is, in the view of those
consulted, clearly closely linked with the SET plan. FP7 (and in time 8) will face a challenge
regarding where it fits into the new landscape, if only because the level of funding it has is
small in comparison to the total of the SET plan (in the order of 5%). The research focused
objectives of the FP also imply that it should not all be used in the SET plan, as this is more
focused on large scale adoption of renewable energy and energy efficiency.
It was reported by one consultee that some current FP applicants / beneficiaries are not happy
about SET as it represents a complete change from what they know, and the procedures and
approaches of the FP from which they have learnt to take maximum benefit. In the opinion of
55
Final Report - FP7 Energy Mid-Term Evaluation
this consultee in order for the SET plan to work it needs the best research centres and
companies to get involved, and this poses a risk to those more middle ranking universities
and companies who currently tend to benefit from the FP.
The very large projects
will be harder for
SMEs and NMs to
enter.
The SET plan will also be a difficult area for SMEs and new member state applicants to
participate in. If there is still a desire to support these groups in low carbon energy research
there is an argument that they may need a different support mechanism. In the view of one
consultee it is not credible to have a programme which aims to support world class, cutting
edge research but that also supports capacity building in SMEs and NMSs. Attempting to
combine this in a particular project imposes costs and risks, reducing effectiveness.
The SET plan is one of
many new and
evolving mechanisms
to support low carbon
R+D in the EU.
The SET plan itself will also need to link with the activities that will be planned under the
auspices of the European Institute of Innovation and Technology (EIT) and its energy
focused Knowledge and Innovation Community (KIC). There are a number of areas where
there is a clear potential for linkages, such as smart cities and the need to define the FP8
priorities / nature etc. This is an issue that will need to be kept under review as the decisions
on SET, KIC and FP8 are all made in the relatively near future.
The profile of the SET
plan is not high,
though this may not
matter.
Profile of the SET plan
There was a general consensus among those consulted that the profile of the SET plan is not
very high. From a MS perspective this lack of profile and a perception of vague aspirations
and lack of clarity on why these issues should be best considered at an EU level has led to a
lack of allegiance to it. Another consultee felt that those who should know about it, research
staff in major companies, were aware of the SET plan but agreed that in terms of political
profile the SET plan is less known. The consultee felt that this low political profile was in
part related to the difficulty the Commission have in communicating their aims, and there
habitual use of mild language – to avoid confrontation. In addition there is the political
problem of the Commission lacking an obvious spokesman for a programme such as the FP,
which contrasts with the US where the president can make bold claims for their R+D
programme.
The economic
downturn may help
attract public
authorities to
participate.
Increasing the profile of the SET plan was thought to need a wider co-operation between
public and private partners. The opinion was raised that the current lack of public funding
(due to the economic downturn) should help attract public authorities and their projects
particularly those looking to replicate technologies. One consultee, felt that there was actually
no need for the SET plan to have a high public or political profile as long as it was known of
an understood by the target audience – energy companies and research groups.
Some perceive SET
as only supporting
demonstration and not
research.
The project coordinators interviewed were asked their opinions on the future links between
the FP and the SET plan. This revealed some interesting and varied perceptions. A
coordinator from a very large research institute raised concerns that the rumours were that the
EC plans to focus their research support solely on large multinational companies and move
away from supporting SMEs. This approach was not popular as it was felt that FP would lose
the ability to support growing but resource constrained SMEs. They believed that continuing
to work with SMEs should be central to the SET plan and the FP. Another coordinator was
also under the impression that the SET plan objectives were very focussed on demonstrations
and as such he felt that funding and focus should also remain for fundamental R&D. There
was also concern that too high of a reliance on private funding would put too much emphasis
on cost effectiveness which could be detrimental.
Final Report - FP7 Energy Mid-Term Evaluation
56
Technology platforms
should continue to
influence priorities.
One coordinator felt that the EC technology platforms were a good mechanism for
influencing the future direction of FP and SET and that the current process offers plenty of
chances to influence/contribute to the policy debate.
The FP should retain
its focus within SET.
One coordinator felt that the FP funds should be used as part of the SET plan and should
focus support on the best technological solutions independently from their proposers (SMEs
or Large Enterprise). The involvement of MS funding was considered necessary in order to
create a “fund association” with the European Commission and others creating Public Private
Partnerships in order to support large scale demonstration projects as well as smaller projects.
They referenced the Marguerite6 programme as being a good example of such an approach.
7.2.2
The success of the
SET plan is nit
guaranteed.
Projects with
significant EU added
value offer a role in
SET for the FP.
The SET plan contains
research as well as
demonstration goals.
SET can and should
learn from the FP.
The FP should retain
its identity within SET.
Officer Views of the SET plan
All of the officers consulted agreed that the SET plan, if at all successful, would be the major
influence on the future of energy research under the FP. However they also pointed out that
the success of the SET plan is not yet guaranteed and the response from industry and member
states will only become apparent over the next few years. For example the fact that not all
MSs are (yet) actively participating in SET complicates the issue. Synergy and avoidance of
duplication, with national programmes would be good as the case could be made that some
FP projects could have been funded by MS funds. Though EU added value projects will
remain an area that MSs are unlikely to fund, which should help avoid duplication and leave
a ‘space’ for the FP to fill.
A number of those consulted pointed out that the SET plan has two main aspects, the
Industrial Initiatives and EU Research Alliance. It was thought that the FP has a role in both.
With regard to the industrial initiatives, given the very large scale of funds envisaged it is
clear that FP funds can only be a relatively small contributor and the risk exists of diverting
the FP from its original aims. However it was suggested that the FP has a potentially
important role in helping to influence the direction of the industrial initiatives. One officer
described this role as being the ‘brain’ of the programme, for example helping to identify
relevant projects and helping to define indicators of success. Another officer echoed this
point suggesting that there should be lessons to be learnt from the history of FP to help the
SET plan.
It was suggested that in order to retain its focus on innovation the FP should support projects
(possibly within SET) which are higher risk, related to proof of concept and higher EU value
added (i.e. with potential for impact and replication across the whole EU). Such an approach
was suggested as being one which would help clarify where FP sits in relation to SET. It was
also pointed out that given the intention of the Industrial Initiatives in the SET plan to
continue to target future funds on large demonstration projects, it will become even more
difficult to try and give an equal spread of funds across the member states, as the large
companies who will tend to dominate large demonstration projects are more common in
certain MS, typically in the EU 12. Though some opportunities may arise from this approach
to SMEs and NMSs as useful consortium members,
6
The Marguerite fund is a pan-European infrastructure fund for long term institutional investors to finance the implementation of
strategic European policy objectives and projects in the Transport, Energy/Climate and Renewables sectors. The main objective
of the fund is to contribute to the European Economic Recovery Plan by financing the implementation of strategically important
European policy objectives in Energy/Climate, Renewable and Transport sector infrastructures.
57
Final Report - FP7 Energy Mid-Term Evaluation
There was also support for the EU Research Alliance aspect of the SET plan as there was a
strong belief that there remains a need to support more fundamental research, with the
research capacity it enables, and the longer term technologies it can invent and research.
These aims were described as being closer to the traditional aims of the FP.
A number of those
consulted felt the FP
should retain some
funds outside of SET.
7.2.3
A number of those consulted also raised the suggestion that some FP funds should be held
back from SET related projects and used to support other types of intervention. However,
these other interventions should be of a similar nature to what the FP has traditionally
supported rather than any refocusing away from this, e.g. it should not focus more on SMEs
and should continue to support projects led by major companies.
Framework Programme Development
A number of issues related to the development of the FP as a whole were also mentioned by
those consulted. The ongoing FP wide efforts to simplify procedures were mentioned and
generally supported, with the caveat that there is an inevitable balance to strike between
reducing burdens on applicant s and beneficiaries but also ensuring that public money is
wisely and honestly spent.
It was also mentioned that there are a number of other EU innovation support programmes of
relevance, which the FP needs to complement: These include:
There are multiple EU
led initiatives of
relevance to FP
ENER.
•
•
DG ENER needs to
keep its involvement in
the development of
these policies
•
•
7
New research and innovation strategy (under DG RTD was DG Enterprise) which is
developing sectoral roadmaps, e.g. energy efficiency in buildings. There are
possibilities here to combine supply and demand side funds, which for DG ENER
implies FP and IEE plus others.
The European Institute of Innovation and Technology (EIT) and its three Knowledge
and Innovation Communities (KICs), one on Sustainable Energy and one on Climate
Change Adaptation and Mitigation, were also mentioned as a recent development of
clear relevance. The KICs are virtual networks of research excellence who will be
seeking funding from a combination of funding (national, private etc.). It can be
expected that these groups will be bidding for, and seeking to align with the activities
of the SET plan and the future FP.
The Joint Research Centres – clearly remain relevant and are strongly involved in the
SET plan, for example they are running SETIS - the SET plan information system.
The purpose of which is to “efficiently collect, harmonise, validate, analyse and
disseminate information on the priority energy technologies identified by the SET
Plan, across the EU. The goal is to provide undisputed and transparent data and
methodologies to support the strategic planning, monitoring and evaluation of the
European Energy Technology policy”7
There continue to be other areas of the FP which share some objectives with the part
supported by DG ENER, For example the DG ENV areas covering eco innovation,
resource efficiency and environment, although formally excluding energy there will
inevitably be some common areas of interest..
See: http://setis.ec.europa.eu/setis-main
Final Report - FP7 Energy Mid-Term Evaluation
58
Participant
suggestions for future
focus areas include:.
The current participants were asked their opinions on where DG ENER should focus its
future FP funding. Their suggestions fell into the following categories
Specific technology /
follow ons.
Specific Technologies – typically follow on ideas from existing projects
Ø Solar generation from industrial processes;
Ø Explore the use / upgrade of naphtha by products from bio refining.
Ø Collecting, handling and pre-treatment of residues for lignocellulosic ethanol
Ø CO2 capture using algae
Ø Biofuels from high cellulose materials;
Wider areas – PV, fuel
cells, wave, multiple
solar ,including
manufacturing
expertise.
Wider Technology Areas – again typically related to their activities, though the need to
support and develop manufacturing abilities was mentioned by a number of participants.
Ø All aspects of PV development. Particularly manufacturing efficiency, excellence,
productivity, low cost, competitiveness etc;
Ø Fuel cells, in particular advanced industrial manufacturing capacity;
Ø New production technologies and control systems – to reduce costs and improve
efficiency;
Ø To move towards a standard approach / design for wave / ocean energy. In the same
way as happened for wind;
Ø Nuclear - new and more secure nuclear technologies, address the problem of
radioactive waste, etc;
Ø A solar energy focussed call combining heating, chilling and power to reflect the
scale of potential demand for this technology in hot countries.
Ø Grid asset management, particularly in Eastern Europe.
Cross cutting areas
including networks and
biofuel – h – fuel cell –
electric car links.
Cross cutting - enabling technologies or those linking energy technologies
Ø Smart networks - ICT-related developments, grid connections;
Ø Synergy between biofuels / hydrogen and electric cars. E.g. Fuel cell cars using
hydrogen from gasification and shift conversion processes.
Non technical issues – not typically issues which the FP can directly influence.
Ø Stability of the legislative framework including long-term targets – e.g. the strict
retention of the 10% biofuel requirement for 2020 in the Renewable Energy
Directive by all Member States..
Nature of the Programme
Ø In terms of future areas / technologies, the Commission should give further
consideration for potential FP calls to mix member states with developing countries
(outside EU). There is an opportunity to build markets / products with these
countries where the future demand for energy is likely to be higher.
• There was some direct contradiction between project coordinators in terms of
whether the focus should be close to market or more fundamental research
7.2.4
Longevity of Impacts and Improving Dissemination
The project coordinators had the following comments on the probable persistence of the
project effects. One coordinator felt that dissemination is generally very good but
commercialisation can be more 'hit and miss,' depending on participation and how the outputs
link to the business of the industrial partners. Another coordinator was more critical stating
that he thought “ the programme will not really produce long-lasting effects for a number of
59
Final Report - FP7 Energy Mid-Term Evaluation
reasons including the fact that SME participation is not as strong as it could be and also
because there is a tendency to avoid high risk projects (which are likely to have a greater
potential impact)". A third coordinator thought that dissemination was good and that it was
easy to find information on projects if you look for it.
Inter FP project
networking works for
some but does not
appear a strong
prospect.
The project coordinators were asked for their opinions and suggestions on how project
dissemination and long term impacts could be improved. One suggestion that was tested out
with a number of the coordinators was the idea of increased inter project networking. There
were a variety of contrasting views on the benefits that such an approach could bring. One
coordinator did not see any benefits from inter-project networking, citing his own experience
of trying to network four laser projects which failed due to a combination of competition
between the projects and a lack of synergy. He felt that existing networking via academic
channels and conferences was sufficient which was an opinion echoed by other consultees.
In contrast another participant thought there should be more inter-project networking and had
never been involved in any previous attempts. Another coordinator also thought that the idea
of joint conferences was a good one and would offer the benefit of increased networking
between projects. A third coordinator suggested a two-day workshop where all relevant FP7
energy projects were presented. A coordinator who had been involved in a joint conference
involving CCS researchers felt that it had been useful and was a good idea for certain
research areas.
Existing procedures
and routes suffice for
academic
dissemination.
With regard to conferences another coordinator stated that discussion was limited to
published results due to confidentiality and IPR protection, though this was the accepted
norm and he thought that any requirement on FP projects to network with each other would
be a somewhat artificial approach. The retention of some IPR by project partners was seen as
a very valuable benefit from participation and one that should not be lost in efforts to more
widely publicise results. However another project participant commented on a reluctance of
industrial partners in his project to share knowledge and learning with other partners. He felt
that there was a need for stronger guidelines at application stage on knowledge sharing. The
issue of disclosure was described as complex, as if FP7 requirements are more towards full
disclosure, it would put partners off participating from fear of losing their IPR. However, he
thought that the balance doesn't seem to be quite right at present.
Some
misunderstanding of
the availability and
purpose of additional
funds to complete
projects.
One participant raised the idea of additional follow on funding to assist in dissemination and
commercialisation. The amount he thought would be required need not be large (may be
only a few thousand Euros) but it could make the difference between a project stopping or
actually achieving market entry and as such it would be an investment worth making.
Another coordinator was less sure on the idea of follow up funding , stating that: “There may
be good reasons why a project shouldn't continue or why it should be considered as a 'testbed' and should be taken forwards outside of EC FP activity”. They also pointed out that
there is already some flexibility to secure funding extensions, though they believed that this
was not consistently applied to all projects. One current participant felt that a separate
contract for post project activity would be a complex arrangement
Good support for a
combined promotion of
projects at high level
trade shows /
conferences.
One coordinator suggested that the EU should develop a dissemination strategy when
products are mature and use highly recognised international events like Inter Solar or Solar
Energy to present their FP7 projects. He felt that this would improve the credibility of the
projects and help commercialise them. Such a collective approach was thought to be more
efficient, in terms of impact per € spent than lots of individual project actions. This
Final Report - FP7 Energy Mid-Term Evaluation
60
suggestion appears strong and was described as one which the CT in the UK have adopted
with some promising results.
One coordinator stated that his project has agreed to keep the project website alive for 1-2
years following project end to assist dissemination, he felt that this was good and it may be a
good idea to oblige other projects to do this.
One project coordinator mentioned security concerns regarding publicising the results of his
project as it concerned the electricity infrastructure which is seen as an area of concern.
61
Final Report - FP7 Energy Mid-Term Evaluation
8 Conclusions and Recommendations
The conclusions for each evaluation question are presented at the start of each chapter. These
recommendations stem from those conclusions and have been presented under sub headings
in line with the programme life cycle. Each recommendation is followed by a brief discussion
and suggestions as to how it could be practically realised.
8.1
Programme Relevance / Strategic Fit
1. DG ENER should maintain an input to the focusing of DG RTD calls and vice –versa
in order to maintain coherence between the research focus of RTD and the
demonstration focus of DG ENER.
This reflects the need to maintain a link between the energy demonstration projects and the
more fundamental research projects, which could one day become demonstration projects,
and / or which needs to reflect any barriers highlighted by demonstration projects. Ideally this
link / input needs to occur at several levels, ranging from the technology platforms, to head if
unit to informal gatherings between project officers.
2. The SET Industrial Initiative implementation plans and their KPIs, plus other
relevant SET plan documents that are developed over time should be clearly
referenced in the future FP calls, with alignment expected to increase as the plans
become detailed and the level of success of SET becomes more apparent.
This reflects the support for the alignment of the FP and SET and the increasing detail
available on the SET plan targets, such as the Industrial Initiative plans. The call for
proposals needs to have clear reference to the technology specific SET plans and make it
clear that applications should attempt to show how their projects will contribute towards the
KPIs and other SET plan targets.
3. Energy efficiency needs to be clearly articulated as an objective of the ENER FP
programme. The whole of the FP should be analysed to ensure that that other parts of
the FP are indeed picking up relevant projects.
Energy efficiency is a vital high level objective of energy efficiency so any projects or ideas
that can genuinely help should have some chance of receiving FP funding. There was a
widespread opinion that energy efficiency is being covered elsewhere in the FP but this
should be verified.
8.2
Information to Applicants
Final Report - FP7 Energy Mid-Term Evaluation
62
4. More effort should be made to clearly articulate the programme objectives to
applicants in a clear and simple way in the application pack.
This is to help ensure that the applicants have as good an understanding as possible of the
mix of objectives that the programme is intended to achieve. This could be achieved by
including the objective tree with a few simple examples of where typical projects match the
objectives. For example: A project to demonstrate a new design of wave energy device –
would help demonstrate the feasibility of a commercial scale application, which would help
lead to an increase in low carbon power generation capacity, the support and development of
EU manufacture, installation and maintenance expertise and the improvement of practical
and scientific knowledge on performance. A project to develop real time information on grid
demand supply from household level and match it to generating capacity (i.e. a smart grid
project) would develop approaches and technologies which could enable increased use of
renewable (i.e. low carbon) generation, thus enabling CO2 emission reductions and
supporting those sectors that produce such generating capacity.
5. There appears to be something of a misunderstanding among current FP participants
that the SET plan will only support large scale demonstration projects led by large
companies and will do nothing to support more fundamental research and SMEs.
This misconception should be addressed.
This could be achieved via clear explanations in the programme documentation but is also an
issue which DG ENER should feed into SET plan development and publicity.
8.3
Programme Management
6. There were a number of detailed suggestions made that could help improve the
efficiency of programme operation, some of which were well supported by other
consultees and appear to be worthy of uptake, if only on a trial basis. These are:
a. A switch to remote as opposed to on site evaluations – as most officers
consulted felt the ability they offer to attract better experts (due to increased
flexibility) outweighed the concerns over document security.
b. Formalised pre submission meeting procedure – to ensure a level playing
field between applicants. Clarify the procedures at information days.
c. Formalised applicant interview, particularly for large projects – to enable a
better understanding of the project objectives and avoid easily explainable
misunderstandings. For projects over a certain budget with the choice of
which partner staff left to the coordinator.
d. A more extensive end of evaluation meeting – involving external experts and
extended discussions to arrive at the agreed approved project list. If the
resources exist to support such a meeting.
e. A set of project management KPIs – to enable easier and more formalised
ongoing assessment of project progress. A mix of procedural and output
quality indicators.
f. Increased use of external advisors to assess interim and final reports – to
bring additional technical expertise and scrutiny to the reviews of project
progress. Clarify the availability and increase if required (and affordable).
63
Final Report - FP7 Energy Mid-Term Evaluation
8.4
Project and Programme Impact
7. In order to enable the vital assessment of project impact and success that virtually all
consultees agree is needed, the authors of this report suggest that some form of
relatively low cost payment could be offered to project coordinators as an incentive
to provide short update reports at fixed periods after project completion.
This requirement should be part of the original contract and be structures such that some
relatively small fixed amount (say €1000 - €2500) is paid to the coordinator each year (or
every other year) for some 5 to 6 years after project completion to supply answers to a short
pro-forma on post project developments. The questions should include: current status –
ranging from all work stopped, through continuing research – directly or on a particular
aspect, to small scale production and mass production. Questions on related patents should
also be asked as should questions on employment and income, as well as scientific and
commercial collaborations with contacts made directly (and indirectly) through the project.
8. The majority of those consulted felt that given the importance of CO2 savings, and
the clear link which should exist between demonstrating low carbon technologies and
large scale replication, project applicants should be very clearly requested to estimate
the scale and nature of the CO2 savings that their projects could enable. There should
be simple standardised methods on how applicants should do this and it should
receive no more emphasis in the evaluation procedure than it already does.
It should be made clear to the applicants that a high figure will not necessarily increase the
likelihood of the project being scored highly. It is more important that the submission
illustrates their understanding of how the work leads to carbon savings and the technologies
and competitors it will face if the supported project is successful.
9. Trial the collective presentation of completed and ongoing FP projects at a well
regarded trade show / conference.
The current portfolio of project should be reviewed for a group of successful recently
completed / very near to completion projects in a related field and the coordinators contacted
to ascertain their interest in such an approach.
10. The procedures and circumstances, under which additional funds can be provided, at
project end, should be clarified. Such funds may be useful to aid dissemination and
commercialisation.
The circumstances under which project extensions are allowed should be highlighted to the
approved projects and those projects going through the negotiation phase. It should be made
clear that there is no guarantee of such funding and that it is awarded only under exceptional
circumstances.
Final Report - FP7 Energy Mid-Term Evaluation
64
8.5
Programme Future Direction
11. There was wide consensus that the technology roadmaps under development via the
SET plan Industrial Initiative should be more explicitly used by the FP7.There is a
debate to be had on how and if the FP should retain its identity within SET.
This recommendation overlaps with nos. 2 and 3 with the additional implication here being
that DG ENER need to ensure that their voice and opinions are continued to he heard in the
ongoing development and operation of the SET plan. The ENER FP could direct all its
funding into SET, or target those projects with match FP ideals, e.g. EU added value, higher
risk, more SMEs involved, or it could give some funds to SET and retain some separate
funds. These are complex issues and the optimum choice depends on the policy objectives
that are required and how they are prioritised.
12. There is a clear hope that the SET plan can learn from the experience of the FP in
terms of programme direction, procedures and operation.
This recommendation overlaps with recommendation number 10 and illustrates the extent to
which the future of FP funding by DG ENER is linked to the SET plan. Therefore it is clearly
in the interest of both if the lessons learnt by DG ENER are utilised in order to help deliver
the SET plan.
13. There are a number of emerging and evolving policy mechanisms for low carbon
research and development. If the FP wants to retain its identity it needs to be actively
involved in the policy debate around these mechanisms.
A number of DGs and programmes are becoming involved in this area. In order to avoid
duplication and maximise synergies efforts need to be made to ensure that the information
flows between the DGs are as free and deep as possible.
65
Final Report - FP7 Energy Mid-Term Evaluation
Annexes
Final Report - FP7 Energy Mid-Term Evaluation
66
1 Context
1.1.1
High Level Policy Drivers
In order to define the objectives of the group of projects which are being evaluated it is
necessary to review all levels of relevant EU policy.
The DG Energy supported energy research component of the Seventh Framework
Programme (FP7) needs to be seen within the overarching content of the EU’s objectives as a
whole:
“The Community shall have as its task to promote throughout the Community a harmonious,
balanced and sustainable development of economic activities, a high level of employment and
of social protection, equality between men and women, sustainable and non-inflationary
growth, a high degree of competitiveness and convergence of economic performance, a high
level of protection and improvement of the quality of the environment, the raising of the
standard of living and quality of life, and economic and social cohesion and solidarity among
Member States.8
EU supported research goals are therefore aimed at serving “higher” economic, societal and
environmental goals as well as the theme specific objectives, including meeting the challenge
to turn Europe into a knowledge-based society: a society where the knowledge triangle of
research, education and training, and innovation is fully mobilised. Some of the most
relevant related EU policies and programmes include the:
The EU Sustainable Development Strategy: adopted in 2006 is the overarching strategy for
all EU policies and sets out "how we can meet the needs of present generations without
compromising the ability of future generations to meet their needs." The strategy covers 7 key
areas including climate change and clean energy. Among the principles highlighted are the
need to involve business and citizens and to work towards an innovative knowledge based
economy. This is at the root of all ERA work in the energy sphere.
EU Energy and Climate Change package: which sets out the 20-20-20 by 2020 agenda for
a 20% reduction in CO2 emissions, renewable energy to make up 20% of all energy consumed
and 20% gains in energy efficiency by 2020. Innovation, research and education and training
will all play key roles in the development, installation and operation of the technologies
necessary to meet these targets.
The Environmental Technologies Action Plan (ETAP): is very closely related to the
67
Final Report - FP7 Energy Mid-Term Evaluation
successful creation of an ERA. Its objectives are to improve eco innovation and the
competitiveness of EU environmental technology industries. By creating standards,
technology centres, networks and supporting markets it expands the market for the research
being sponsored under ERA. It also works at the next steps beyond ERA in fully
commercialising technologies. Ecorys have carried out a review of the implementation of
ETAP.
Internal EU gas and electricity market: the EU's stated policy goal is "to set up a truly
competitive internal market for gas and electricity to offer consumers a real freedom of
choice at fair, competitive prices, to stimulate clean energy production and to improve
security of supply." Deficiencies in progress towards this goal have been identified and led to
the recent adoption of a third package of measures with the primary aims to continue to open
markets to competition. While ERA outputs may be focused on technological issues for
networks there is also scope for increased cross-border research as the market increasingly
becomes trans-national.
European Energy Programme for Recovery: this initiative by the commission is in
response to the economic downturn and has the goals of stimulating economic activity while
contributing to strategic energy policy objectives. €4 billion of funding until 2010 has been
allocated in 3 areas, gas and electricity infrastructure, offshore wind and carbon capture and
storage (CCS). These areas, particularly CCS, will all draw upon new research and innovation
stimulated within the ERA.
Trans-European Energy Network Programme (TEN-E): is an initiative targeted
specifically on energy infrastructure that aims to improve the internal EU energy market by
encouraging energy diversification and interoperability to increase energy security, promoting
and strengthening economic and social cohesion and supporting sustainable development. By
focusing funding on feasibility studies there is a close relationship to the utilization and
demonstration of energy technologies such as those developed through the ERA network. An
evaluation of the TEN-E programme has been carried out by Ecorys.
The Seventh Framework Programme (FP7) is a key pillar in delivering the wide policy goals
described. Its broad global objectives are grouped thus9:
• Cooperation – support for transnational activity at an appropriate scale across the EU and
beyond, in thematic areas corresponding to major fields of knowledge and technology
where research should be supported and strengthened to address European social,
economic, environmental, public health and industrial challenges, serve public good and
support developing countries.
• Ideas – activities to be implemented by the European Research Council (ERC) in
developing “high-level frontier research” at EU level, building on excellence in Europe
and raising its profile at international level.
8
9
Treaty Establishing the European Community (Consolidated Text), Official Journal C 325 of 24.
European Parliament and Council Decision No. 1982/2006/EC of 18 December 2006 concerning the Seventh Framework
Programme of the European Community for research, technological development and demonstration activities (2007-2013)
Final Report - FP7 Energy Mid-Term Evaluation
68
• People – individuals to be stimulated to become professional researchers, be encouraged
to remain in Europe and the best researchers from outside Europe attracted here. Mobility
is a key dimension; for professional and personal development for also for knowledge
sharing and transfer of knowledge between countries.
• Capacities – the use and development of research infrastructures should be optimised,
innovative capacities of SMEs strengthened and regional research-driven clusters
supported. Science and society should be brought together in European society and
support given to coherent research policies at national and Community level and
horizontal actions and measures undertaken in support of international cooperation.
It is important that our approach reflects the overarching objectives of the FP7 as a whole,
(since judgements on the success of the energy component must also concern its contribution
to the programme as a whole), without loosing its focus on energy. The general FP7
objectives are diverse and reflect a very wide range of policies and strategies. The ex-ante
assessment of FP7 provides a useful starting point in understanding them:
It is also important to bear in mind the relevance and potential cross over with a multitude of
other relevant EU policy activity. A recent European commission communication10
'Reviewing Community innovation policy in a changing world' referenced the FP as well as
the SET-Plan, the Action plan on Sustainable Consumption and Production and Sustainable
Industrial Policy and the Eco-design Directive as key policies in triggering market uptake of
innovative products. The same communication also mentions standardisation policy, public
procurement and the Lead Market initiative as being important in this area. This will lead to
the adoption of an EU Innovation Action Plan in 2010. In terms of financial support for
research innovation the FP was highlighted along with Cohesion policy, Rural development
policy and the CIP. There are also clear links between innovation and policy activity in
education and skills where the key aims are also enhancing competitiveness and promoting
and supporting the knowledge economy.
1.1.2
Energy research under FP7
Energy in FP7 covers a wide range of subject areas and disciplines, reflecting the
multidisciplinary and cross-sectoral research required to support policy-making in this
important field. The DG Energy specific energy component of FP7 is described in the table
below:
Table 1.1 Energy research in the Seventh Framework Programme (2007-2013)
Seventh Framework Programme (2007-2013) (EUR 53.2 BIO)
EC Programme (EUR 50.5 BIO)
Cooperation (EUR 32,365 MIO)
Energy (EUR 2,350 MIO) DG TREN and Research
10
Communication from the Commission to the European Parliament, the council, the European Economic and Social committee of
the Regions. 2.9.2009. Com (2009) 442 final.
69
Final Report - FP7 Energy Mid-Term Evaluation
•
•
•
•
•
•
•
•
•
Hydrogen and fuel cells
Renewable electricity generation
Renewable fuel production
Renewables for heating and cooling
CO2 capture and storage technologies for zero emission power generation
Clean coal technologies
Smart energy networks
Energy efficiency and savings
Knowledge for energy policy making
Energy - DG Energy (EUR 501.4 MIO in the first 4 calls)
FP7-ENERGY-2007-2-TREN
128 mio
•
45 topics
FP7-ENERGY-2008-TREN-1
147 mio
•
18 topics
•
Post Strategic Energy Technology (SET) plan. (7
key energy technologies to 2020, 7 other
technologies to 2050).
•
More industry commitment
•
Focus on core topics
•
Intensive consultation on needs and bottlenecks
•
More funding per project
FP7-ENERGY-2009-2
100 mio
•
10 topics
FP7-ENERGY-2010-2 (open)
126.4 mio
•
8 topics
•
Call opened July 09.
•
Deadline for applications March 2010
The FP7 research programmes have been constructed thoroughly and carefully, and it is
therefore important for the evaluation that our team has a genuine knowledge of the rationale
for inclusion of the various research domains; and is familiar with current research and
development, together with the relevant critical issues and debates.
The themes supported by DG Energy have evolved over the life of FP7. This development
during FP7 reflects past practice, for example during FP5 the decision was taken to stop
supporting projects designed to improve the extraction of oil and gas, on the basis that the oil
price had risen to the extent where subsidising oil companies could not be justified. The calls
are described in more detail in section 3.
1.1.3
Division between DG Energy and DG RTD
The Directorate General (DG) Energy – formerly Transport and Energy (TREN) - and
Research both fund a variety of energy research programs. They both support FP projects
however they focus on different aspect. Below we describe some of the distinguishing
aspects of both DGs in terms of the research they support.
DG Energy
DG Energy provides and manages the funding for a variety of energy research initiatives for
trans-European networks and technological development (like the pilot cases for CCS) and
innovation. Fundamentally the focus of DG Energy is on commercialisation and
Final Report - FP7 Energy Mid-Term Evaluation
70
demonstration projects. This means that from a typical technology life-cycle perspective the
technology development is past the basic research and pilot phase. As the technology is close
to commercialisation private organizations are more willing to take the lead in this stage of
the development process. Industries commit themselves to the technology as they see the
potential for it to result in profits for them. Moreover, the risks of investment shrink as a
result of (successful) pilot projects. However, as there are still a large number of risks
(technical, economic, etc.) the public funding through DG Energy is still very important.
Furthermore, the technologies funded under DG Energy are expected to penetrate the market
in the relatively short term. The scale of the technology at this point in the life cycle is
potentially being adopted at such a pace that it would qualify as widespread technology
diffusion. This implies that these industrial led projects can potentially have a significant
impact on the 2020 targets relating to emissions and efficiency.
DG Research
DG Research provides and manages the funding for a variety of energy research and
technological development initiatives. Fundamentally DG Research focus on the first stages
of the life cycle of technology development. This means that fundamental research would be
eligible for support. As such research is relatively far from commercialisation (compared to
DG Energy projects) it takes a longer term perspective in what it supports. Furthermore, it
implies that these research projects focus on future technologies.
1.2
Lessons from previous evaluations
When approaching evaluation assignments, some of the key reference documents are the
most recent previous reports in the evaluation cycle of the programme in question and of
closely related programmes. There are a number of reports of such relevance to this
assignment, as follows:
Ex post of FP5 and FP6 DG RTD Energy projects has recently been completed - summary
The Ex-Ante Evaluation of the 7th Framework Programme for RTD in the field of Energy
and Transport (2006-2010) was carried out by ECORYS in 2005.11 It looked at the whole
programme of DG TREN FP7 activities, including Nuclear and Transport. In setting out the
rationale, objectives and delivery mechanisms for the programme it made a number of key
recommendations and developed the underlying intervention logic. The ex-ante evaluation
aimed to support the discussion on DG TREN’s role in FP7 and the contribution of the
research programme to the objectives of European transport and energy policies. The main
question related to the value added for DG TREN of being involved in FP7. Five intervention
strategies were defined according to the level of involvement: full package, focussed
approach, programming only, independent and no RTD. The pros and cons of these strategies
were assessed with a multi-criteria analysis. For the first four intervention strategies
recommendations were formulated to assure an appropriate effect. For the latter intervention
strategy the recommendation was to not adopt it. The recommendation was that with a little
11
ECORYS (2005) Ex ante Evaluation of the 7th Framework Programme for Research and Technological Development in the field of
Energy and Transport (2006-2010), Rotterdam. Delivered under the previous evaluation framework contract
71
Final Report - FP7 Energy Mid-Term Evaluation
additional effort the strategy could be ‘programming only’ which clearly generated additional
benefits.
Parallel to this study a number of research priorities for FP7 were established containing new
orientations that envisaged an improved response to the developments in the energy and
transport sector. This also applied to the domains that were to be continued in FP7, such as
CIVITAS and CONCERTO, road safety, intermodal transport and intelligent transport
systems. The research themes (priorities) that were defined were:
•
•
•
•
•
•
•
•
Energy efficiency and renewable energy sources
Clean coal technologies
Nuclear waste management
Air traffic management
The clean and safe vehicle
European systems for satellite navigation
Efficient and interoperable rail transport
Measures to profit for the findings of research
Other main conclusions of the study involved the main critical success factors in translating
successful projects into useful results and impacts, these were:
• Policy take up of RTD results
• User involvement and attention to commercialisation and regulatory aspects
• Dissemination of results.
• Alignment of EU policies versus national interests and interest of participants.
• Continuity of knowledgeable EU staff
The Ex-Post evaluation of Non Nuclear Energy (NNE) Projects supported by DG TREN
under FP5 was carried out in 2007 by Ecotec (Ecorys in the UK). The sector and technology
scope of the FP5 ex-post evaluation is the same as for this mid-term evaluation, reviewing
non-nuclear energy RTD projects funded under a framework programme.
The key conclusions from this report were:
• Effectiveness: that there was a good match to objectives with only minor issues
around improving awareness in new member states and data collection.
• Efficiency: in balancing programme administration it was difficult to satisfy both
applicant and audit requirements. Issues were raised around the with-holding of
funds.
• Utility: A good match was found between project effects and needs and specific
value added was found in encouraging trans-national research co-operation and
enhancing the global standing of European energy research.
• Sustainability: that projects that took a holistic approach to research and development
were in general more successful than those focused on single issues, that the energy
price and relative market subsidies were major factors in success and that firms
directly involved and with a commercial interest in the market were also more
successful.
From this the following key recommendations were made:
•
•
To definitely continue and potentially expand the programme.
To continue to work closely with industry led technology platforms and let applicants
know of the flexibility in the process.
Final Report - FP7 Energy Mid-Term Evaluation
72
•
•
•
•
•
•
•
To improve data management and collection by DG TREN.
That marketing did not need to expand but could be targeted on newer member
states.
The benchmarking indicators around administration, particularly payment timings
should be developed.
To investigate with DG Research the use of external project technical assistants and
agency to manage the programme more effectively and efficiently.
Strengthening the risk appraisal process and to consider the use of break-point
contingencies and making the extension process clearer.
To consider additional follow-up activity 2-3 years after ex-post to significantly
improve evaluation of sustainability objective.
Areas for further research to include, SME participation rates, more sector specific
analysis, benefits of an agency approach and optimum methods to integrate new
member states.
It is clear from the ToR that many of these issues remain relevant (e.g. integration of SMEs
and new member states) and that the evaluation of measures implemented as part of FP7 to
address these issues is an important element of this mid-term evaluation.
The intervention logic developed as part of the ex-post evaluation is shown below. This is
largely similar to the objective tree in the ex-ante of FP7 (pp. 21) but it more clearly
highlights the objectives around industrial competitiveness and R&D policy and delineates
the objectives at the various levels. The literature review in the ex-post evaluation of FP5
reflected that there was still scope to improve the intervention logic being used and that a
more systematic approach could improve the quality and usefulness of subsequent FPs and
evaluations.
73
Final Report - FP7 Energy Mid-Term Evaluation
Figure 1.1 Intervention logic from Ex-post evaluation of Non-nuclear Energy Projects supported by DG TREN under FP5
Global Objectives
Reduce global warming and
environmental impacts of energy
Secure Diverse, high quality
and low cost energy services
Sustainable development EU and worldwide
Intermediate Objectives
Reduce Energy
demand and
imports
Minimise environmental impact
of energy production and use
Competitive
energy markets
Increase uptake of
RES
Specific Objectives
Energy efficiency of
equipment and services
RES levels - %
and absolute
Lower CO2 and
other emissions
Transparency and diversity
of energy market
Improved industrial
competitiveness
Increased levels and improved
quality of R+D, displaying
- Community added value
- Social contribution
- In areas of economic
development with prospects
Critical mass (human and financial
resources) in R+D
R+D contrbuting to implemeting
community policy and addressing
EU level issues
R+D contributing to employment,
quality of life and health
R+D in expanding areas with good
pros[ects, in areas where EU
businesses can and must improve
their compet
Operational Objectives
New product / process
with improved efficiency
/ reduced impact.
Improved awareness /
uptake of RES and
energy efficiency
Cost
savings
achieved
Strengthening EU
energy industry
Improvement of
technical
knowledge /
perforamnce
New contacts made and follow on
collaborations - strengthening of
capacity , improved networking
R+D which overlaps with other policy
goals
Enhance skills of R+D staff, develop
skills in target areas
Registration of patents, publication of
journal articles
The ex-ante evaluation in particular discusses the overall need for the FP and the rationale for
selecting the specific thematic areas and so will be especially useful input for this task. The
ex-post evaluations of FP6 (including the Report of the Expert Group published in February
200912) also contains a useful discussion on intervention logic, noting that, “a more explicit
‘programme logic’ would have produced a more robust overall FP design”.
The Ex-Post Evaluation of FP6 by an Expert group report (all themes) – was released in
February 200913. The evaluation recognised the common issue that longer term impacts are
difficult to identify, even for completed projects, with the accepted compromise being that
the link between outputs and results and how these lead to outcomes can be described. This
confirms our experience and assumption that given the interim nature of the evaluation we
are bidding for the impacts will be even harder to identify - as none of he projects will have
completed yet.
12
13
http://ec.europa.eu/research/reports/2009/pdf/fp6_evaluation_final_report_en.pdf
http://ec.europa.eu/research/reports/2009/pdf/fp6_evaluation_final_report_en.pdf
Final Report - FP7 Energy Mid-Term Evaluation
74
The report also stated that dissemination and exploitation remain key to the achievement of
impact and as such they need to be well resourced. This implies that our evaluation should
specifically investigate how well this issue is planned into the projects we review and if any
efforts have been made to begin dissemination from the start of a project, rather than leaving
it all to the end as is commonly the case. Another finding of the report was that active links
and networks between project partners, and between projects (including networks of
Excellence), are key to technology transfer, project success and behavioural change- this
confirms the value in the use of network analysis tools (which will be discussed in the next
section).
The report recommendation highlighting post project funding for dissemination is an
interesting and valuable idea but not one which an interim evaluation is well placed to
examine - as no FP7 projects have completed yet.
This report (and others) provide evidence of overburdening of Commission project officers –
this will be a key issue for the efficiency aspect of our evaluation and is a key aspect at
interim stage. The finding that past project coordinators and participants have a great deal of
knowledge on best practice, raises the question of the degree of re-participation from these
past applicants and the extent to which new partners are able to learn and benefit from their
experience.
At both project and programme level there are methods to enhance the policy-research links,
both for policy makers to pass on the areas of research they would benefit from and for
researchers to make, and present, their work in a more policy relevant manner. This is an
issue that the relevance and sustainability evaluation questions need to address.
Although it covers DG Research as opposed to DG TREN projects the report Evaluation
and Impact Assessment of the European Non Nuclear Energy RTD Programme14 has a
number of points of relevance to this work, such as:
The differences between FP5 and 6, summarised as in FP6 the whole value chain was
addressed via larger projects with multiple participants, whereas FP5 projects dealt with one
issue at a time. The overall RTD budget decreased from FP5 to 6 by some 30%. Industry
participation reduced from 42% to 31%.
In terms of results the continuity of support for technologies was identified as an important
factor in project (and area) success.
In terms of relevance the general conclusion was that the larger scale of the FP6 projects was
not as productive as the more focussed nature of FP5.
In terms of policy impacts the production of roadmaps was regarded positively partly due to
its ability to internationalise the research agenda. The positive contribution of projects and
the programme to the policy debate in this area was also highlighted.
14
Evaluation and Impact Assessment of the European Non Nuclear Energy RTD Programme. Development and implementation of a
methodology for evaluation and impact assessment of the energy programme of the fifth and sixth Framework Programme of the
European Community for RTD activities. EPEC. October 2009
75
Final Report - FP7 Energy Mid-Term Evaluation
In terms of economic impact the presence of industrial engagement was highlighted as a
positive factor. Distinct variations were also reported between the sectors with solar energy
performing well.
1.3
Programme History and Evolution
Energy in FP7 covers a wide range of subject areas and disciplines, reflecting the
multidisciplinary and cross-sectoral research required to support policy-making in this
important field. The DG Energy specific energy component of FP7 is described in table 1:
Table 1.
Energy research in the Seventh Framework Programme (2007-2013)
Seventh Framework Programme (2007-2013) (EUR 53.2 BIO)
EC Programme (EUR 50.5 BIO)
Cooperation (EUR 32,365 MIO)
Energy (EUR 2,350 MIO) DG TREN and DG Research
• Hydrogen and fuel cells
• Renewable electricity generation
• Renewable fuel production
• Renewables for heating and cooling
• CO2 capture and storage technologies for zero emission power generation
• Clean coal technologies
• Smart energy networks
• Energy efficiency and savings
• Knowledge for energy policy making
Energy - DG Energy (EUR 501.4 MIO in the first 4 calls)
The themes supported by DG Energy have evolved over the life of FP7. This evolution
continues past practice, for example during FP5 the decision was taken to stop supporting
projects designed to improve the extraction efficiency of oil and gas, on the basis that the oil
price had risen to the extent where subsidising oil companies could not be justified.
The Directorate General (DG) of Energy and DG Research both fund a variety of energy
research programs. They both support FP projects however they focus on different aspects.
Fundamentally the focus of DG Energy is on commercialisation and demonstration projects.
This means that from a typical technology life-cycle perspective the technology development
is past the basic research and pilot phase. As the technology is close to commercialisation
private organisations are more willing to take the lead in this stage of the development
process. Industries commit themselves to the technology as they see the potential for it to
result in profits for them. Moreover, the risks of investment shrink as a result of (successful)
pilot projects. However, as there are still a large number of risks (technical, economic, etc.)
the public funding through DG Energy is still very important. Furthermore, the technologies
funded under DG Energy are expected to penetrate the market in the relatively short term.
DG Research focus on the early stages of the life cycle of technology development. This
Final Report - FP7 Energy Mid-Term Evaluation
76
means that fundamental research would be eligible for support. As such research is relatively
far from commercialisation (compared to DG Energy projects) it takes a longer term
perspective in what it supports.
77
Final Report - FP7 Energy Mid-Term Evaluation
2 Strategic Energy Technology (SET) Plan
2.1
Overview
FP7 has evolved over its life with the decision to make the calls more focussed and
influenced by the priorities of the Strategic Energy Technology (SET) Plan. The SET Plan
was put forward in 2007 by the Commission in order to accelerate the development and
deployment of cost-effective low carbon technologies. It comprises of measures relating to
planning, implementation, resources and international cooperation in the energy technology
field. The SET plan proposes a new method for managing research on energy technologies
based on joint strategic planning. The policy measures are designed to stimulate a more
intensive international collaboration strategy.
The SET Plan is focused on both short term research close to commercialisation as well as
supporting long term technology developments in the field of low carbon technologies. With
regard to shorter term research, the SET Plan aims to lower costs, improve performance and
stimulate commercialisation of existing technologies. With regard to longer term research the
SET plan supports the development of next generation technologies relevant to low carbon.
Actions are focused on improving specific technologies such as renewable technologies,
energy storage, fusion energy, etc.
In October 2009 the European Commission called for governments, firms and researchers to
combine efforts to develop energy technologies (in line with the priorities of the SET Plan for
a number of reasons: tackling the climate change problem, safeguarding the energy supply in
the EU and to guarantee the competitiveness of the EU economies. It would, furthermore,
contribute to advancing the fast growing industry in the EU as well as creating jobs. The
Commission estimated an additional investment of approximately EUR 50 billion for
research on energy technologies. It was suggested that this finance should come from
different sources, including government and the private sector at both the national and EU
level (primarily from the Framework Programme) and the allocation should be centrally
coordinated.
The technologies targeted in the SET Plan were divided into three groups, in order to cater
for the variety of needs. These technology groups were:
• Close to market competitiveness. Technologies in this groups are on-shore wind,
solar heating, solar photovoltaics (c-Si), first generation biofuels, electricity networks
(transmission), nuclear fission (G III+), combined heat and power (CHP) and energy
efficiency in buildings, transport and industry.
• Emerging technologies on the verge of mass market penetration. These include offshore wind, solar photovoltaics (thin film), concentrated solar power generation,
carbon capture and storage, second generation biofuels and electricity networks
(distribution).
Final Report - FP7 Energy Mid-Term Evaluation
78
•
New technologies, such as nuclear fission (Generation IV), hydrogen and fuel cells,
ocean energy and geothermal.
There are seven SET plan initiatives described in the SET-Plan Communication of November
2007:
• European Wind Initiative, where the focus is both on- and off-shore applications. In
this initiative rapid expansion of wind will be supported across Europe. This will be
done by coordinated measurement campaigns, development of testing facilities for
new turbine components, demonstration projects of next generation turbines, and
testing of new logistics strategies and erection techniques in remote environments.
• Solar Europe Initiative, including photovoltaics (PV) and concentrated solar power
(CSP). The aim is to become more competitive and gain mass market appeal in this
technological area. To support the development of PV, long-term and large-scale
demonstration projects will be undertaken, with up to five pilot plants for automated
mass production and a portfolio of demonstration projects for centralised and
decentralised PV electricity generation. For CSP, the initiative mainly consists of
building up to ten demonstration power plants.
• European Electricity Grid Initiative responding to three interrelated challenges:
creating a real internal market, integrating a massive increase of intermittent energy
sources and managing complex interactions between suppliers and customers. The
focus in this initiative is on the development of a smart electricity system, including
storage, and on the creation of a European centre to implement a research programme
for the European transmission network.
• Sustainable Bio-Energy Europe Initiative, to bring the most promising bio-energy
technologies to commercial maturity. Up to 30 pilot plants will be developed for
demonstration.
• The European CO2 capture, transport and storage initiative. Carbon Capture and
Storage (CCS) technologies will be widely commercialised. The full CCS chain for a
representative portfolio of different capture, transport and storage options will be
demonstrated at an industrial scale. In addition, a comprehensive research
programme will deliver improved components, integrated systems and processes to
make CCS commercially feasible.
• The sustainable nuclear fission initiative, which is mainly focused on the
development of Generation IV reactors. These reactors will be designed to maximise
inherent safety, increase efficiency, produce less radioactive waste and minimise
proliferation risks.
• The Smart Cities Initiative will support ambitious and pioneer cities that would
transform their buildings, energy networks and transport systems into those of the
future, demonstrating transition concepts and strategies to a low carbon economy.
Participating cities and regions will be expected to test and demonstrate the
feasibility of going beyond the current EU energy and climate objectives.
The cost estimates of these initiatives are as follows:
• European Wind Initiative (EUR 6 bln),
• Solar Europe Initiative (EUR 16 bln),
• European Electricity Grid Initiative (EUR 2 bln),
• Sustainable Bio-Energy Europe Initiative (EUR 9 bln),
• European CO2 capture, transport and storage initiative (EUR 10.5 – 16.6 bln),
• Sustainable Nuclear Fission Initiative (EUR 5 – 10 bln),
• Smart Cities Initiative (EUR 10 – 12 bln).
79
Final Report - FP7 Energy Mid-Term Evaluation
Additionally a set of technology roadmaps were presented by the first European Industrial
Initiatives (EII's), the Initiative on Smart Cities and the Energy European Research Alliance
(EERA) during the next 10 years. The roadmaps cover a number of key research areas with
cost estimates, as follows:
•
•
•
•
•
•
•
Wind energy (EUR 6 bln),
Solar energy (PV and CSP) (EUR 16 bln),
Bioenergy (EUR 9 bln),
Carbon capture and Storage (EUR 10.5 – 16.6 bln),
Electricity Grid (EUR 2 bln),
Sustainable Nuclear Energy (EUR 5 – 10 bln),
Smart Cities (EUR 10 – 12 bln).
Notes
The Wind, Solar (PV and CSP), Electricity grid and CCS European Industrial Initiatives were
launched in June 2010. The launch signalled agreement and commitment to the technology
roadmaps, implementation plans and governance arrangements. The implementation plans
and technology roadmap highlight a number of key objectives, targets and indicators for each
technology, these can be summarised as follows:
2.2
Wind
Technical KPI's have also been developed from the technology roadmap released in October
200915, the summary technical KPIs for wind from this are shown in the table below.
Activity / Area
KPI
Average wind energy electricity production cost reduced
by 20% by 2020
N.B. – EII IP – this is to be calculated from Levelised
Strategic Key Performance Indicator:
Cost of Electricity and separately for onshore and
offshore.
Activities Key Performance Indicators
Manufacturing costs of wind turbines and their
components reduced by 20% by 2020
New turbines and Components
Transport and erection costs of on- and offshore wind
turbines reduced by 20% by 2020
15
SEC (1295) available at
http://ec.europa.eu/energy/technology/set_plan/doc/2009_comm_investing_development_low_carbon_technologies_roadmap.p
df
Final Report - FP7 Energy Mid-Term Evaluation
80
Activity / Area
KPI
Installation costs of offshore wind turbines reduced by
Offshore structure related technologies
20% by 2020
Maintenance costs of offshore wind turbines reduced by
20% by 2020
Grid integration
Resource assessment and spatial planning
Virtual capacity factor of wind farms reaching 80%
Wind resources and conditions predicted with an
uncertainty of less than 3%
Source: SEC (1295) – these summary KPI's further developed and detailed in Wind EII Implementation Plan
The wind energy EII implementation plan16 sets out a detailed action plan to achieve the
objectives set out in the technology roadmap. The implementation sets out one overarching
Key Performance Indicator (KPI) for the Wind Energy roadmap which will be used by
SETIS to monitor its implementation. This is the Levelised Cost of Electricity (LCOE),
which will be calculated separately for onshore and offshore wind. A complex formula
drawing in all relevant elements has been utilised to calculate the LCOE, this is fully detailed,
with assumptions, in the implementation plan document.
The implementation plan has significantly expanded on these objectives, with a large number
of draft (to be validated by the wind EII) technical objectives and related KPIs listed in the
implementation plan. Action points and a limited number of more project specific KPIs have
been devised. The majority of these indicators contain numerical and chronological targets
such as increased rotor blade length, weight to power ratios, turbine generator capacity and
efficiency gains by specific dates. The measures / indicators can be grouped under the
following headings:
•
•
•
•
•
•
•
•
•
16
Turbine improvements and design development
Reliability of turbines and wind farms
Optimisation and testing of wind turbines new different terrains.
Standards for large scale wind turbine components
Manufacturing and logistics for large scale turbines.
Demonstration sites
Safety Standards
Grid Connection and Interconnection between MS countries
Offshore and Onshore Wind Solutions
http://ec.europa.eu/energy/technology/initiatives/doc/wind_implementation_plan_final.pdf
81
Final Report - FP7 Energy Mid-Term Evaluation
•
•
•
2.3
Power Plant requirements
Market Integration and Penetration
Research
Solar (PV)
Solar Energy – PV roadmap set a strategic objective of establishing photovoltaic's as a clean,
competitive and sustainable energy source which can provide up to 12% of electricity
demand by 2020. The technology roadmap set this objective, along with technical KPI's in
October 2009. A summary of the technical KPIs for solar PV from this report are highlighted
in the table below.
Activity / Area
KPI
Activities Key Performance Indicators
Reduced conventional turnkey PV system cost to
<1.5€/Wp by 2020
Reduced concentrated PV system cost to <2€/Wp
by 2020
Increased PV (module) conversion efficiency to
PV system
>23% by 2020
Increased conversion efficiency to concentrated PV
to >35% by 2020.
Increased crystalline silicon and thin film modules
lifetime to 40 years.
Increased inverter lifetime to >25 years by 2020
Integration of PV-electricity generation
Battery storage cost <0.06€/kWh and life >25 years
In May 2010, an up-date to this report was launched but the Solar Implementation Plan17
(Solar Europe Industry Initiative) has not significantly expanded and/or refined the original
KPIs established in 2009. The primary objective of the KPIs is to support the original
strategic objective and support increased market penetration by reducing the cost of PV
within the commercial market.
Instead, five year targets have been established for each of the indicators in order that real
goals for PV technology and electricity can be achieved, and support the interaction between
other energy sources.
2.4
Solar (CSP)
Concentrating Solar Power (CSP) has a strategic objective set out in the Roadmap of
contributing around 3% of European electricity supply by 2020 with a potential of at least
10% by 2030. Technical KPI's for CSP have been developed from the Roadmap which was
17
http://ec.europa.eu/energy/technology/initiatives/doc/pv_implementation_plan_final.pdf
Final Report - FP7 Energy Mid-Term Evaluation
82
released in October 2009, the summary technical KPIs for wind from this are shown in the
table below.
Activity / Area
KPI
Activities Key Performance Indicators
Increase solar to electricity conversion efficiency by
at least 20% (relative).
Increase efficiency & Reduce cost
Reduce cost of installed products and O&M by at
least 20% compared with state of the art in
commercial plants in 2009
Increase dispatchability
Increased performance of storage and hybridisation
by at least 20%
Substantial reduction of water consumption with only
Improve the environmental profile
minor loss of performance
Substantial reduction in land use per MW installed
The CSP EII Implementation Plan18 (Solar Thermal Electricity European Industrial Initiative
STE-EII) was launched in May 2010. The implementing plan aims to expand on the original
technology roadmap objectives, and groups CSP work under four strategic objectives which
are; Reduction of Generation, Operation and Maintenance Costs, Improvement of
Operational Flexibility and Energy Dispatchability, Improvement in the Environmental and
Water-use Footprint, and Advanced Concepts and Designs. The plan has reformulated the
CSP but there has been limited variation from the KPIs set out above. Instead, they have
been refined and expanded upon into the following two categories:
Activity / Area
KPI
Activities Key Performance Indicators
Increased efficiency:
Increase solar to electricity conversion efficiency by
at least 5% (relative).
Increase Heat Collecting Fluid Steam Temperature.
Increase efficiency & Reduce cost
Reduce costs:
Reduce cost of installed products and O&M by at
least 10% compared with state of the art in
commercial plants in 2009
Lifetime Levelized Electricity Cost €/MWh
Number of 'down-time' hours per year (plant
reliability).
18
http://ec.europa.eu/energy/technology/initiatives/doc/csp_implementation_plan_final.pdf
83
Final Report - FP7 Energy Mid-Term Evaluation
Activity / Area
KPI
Increased performance of storage and hybridisation
by at least 5%
Investment cost of storage, €/MWht of stored energy
Increase efficiency of storage, % as well as time
dependency
Increase dispatchability
Decrease size of storage, M3/MWht
Increase number of operating hours, based on
maximum storage capacity
Decrease the cost of produced energy (compared
with a similar plant without storage
The recently launched implementing plan demonstrates that the original KPIs have been
refined and developed to reflect the specific of the Solar Thermal Electricity sector. The
targets have been reduced but still aim to contribute to the 2020 targets and enhance market
penetration of the CSP sector.
2.5
Electricity Grids
Resulting from the Technology Roadmap KPI's have been developed for the Electricity Grid
sector 2009 and are shown in the table below.
Activity / Area
KPI
Activities Key Performance Indicators
Number of customers involved (at least 1.5 million)
Greatly increased capacity to host RES electricity
from central and distributed sources (to at least 35%
of electricity consumption) including readiness for
massive offshore wind integration
Increased overall quality of electricity supply (by a 210% reduction of energy not supplied)
Reduced peak to average load ratio (by 5-10%) and
thus reduced need for investments
Full integration of customers in market mechanisms
promoting energy efficiency and active demand
practices.
The European Electricity Grid Initiative (EEGI) Implementation Plan19 has sought to build
upon the original KPIs but has yet to finalise the draft indicators set out in the table below.
19
http://ec.europa.eu/energy/technology/initiatives/doc/grid_implementation_plan_final.pdf
Final Report - FP7 Energy Mid-Term Evaluation
84
The EEGI considers there is a need to distinguish between the costs/benefits for grid users
(internal) and cost/benefits for externals. The benefits and KPIs will be further examined
with the SETIS (SET Information System.
Benefits
Increased Sustainability
Potential KPIs
Quantified reduction in CO2 emission
Hosting capacity of DER
Adequate transmission / distribution grid capacity
Reduction in DER cut-off due to congestion
Allowable maximum injection of power without
congestion risks
Harmonization and standardisation of grid
Reduction in time to connect new user
connection
Uniform grid connection rules
Reduction peak demand ratio
Increased share of renewables
Reduction in interruptions per customer
Increased voltage quality performance
Higher security and quality of supply
Increase in coordinated operation between TSOs
and DSOs
Increased efficiency in preventive and emergency
control
Coordinated restoration after emergency
Reduction in network losses
Increased demand side participation
Enabling energy efficiency by end users
Enhanced efficiency and better service
Hosting capacity of EV
Increased availability of network components
Actual availability of network capacity
Availability of ancillary services across transmission
and distribution grids
Effective support for pan-European electricity
markets
Increase of cross-border interconnection capacity
Once complete, the KPIs will represent; Level 1 (2020 EU Energy Policy), Level 2 (The
Programme), and Level 3 (The Projects).
2.6
Carbon Capture Storage (CCS)
The Technology Roadmap proposed and agreed in October 2009 set a twin strategic objective
for CCS EII of enabling cost competitive development of CCS beyond 2020 and assist
further develop technologies to allow application in all carbon intensive industrial sectors.
To support this, technical KPI's were developed from the technology roadmap released in
October 2009, and a summary of the technical KPIs for CCS is presented in the table below.
85
Final Report - FP7 Energy Mid-Term Evaluation
Activity / Area
KPI
Cost of CCS technology (including capture,
transport and storage) in power plants reduced by
Strategic Key Performance Indicator:
30-40%
Activities Key Performance Indicators
Average annual load factor of at least 80% in CCS
power plants by 2020
Average annual CO2 capture rate of at least 90% in
CCS power plants by 2020
Net efficiency of coal fired power plants equipped
with CCS (pulverised coal with post-combustion,
IGCC with pre-combustion capture or oxyfuel)
higher than 40% by 2020.
Net efficiency of conventional coal fired power plants
higher than 50% by 2020.
Publication of an accurate atlas of CO 2 storage sites
in Europe and an outline of the European transport
infrastructure by 2020
First commercial deployment of CCS technologies to
energy intensive industrial applications by 2025.
The CCS Implementation Plan20 seeks to expand on the original objectives and associated
KPIs. The KPIs are yet to be validated by the CCS team but can be grouped as follows:
Activity / Area
KPI
Cumulative number of Final Investment Decisions
taken
Gross installed cumulative capacity of CCS projects
across Europe
Specific capital investment
Fixed and variable operating costs
Demonstration and Lighthouse Projects
Availability
Construction time
Plant efficiency
CO2 capture rate
CO2 avoided
CO2 stream composition
20
http://ec.europa.eu/energy/technology/initiatives/doc/ccs_implementation_plan_final.pdf
Final Report - FP7 Energy Mid-Term Evaluation
86
Activity / Area
KPI
Length of pipeline network built
CO2 transport and storage
Number of Final Investment Decisions
Assessment of European Storage Potential
Amount of CO2 stored
Public awareness and support
Eurobarometer poll
Health Safety, Environmental and Knowledge
Number of permits to CCS projects
Sharing KPIs
Number of projects in the Project Network
87
Final Report - FP7 Energy Mid-Term Evaluation
From COM (519) - SEC (2009) 1295 Investing in the Development of Low-Carbon
Technologies (SET-Plan) A Technology Roadmap
Final Report - FP7 Energy Mid-Term Evaluation
88
89
Final Report - FP7 Energy Mid-Term Evaluation
3 Evolution of calls
The table below lists the subject areas open for bids in each of the four calls launched to date.
Table 2 summarises the key points of this:
Table 2 – Summary of Calls
Call
Budget (€m)
No. subject areas open
Projects approved
FP7-ENERGY-2007-2TREN
128
45
26
FP7-ENERGY-2008TREN-1
147
18
23
FP7-ENERGY-2009-2
100
10
11 under negotiation
FP7-ENERGY-2010-2
Deadline March 2010
126.4
8
15 under negotiation
The change in programme focus following the first call is clear in that the number of subject
areas ‘open’ for submissions reduced dramatically. The detail of the subject areas open in
each call is shown in annexe one.
Table 3: Calls For Proposals by Activity, Area and Topic
Level 1
Level 2
2
Renewable
energy
generation.
2007
Called?
Level 3
2.1.1 Enhancing strategic international cooperation
initiatives in the field of concentration photovoltaics
2.1.2 Manufacturing and product issues for thin-film
photovoltaics
2.1.3 multiple benefits of PV systems.
2.1.7: secure, reliable and affordable supply of
feedstock for the pv industry
2.1.8: improved production equipment and cost
reduction
2.1.9: innovative/improved pv manufacturing
processes
2.1.10: development and demonstration of
standardized building components
2.1.11: multiple benefits of pv systems
Final Report - FP7 Energy Mid-Term Evaluation
90
2009
Called?
x
x
2.1.6 Alternative approaches for crystalline silicon PV
2.1
Photovoltaics
2008
Called?
x
x
x
x
x
2010
Called?
Level 1
Level 2
2007
Called?
Level 3
2.2.1 Biomass to electricity from energy crops and
recovered fuels
2.2 Biomass
2.3 Wind
2.4.
Geothermal
2.5:
Concentrated
solar power
2.6.: Ocean
energy
2.7 HydroElectric
2.9. Crosscutting issues
3
Renewable
fuel
production
3.1: First
generation
biofuel from
biomass
3.2 Second
generation
biofuels from
biomass
3.3
2.2.4: large-scale cofiring
x
2.2.5: novel solid biofuels for electricity generation
2.2.6: high-efficiency medium-to-large scale
electricity generation from biomass
2.3.1 Demonstration of large scale systems for onand off-shore wind farms including their cost
effective grid integration
2.3.2: Support to the coordination of stakeholders’
activities in the field of wind energy
2.3 Testing, standards and certification for wind
energy systems
2.3.4: demonstration of large scale systems for onand off-shore wind farms
2.3.5: integration of collaborative project
2.3.6: wind mapping for offshore applications
2.4.1 Increased electricity production from Enhanced
Geothermal Systems and from low enthalpy
geothermal sources
2.5.1.:Improve the environmental profile of the csp
installations
2.5-02 Using CSP for water desalination
2.5.3: low cost, high efficiency daily storage systems
2.5.4: improve the environmental profile of the csp
installations
x
2.5.5: csp: innovative heat transfer concepts
x
2.5.6: intermediate size, lower concentration ratio
csp systems
2.6.1. Ocean: demonstration of innovative full size
systems
2.6-03 Pre-normative research for ocean energy
2.7-01 New or improved hydro components and
concepts
2.9.1. Storage for intermittent electricity
2.9-1: Demonstration of innovating multipurpose
solar plants
2.9.1 Deep off-shore multi-purpose renewable
energy conversion platforms for wind/ocean energy
conversion
3.1.1:Biofuels from high moisture content biomass –
biomethane production
3.1.2: biodiesel from oil crops, animal tallow and
used cooking oils
3.2.1 - 07 Identifying research needs and
technological opportunities for biofuels production
in Latin America
3.2.2.:Bioethanol production from lignocellulosics
3.2-03 High purity syngas cleaning technologies for
biofuels
3.2.5: synthetic biofuels via gasification
3.2.6: hydrogenation of oils and fats
2008
Called?
2009
Called?
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
3.3.1 Sustainable Biorefineries
91
2010
Called?
x
Final Report - FP7 Energy Mid-Term Evaluation
Level 1
Level 3
Biorefineries
(Part
supported by
TREN)
3.4 New
energy crops
3.5
3.6: biofuel use
in transport
3.3.2 Enhancing exchange of information, synergies
and crossfertilisation between projects in the field of
Biorefineries
3.3.3 Developing biorefinery concepts
3.7: crosscutting issues
4.1
Low/medium
temperature
solar thermal
energy
4.
Renewables
for heating
and cooling
4.2: Biomass
4.3:
geothermal
energy
4.4: crosscutting issues
4.5. Cross
cutting issues
5. ?
5.1 ?
5&6.1:Power
generation
technologies
for integrated
zero emission
solutions
5&6
5&6.2: cross
cutting and
regulatory
issues
6. Clean coal
technologies
2007
Called?
Level 2
6.1 Conversion
technologies
2008
Called?
2009
Called?
x
3.4-1: Biofuels from algae
3.5-01 Fuel production using solar radiation
3.6.1: demonstration of liquid and gaseous biofuels
use in transport/ vehicles
2010
Called?
x
x
3.7.1 Harmonisation of biomass resource assessment
3.7.3: standardisation and sustainability issues
3.7.4: promotion and dissemination
4.1.1 Low/medium temperature solar thermal
systems for industrial Process Heat
4.1.1: collector design and components
4.1.2: small scale thermal cooling units
4.1.3: small distributed systems for seawater
desalination
4.1.4: large scale systems for industrial heat
processes
4.2-1: demonstration of a new generation of boilers
and stoves
4.3.1: improved ground source heat pumps
x
x
4.3.2: improved underground systems
x
4.4.1: advanced compact storage systems
x
x
x
x
x
x
x
x
4.5.1 Hybrid systems based on solar thermal
heating/ cooling, backed up by biomass or
geothermal to compensate heat load intermittence
4.5-1: support to the coordination of stakeholders'
activities in renewables for heating and cooling
5.1.3 Advanced separation techniques
5.1.4 Separation techniques in gaseous fuel power
generation (RTD)
x
x
5&6.1.1. :Feasibility and engineering study for
development of an integrated solution for a large
scale zero emission fossil fuel power plant
5&6.2-1: extending the value chain for ghg emissions
other than co2 collaborative project
5&6.2.2: support to regulatory activities for zero
emission power generation
5.2.3 CO2 capture and storage - public acceptance
5&6.2.4: initiating a CO2 value chain in the energy
sector using early opportunities
5&6.2.5: extending the value chain for ghg emissions
other than CO2 associated with coal production and
use
6.1.1 Efficiency increases in existing and new build
pulverized coal power plants with a view to CCS.
Final Report - FP7 Energy Mid-Term Evaluation
92
x
x
x
x
x
x
x
Level 1
Level 3
for zero
emissionpower
generation
6.1.1. :Advanced fluidized bed combustion
technology
6.1.1: solid fuel gasification development –
improvement of gasifier technologies
6.1-1: efficiency improvement of oxygen-based
combustion collaborative project
6.1.3.:Efficiency improvement of oxygen-based
combustion
6.1.4.:Advanced gas turbines for solid fuel
gasification processes
6.2.1: poly-generation concepts for coal fired power
plants
7.1.1: Optimisation of the electricity grid with large
scale renewables and storage
6.2: coal-based
polygeneration
7.1:
Development
of inter-active
distribution
energy
networks
7. Smart
energy
networks
7.2: Pan
european
energy
networks
7.3: cross
cutting issues
and
technologies
8: Energy
efficiency
and savings
2007
Called?
Level 2
8.1: efficient
energy use in
the
manufacturing
industry
2009
Called?
x
x
x
x
x
x
x
7.2.1 Simulation and state estimation of smart
electricity transmission networks
7.2.3.: Diagnostics, Surveillance, Maintenance and
Control of Power Transmission and Grid Connections
7.2.4.: Assessment of needs for reliable and flexible
future European gas networks
7.3.1 HTS Devices for Electricity Networks
x
x
7.3.3 Stepping up the cooperation of national and
regional research activities on Smart Energy
Networks
7.3.4: analysis and scenarios of energy infrastructure
evolution
7.3.5: more efficient integration of renewable energy
into future infrastructures
8.1.1: Energy efficiency in energy intensive industry
8.1.1: manufacturing industry: wastes and waste
heat recovery and transfer
8.1.2: manufacturing industry: smes energy
innovation
8.1.3: manufacturing industry: innovative energy
efficient manufacturing processes
8.2.1.: High efficiency polygeneration - renewable
energies for applications in industry
8.4:
8.4.1.: CONCERTO communities: the way to the
future
8.5: innovative
strategies for
clean urban
transport:
Civitas-plus
8.5.1: testing innovative strategies for clean urban
transport
8.6.1: support action for evaluation and monitoring
civitas-plus
93
2010
Called?
x
7.1-1: large scale demonstration of smart
distribution networks with distributed generation
and active customer participation
8.2: High
efficiency
polygeneration
8.6: socio
economic
research and
2008
Called?
x
x
x
x
x
x
x
x
x
x
x
Final Report - FP7 Energy Mid-Term Evaluation
Level 1
Level 2
2007
Called?
Level 3
2008
Called?
2009
Called?
2010
Called?
innovation
9:
Knowledge
for energy
policy
making
8.7: thematic
promotion and
dissemination
8.7.1: promotion and dissemination
8.7.2: support action for coordination and
dissemination
9.1.1 European energy infrastructure networks and
systems transition planning
9.1 ?
9.1.2 Energy behavioural changes
9.2: scientific
support to
policy
x
x
9.1.3 Energy technological foresight and scenario
development
9.2.1 European scientific multidisciplinary "thinktank" to support energy policy and to assess the
potential impacts of its measures.
9.2.2: energy technology watch
x
x
10.1.1 Future Emerging Technologies (FET)
10. ?
10.1 ?
10.1.2 Novel materials for energy applications (Joint
Call NMP)
10.1.3 Trans-national co-operation among NCPs
SETPLAN
TOTALS
Final Report - FP7 Energy Mid-Term Evaluation
45
94
18
10
8
4 Participant Survey
A high response rate was achieved in the survey of current FP7 Energy participants from the
2007 and 2008 calls. Of 477 valid contacts, 221 responded, equal to a 46% response rate. The
following table presents a breakdown of the respondents by type.
Partners
TOTALS
As% of total
respondents
CoOrdinators
Non-
SMEs
SME's
Companies
NonCompanies
191
30
57
164
144
76
86.4%
13.6%
25.8%
74.2%
65.2%
34.4%
455
48
118
385
-
-
42.0%
62.5%
48.3%
42.6%
-
-
TOTAL
221
TOTAL in
FP7 2007 &
2008
Respondents
as %
This shows that a response rate of at least 40% was achieved in the partner/coordinator and
SME/Non-SME sub-groups. The co-ordinators subgroup with a sample of only 30 would
ordinarily be of limited value as it is nominally only a small figure, but as it represents over
62% of the whole sub-group this is mitigated, so that the findings remain strongly relevant.
Non-companies in this case encompass all sorts of publically funded institutions including
local municipalities, universities, research institutions and NGOs. There is some uncertainty
with the data and therefore the definition of respondents in this regard, this should be borne in
mind when considering this factor.
4.1.1
Project Information
The first 4 survey questions established the basic project information relevant to the survey
including the respondents name, organisation, the funded project and project website.
Questions 5 and 6, presented below with their results, were designed to provide valuable
insight into the various core dimensions of the projects. This is important to enable evaluation
of the types of projects being supported against the FP7 Energy objectives.
Q5. How would you describe your project in terms of the following dimensions?
• Strategic importance for society
• Strategic importance for your organisation
• Technical complexity
95
Final Report - FP7 Energy Mid-Term Evaluation
510
•
•
•
Commercial risk
Technical risk
Cost
How would you describe your project in terms of the following
dimensions?
Low
Medium
High
N/A
Strategic importance for
society
Strategic importance for your
organisation
Technical complexity
Commercial risk
Technical risk
Cost
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Q6. How would you describe your project in terms of its proximity to market?
How would you describe your project in terms of its proximity
to the market?
Fundamental
research
4%
Applied research
24%
Demonstration
46%
Product testing
4%
Final Report - FP7 Energy Mid-Term Evaluation
96
Prototype
development
22%
100%
It was expected that a number of trade-offs e.g. cost v risk to become evident from these
questions. Among the clearest results identified is that the funded projects are felt to have
high strategic importance for society and the participant organisation. The strategic
importance for society was felt to be slightly higher than that for the organisation itself,
perhaps highlighting that funded projects are felt to be targeted more closely on societal
objectives rather than organisational objectives. Over 95% felt their project was of medium
or high technical complexity. A significantly smaller proportion (less than 70%) felt that their
project was of medium or high commercial risk, while around 80% felt their project was of
medium or high technical risk. These findings may point to confidence among researchers
that despite projects being complex they still have the resources to overcome the technical
and commercial hurdles they face. In respect of cost around 60% of respondents felt their
project was of medium cost, 10% low and almost 30% high.
In respect of proximity to market it was clear that the 3 main project types supported are
demonstration, prototype development and applied research projects, together constituting
over 90% of the total. Almost half of respondents considered themselves to be at the
demonstration stage, which is in line with the project types as classified in the call analysis.
4.1.2
Relevance and Effectiveness
This section examined participants understanding of the nature and objectives of FP7 and
how their project fits with this. The intention of Question 7 was to establish the level of
understanding of programme participants when applying, to evaluate the relevance of the
programme and how well the objectives of the FP7 Energy programme are presented and
known. Questions 8, 9 and 10 examined the level of match between the project and different
sets of wider objectives. Identifying the level of match is important, as if the FP7 Energy
programme is to achieve its objectives, there should be a match between the objectives of the
projects it funds and the programme objectives. The identification of which project objectives
most closely and commonly match programme objectives is also interesting.
As a follow-up to this Questions 11 and 12 provide direct comment from respondents on the
relevance of the FP7 Energy programme to achieving the stated objectives and how this has
evolved over time. This final element is particularly interesting to note the perceived
evolution of the programme and what respondents views on this are. This is useful to
evaluate the perception among respondents of any changes due to increased alignment with
the SET plan and what they think of these changes – this is also relevant to evaluating FP7
Energy consistency and acceptability.
Questions 13 and 14 provide information on whether the respondent has been a previous
recipient of European or national level funding for energy research and development. This is
important to identify the extent to which past participants re-apply and the relationship
between national and EU funding. This can inform an evaluation of additionality and the
complementary nature of FP7 Energy support.
97
Final Report - FP7 Energy Mid-Term Evaluation
Q7. How well did you feel you understood the FP7 Energy programme objectives when
applying?
How well did you feel you understood the FP7 Energy
programme objectives when applying?
Not at all
0%
Poorly
1%
Basic
understanding
23%
Very well
31%
Moderately
well
45%
It was expected that participants would have at least a basic understanding of the FP7 Energy
programme, the results show that almost 99% of participants claimed to have at least a basic
understanding of the programme, almost 1/3 believed they understood the programme
objectives very well. This points to widespread recognition and understanding of the FP7
Energy programme objectives among applicants.
Q8. How closely do your project objectives match the following high level FP7
objectives?
•
•
•
Achieving a secure and diverse EU Energy Supply
Reducing GHG emissions and increasing the sustainability of energy supply
Improving the economic competitiveness of EU industries
Final Report - FP7 Energy Mid-Term Evaluation
98
How closely do your project objective match these high level
FP7 objectives?
No match
Slight match
Some match
Good match
Very close match
N/A
Improving the economic
competitiveness of EU
industries
Reducing GHG emissions and
increasing the sustainability of
energy supply
Achieving a secure and diverse
EU Energy Supply
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
It would be expected at this high level that there should be at least a good match between
high level FP7 objectives and ultimate project objectives. Responses from participants
showed a belief that high level objectives matched very closely, particularly in respect of
reducing GHG emissions and achieving a secure and diverse energy supply. Responses were
more mixed on the match with improving economic competitiveness but still over 75% felt
there was a good or very close match between project and programme objectives. These
results point to a high level match in purpose between project and programme objectives and
therefore that the funded projects should make a contribution towards achieving these high
level FP7 Energy objectives.
Q9. How closely do your project objectives match the following energy specific FP7
objectives?
• Increase energy generated from renewable energy sources
• Improved energy sector competitiveness
• Contribute to achieving wider energy policy goals e.g. 20-20-20 targets
• Developing technologies that can achieve short term market penetration
• Increase whole energy generation system efficiencies
• Build interfaces between new and existing technologies
99
Final Report - FP7 Energy Mid-Term Evaluation
How closely do your project objectives match the following energy specific
FP7 objectives?
No match
Slight match
Some match
Good match
Very close match
N/A
Build interfaces between
new and existing
technologies
Increase whole energy
generation system
efficiencies
Developing technologies
that can achieve short term
market penetration
Contribute to achieving
wider energy policy goals
e.g. 20-20-20 targets
Improved energy sector
competitiveness
Increase energy generated
from renewable energy
sources
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The different level of objectives in this question would be expected to produce a more
nuanced set of results and draw out particularly strong or weak objective matches. Against
the energy specific objectives, in every case at least 65% of respondents felt there was a good
or very close match between their project and the FP7 programme objectives. The objective
with the lowest % of good/very close matches is the 'developing technologies that can
achieve short term market penetration' objective, highlighting that this is not necessarily a
primary project objective. This is understandable given the results of Q6 which although
showing high proportions of projects at demonstration and product development stages also
show a significant minority of around 30% that are at fundamental or applied research stages
and therefore further from market. An interesting result is that only 13 of the 222 respondents
to this question believed there was no match between their project and the objective of
increasing energy generated from renewable energy sources. Given that around 40% of
project funding is directed at non-renewable energies the proportion here could have been
expected to be higher. This may point to projects in areas such as energy efficiency and smart
networks seeing their role as an enabler for renewable energy.
Q10. How closely do your project objectives match the following commercial and
research related FP7 objectives?
•
•
•
•
•
•
Enhance EU research and development capacity
Leverage private investment
Support trans-national co-operation
Provide tools and information for policymakers
Securing industry commitment
Specific technology development
Final Report - FP7 Energy Mid-Term Evaluation
100
How closely do your project objectives match the following
commercial and research related FP7 objectives?
No match
Slight match
Some match
Good match
Very close match
N/A
Specific technology development
Securing industry commitment
Provide tools and information for
policymakers
Support transnational cooperation
Leverage private investment
Enhance EU research and
development capacity
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The results from this set of objectives show varying levels of match. The strongest objective
match, with over 88% seeing a good or very close match, is supporting transnational cooperation. Specific technology development and enhancing EU research and development
capacity were also believed to strong matches, with around 75% seeing a good or very close
match in each. The other 3 objectives each saw this good / very close match level fall to
around 60% of respondents with providing tools and information for policymakers seeing the
lowest match. The policymakers information objective also saw the highest combined levels
of no or only slight match at 14% of respondents, highlighting that this seemed to be the least
relevant objective at project level. Leveraging private investment saw the highest numbers of
respondents (11 out of 222 or 5%) declaring that there was no objective match.
Q11. What is your view on the appropriateness of the FP7 Energy programme to
achieving the objectives above, given the wider economic, social and environmental
context?
101
Final Report - FP7 Energy Mid-Term Evaluation
What is your view on the appropriateness of FP7 Energy to achieving
the objectives above, given the wider economic, social and
environmental context?
Not appropriate
Moderately appropriate
Highly appropriate
Don't Know
Appropriateness
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The results show that over 70% of respondents feel that the FP7 Energy programme is
appropriate to achieve the objectives listed in the previous questions. None of the respondents
felt the FP7 programme was inappropriate. This would appear to be a very strong
endorsement of the FP7 energy programme as an instrument to achieve the stated objectives.
Q12. And how has this changed since:
•
•
Previous Framework Programmes (FP4, FP5, FP6)
The start of FP7
And how has this changed since:
Become less appropriate
No change
Become more appropriate
Don't Know
The start of FP7
Previous Framework
Programmes (FP4, FP5,
FP6)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The results show that a significant proportion of respondents, over 50% in the case of
previous Framework Programmes and over 40% in the case of FP7, are unsure of any
changes in appropriateness. Of those that did have a view, in the case of previous Framework
Programmes over 75% believed that FP7 had become more appropriate than previous FPs.
Final Report - FP7 Energy Mid-Term Evaluation
102
Since the start of FP7, those that had a view were split approximately 50-50 between those
that believed the programme had become more appropriate and those that believed there had
been no change. These results point to an understanding among previous participants that the
programme is becoming more appropriate to achieving the stated objectives but that a similar
change since the start of FP7 is less clear. At the same time a large proportion of participants
don't know either way how the appropriateness may have changed, this may in part be due to
only participating in FP7, or that they feel it is too early to give a view on changes since the
start of FP7.
Q13. Has your organisation received any EU Energy research funding prior to your
FP7 project?
Response
Percent
Response
Count
No
37.4%
83
Previous FPs (FP4, FP5, FP6)
54.5%
121
Other energy programmes (e.g. Intelligent Energy
Europe)
22.5%
50
Other programmes (please specify)
12.6%
28
Answer Options
answered question
222
The results show that over half of the respondents had taken part in previous FP programmes
and that in total over 60% of respondents had received some form of EU Energy research
funding prior to taking part in FP7. It also shows that over 1/3 of participants are new to EU
Energy research funding.
Q14. Has your organisation received any national funding for energy related research
in the five years prior to this application?
Response
Percent
Response
Count
No
26.1%
58
Yes, 1-5 projects
43.7%
97
Yes, 6-10 projects
4.5%
10
Yes, > 10 projects
18.0%
40
Don't Know
7.7%
17
Answer Options
answered question
222
The results show that almost 75% of respondents had received some form of national funding
for energy related research in the last 5 years. Nearly 1 in 5 respondents had received funding
for over 10 projects.
Summary
Overall the results of Q8, 9 and 10 showed that the level of match between project and
programme objectives was rated good or very close was over 80% in most cases, and the
lowest level of match at this level was still 58% (for providing tools and information for
103
Final Report - FP7 Energy Mid-Term Evaluation
policymakers). It could therefore be clearly said that respondents felt their projects had a
good or very close match to FP7 Energy objectives in the great majority of cases. Questions
11 and 12 indicated that the FP7 Energy programme is strongly felt to be an appropriate way
of achieving the stated objectives and that it is more appropriate than previous framework
programmes. The results from questions 13 and 14 show that around 1/3 of participants are
involved in an EU Energy related funding programme for the first time, but also that over
half have been involved in a previous FP programme, while almost 75% had received
funding nationally. The high proportion of those with previous FP experience points to this
being beneficial to making a successful application, but at the same time the significant
minority without any prior FP experience, highlights that prior experience is not an essential
requirement and that the programme remains accessible to new entrants.
4.1.3
Efficiency
The questions in this section were designed to produce data on the administrative load of
preparing a bid for FP7 Energy funding and, following participants success in this, in project
delivery. This is useful to examine the level of administrative burdens which will be
indicative of the efficiency of programme delivery. Comparison to previous FP programmes
will enable analysis of the trend for efficiency. To achieve this questions 15 and 16 query
time spent in preparing the FP7 Energy project application and questions 17 and 18 on time
spent to administer the project since it began. Question 19 explores participants satisfaction
with specific elements of programme implementation and question 20 how participants
experience of the programme compares to previous Energy Framework Programmes. The
qualitative measures used in a number of these questions allows for analysis of project
participants relative perception of administrative burdens, which is also an important factor in
evaluating efficiency. The questions in this section can also assist in evaluating relevance and
effectiveness
Q15. How much work was required to prepare the application? (Approximate number
of person days)
Answer Options
Response Count
answered question
Average
Median
201
45 days
25 days
Q16. Regarding the time you spent on the application, did you find this?
Answer
Options
Did you find
this?
Low
As
expected
High
Don't
Know
Response
Count
4
121
75
11
211
The results of the questions on the time taken to prepare the applications show that on
average each organisation had to spend an estimated 45 person days to prepare their
applications. This figure is somewhat skewed by a number of high responses including one
respondent that took almost 2 years (730 person days) to prepare their application. It is
noteworthy that 20 respondents reported requiring over 100 person days to prepare their
Final Report - FP7 Energy Mid-Term Evaluation
104
contributions to applications, yet 58 respondents required 10 days or less. Taking the skew
from the highest estimates into account the median value of 25 person days may give a more
representative view on approximate person days per application.
Over half of respondents felt that the time spent on the application was in line with their
expectations, but around 35% found the time spent higher than expected.
Q17. How much work has been required so far by your organisation relating to
Commission required management and administration of the project? (Approximate
number of person days)
Answer Options
Response Count
answered question
Average
Median
191
44 days
20 days
Q18. Regarding the time you have spent on administration, did you find this?
Answer
Options
Have you
found this?
Low
As
expected
High
Don't
Know
Response
Count
5
129
68
9
211
The results of the questions on the time taken to satisfy FP7 management and administration
requirements show that on average each organisation had to spend an estimated 44 person
days annually. This figure is again somewhat skewed by a number of high responses
including two respondents that took around 2 years (700 and 800 person days) to administer
their projects, though both of these regarded this length of time 'as expected'. It is noteworthy
that 23 respondents reported requiring over 100 person days for project administration, yet 75
participants required 10 days or less. Taking the skew of these higher entries into account the
median value of 20 person days may give a more representative view on approximate person
days for project administration.
Over 60% of respondents felt that the time spent on project administration was in line with
their expectations, but around 30% found the time spent higher than expected.
Q19. How satisfied have you been with the following aspects of programme
implementation?
•
•
•
•
•
•
•
•
Application procedures
Proposal evaluation procedures
Time from project approval to contracting.
Project payment arrangements
Overall project administration and communication
Strategic orientation
Scientific and technological areas covered
'User friendliness' of FP7 compared with other schemes
105
Final Report - FP7 Energy Mid-Term Evaluation
How satisied have you been with the following aspects of programme implementation?
Very dissatisfied
Dissatisfied
Neither Satisfied or Dissatisfied
Satisfied
Very Satisfied
No Opinion
User friendliness' of FP7
compared with other schemes
Scientific and technological areas
covered
Strategic orientation
Overall project administration and
communication
Project payment arrangements
Time from project approval to
contracting
Proposal evaluation procedures
Application procedures
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The results from this question show that generally participants are satisfied or very satisfied
with these aspects of programme implementation, with combined satisfaction rates of over
60% in all but 2 of these areas. Participants were most satisfied with the scientific and
technological areas covered, the strategic orientation of the programme and project payment
arrangements, with over 70% either satisfied or very satisfied.
The 2 areas with lower satisfaction reported are the time from project approval to contracting
and 'user friendliness' of FP7 compared with other schemes. While the lower satisfaction
rates for user friendliness can largely be explained by a large proportion of respondents with
no opinion (n.b. over 60% of those with an opinion were satisfied), the time from project
approval to contracting appears to be the main area of dissatisfaction. Only just over 40%
were satisfied or very satisfied with the time from project approval to contracting, while 22%
were either dissatisfied or very dissatisfied. Although these figures appear high it is
interesting to compare them with the replies to the same questions from out ex-post of TREN
FP5 energy projects. In that survey 39% were ‘not satisfied’ with the payment arrangements,
significantly more than the 22% not satisfied in this survey.
Q20. Overall, how does this compare to your past experiences, if any, of the FP Energy
programme?
Final Report - FP7 Energy Mid-Term Evaluation
106
Overall, how does this compare to your past experiences, if any, of
the FP Energy programme?
Much worse
Worse
No Change
Better
Much better
Of the 119 respondents with past experience of FP Energy funding over 57% felt their
experience this time was better, and for 6% much better. So around 2/3 of respondents
believe their experience of programme implementation has improved. Around 30% believed
there had been no change from their past experience. Around 6% believed their experience
was worse than past experiences, but not believed their experience was much worse. These
suggest the majority view among participants that their experience of the programme is
improving, whether this is down to their own learning from past experience or improvements
in programme implementation is unclear from this data.
Summary
The findings of the survey indicate that participants in the FP7 Energy programme typically
spend between 20-45 person days each in preparing their application and meeting the
management and administration requirements of the programme. For many the burden is
lower but for a small minority of participants (around 10%) the time required is over 100
person days.
Overall the programme participants appear broadly satisfied with the various elements of
programme implementation and particularly with the scientific and technological areas
covered, the strategic orientation of the programme and project payment arrangements. There
was one clear area where satisfaction was much lower and this was the time from project
approval to contracting. For those with prior experience with the Energy Framework
Programmes a majority believe that their experience of FP7 is better than in the past and only
a very small minority believe it is worse.
The consistency in categories and questions asked here will allow further analysis through
cross-referencing the findings of this survey against previous FP evaluation surveys, to chart
any improvement or decline in programme efficiency over time.
107
Final Report - FP7 Energy Mid-Term Evaluation
4.1.4
Results and Outputs
The questions in this section were designed to give insight into the nature of expected results
and project outputs. It is important to have views on the expected outputs at project level as
these should correspond to the programme level intervention logic. Looking beyond this to
the potential scaling up or global impacts a project could have is also an important objective
and therefore evaluation criteria. This is particularly relevant in respect of the top level EU
and FP7 Energy objectives. Questions 21 and 22 provide insight into the types of expected
outputs, these are key factors in evaluating overall programme effectiveness, efficiency,
utility and distributional effects. Questions 23 and 24 explore the potential energy saving
and/or CO2 reduction benefits of projects, and also the potential for the scaling up of these
benefits.
Q21. What specific types of output(s) is your project likely to produce?
Response
Percent
Response
Count
Prototype, Demonstrator or Pilot
69.5%
146
New or improved process or technique
65.7%
138
Publication in refereed journal (peer reviewed)
50.5%
106
Conference
46.2%
97
New or improved product
41.4%
87
Policy and/or technical advice or recommendations to
external bodies
40.5%
85
Exploitation product (e.g. patent, copyright, trademark,
license, registered design)
29.0%
61
Production of electronic outputs (please specify reports,
data sets, codes, shareware, other software via web,
CD-ROM etc.)
29.0%
61
New or improved service
24.3%
51
Other (please specify)
12.4%
26
Answer Options
Other
Sharing of knowledge and experience, building of networks
Others were provided but have not been reported in order to protect anonymity.
The results of this question showed that almost 70% of respondents expect to produce a
demonstrator, prototype or pilot as a result of their project. Around 65% expect their project
to produce a new or improved process or technique, while over 50% expect their project to
produce articles published in peer reviewed journals. Among the other expected outputs an
interesting result was that only 29% expect an exploitable product to be among the project
outputs. There are a variety of reasons that this may be the case, which may be worth
exploring further. Among the other types of outputs listed by respondents the key outputs
were informational – items such as training, guidelines, country reports, case studies and
standards.
It is interesting to compare these expectations of results with the actual results reported by
FP5 participants. In the FP5 survey some 40% had published in a peer reviewed journal, so if
Final Report - FP7 Energy Mid-Term Evaluation
108
the 50% who expect this to occur do achieve it this will be an improvement. If 30% of the
products are able to lodge a patent this will be a significant improvement from the actual
results of FP5 where only 13% did this.
Q22. In your expectations, what will be the main technical result(s) of your project?
Response
Percent
Response
Count
Bring demonstration technology closer to large scale
deployment
56.2%
118
Validation of energy technology concept through
demonstration at full scale
51.9%
109
Improvement of existing commercial energy
technologies
32.4%
68
Integration of new and existing technologies e.g.
Buildings integration of Solar PV
31.9%
67
Advances over state of the art – i.e. completely new
technology, significant breakthrough
25.2%
53
Other (please specify)
8.1%
17
None
3.3%
7
Don't know
2.4%
5
Answer Options
Other
assessment of new technologies for long term emissions abatement
Others were provided but have not been reported in order to protect anonymity.
In narrowing the scope of the question to the expected technical outputs the 2 most expected
outputs relate to demonstration technologies, which as Q6 showed is the most numerous type
of project funded. Over 56% of participants expect their project to bring a technology closer
to large scale deployment and around 52% expect their project to validate a technology
concept through full scale demonstration. Around 30% each feel their technical outputs will
be improvements of existing commercial technologies or the integration of new and existing
technologies. Just over ¼ of participants feel that their outputs will represent an advance over
state of the art, representing a significant breakthrough or new technology. Interestingly
around 5% of respondents expect no technical outputs or don't know what their outputs will
be.
The shift in the nature of the programme towards demonstration is apparent when comparing
these answers to those recorded for FP5 where it was the case that many more of the projects
thought they had produced an advance over state of the art.
Q23. Are you able to estimate the CO2 emissions savings and / or the energy savings
your project itself will enable (if successful)?
Answer Options
Yes
109
Response
Percent
Response Count
30.0%
63
Final Report - FP7 Energy Mid-Term Evaluation
70.0%
No
If Yes, can you provide an Approximate figure (tonnes of CO2 or GWh)
147
49
Approximate figures
60 Mte/annum of CO2
CO2 savings 500 to 750 megaton per year in 2050
Not yet, too early
5000t CO2/y
70% GHG emissions reduction in the Hybrid concept (existing facility plus 2nd generation)
shall provide the reference economic evaluation method & tool for CCS projects: without a good
tool, no CCS.
5 GWh, assuming 10 year lifetime
No
12 million tonnes of CO2 per year for Sweden, about 36 million tonnes for EU
112 MWh/day
CO2 emission savings: - 2 ton CO2/ton EtOH
50'000 tCO2 per year per plant (of 350MW size)
0,6 GWh
~ 90.000 tn/year
at least 1 000 000 tonnes CO2
At end of project when sustainability modeling tools are ready
54.000 t CO2 per year
~50000 tonnes CO2/year
13000 tons/year
640 tonnes of CO 2 / year
900000 t/year
25000 CO2-e/year at full capacity
Demonstration plant 30-40 000 tons, dissemination of the technology, millions of tons
7000 ton/year
3000 tons CO2 / year
there will be very high savings but it is too early now to provide figures
4000
Up to 40% energy and CO2 savings compared to conventional technologies.
80% per tractor
In commercial plant over 90% of CO2 produced will be captured
Presented by VTT in the application
"Well-to-wheel" analysis performed for full-scale technology (~95 CO2 reduction), but not for pilot
scale size.
Tens of Millions of tons when applied
Final Report - FP7 Energy Mid-Term Evaluation
110
not yet, it depends on the technology
5000CO2 t/a
1800 tons
Reduce the carbon footprint of Wave Energy Converter manufacture/installation by a factor of
50% due to the anticipated increase in annualised average electricity outputs mean fewer devices
will be required. The number will depend upon the footprint of each specific device.
10 000 000.00
Unknown yet
4000
40.000 tonnes of CO2/year, 6,2 tonnes/person 6-7.000 new CO2-neutral citesents
30 million tonnes CO2 / a
5 tonnes per year
I would prefer co-ordinators answer to this
21,3 GWh/year for the demonstration in Italy
0,7 tCO2 avoided per MWe
200000 tonnes of CO2
3% total energetical fossil fuel reduction (e.g. about 2 Mio. t Diesel/a in Germany)
N/A yet
The results to this question show that only 3 out of 10 respondents were able to estimate what
the carbon or energy savings that may occur as a result of their project. Of the 49 respondents
that provided an estimate of some form, only around 20 of these provided any useful
quantitative estimates. For only 10% of programme participants to be able to estimate in this
way is somewhat worrying, especially considering that reducing carbon emissions is one of
the key, perhaps the most important, high level objectives of FP7 Energy.
Further analysis of the responses for this question shows that of the 30 project co-ordinators
that responded to this question, 13 of them (43%) were able to estimate the CO2 reductions or
energy savings that would result from their project. Of the 191 partners that responded, 49
(26%) were able to estimate the CO2 reductions or energy savings. This appears to show that
project co-ordinators have a better understanding than project partners of the likely outcomes
of their project in terms of CO2 reduction or energy saving, though less than half were still
able to provide an estimate.
Q24. What is your view of the approximate EU and global potential scale of replication
of your project?
Answer
Options
x1-10
x10-100
x100-1000
x1000 +
Don't
Know
Response
Count
National
80
27
19
12
66
204
EU
26
59
27
31
66
209
Global
14
37
32
45
74
202
111
Final Report - FP7 Energy Mid-Term Evaluation
14
Other (please specify National / EU / Global)
Other
Not applicable: our project was a concerted action aimed to share existing information
Others were provided but have not been reported in order to protect anonymity.
This question also highlighted a gap in knowledge among participants, with around 35% of
respondents in each case unsure (don't know) of the replication potential of their projects.
There were indications from respondents that some did not understand the question. Of those
that were able to estimate the replication potential of their project the answers showed that
most respondents (58%) felt that nationally the replication scale of their projects was only x110. At the EU level it was felt by over 40% of participants that replication scale was in region
of x10-100, though over 20% put this at x1,000+. At the global level over 35% felt that
replication scale was over x1,000+, and over 25% each in x10-100 and x100-1,000.
The survey returns in questions 23 and 24 are not sufficient to provide a clear view of overall
potential carbon or energy savings of FP7, given the survey audience there is potential double
counting and other factors to consider. These questions though potentially do give insight
into some important issues around projects not having clear estimates of the carbon or energy
saving potential of their project or the wider potential of the project outputs.
It is probable that a number of respondents did not understand the question, this in itself is
worthy of comment as it suggests that the idea of the project being to ultimately reduce
carbon emissions is not the way in which they think about their project.
4.1.5
Project Impacts
These questions were designed to capture the expected longer term impacts of the projects
and although for most participants it will be early in their project life, they will still have
expectations of impacts. Evaluating the fit between these and the programme intervention
logic and objectives is important. In addition, aggregation of impacts at EU level in areas
such as high level EU policy, science and technology, commercial competitiveness and
environmental policy are also important evaluation criterion. Given the nature of the projects
supported by FP7 Energy, i.e. relatively near to market, it is expected that there would be a
spread of benefits across the various areas, with the commercial elements expected to be
particularly relevant. These elements will be central to evaluating the programme against the
economic and competitiveness objectives.
In evaluating the broader impacts of the programme it is necessary to consider that at an
individual project level the responses will reflect low levels of significance but at an
aggregate level these effects may be multiplied.
Questions 25 and 26 explore the expected significance of the impacts of projects on
participants organisations in both the scientific/technical and commercial dimensions.
Question 27 further explores the commercial benefits, if any, of the FP7 Energy project.
Questions 28, 29 and 30 go beyond the organisational benefits expected from project
participation and explore the extent of wider impacts in EU and FP7 Energy policy and
objective areas.
Final Report - FP7 Energy Mid-Term Evaluation
112
Q25. How significant do you expect the impacts of the project on your organisation to
be in the following scientific/technical areas?
•
•
•
•
Enhanced reputation and image
Ability to carry out new activities or enter new areas
Enhanced ability to produce or deliver new products, processes or services
Other (please specify)
How significant do you expect the impacts of the project on your
organisation to be in the following scientific / technical areas?
N/A
Insignificant
Low
Medium
High
Don't know
Enhanced ability to produce
or deliver new products,
processes or services
Ability to carry out new
activities or enter new areas
Enhanced reputation and
image
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Other field of impact
Enhance cooperation with other EU research organisations
Others were provided but have not been reported in order to protect anonymity.
The results to this question show clearly that participants believe that their project will have a
significant impact on their organisation with over 80% expecting a medium or high impact, in
each of the three scientific and technical areas, and particularly in enhancing their
scientific/technical reputation and image.
Looking more closely at the responses to detect any variation in response by whether the
organisation was a partner / co-ordinator, SME or Non-SME, or Private / Public shows that in
every case the project co-ordinators are more confident (though not by a lot) than project
partners in positive outcomes in the science and technical areas. The same is also the case for
public organisations compared to private companies. This is even more so in comparison
between public organisations and private companies, with the public organisations 15-30%
more confident of highly significant outcomes. In contrast to this, for SME's there is little
noticeable difference in expected outcomes, only in the case of enhancing the ability to
113
Final Report - FP7 Energy Mid-Term Evaluation
deliver new products, processes or services was a small difference found, with non-SME's
slightly more confident of medium significance outcomes.
Q26. How significant do you expect the impacts of the project on your organisation to
be in the following commercial areas?
•
•
•
•
•
•
Improved competitive position through increased market share or access to new
markets
Increased profitability
Enhanced productivity
Improved linkages with other organisations
Formation of new business entities
Other (please specify)
How significant do you expect the impacts of the project on your
organisation to be in the following commercial areas?
N/A
Insignificant
Low
Medium
High
Don't know
Formation of new business
entities
Improved linkages with other
organisations
Enhanced productivity
Increased profitability
Improved competitive position
through increased market
share or access to new
markets
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Other field of impact
Remark: we are a government (and hence a non-profit) organisation
Others were provided but have not been reported in order to protect anonymity.
The results to this question show a more mixed view of the potential commercial impacts of
the participants project involvement for their organisation, this is not unexpected with many
of the organisations involved being publically funded or non-profit universities or research
institutions. The most significant impacts are expected in improving linkages with other
organisations and improved competitive position through increased market share or access to
new markets, with 87% and 62% respectively, expecting impacts of medium or high
significance. The transnational and consortium approach of the FP7 programme would be
Final Report - FP7 Energy Mid-Term Evaluation
114
expected as a matter of course to improve organisational linkages, it is interesting to note that
participants believe this will have significant impact on their organisations. The widespread
expectation of significant impact on competitive position is also an interesting result,
suggesting the FP7 Energy programme will have impacts on high level competitiveness
objectives.
Set against this are more mixed expectations in respect of the significance of impact on
increasing profitability, enhancing productivity and the formation of new business entities. In
each of these areas the number of participants who feel that the impacts will be low or
insignificant is around 35% while those who believe the impacts will be medium or high is
around 45%. So although there is some bias towards significant impacts there is a large
minority of participants that don't expect significant impacts.
Further analysis on the basis of respondent type reveals the following against the various
commercial areas:
•
•
•
•
•
Improved competitive position through increased market share or access to new
markets – Project partners were a little more confident than project co-ordinators of
medium to highly significant returns in this area (76% to 61%), the same is also true
for SME's in comparison to non-SME's (70%-60%). A significant difference was
found between private companies and non-companies, with private firms much more
confident than non-companies of medium or highly significant impacts in this area
(71% to 47%).
Increased profitability – Project partners were much more confident than project coordinators of medium or highly significant impacts in terms of profitability (60% to
32%). SME status had very little impact on the responses. In this case companies
were slightly more confident of medium to highly significant impacts than noncompanies (50% to 40%)
Enhanced productivity – there was little variation from the total proportions within or
between the subgroups in respect of productivity.
Improved linkages with other organisations - there was little variation from the total
proportions within or between the subgroups in respect of improved linkages with
other organisations.
Formation of new business entities - there was little variation from the total
proportions within or between the subgroups in respect of the formation of new
business entities.
Overall, it can be said that project partners are more confident than project co-ordinators of
significant commercial impacts in respect of improving competitive positions and increasing
profitability. A similar difference is also noted between private companies and noncompanies, with private firms more confident of significant positive impacts against these 2
commercial objectives. SME status appeared to have little or no effect on the respondents
expectations for commercial gain. In respect of productivity, linkages with other
organisations and the formation of new business entities there was very little variation based
on sub-groups.
Contrasting these results with the similar analysis for Q25 appears to show that SME status
has very little effect on the expectations of impacts of involvement in FP7 in relation to
science, technical or commercial goals. A difference that is noted is that project co-ordinators
115
Final Report - FP7 Energy Mid-Term Evaluation
seem more positive about scientific/technical impacts than commercial, while the reverse is
true for project partners. As would be expected given the classifications, non-companies
(public) organisations are more confident of significant scientific/technical impacts, while
private firms are more confident of significant commercial impacts.
Q27. What type of commercial impact(s) do you expect your project to enable in the
following markets?
None
Improved
market
share
Improved
profits
New
partners
to work
with
No direct
commercial
return
Don't
know
Response
Count
Local
47
35
28
64
40
36
205
Regional
36
40
36
81
35
34
204
National
18
65
46
98
26
30
207
EU
13
70
46
120
23
29
208
Global
17
51
36
80
26
55
201
Answer
Options
The results show that at the Local through to the EU level around 15% of participants don't
know what the commercial impacts of their project to be, at the Global level this increases to
over 27%. Of those that did have a view on commercial impacts over 25% expected no
impact at the local level, 20% none at the regional level and around 10% at all other levels.
Among these, the EU level was the level with the lowest number (7%) that expected no
commercial impact. The results here are broadly similar to those who expect no direct
commercial return from their project involvement, which is a logical match.
The most widely expected commercial impact was that project involvement would provide
new partners to work with, this was particularly expected at the EU level (58% of
respondents). The next most expected commercial impact was for improved market share but
with only 17-34% of respondents expecting this at the various levels, again with impact
expected to be greatest at EU level. Improved profits at the various levels were only expected
by 14-22% of respondents.
The results here point to commercial benefits being somewhat limited beyond improving the
potential for partnership at various geographic levels.
Q28. How significant do you expect the wider impacts of the project to be against the
following high level EU policy objectives?
•
•
•
Employment
Education
Quality of life and health and safety
Final Report - FP7 Energy Mid-Term Evaluation
116
How significantdo you expect the wider impacts of the project to be
against the following high level EU policy objectives?
N/A
Insignificant
Low
Medium
High
Don't know
Quality of life and health and
safety
Education
Employment
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The results of this question show that against each objective over 50% of those surveyed
expected the wide impacts of their project to have an impact of medium or high significance.
The expectation of an impact of this size is highest in respect of Quality of life and health and
safety at 72%, while lowest for employment at 56%. For each of the 3 objectives the
expectation of medium impacts was the same at around 35%, the main difference arises in
expectation of high impacts. In each case around 7-10% of respondents didn't know what
wider impacts their project may have against the high level objectives.
Q29. How significant do you expect the wider impacts of the project to be against the
following science and technology related EU policy objectives?
How significant do you expect the wider impacts of the project to be
against the following EU science and technology related policy
objectives?
N/A
Insignificant
Low
Medium
30%
40%
High
Don't know
Enhancing EU Energy
technology leadership
Development of world class
expertise in a research area
Enhance skills of RTD staff
Develop international science
and technology co-operation
Improved public acceptance
of new technology / approach
Equal opportunites for all and
gender balance in science
0%
10%
20%
50%
60%
70%
80%
90%
100%
The results from this question show that for every objective, except Equal opportunities for
all and gender balance in science, over 75% of respondents believed that their projects wider
117
Final Report - FP7 Energy Mid-Term Evaluation
impacts would be of medium or high significance. This points to a very strong belief among
respondents that their projects will make significant contributions to achieving the EU's
science and technology policy objectives.
Q30. Do you expect your project to have any benefits in terms of the following areas of
environmental policy and what is the scale of this benefit?
Do you expect your project to have any benefits in terms of the
following areas of environmental policy and what is the scale of this
benefit?
Disbenefit
None
Low
Medium
High
Don't know
Reduced CO2 emissions
Reduced impact of energy
production/use/waste
Reduced impact on
biodiversity, soils, forestry
Increased share of new and
renewable energy source
Reduced energy
consumption / demand
Improved energy efficiency
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
It would be expected that the energy projects funded by FP7 will have positive impacts on the
environment. The results to this question highlight that in all areas, but to varying degrees,
respondents believe their project will have environmental benefits. Importantly in the area of
reducing CO2 emissions, over 85% of respondents expect their project will have medium or
high benefits, with almost 68% of the total believing the benefits will be high. This is very
strong backing for an expectation of carbon savings and contrasts with the findings of Q23
and Q24, highlighting that while estimates of savings and potential for scale-up are not clear
to respondents, they expect their project to result in large scale CO2 reduction benefits.
Similarly high proportions also expect their projects to have significant benefits in improved
energy efficiency and increasing the share of new and renewable energy sources.
Interestingly 3 respondents believed their project would be a disbenefit to the new and
Final Report - FP7 Energy Mid-Term Evaluation
118
renewable energy sources objectives, closer investigation reveals that these respondents are
projects under the Clean Coal activity area.
More mixed views on environmental benefits were found against the reduced energy
consumption / demand and reduced impact on biodiversity, soils and forests objectives. In
each over 50% still expected medium to high scale environmental benefits but at the same
time over 35% believed that the environmental benefits would be either dis-benefits, no
benefits or low benefits. The reasons behind these lower expectations is potentially an
interesting area for further analysis.
Overall there was evidence of widespread expectation of significant environmental benefits
resulting from the projects.
4.1.6
Other comments
Question 31 asked respondents if they were willing to take part in an in-depth project
profiling, over 50% of respondents indicated they would be happy to do this.
Q32. Do you have any other comments on your experience with the FP7 programme
and DG ENERGY?
This question elicited over 30 responses as highlighted below.
Any other comments.
I find that in some cases the delays in payment is much too long.
I also found (in one project unrelated to the
...
Others were provided but have not been reported in order to protect anonymity.
4.2
Past Beneficiary Survey
This shorter and smaller survey was designed to examine the reasons that participants in the
FP6 Energy programme have not been involved in the FP7 Energy programme.
The first 3 questions established the respondents name, organisation and FP6 project,
Question 4 confirmed that they participated in FP6 Energy but not FP7 Energy. It was also
asked what the reasons for this were.
Q4. Can you please confirm that your organisation participated in the FP6 Energy
Programme but has not participated in the FP7 Energy Programme.
Response
Percent
Response
Count
Yes, participated in FP6 Energy but not FP7 Energy
80%
32
No - Please specify reason below (e.g. we did not participate
in FP6 Energy, we are participating in FP7 Energy or a nonEnergy FP7 project)
20%
8
Answer Options
119
Final Report - FP7 Energy Mid-Term Evaluation
answered question
Response
Reason
NO
We are part. in FP7
FZJ is largest research centre in Europe: other unit might participate
We are also participating on FP7 through the PV TP-SEC/ Grant agreement no.:
241377
We participate both in FP6 and FP7 projects
We are just under negotiation with a FP 7 Demoproject
We are participating in non-Energy FP7 project
we are looking to present a new proposal
Yes, Other
We are mainly involved in NMP
Others were provided but have not been reported in order to protect anonymity.
Q5. Please rate the following factors as reasons for not applying to the FP7 Energy
programme.
•
•
•
•
•
•
•
•
•
•
•
•
•
No appropriate project has arisen
Organisational focus has moved away from R&D
Now focussing on private sources of R&D funding
Now focusing on national (public) sources of R&D funding
Now focusing on other EU sources of R&D funding
No FP7 funding in your area of research
Insufficient funding available from FP7
Scale of FP7 projects now too large for your requirements
Scale of FP7 projects now too small for your requirements
Unable to find trans-national partners
Requirement for a consortium approach is a disincentive
Experience of FP6 is a disincentive
Other (please specify)
Final Report - FP7 Energy Mid-Term Evaluation
120
40
Please rate the following factors as reasons for not applying to the FP7 Energy
programme
Don't Know
Not a factor
Minor factor
Major factor
N/A
Experience of FP6 is a
disincentive
Requirement for a consortium
approach is a disincentive
Unable to find trans-national
partners
Scale of FP7 projects now too
small for your requirements
Scale of FP7 projects now too
large for your requirements
Insufficient funding available
from FP7
No FP7 funding in your area of
research
Now focusing on other EU
sources of R&D funding
Now focusing on national
(public) sources of R&D funding
Now focusing on private
sources of R&D funding
Organisational focus has
moved away from R&D
No appropriate project has
arisen
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Other (please specify)
Too much bureaucracy: waste of time and resources: major factor
We have applied for two projects unsuccessfully are two other proposals are being considered. Very few
possibilities exist within the calls.
No project opportunities for cities, no more Civitas or Concerto
I was not invited to participate to any project
Main reason for not participating was that either no appropriate topic were mentioned in the call or the
way how they were mentioned was not appropriate.
Lack of internal resources to work up ideas to project stage.
General economic situation in ….does not give room for participation.
The results of this question show that by far the most important major factor in not applying
to the FP7 Energy programme is that no appropriate project has arisen. A less major factor is
that there is no FP7 funding in their research area. A number of the issues were identified as
minor factors in not applying to FP7 energy including a focus on national (public) sources of
R&D funding, that insufficient funding is available from FP7 Energy, finding a transnational
partner and the requirement for a consortium approach. Some items were specifically
identified as not a factor in not applying for FP7, particularly the scale of FP7 being too small
but also organisational focus moving away from R&D and focusing on national funding.
121
Final Report - FP7 Energy Mid-Term Evaluation
Interestingly, 60% of respondents said their experience of FP6 was not at all a disincentive to
apply for FP7 funding, through around 20% said it was a minor factor and 15% a major
factor in not applying.
Overall the most influential reasons for not applying appear to be based around specific
project details such as appropriateness to the organisation or projects research areas called.
Some operational elements such as finding partners and needing a consortium approach were
highlighted as minor factors. Previous experience of the Energy FP did not appear to act as a
negative factor to the majority of respondents.
6. How likely do you think it is that your organisation will participate in a future FP7
Energy programme?
How likely do you think it is that your organistation will particpate in a future
FP7 Energy programme?
Don't Know
Very Unlikely
Unlikely
Neither Likely or Unlikely
Likely
Very Likely
Likelihood of future FP7
participation
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The results of this question show that over 55% believe they are likely or very likely to
participate in a future FP7 Energy programme. While around 13% thought it unlikely they
would participate in a future FP7 Energy programme no respondents felt it was very unlikely.
Q7. Do you have any other comments?
Any other comments
I think that if my city could implement in FP7 Energy - investments project, not research like is now, it
would be more interesting for my authorities
Others were provided but have not been reported in order to protect anonymity.
Final Report - FP7 Energy Mid-Term Evaluation
122
5 List of Consultees
The expert panel members are as follows:
• Garry Staunton – head of low carbon innovation and the Carbon Trust.
• Collette Lewiner – Cap Gemini Energy and Utility Director
• Jesus Isoird – as a representative of Euroace (employee of Accione – Spanish
construction and energy developer)
• Ronnie Belmans – University of Leuven
• Emilia Panayotova-Björnbom, Royal Institute of Technology, Stockholm (retired)
The stakeholders consulted are as follows:
Commission
Stefan Tostmann
DG Energy Head of unit
Glyn Evans
RTD Deputy head of unit K1
DG Energy
Ms Constantina Filiou,
DG Energy
Mr Roberto Gambi,
DG Energy
Mr Alexandros Kotronaros,
DG Energy
Mr Kyriakos Maniatis,
DG Energy
Mr Pietro Menna,
DG Energy
Mr Jose Riesgo Villanueva
Technology Platforms
Mr Fichaux, Ms Radvilaite
Mr. Landolina
Advisory Group on Energy
Ms. Haug
Ms. Bach
Programme committee
Mr. Martin
Mr. Vaucher
EWEA
Eurec
University Hohenheim
Arsenal research
DECC – UK Government
Euresearch
The FP Projects chosen for in depth interviews are as follows:
Project Acronym
Interviews
Co-ordinator
Partner
1 METAPV
3E. (Be)
2 SOLASYS
Fraunhoffer(De)
BP (UK)
3 STANDPOINT
Wavebob (UK)
4 LED
Abengoa, Spain (ES)
TNO (Be)
5 OPTIFUEL
Inensys (De)
-
123
Final Report - FP7 Energy Mid-Term Evaluation
6
7
8
9
10
ALONE
MEDIRAS
DECPO
ECCO
SEETSOC
Florence University (It)
Franuhoffer (De)
ENEL (It)
Sintef (Norway)
City University (UK)
Final Report - FP7 Energy Mid-Term Evaluation
124
DLR (DE), Eurac, Solitem (De)
PSE (De)
Statoil (Norway)
NTU (EL), EMS (Serbia)
6 Stakeholder Interviews
6.1
Evolution of calls in FP7
Our consultations have revealed a number of interesting points regarding the way in which
the calls under FP7 have evolved. The first call was described as being drafted with an FP6
mentality which led to a relatively high number of projects with a high number of participants
being supported.
There is no formal, specific evaluation of the projects supported by DG TREN under FP6 but
it is clear from the opinions gathered that there were a number of problems. These problems
can be summarised as follows:
•
•
The high number of projects and the high number of partners within these projects
resulted in a high level of administrative requirements for TREN, for example more
partners increased the chances of partner changes, which is an admin heavy task.
The high number of partners implied a low budget per partner which can increase the
likelihood of the project receiving little attention.
One external consultee raised the point that the decision (to reduce the number of areas open
and go for fewer, larger projects) could be justified solely for administrative efficiency
reasons. These related to the very poor perception among some beneficiaries and applicants
that developed regarding FP6. These could be summarised as stemming from lots of small
projects, with lots of partners, leading to diluted efforts which acted as a disincentive to many
applicants. The creation of this disincentive needed to be addressed and the approach adopted
has achieved this.
6.1.1
Fewer Projects with Less Partners
The switch to fewer projects, with fewer partners, which began in the second call and has
been retained since, raised a number of points.
A benefit which was mentioned by both Commission staff and external consultees was the
reduction in the administrative workload of commission staff. This brings a number of
positives, including:
•
•
•
The project officers have a more reasonable workload, so can do a better job.
The officers are able to focus more closely on the project, so learn about it, and from
it, more, for example each project should now get at least one visit from the project
officer.
Learning from the projects is useful as the project officers are involved in policy
development so this direct and practical involvement with research is of clear benefit.
125
Final Report - FP7 Energy Mid-Term Evaluation
The ideal situation, in terms of project workload on an officer, was described as a portfolio of
some large and high profile projects plus some smaller ones.
The generally increased scale of projects (average project value up from €6m to €15m)
means that the projects should be of sufficient scale to achieve high impact and visibility on
the research landscape. Larger projects are also more likely to attract attention within the
Commission which helps give profile to the policy goals and activity of DG Energy and can
even help in terms of speeding up Commission administration.
Industrial participation in the DG ENER part of FP7 is high, currently reported as over 70%
which is much higher than the rates achieved under previous FPs. The high level of industrial
participation has led to large sums of private funds being attracted to match the commission
funds. This is believed to build in a positive cycle in that a large cash contribution from a
private firm will increase and improve the management and input from the company, which
makes for better projects with less management input required from the Commission.
The major reduction in the number of topics open per call was also described as having the
following benefits. As the total budget remains the same, the amount per open topic has
increased. As the number of potential, credible, participants in a particular field is limited the
implication, and reality, of the situation has been larger budgets per project and larger
budgets per consortium member. This increase in budget is perceived as being attractive to
potential applicants. This issue is illustrated by the reduction in the number of bids and the
subsequent increase in the chances of a successful application.
One consultee, who is employed by a large company that has participated in FP7 projects,
agreed that fewer and smaller projects resulting in a higher success rate (for applicants) does
make the FP more attractive to large companies, as they see a better chance of winning and
the effort they expend is more likely to result in funding.
The improved chances of success when bidding for funds, narrowing from 1 in 10 previously,
to 1 in 5, or even lower, is more in line with the chances of success companies typically work
with when bidding for commercial contracts. This motivates better bids, particularly from
commercial organisations. Improved motivation also relates in part to the significant time
input required in preparing a bid of this nature, for example the need to identify
demonstration sites usually requires the agreement and buy in of a third party such as a local
authority.
6.1.2
Large vs. small projects
With regard to the question of a general approach of many small vs. fewer and larger projects
there were a number of interesting points raised. While it was agreed that in general projects
with lots of partners are more difficult to manage, there are some cases and target markets
where this approach is beneficial. For example in some building related projects the ideal
consortium would have a representative from each part of the long (in comparison to
renewable energy) supply / value chain. Although this implies a high number of partners
many will only have a small, but important, involvement
Final Report - FP7 Energy Mid-Term Evaluation
126
6.1.3
Mix of technologies in approved projects and calls
Those consulted raised the following points regarding the mix of technologies covered in the
calls and approved projects.
The alignment with the SET plan was described as a sensible arrangement and one that has
made the programme management more straightforward as it provides a justified framework
for structuring the calls.
The fact that each call description is presented as a combined document between DG Energy
and Research is intentional as it gives a picture of a coordinated and larger approach than
separating the focus areas between the two DGs.
Many of those consulted felt that the current approach of supporting large, demonstration
focussed projects presents an easier opportunity for renewable energy focussed projects than
for energy efficiency focussed projects. This was believed to partly relate to the nature of the
projects, for example a large scale installation of a new wind turbine is a simpler prospect
(one site and one technology) than installing multiple small scale efficient appliances. The
Concerto approach (considering multiple low carbon solutions on a spatial basis) was
mentioned as a way in which this issue had been addressed in the past.
With regard to energy efficiency, another project type which has been stopped is ecobuilding,
the main reason for this was understood to be that it is difficult to judge how one building
project is better than another. However there has been a recent call for proposals called the
E2B roadmap. This is a repackaging of FP7 money, branded as a response to the economic
downturn with a total value of €500m including €125m from DG energy. The call asks for
projects focussed on particular building sectors e.g. ‘a school of the future’ concept.
With regard to funding projects focussed on industrial energy efficiency, another problem
with these is that there is a perception that industry should be pursuing these opportunities
anyway, particularly as energy costs are so high, leading to a significant risk of a lack of
additionality. Despite this concern some projects are funded, about 20% of the project
portfolio. It was suggested that the ideal solution would be a cluster of projects to help get
scale and momentum.
6.1.4
Mix of Technologies and SET Plan Alignment
The question was raised of balance between the objectives, with the assumption that a
portfolio approach, where projects covering all potential areas of research are requested, will
allow comparison between the technologies and multiple approaches to the same technology.
This is a first step to 'picking winners' both within a technology area and between technology
areas. Such a portfolio approach also allows cross sectoral issues to be picked up.
This leads on to questions around the fundamental targeting of the programme, which
revealed a number of alternative approaches and opinions, some of which are energy specific
and some of which are more generic, though still relevant.
The consultee from the UK’s Carbon Trust (CT) provided an interesting comparison between
their approach and their perception of how the FP works. He described a high level split in
the approach, such that a programme could either focus on known areas of strength or
improving areas where the EU performance is currently weak, but improvements are
127
Final Report - FP7 Energy Mid-Term Evaluation
necessary / thought to be beneficial. The CT approach was described as being to offer a range
of schemes to fit the variety of R+D needs. This approach was felt to differ from that of the
FP which is perceived as starting from a final objective and working in a top down manner to
specify the research it wishes to support.
This consultee also raised the issue of the risks involved in funding a project and on
commercialising low carbon technology. He raised the point that the perception of risk is
industry is often not what public funders expect and that industry put a large weight on the
perceived risk at certain stages on the route to market, such as the classic ‘valley of death’
risk profile as innovations come close to market – at which point the public sector sees the
risk as diminishing but the private sector see it as very high. It is important that public
supporters of R+D are aware of this difference in perception and are willing to accept how
industry feels, and do what it can to help them. The CT aim to adopt a more private sector
view on risks, and as such, the vast majority of their staff have a private sector background –
and try to focus more on the issues of importance to the companies involved.
The additionality that public R+D support should have is two fold – enabling projects which
would not otherwise happen and speeding up others. For projects where the additionality is
‘speeding up’ the recipients are relatively passive in that they need help to do more of the
same. Where the desire is to accelerate activity the assistance needs to look at the key
barriers, often this shows areas where things should be happening but are not. The technology
accelerator approach, that the CT use, is designed to help fill such gaps by speeding up the
transition from university research to the market.
One consultee described the current situation in terms of technology choice in low carbon
research as still being at the ‘let all flowers bloom’ stage, i.e. the problem is so large and
important, and the potential solutions so diverse that it is unwise to try and focus on particular
technologies to the exclusion of others. Another reason not to remove any technologies and to
continue to support all is that the policy environment in terms of member state and industry
commitment to low carbon is still fragile. It is also the case that the technologies which
eventually emerge as the most important may not be those which appear best placed at the
moment, or those which are technically / theoretically the best, as this is how technology
development works.
With regard to the focus of the projects, one consultee stated that the focus on near to market
research was understandable, as industry is motivated by the prospect of profits and there is a
need to spur carbon savings as soon as possible. This needs to be balanced with a recognition
that innovation does not finish when a product comes to market. Ideally there should be some
funds available to support all stages of the research cycle, including product development.
This point is of relevance to reducing energy use in the buildings sector, as here research
needs to focus on the cost reduction of relatively mature technologies - i.e. more like product
development but still research. Industry needs to have confidence that they can look for
support to help them plan investments.
With regard to the split between RES and EE one DG officer made the comment that if this
split is done by project numbers it can be misleading as the large CONCERTO projects might
have been 3 to 4 separate projects in the past but the focus on larger projects has led to them
being combined. Analysing by funding can also be misleading as some of the efficiency
projects will have an impact on a lot of buildings despite being relatively low budget in
Final Report - FP7 Energy Mid-Term Evaluation
128
comparison to RES projects. The building specific calls have helped to keep the balance –
which is better than it appears.
In general terms, heat related projects are less supported than electricity related projects, if
the programme wishes to reflect the use of energy, this is an under representation. This also
relates to the fact that heat is not a traded and transported commodity like electricity.
Improving the efficiency of its use will require approaches that are attractive to consumers,
which has been something of a market failure to date. The technology platform on heating /
cooling should help develop such concepts.
Energy efficiency is more cross sectoral than RES in terms of the technologies being
developed. Much of the research / improvements available relate to controls and ICT and
these projects would typically not come under DG ENER. Energy use in buildings is complex
and many of the FP projects / calls are pursued in conjunction with other DGs, (2 calls of this
nature were mentioned). One officer raised the logical point that any relative (to renewables)
lack of energy efficiency demonstration projects, fundamentally relates to the lack of a
winning commercial business model for energy efficiency. The technology is mostly well
known and proven, but the economic models and incentives are still to be properly devised.
Some of the DG officers felt that ideally the programme should look to retain some balance,
within each project and across projects, between demonstration for commercial benefits and
scientific benefits, with the projects selected for the scientific benefits typically being more
cutting edge and riskier. This balance needs to start in the calls for proposals. On a similar
note one officer felt that it would be good to retain some of the budget to support smaller
projects. Another officer agreed that supporting some smaller projects reflects the industry
approach as they will carry out smaller scale testing prior to large demonstration.
With regard to supporting cross cutting technologies, a relevant technology area that was
discussed was Carbon Capture and Storage (CCS). It was pointed out that CCS projects are
also supported under the European Energy Programme for Recovery (EEPR) via grant aid of
€1bn to be invested in six CO2 capture and storage projects. This was described as being in
recognition of its major importance for the EU economy and also as an important bridging
technology to a future renewable and hydrogen energy system.
In order to help applicants clearly understand what ENER mean by ‘demonstration’ projects
it was suggested that the definition used in the call for proposals needs to be very clear. This
will avoid applicants submitting and presenting research projects as demonstration (not to say
that research projects shouldn't be funded). The attempts in previous calls to define
‘demonstration’ could be improved.
The EU has in the past lacked an overall strategy, in terms of which energy technologies to
support and how, the SET plan has been the first attempt to address this. Some alignment of
FP Energy funding with the SET roadmaps has already taken place and is likely this will
increase over time.
With regard to the balance between renewable energy and energy efficiency projects, the
point was raised that many technologies that can improve energy efficiency are already
developed and available, and that take up is lacking due to the market, rather than technology
issues. Renewable energy is also generally perceived as more glamorous than energy
efficiency.
129
Final Report - FP7 Energy Mid-Term Evaluation
With regard to the balance between demonstration and research, the view was given that it
was appropriate to focus more on demonstration. Given the large investment such projects
require it was to be expected that large companies should lead projects. SMEs can still take
part but they should not lead projects.
From the DG RTD perspective it was felt that DG ENER now managed the demonstration
projects and DG RTD the research focused projects. It is important that research is continued
to be supported as it drives innovation. With regard to putting both research and
demonstration projects under DG Energy, this approach would be resisted by DG RTD as
other DG's may also seek control of both aspects of the parts of the Framework Programme
of relevance to them, e.g. DG Environment.
One officer felt that the clear focus on demonstration had helped clarify the division between
DG Research and DG Ener – this split (in terms of distance from market) had not always
been obvious to external observers.
The alignment of FP7 Energy with the emerging priorities of the SET plan was generally
supported, with it being described as a way of adding credibility to both. The technology
platforms continue to support the alignment with the SET priorities.
However the change to fewer and larger projects is recognised as having a number of down
sides. These include:
• A perception of a focus on large companies, though in terms of SME participation
rates this does not appear to be true.
• Some past participants are unhappy that their opportunity to participate has reduced.
This is most often the case for smaller research institutes, which are more
concentrated in certain member states.
• There is a perception of decreased participation from new member states, partly due
to potential applicants there lacking the administrative capacity and experience to
submit bids.
• With regard to projects which are not focussed on demonstration, the overall
approach of less and bigger projects is less obviously beneficial. In these more
research based areas narrower calls will lose potential ideas and approaches. Bids of
this nature are often easier to put together than large demonstration type projects, for
example because there is no need to agree large sites, so applicants are more likely to
be willing to accept a lower chance of winning.
• It is recognised that, although improved, the flexibility in the financial arrangements
could still be better.
6.2
Quality of Supported Research
Views on the quality of the research supported were sought from Commission Officers and
project participants.
The global ranking of energy R+D in Europe was reported as an issue which is always
considered when assessing applications. The point was also raised that as there are now fewer
but larger) projects it becomes even more important, because the numbers of projects are
lower, so more is more pressure on each individual project to deliver something of global
significance.)
Final Report - FP7 Energy Mid-Term Evaluation
130
With regard to biofuel research and demonstration, the opinion was raised by a DG officer
that FP support has helped put the EU at the front of global research. However, it is difficult
to compare EU research with the position in the US, as their approach is different. The US
approach involves more funding and more concentrated efforts on a technology once it has
been selected, e.g. ten projects funded at one time in biofuels in order to increase the chances
of success. This gives more intense coverage in some areas than the EU, but less in others.
6.3
Increased support for International (non EU) partners
The question of whether the quality of projects would be improved if it was made easier for
non EU applicants to participate in projects drew a variety of comments. It was pointed out
that there are already a number of other parts of the FP, and other EU funding sources
specifically targeted on cooperation with partners from outside the EU. For example a joint
RTD/ENER topic, under the Specific International Cooperation Action, focused on EU
cooperation with India. There are also overseas aid programmes which can support energy
infrastructure development and it should be remembered that the FP has a very different
purpose to the aid budget.
On international partners, the benefits of additional inclusion were felt to vary by sector.
Where there are clearly global leaders who are not EU based, (e.g. biofuels in Brazil) there is
a good case, but where the EU leads (e.g. wind), or where the project will result in
developments outside of Europe the case is much less strong. For new sectors (e.g. wave) all
developers have the same initial goal (i.e. a working large scale prototype) so it is possibly
easier and more sensible to collaborate.
Where non EU cooperation is supported it should be done on an equal basis (in terms of
funding support) and care needs to be exercised with regard to leakage of technology and
expertise, especially to countries who are actually at the cutting edge themselves, e.g. current
coal generation plant installations in China are more efficient than in the USA and emit less
CO2 per kWh produced. Contractual complexity also increases with the inclusion of non EU
partners, which is another reason for limiting it.
In conclusion the most commonly held opinion was that international partners can improve
the quality of projects, but also bring risks and downsides so the current flexibility is a fair
compromise.
6.4
Measuring progress
The DG officers that were consulted raised a number of interesting points concerning the
measurement of project, and programme achievement.
The following measures of success were suggested, with the proviso that they can only be
measured some time after the project funding period has passed.
•
•
•
The speed and scale of technology deployment.
The establishment of European companies as world leaders in their field (as has
happened with wind energy).
Achieving / contributing to the policy objectives, i.e. replication and the CO2 savings
each replication achieves.
131
Final Report - FP7 Energy Mid-Term Evaluation
•
•
Improving the cost effectiveness of the technology, which requires clarity on the cost
reductions achieved, even if only a forecast.
Progress towards market for the technologies supported, this is more complex to
measure and requires an in depth analysis by field.
Fundamentally each project should be evaluated on whether it achieved its goals. For
example, for a project demonstrating a new plant – did it get the plant up and running? Did it
achieve its targets (e.g. % efficiency)? Has it reduced the production cost? Has it led to
replicate plants in the EU? However even if some (or all) of these goals have not been
achieved the project can still produce useful results, although this would require an in-depth
and project specific review.
It was generally recognised that these issues are not very well picked up in the project
monitoring process and that the applicants can be somewhat weak on specifying these
measures of success in their applications. There was some concern that the follow up of
projects was lacking and that the focus of attention is much more on launching and running
projects and programmes rather than looking for their longer term / ultimate impact. It was
suggested that metrics such as an index on related growth or 'build and run' related indexes,
would help address this and illustrate programme sustainability. It was suggested that ex-post
monitoring of impacts, could be a contractual obligation on the projects but this would imply
long contractual periods which would be difficult to enforce once payment (for the bulk of
the project) has been given.
On a general level it was also pointed out that attempting to predict project success in
advance is not really possible, partly because many of the factors which determine success or
failure are external to the project.
The use of patents and citations (resulting from FP funded projects) although interesting and
positive, was not felt to be a very strong or suitable indicator of success for the projects
funded. There were a number of logical reasons given for this opinion, as follows:
•
•
•
•
•
•
Spurious innovations can be patented if this is seen as an end in itself.
Citations can also be manipulated.
For the demonstration projects which are now the focus of FP ENER, it would be
expected that much of the innovation being demonstrated is already patented.
Sometimes the innovation in a project is the scale, which is not really a patenting
issue.
Some innovations achieved by programmes cannot be effectively or efficiently
patented.
Increasingly ideas are not patent protected anyway, due to the difficulty of
international patent protection.
The guidance documents on completing the application form ask about CO2 savings
(although there is no explicit question in the form) and this is an important factor in assessing
bids. Replication potential is also raised as relevant but it is recognised that this a more
complex issue than CO2 as it requires a commercial understanding of EU market. Some
officers also thought that estimating CO2 savings was also too difficult for applicants to
estimate. These markets vary a lot by technology, e.g. retrofit vs. new and by MS and even
region. For example with biomass the replication potential depends on resource availability –
this varies widely and at a very local level – and is therefore very hard to know for the whole
Final Report - FP7 Energy Mid-Term Evaluation
132
of the EU. Estimation of replication also requires some prediction of the future, which is
complex, though this could be improved, for example a view on future fossil fuel prices is
key to estimating replication potential for the next 10-15 years.
The majority of those questioned thought that the CO2 savings question is important as it
quantifies the impacts and clearly links to the priority target for ENER – 20/20/20. Even
though it is not necessarily an easy question to answer, applicants should be obliged to
attempt to do so, as long as the method to do this is clearly explained, a rational view taken
on any claims which appear extravagant, and it was not used as the only criteria for judging
applications.
It was also recognised that the ease and speed with which a low carbon technology becomes
commercial is strongly influenced by factors other than the technology, i.e. financial
incentives. This reflects the fact that at the present time most technologies targeted by the
programme are still not competitive with conventional (fossil) solutions, though some are
getting close (e.g. wind). The length of time that such financial incentives will be required is
very closely linked to the fossil fuel price. However the support provided by the FP will
improve the technologies and make them more competitive with fossil fuels. A number of the
officers described it as a balance to strike between supporting the ideas which are very close
to commercial and those which are higher risk. If projects are too close to market the support
becomes subsidising what the companies should do themselves, if too risky, the
implementation may be difficult to achieve and too few of the projects succeed. One officer
felt that there are good examples of the balance being achieved in the past, the example the
support given to the Concentrated Solar Power sector by the FP helped the sector get started
by funding three large demonstration projects, two of which were successful and a now
commercially successful company (Abengoa Solar) has been created.
The reality of market and technology interaction driving large scale replication and market
uptake of low carbon technologies is intended to be reflected in the SET implementation
plans. The key performance indicators (KPIs) being developed in these plans should offer a
clear future mechanism to help monitor this and as such it was suggested by a number of
consultees that they should be clearly referenced in future FP calls. Using the SET roadmaps
is the assumed intention for the future of FP Energy funding. This could also help avoid MS
duplication, but is hindered by the fact that not all MSs are actively involved with SET.
DG RTD also recognised a need to improve the measurement of project success, and hoped
that this would improve under FP8, for example with the utilisation of the KPIs developed by
the SET plan.
133
Final Report - FP7 Energy Mid-Term Evaluation
7 Project Participant Interviews
7.1
7.1.1
Relevance / Utility
Global Competitiveness
Most of the coordinators interviewed felt that FP7 put Europe at or near the top of the world
rankings in terms of energy research, and that it was important, indeed critical, for EU
competitiveness that that was the case, though one project co-ordinator felt Europe lagged
behind global competitors somewhat. He did comment, though, that FP7 was helpful in
somewhat redressing the imbalance.
That said, few felt they had a sufficient grasp of the overall balance of energy R&D funding
globally on which to make comparisons, though in their particular topics and fields of
expertise, most felt the FP7 projects were cutting edge. A number commented that on
renewables development and deployment, Europe was world leading, and particular
applications such as laser treatment of photovoltaics was global state of the art. Another
consultee also felt that the EU was currently leading in CCS but the US were now dedicating
large resources to this area and could overtake the EU. The comparison to the US was made
by another consultee who felt that the FP7 has smaller budgets than the US Department of
Energy research programmes, both for research and demonstration focussed projects. One
interviewee pointed out that there was a global increase in the level of interest in, and
resources directed towards renewable energy sources, which he characterised as a global shift
away from IT R&D towards renewable energy.
This opinion was endorsed by another consultee, a large commercial partner, who
commented that the absolute importance of EU or Member State funding of research had
reduced somewhat in recent years, as companies have a sufficiently strong commercial driver
to undertake research in house or with partners but without public funding, the benefits to
firms being greater control over the research, the ability to progress more swiftly (not having
to wait for a decision on funding award and subsequent contract negotiation), less
bureaucracy and, perhaps most importantly, the ability to keep highly commercially sensitive
work secret. The downside of purely commercial work is that results don't get shared. The
implication is that the terms for commercial partners to participate need to be made
sufficiently attractive such that strategically important work could be done within the context
of an EU public/private project.
One co-ordinator from an academic institute with extensive experience of FP projects
considers that the FP7 programme has been very effective at meeting the goals of energy and
R&D policy. It has been able to develop projects which are assisting in taking forward big
step changes in technological developments. The finance provided by the Commission
enables good sized projects to be devised but they feel the big ambitious goals are inhibited
by too much emphasis on demonstration and not enough on the R&D element. In addition,
Final Report - FP7 Energy Mid-Term Evaluation
134
because FP7 only provides 50% of the funds, trying to secure the remaining leverage from
industry or the private sector is viewed as very difficult. The particular technology in the
project for this co-ordinator is a long way from market entry and the project is relatively high
risk.
7.1.2
Measuring the Impact
There were very mixed views among the coordinators on the best metrics to use to measure
the impact of the programme. For every participant who suggested that one or other of the
proposed metrics (citation index, patents, overall R&D spend, and R&D spend per capita
GHG emissions) is a good indicator, three or four other participants cited drawbacks with
each one:
Ø Citation index might be fine for an academic partner, but not for a commercial
partner;
Ø Not all areas of research can generate the same level of patents – particularly those
projects that do not lead to a commercial product. Also, given the programme is
focused nearer to demonstration, there may be fewer patents emerging (particularly
where a project is focused on process improvements);
Ø Overall R&D spend was considered crude – it's not about the money, it's about the
results;
Ø As far as spend per capita GHG emissions, there are many factors affecting
emissions, and not all the change is driven by R&D. Also, this would favour large
scale, near to market technologies such as CCS / Nuclear but will work against those
such as ocean energy which are at an earlier point of scale and technological
development.
Ø Cost is not the only barrier to technology deployment, for some technologies public
acceptance is also key.
Picking up the theme of the importance of results, 6 interviewees said that the true impact of
the programme can only be measured by the downstream impacts, be that sales volumes,
number of production plants constructed, increased turnover of participating companies,
number of new start up/spin off companies created, or GWh of energy saved or GHG
emission reduction.
7.1.3
International Co-Operation
When questioned on international co-operation with organisations outside the EU,
coordinator interviewees were split into two camps – those that felt in would be a positive
development, and those who were against it. The main concerns of those opposed to
international co-operation centred on the leakage of intellectual property and manufacturing
capacity out of the EU. One of the commentators against co-operation argued there would be
more value from encouraging more cross-country co-operation spanning different areas of
Europe e.g. Scandinavia and the Balkans. He felt that Europe is very diverse and it would be
beneficial for different areas to work together and learn from each other.
Two interviewees in favour of international co-operation argued that "artificial geographic
limits" do not reflect the reality of teams and production capacity being placed anywhere
globally, even if the parent company is European. One argued that, given the global nature of
the GHG problem, it was important to have global co-operation on solutions.
135
Final Report - FP7 Energy Mid-Term Evaluation
One partner in a project focused on increasing deployment of renewable energy in South East
Europe felt that it was important politically as well as environmentally (given the large
renewable potential) to involve neighbouring and accession countries such as Turkey.
However, this partner felt that nothing would be gained by including non-neighbouring
countries such as India and China, a view echoed by the co-ordinator of another project, who
felt that opening up the programme to partners from the Far East would risk EU companies
NOT participating, as they would be wary of the risk of knowledge transfer outside the EU.
One partner felt there was a real benefit in keeping the programme within the EU as the
distance and time to deployment of successful projects would be minimised.
Conversely, one interviewee felt that collaboration with China would be very useful, given
they have expertise in implementation at large scale.
Of those in favour of international co-operation, a number commented in detail:
Ø "Co-operation with non-EU partners is very important. We are starting to have many
contacts with India, Russia, South America, Libya, Tunisia Morocco etc. The
Russians, for example, pay all the contribution costs while for example with India it
is a real co-operation project. The EU brings the intelligent design dimension while
the Indians can provide the workforce for developing and testing a component that in
EU would cost too much. Such examples would double the funding and at the same
time oblige the partner country to bring its added value to the project. But you need
to ensure it is a proper partnership – my experience in partnering with [a North
African country] was not a positive one."
Ø "The energy problems are global, so worldwide research should be complementary.
As suggested, foreign research (i.e. outside of the EU) is to have two governments
making agreements on the ways of funding. In the present FP7 programme, it is
possible to include a foreign institute, but only as a subcontractor. This means in
practice EU money is flowing externally. An example of such an alternative
structure is the IBEROEKA program, where the Latin-American authorities fund
research in Latin America while the European authorities fund European research on
their part accordingly (some specific calls include Japan).
Ø "The presence of international co-operation funds could improve the quality of future
projects. For instance, CCS (Carbon Capture and Storage) projects can address
problems which go beyond European borders. In this case a mixture of funds could
help the birth of cooperation projects which can put together international know-how.
For example, in the specific field of CCS an important contribution could come from
MIT (Massachusetts Institute of Technology), who are at this moment a very
important partner. But other important international institutions can make a valuable
contribution as well."
Ø A wider breadth of participating countries would help spread the costs, particularly
for very high cost technologies, such as CCS.
Ø Involving partner s from ‘hot’ countries (e.g. Egypt, Algeria, Morocco) is appropriate
for solar technologies which would have a large potential market in these countries.
Siting a demonstration project in these countries would be ideal when the potential
customers are there.
Final Report - FP7 Energy Mid-Term Evaluation
136
Ø There are MS level programmes which are designed to enable direct cooperation
with non EU partners. E.g. the Dutch BE-Basic programme regarding bio-refineries
which supports cooperation with Brazilian firms.
Ø One project reported difficulties with Moroccan and Jordanian partners.
7.2
7.2.1
Efficiency
The Application Process
In general, participants felt the application process was reasonable, though most had
comments on specific areas where the process, or their experience, could have been better.
Two co-ordinators admitted to using external experts to manage the application process. The
comments below illustrate the wide ranging nature of comments on the application
experience – with different applicants sometimes presenting opposite views on certain
aspects:
Ø "I found the application process straightforward with clear instructions. However,
once the project had been recommended for funding, the process became more
complex as it was necessary to make some adjustments to the original proposal (in
order to rectify errors) and a detailed justification was required. The support received
from the EC officer was outstanding, as this person helped to identify issues and also
resolve them."
Ø "I felt that the structure of FP7 projects is still very much focused on companies with
national organisations and does not reflect the business processes of companies that
are already organised on a European level. This results in an additional
administrative burden during the proposal phase as well as during the execution
phase, because the different R&D locations in other countries in Europe have to be
included as third parties. This is not a substantial obstacle but it does not reflect the
European concept and adds additional, unnecessary administrative burden without
yielding any benefits in return."
Ø "There were some problems with the participant identification code – mistakes in the
coding process had led to some of the participant numbers being linked to the records
of different companies. This was frustrating and it took some time to get a response
from the IT helpdesk and resolve the issue."
Ø "This was my first FP project (although the university has previous experience). My
initial task was to coordinate the proposal inputs and build the consortium. The
application process was not that bad. I had previously applied for US grants and
found the process for these was more difficult."
Ø "Meeting the partnership conditions was challenging – assembling partners in the
Balkan region can be difficult as there is interaction between business and politics.
Their approach was to make contact with the industrial partners via the academic
partner for that country. This worked well."
Ø "The new application process for FP7 is better than for FP5 & 6. It is much easier
doing it electronically."
Ø "The two-step procedure is helpful as it reduces effort for 1st stage. However, the
evaluation procedure for the 1st step is not clear as it doesn't give a score to show
ranking position (it's just a pass/fail)."
137
Final Report - FP7 Energy Mid-Term Evaluation
Ø "As a partner, I wasn't involved in application process itself, so can't comment. I
understand the Commission might want a mix of countries and indeed genders, but it
could be too rigid/limiting. I could envisage a project where, all the best partners
were from one country, but that wouldn't be allowed, so putting any form of artificial
constraints do limit the absolute quality of the project in terms of its net impact."
Ø "From my perspective, the process and structures related to FP7 are generally fine
and appropriate. It's important for FP to have some relatively strict, rigid conditions
which all projects need to meet, otherwise how else do you select projects to either
approve or reject?"
Ø "The application process was a severe burden. Not so much the actual application,
but collecting all the forms and documents is: it is considered to be a full-time job.
The consortium decided to subcontract an external agency to do this (hence, it is also
difficult to evaluate the application process from the view of the partners). It is
stressed that the negotiation part (after submission) is unsatisfactory as well. Because
of this, it is not known in advance what exactly will be funded at which conditions,
but the consortium has to sign a priori."
Ø "I felt the application process for FP7 was not too complex but we hired in somebody
who had the experience with these procedures. It is important to note that the
administrative process, especially the contract negotiations, takes a considerable
length of time – which seems incomprehensible for an SME like us that is applying
for the first time."
Ø "We found the application process fairly consistent between FPs but negotiation and
management of projects is very time consuming and demanding. The negotiation
process has very specific needs and this can get very difficult for SMEs. The move
from FP6 to FP7 with a unified database is an improvement."
Ø "The procedures are too long, complex and risky: it is not easy for a small firm to
work with FP7: a university has the capacity to put people to work for 2-3 months to
prepare a proposal while for small firms this is impossible. This kind of procedure
can last a year with the risk of not being awarded the project. There should be a preevaluation proposal of no more than 3 pages. The EC could already select the ideas
which are in line with the project. In this way if an idea is not likely to be selected
time and money will be saved. The pre qualification paper would help to select the
good ideas that could come from smaller firms."
Ø "Another problem is that we never know by whom we are evaluated and some
projects are not even taken into consideration while the same project presented for
similar calls receive very good ranking."
Ø "I did not feel the application process was too complex or rigid. In fact, stricter
requirements, such as having a signed consortium agreement before contracting
would have helped, as in our case, one partner withdrew after the project was
contracted with the Commission."
Ø "I see the application process as not too complex but of course there are too rigid
conditions. One of them is for example the number of partners required. My opinion
is that a proposal if well prepared and focused on technological improvement could
be presented even to only one company instead of a large consortium. Provided that a
strong commitment to dissemination and to publish results to the European
Commission is assured (of course this must be secured by the European rules of the
programme)."
Ø "We did not find the time to contract a problem. Rather, we felt that this was more a
pressure from the Commission side with achieving 2020 goals in mind. In certain
Final Report - FP7 Energy Mid-Term Evaluation
138
rapidly changing sectors e.g. Solar PV, a delay in contracting could be problematic in
terms of being left behind and that there was a need to act fast in these cases."
Ø A two stage application process would reduce the risk of preparing a large and
detailed full application based on the relatively brief information contained within the
call for proposals, only to find that the application was not in line with Commission
expectations / requirements.
Ø It is become harder to gain access to Commission officers prior to submission.
Ø The application process is not too complex and the inputs required are reasonable.
One participant felt the financing regulations were too strict. Shared activities in various
Member States lead to an administrative burden; for instance (external) auditing of salaries is
required for each country. It was suggested that more flexibility in resource allocation (leave
it to the partners) should be pursued. Another suggestion was to have a dedicated project
manager at the side of the Commission (being present at meetings, knowing the content, etc.)
rather than an administrative officer."
7.2.2
Project Size
There were mixed views among coordinators about the move to larger projects, with the
balance slightly in favour of large projects. Among those in favour of larger projects, the
following comments were made:
Ø "There are no real downsides to larger projects. However, if the trend is more to
demonstration projects, then R&D centres will find it more difficult to participate as
can't find the match funding from own resources."
Ø "I prefer the funding of fewer but larger projects. It is possible to do more with a
larger project and get better, more meaningful results. However, a downside is that
there is now more competition – the larger grants, along with the current economic
crisis, make the programme more attractive to industry."
Ø "Larger projects are definitely more attractive, given the nature of the project e.g.
many technological firsts and physical scale of demonstrator – project has to be big. I
feel the FP7 budget overall is large enough, but were some technologies e.g. CCS,
Nuclear receiving more than their fair share? Wave energy should get more."
Ø "The reshaping towards larger projects is positive. The proposal quality is going to be
improved and consortia will be stronger. A consortium including 9 to 10 partners is
optimal (in the past, some consortia included even up to twenty partners, which is
considered too much)."
Ø "The reshaping of FP7 energy towards larger projects makes it more attractive to
industry and I think it generates higher profile projects. I do not see downsides,
though I cannot judge the influence on SMEs or small R&D centres."
Ø "I'm positive about the evolution into the direction of fewer, but bigger projects and
also the fact that you need fewer partners."
Ø "The budget is appropriate. A larger budget can produce a larger number of projects
and more new and technological target solutions. The European Union must
139
Final Report - FP7 Energy Mid-Term Evaluation
encourage MS to reinforce National R&D programmes and budgets in order to offer
further financial resources to the energy market (in particular the public funding
combined with the development and sharing of common strategic research target will
stimulate the mobilisation of industrial actors on larger projects)."
Ø Larger projects are preferred as the administrative burden (for applicants) stays the
same what ever the project size.
Ø Larger projects are more attractive to industry as the scale involved is more realistic
with regard to actual deployment.
Ø The project might well have been 5 separate, small projects under the old approach,
so one large project is more efficient and increases the chance and scale of learning
between the partners.
Those less keen on the shift to larger projects commented:
Ø "In terms of reshaping towards larger projects, one downside is the number of project
partners required to form an FP7 programme. The only impact this creates is an
extra administrative burden, which moves from the Commission and onto each
project. It is almost impossible for an FP7 project not to designate / appoint a
professional coordinator. The other key impact it creates (due to the number of
project partners) is it only takes one partner to slow down or even stall the project, so
more participants increases that risk."
Ø "Larger projects are not necessarily more attractive to industry. The important thing
is they must be relevant to their needs, and able to move quickly – size is not that
important. Larger does give higher profile projects for EU to shout about, but they're
not necessarily better. We were involved in a previous FP project – it was a huge
project, involving everyone under the sun. The overheads involved in organisation
were considerable. Likewise, communication within the project was a big challenge,
as was keeping the focus on the project goals – too many people doing their own
thing and going down their own little paths."
Ø "Small projects can be more effective and more focused – they generally work more
efficiently. You get much better interaction between participants. Also it's better for
the R&D centres – they actually do the research, rather than focus a lot of effort on
project management. As far as risk is concerned, it's not immediately obvious it's
more or less risky depending on size."
Ø "In future, the EC should refocus the FP7 programme towards smaller projects (with
a maximum of 5-7 project partners), with streamlined / standardised conditions or
terms of reference for approval. This would improve the efficiency of FP
programmes and enable a more effective and speedy approval and operation
process."
Ø The larger scale can discriminate against newer ideas as it is not feasible to go
straight to large scale demonstration with untried ideas.
And some saw both sides of the argument:
Final Report - FP7 Energy Mid-Term Evaluation
140
Ø "We need 2 streams of projects: (a) long term, very large projects (say €100M) on
visionary future goals, which need 5-10 year horizons. An example might be a totally
new battery technology needs longer. Within these large "framework" projects, there
can be smaller projects, as you can't map out a 10-year programme in advance in
detail, so it's got to operate more like a platform, within which there would be
specific projects and calls for proposals. (b) The second stream should be focused of
smaller targeted projects, say €1-10 M. The two stream approach means you don't
put all eggs in one basket (technology) but you enable step change progress for
longer term potential. Larger industry players will have the longer horizons. SMEs
of course won't – can't commit to 5+ years, but they can join individual projects. Risk
is inherent in both types, but these vary by the two streams."
Ø "As regards higher profile projects, I agree they should be part of the mix, but don’t
do it to the exclusion of smaller R&D."
Ø "I am unsure about the impact of the move towards funding fewer but larger projects.
Larger projects are more attractive to industrial partners. However, they can be more
difficult to review, quantify impacts and achieve cohesion/integration. Large projects
are still attractive to SMEs and they can bring unique skills to partnership. SMEs are
included in partnerships but it would be interesting to explore what proportion of the
funding they receive. On balance, I think it is necessary to find a balance between
large projects and small high-tech projects.
Ø "I believe the reshaping of FP7 energy into larger fewer projects has made it more
attractive for those planning demonstration projects but less attractive to more R&D
focused type projects. It hasn't put us off applying, though." (from an R&D partner)
7.3
7.3.1
Effectiveness
FP7 Objectives
Not all coordinator interviewees agreed that the most important objective of the FP7 energy
programme is the achievement of GHG emission reductions. Most recognised it as an
overarching ambition, but one which was a by-product of the R&D activity. A significant
minority thought the number one objective to be industry-focused – either strengthening
manufacturing capacity within Europe, improving competitiveness, or addressing security of
supply.
Whatever their views on the priority objectives, many all interviewees were agreed that
quantification of CO2 or GHG savings was either extremely difficult, or impossible, at the
application stage. Comments included:
Ø "It is not straightforward to produce an estimate of the emissions savings that a
project would generate. In general, this would require two simulations to be run to
predict the emissions with the project and also without, but this is not easy.
141
Final Report - FP7 Energy Mid-Term Evaluation
Ø "It's very difficult to quantify the CO2 savings as they arise downstream. Of course,
you can make an estimate of the forecast volume of sales as a proxy, but this is prone
to massive error."
Ø CO2 savings could be easily estimated for our project, requiring a combination of
technical and market analysis.
Ø "CO2 wasn't a requirement in application process. Still now, half way through
project, we can't put a figure on CO2, but can describe qualitatively (in the project
impact section). We don't think that CO2 should be part of the ranking process."
Ø "We don't know what CO2 savings are, and don't think it's that important to know
precisely. If using it as a proposal selection criterion, the problem is that without a
strictly controlled process for calculating the savings, you could create perverse
incentives as to how optimistic the outcome is to increase chance of success. "
Ø "Yes, CO2 savings should be included in ranking but would need clear procedure on
how to do calculations for CO2 – maybe independent of the proposers."
Ø CO2 savings are the most important objective, though there is clearly a debate on
how to best achieve these.
Ø The current level of focus in the application forms on CO2 is appropriate as it is too
difficult, due to the market variations and uncertainties, to predict the potential
savings in detail.
Ø "We consider it's almost impossible to record or estimate CO2 reductions – there are
too many uncertainties which could influence this result and distort the findings. The
institute have tried to meet internally to give consideration as to how to do it but they
can't agree on the best method to achieve this. It is also considered that there are
more appropriate measures such as (kWh)."
Ø "Each applicant needs to demonstrate a strong, clear argument of how each project
application could achieve GHG reductions. The problem for FP7 is that if nuclear
projects were applying for assistance, whilst they can show huge GHG reductions,
there are other risks (in the case of nuclear) which need to offset this connection. I'm
afraid I can't offer any suggestions as to how this could be alleviated but I'm very
aware of the problems for EC."
Ø "We tend to concentrate on the production rate outputs of technology systems e.g.
kWh – this is our primary objective. From a commercial perspective, the improved
efficiency, flow and operation of production systems from new energy efficiency /
sustainable energy sources is a much more persuasive argument. By concentrating
on kWh rather than CO2 savings the benefits are clearly visible, the market / product
potential looks more viable. We consider there is a very fine line between research
and supporting industry to develop new markets and products. It isn't so easy to
gauge future market success / market growth on CO2 savings made by project alone."
Final Report - FP7 Energy Mid-Term Evaluation
142
Ø "Personally, I don't think that being able to quantify CO2 savings was that important,
as these benefits were likely to flow anyway. What's far more important is the
potential for industrial success and to contribute to energy security."
Ø "I've noticed that CO2 savings is a focus of applications for national level funding,
but don't feel that GHG reductions provides a level playing field for technologies as
some can much more easily describe scaled savings e.g. nuclear, CCS, than a
technology such as wave that is still at early development stages."
Ø "The impossibility to estimate CO2 saving in many projects must be solved because
this could limit the possibility to monitor the possibility to reach the first objective
(GHG reduction). Of course the participants must be put into the condition to give
the correct answers to precise and concise questions. Putting more emphasis on this
issue is very important but I am not sure on what kind of indicator must be used to do
so."
7.3.2
R&D benefits Vs. Commercial benefits
Most coordinator respondents agreed the programme has a mixture of R&D and commercial
benefits, with the balance weighted towards commercial benefits. Predictably, academic or
research institutes put more emphasis on the R&D aspects, while industrial partners were far
more interested in the commercial benefits.
Ø "The focus also depends on the nature of the consortium/project – e.g. if a project is
led by an industrial partner then it would be expected to have a more clearly defined
commercial focus. Where the focus is on R&D, it will take more time to realise the
commercial benefits. In general, it is not possible to separate R&D and commercial
benefits as they are all part of the same process. However, commercial benefits are
generally part of the impact generated by the process of high quality research
activity."
Ø "Projects already place an emphasis on commercial benefits but that the programme
requires immediate rather than long-term effects. Generating important intellectual
benefits requires a more long-term approach."
Ø "It's a combination between R&D partner and commercial players. Main benefit for
R&D partner would be another follow-on R&D project! Of course there has to be a
commercial benefit to get partners on board. So need to get the balance right. Both
are needed."
Ø Given that many of the recipients of funding are researchers they will be more
confident of their projects achieving research, as opposed to commercial, benefits as
they understand research benefits more clearly.
Ø "Only way to reduce GHG emissions is if R&D is commercialised, so these are the
most important aspects of the programme. Purely scientific interest isn't a good
enough justification. It should be about commercialisation through the R&D
process."
143
Final Report - FP7 Energy Mid-Term Evaluation
Ø "In terms of commercial versus R&D benefits – it is a little too early to say for our
project but our institute's previous experience has shown that the commercial benefits
are much harder to achieve."
Ø "As regards the balance between R&D and commercialisation, we're very much
focused on the latter, as we need to attract commercial partners to the project."
Ø "R&D is really important. If we don’t focus enough on R&D when the market is
stagnant we won’t bring any money back. We should be careful not to mix the R&D
and the commercial part. The schema should be:
R&D => Demo-project => Review of product’s design => Commercialisation of the
product.
Nowadays we tend to pass from R&D directly to the commercialisation and even
taking money from R&D for the commercialisation. Japanese have developed the
R&D in the 1990s and only today they are commercialising the products. We should
bring back the manufacturing otherwise we will remain forever China-dependent."
Ø "I'm strongly convinced that commercial benefits should be at the heart of FP7
projects. Projects focusing on improving R&D capacity rather than commercial
benefits were getting things the wrong way round. Projects receiving FP7 funding
should be beyond the R&D stage and be demonstrating something that will create a
platform to market."
Ø "The commercial benefits must be always taken into consideration. If an R&D
project works (I mean it produces results), its results must take in commercial
benefits (or future commercial benefits). To solve this matter it is not necessary to
measure patents but it is sufficient to have complete evidence of the results of the
projects and of their applications from a commercial point of view. An example of a
question that an applicant could be asked to get this commercial view could be to
demonstrate in the next two years after the project – in a commercial phase – to give
evidence of the annual emissions reduction, annual energy saving, CO2 saving, etc."
Ø One new MS participant highlighted, and appreciated, the networking opportunities
that project participation had given him.
Ø One project participant suggested that willingness to pay for the results of a project
was a strong indicator of success.
7.4
7.4.1
Sustainability
Future focus for FP7
Suggestions for future areas of activity typically reflected the particular field of endeavour of
the coordinators interviewed. Areas put forward included:
Ø Smart networks or ICT-related developments;
Final Report - FP7 Energy Mid-Term Evaluation
144
Ø All aspects of PV development. In particular, to strengthen the EU PV industry, it
needs to be more competitive, so future FP calls should be more about manufacturing
efficiency, excellence, productivity, low cost, competitiveness etc;
Ø Batteries;
Ø Fuel cells, in particular advanced industrial manufacturing capacity - current
manufacturing is largely manual or only partially automated. EU has to compete
with manual Chinese manufacturing. We need EU knowledge on highly
industrialised production in new fields such as PV, fuel cells and batteries;
Ø In the future, synergy could be sought with the hydrogen technology. Fuel cell cars
require hydrogen that could stem from the gasification and shift conversion processes
as well. The development of electric cars may go in parallel;
Ø Solar generation from industrial processes;
Ø Explore the use of naphtha that is, alongside biodiesel, another product resulting
from the distillation process. Bio-naphtha requires upgrading to regular gasoline
standards and the upgrading process may be different compared to fossil naphtha.
This is already an R&D activity within this project but this might require further
R&D beyond the schedule of this project;
Ø Grid connection as a priority area;
Ø CCS;
Ø New production technologies and control systems - these could help to reduce costs
and improve efficiency. It is very important to continue to finance demonstration
projects;
Ø For future FP calls, the key issue was to move towards some kind of standard
approach to wave / ocean energy. Comparison was made with wind energy in the
early 1990s, where multiple turbine options were around and that eventually the
shakeout led towards a fairly standard design. A similar process is needed in wave
energy;
Ø A new area for subsequent funding may be on the raw materials side of
Lignocellulosic Ethanol development - collecting, handling and pre-treatment of
residues, CO2 capture using algae, etc demonstrations and first of a kind steps to
commercialise the technology may need different support. Another topic may be the
optimal use of by-products and finally making inquiries on the conversion of
materials containing higher amounts of cellulose;
Ø New solutions for off-shore wind;
Ø New applications of geothermal energy to low enthalpy (binary cycle);
Ø New solutions for hydrogen application;
Ø Marine renewable energy;
Ø Nuclear - new and more secure nuclear technologies, solve the problems of
radioactive waste, etc;
Ø In a world where the utilisation of fossil fuel is predominant, we need new and
definitive solutions to reduce emissions from thermoelectric power plants.
Ø A solar energy focussed call combining heating, chilling and power to reflect the
scale of potential demand for this technology in hot countries.
Ø Grid asset management, particularly in eastern Europe.
Other coordinator comments concerned the nature of the programme, rather than specific
technologies:
Ø In terms of future areas / technologies, the Commission should give further
consideration for potential FP calls to mix member states with developing
145
Final Report - FP7 Energy Mid-Term Evaluation
countries (outside EU). Given the focus of development in these countries and their
need for more sustainable technologies, there is an opportunity to build markets /
products with these countries. It is likely that this is where the greatest demand for
energy technologies will be but much of the production / innovation experience is
found in the EU. Currently, the FP7 programme doesn't enable this to happen as
extensively as it could.
Ø The new calls need to be less 'narrow / constrained' in the view of eligible projects.
Currently it is a little limiting and EC can be slightly 'blinkered' or risk adverse.
Ø Need to also focus on longer term technologies and fundamental research.
One partner in the LED LED (Lignocellulosic Ethanol Demonstration) project made two
important observations that R&D funding on its own was not the solution:
Ø Critical for the development of the second generation biofuels (and first generation as
well) is the stability of the legislative framework including long-term targets – this is
more important than FP7 funding. The 10% biofuel requirement for 2020 in the
Renewable Energy Directive is being loosened by some Member States. As a result,
various biodiesel companies went bankrupt in the recent past.
Ø Rather than fixed processes and complex administrative regulations, it is suggested
the Commission has open discussions with applicants on ideas and to establish
reasonable budgets and prices (i.e. what is a reasonable amount of money to achieve
a desired outcome).
7.4.2
Strategic Future of FP
The project coordinators were asked their opinions on the future focus and nature of the DG
Ener supported FP7 programme. One coordinator reported that he felt “the Framework
Programme should aim for some continuity in its work programme – building on the previous
call but retaining flexibility to develop areas of weakness in order to maximize the benefits.
FP6 appeared less focused and incremental and this may limit its impact. FP7 has been more
successful at getting stakeholders involved and identifying their needs.
There were a number of specific technologies and areas suggested for future support,
comments of this nature included:
•
•
•
•
•
•
There is potential for major innovation in energy markets, e.g. materials and ICT,
which have not yet been explored.
The Future and Emerging Technologies (FET) programme is a good idea and there
would be scope for a similar programme focusing on the energy market.
RES grid connection and the smart grid platform
Local energy generation and distributive energy sources.
Needs to be focused on encouraging EU manufacturing competence &
competitiveness. Most production lines are turnkey, so can be sold anywhere in the
world wherever the skills lie and wherever it's going to be most profitable.
Felt that FP funds should be directed towards projects that can deliver wealth
creation – in respect of this RES should be the main focus, as Europe has an IP lead
in many technologies and there is potential to expand the economic benefit of this.
Final Report - FP7 Energy Mid-Term Evaluation
146
There was some direct contradiction between project coordinators in terms of whether the
focus should be close to market or more fundamental research. The diversity of opinion is
shown by the following:
•
•
•
•
•
•
There should be more focus on research which is further from the market but this
cannot happen given the current proposal evaluation process which penalizes this
type of work for not producing short-term outcomes.
Large 10-year projects should be part of the mix, to address the mega trends.
For EU competitiveness, need to focus on manufacturing capability – so preference
for near-market focus, rather than further from market.
FP programmes should be stepping beyond just 'demonstration' and onto supporting
market stimulation, assisting successful projects in the short-medium term after
completion in order to avoid floundering and failure after FP funding ends.
There is currently a gap between technology and distribution systems, so the
programme should include applied technologies, i.e. more market oriented.
An element of the programme budget for smaller projects, which would help SMEs
and further from market technologies.
With regard to the ability of the FP to leverage in, and encourage more private sector R&D in
this field, one coordinator stated that “it is not possible in a free market to force private
energy companies to divert some profits into R&D”. He went on to state that “ The funding
of R&D must be a prerogative of EU and MSs which are working together with private
companies to find the best innovation for the future. In other words, the funds must come
from the government to try to induce private companies to put their know-how to find these
solutions.”
7.4.3
Longevity of Impacts
The project coordinators had the following comments on the probable persistence of the
project effects. One coordinator felt that dissemination is generally very good but
commercialisation can be more 'hit and miss,' depending on participation and how the outputs
link to the business of the industrial partners. Another coordinator was more critical stating
that he thought “ the programme will not really produce long-lasting effects for a number of
reasons including the fact that SME participation is not as strong as it could be and also
because there is a tendency to avoid high risk projects (which are likely to have a greater
potential impact)". A third coordinator thought that dissemination was good and that it was
easy to find information on projects if you look for it.
7.4.4
Improving Dissemination
The project coordinators were asked for their opinions and suggestions on how project
dissemination and long term impacts could be improved. One suggestion that was tested out
with a number of the coordinators was the idea of increased inter project networking. There
were a variety of contrasting views on the benefits that such an approach could bring. One
coordinator did not see any benefits from inter-project networking, citing his own experience
of trying to network four laser projects which failed due to a combination of competition
between the projects and a lack of synergy. He felt that existing networking via academic
channels and conferences was sufficient which was an opinion echoed by other consultees.
147
Final Report - FP7 Energy Mid-Term Evaluation
In contrast another participant thought there should be more inter-project networking and had
never been involved in any previous attempts. Another coordinator also thought that the idea
of joint conferences was a good one and would offer the benefit of increased networking
between projects. A third coordinator suggested a two-day workshop where all relevant FP7
energy projects were presented. A coordinator who had been involved in a joint conference
involving CCS researchers felt that it had been useful and was a good idea for certain
research areas.
With regard to conferences another coordinator stated that discussion was limited to
published results due to confidentiality and IPR protection, though this was the accepted
norm and he thought that any requirement on FP projects to network with each other would
be a somewhat artificial approach. The retention of some IPR by project partners was seen as
a very valuable benefit from participation and one that should not be lost in efforts to more
widely publicise results. However another project participant commented on a reluctance of
industrial partners in his project to share knowledge and learning with other partners, noting
that some experiments within the project had been done which support a particular company's
agenda rather than the project as a whole and details of the particular application were not
being shared. He felt that there was a need stronger guidelines at application stage. The issue
of disclosure was described as complex, as if FP7 requirements are more towards full
disclosure, it would put partners off participating from fear of losing their IPR. However, he
thought that the balance doesn't seem to be quite right at present. In contrast a participant in
another project reported that dissemination was proceeding well and IPR protection had not
been an issue.
One participant raised the idea of additional follow on funding to assist in dissemination and
commercialisation. The amount he thought would be required need not be large (may be
only a few thousand Euros) but it could make the difference between a project stopping or
actually achieving market entry and as such it would be an investment worth making.
Another coordinator was less sure on the idea of follow up funding , stating that: “There may
be good reasons why a project shouldn't continue or why it should be considered as a 'testbed' and should be taken forwards outside of EC FP activity. They also pointed out that there
is already some flexibility to secure funding extensions, though they believed that this was
not consistently applied to all projects. One current participant felt that a separate contract for
post project activity would be a complex arrangement
One coordinator suggested that the EU should develop a dissemination strategy when
products are mature and use highly recognised international events like Inter Solar or Solar
Energy to present their FP7 projects. He felt that this would improve the credibility of the
projects and help commercialise them. Such a collective approach was thought to be more
efficient, in terms of impact per € spent than lots of individual project actions.
One coordinator stated that his project has agreed to keep the project website alive for 1-2
years following project end to assist dissemination, he felt that this was good and it may be a
good idea to oblige other projects to do this.
One project coordinator mentioned security concerns regarding publicising the results of his
project as it concerned the electricity infrastructure which is seen as an area of concern.
Final Report - FP7 Energy Mid-Term Evaluation
148
7.4.5
SET Plan
The project coordinators interviewed were asked their opinions on the future links between
the FP and the SET plan. This revealed some interesting and varied perceptions. A
coordinator from a very large research institute raised concerns that the rumours were that the
EC plans to focus their research support solely on large multinational companies and move
away from supporting SMEs. This approach was not popular as it was felt that FP would lose
the ability to support growing but resource constrained SMEs. They believed that continuing
to work with SMEs should be central to the SET plan and the FP. Another coordinator was
also under the impression that the SET plan objectives were very focussed on demonstrations
and as such he felt that funding and focus should also remain for fundamental R&D. There
was also concern that too high of a reliance on private funding would put too much emphasis
on cost effectiveness which could be detrimental.
One coordinator felt that the EC technology platforms were a good mechanism for
influencing the future direction of FP and SET and that the current process offers plenty of
chances to influence/contribute to the policy debate.
One coordinator felt that the FP funds should be used as part of the SET plan and should
focus support on the best technological solutions independently from their proposers (SMEs
or Large Enterprise). The involvement of MS funding was considered necessary in order to
create a “fund association” with the European Commission and others creating Public Private
Partnerships in order to support large scale demonstration projects as well as smaller projects.
They referenced the Marguerite21 programme as being a good example of such an approach.
21
The Marguerite fund is a pan-European infrastructure fund for long term institutional investors to finance the implementation of
strategic European policy objectives and projects in the Transport, Energy/Climate and Renewables sectors. The main objective
of the fund is to contribute to the European Economic Recovery Plan by financing the implementation of strategically important
European policy objectives in Energy/Climate, Renewable and Transport sector infrastructures.
149
Final Report - FP7 Energy Mid-Term Evaluation
8 Network Analysis of FP6 and FP7
•
•
•
•
•
•
•
•
•
Separate networks are presented for FP6 and FP7
Networks are the result of the analysis of collaboration intensity among countries and
participants
Data have been cleaned prior to the analysis to avoid doubles (as a result of homonyms and
synonyms) – however, this cleanup mainly on the level of organisations/participants cannot
be considered to be exhaustive (due to the large number of observations)
Methodological note: the counting scheme applies is based on so-called ‘normal’ counting
scheme; this implies that in the cases one projects contain participants of NL, BE, DE, DE,
this would result in collaboration between NL, BE and DE; the two occurrences of DE are not
treated as ‘2’
Countries are noted by 2 digit code level (as appearing in the source data)
Participants are noted by their short name (as appearing in the source data) in order to keep
the maps readable
The titles of the figures presented below contain information on the approach used; e.g.
several network diagrams are the result of limiting the network to collaboration above a
certain threshold (this is needed in order to keep figures readable)
The thickness of the lines reflect the intensity of the collaboration – the position of
countries/organisations are the result of relative positioning and indicate the ‘role’; the more
central the position is the more ‘involved’ an actor is
On the specific figures below:
o Fig 1 - contains the interactions among participating countries for FP6 (all countries)
o Fig 2 - contains the interactions among participating countries for FP6 (all countries)
where there are more than 5 links between countries
o Fig 3 - contains the interactions among participating countries for FP6 (all countries)
where there are more than 10 links between countries
o Fig 4 - contains the interactions among individual participants for FP6 (all
participants) where there is more than 1 link
o Fig 5 - contains the interactions among individual participants for FP6 (all
participants) clustered around ‘CRES’ which is the most active participant in FP6
energy.
o Fig 6 - contains the interactions among participating countries for FP7 (all countries)
o Fig 7 - contains the interactions among participating countries for FP7 (all countries)
where there are more than 5 links between countries
o Fig 8 - contains the interactions among participating countries for FP7 (all countries)
where there are more than 10 links between countries
o Fig 9 - contains the interactions among individual participants for FP7 (all
participants) where there is more than 1 link
o Fig 10 - contains the interactions among individual participants for FP7 (all
participants) clustered around ‘VTT’ (Finland) which is the most active participant
in FP7 energy.
Final Report - FP7 Energy Mid-Term Evaluation
150
8.1
Energy FP6
Figuur 1: Collaboration visualization on country level (FP6, all links)
Figuur 2: Collaboration visualization on country level (FP6, > 5 links)
152
July 2010
Figuur 3: Collaboration visualization on country level (FP6, > 10 links)
Figuur 4: Collaboration visualization on participant/organization level (FP6, > 1 link)
154
July 2010
Figuur 5: Collaboration visualization on country level (FP6, all links)
8.2
Energy FP7
Figure 6: Collaboration visualization on country level (FP7, all links)
156
July 2010
Figuur 7: Collaboration visualization on country level (FP7, > 5 links)
Figuur 8: Collaboration visualization on country level (FP7, > 10 links)
158
July 2010
Figuur 9: Collaboration visualization on participant level (FP7, > 1 link)
Figuur 10: Collaboration visualization on country level (FP7, all links)
160
July 2010