European Union
Regional Policy – Employment, Social Affairs and Inclusion
Draft guidance on monitoring and
evaluation 2014-2020:
Concepts and recommendations
REGIO network meeting with Member States, 20 Oct. 2011
Kai Stryczynski
1
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Findings of April meeting
•
•
•
•
General support from Member States
Use and usability missing
Definitions of common indicators needed
Annex 2 on result indicators not helpful
2
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Section 1: Concepts
• New section on use and usability
– Further development planned in EVALSED
• No other major changes
• Text improvements
– E.g. relation between counterfactual and
theory-based evaluation
• Section on standards suppressed – little
interest, no reactions
3
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Section 2
More changes than in section 1…
4
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Section 2: Quality criteria for
result indicators
• New: Quality criteria for result indicators
• New: ex ante conditionality on statistical
systems and result indicators
• Network could discuss how to deal with ex
ante conditionality in due time
(together with ex ante evaluation?)
5
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Result indicators:
A baseline for what or whom?
• Baseline for the priority axis can be
different from baseline data needed for
evaluation.
• For example:
– Baseline data for all SME in a region when
deciding the programme content
– Only a small part will be supported – the
effect on these enterprises only will be
evaluated.
• See box page 11, example in annex 2.
6
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Targets for result indicators I
• Setting targets for result indicators is
difficult! Reflects effect of programme and
other factors.
• Can be of quantitative or qualitative
nature.
• What is a qualitative target?
– Range of expected values,
– the expected direction of change and the
expected pace of change
7
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Targets for result indicators II
Attention – two different issues
• the estimate of a future value of the result
indicator - influenced by programme and
other factors. It is this stated aim that is
meant by the legal text.
• an estimate of the contribution of the
programme to the change of the result
indicator (the effect or impact of the
programme).
8
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Targets for output indicators
• Based on past experience (past unit costs)
• Recommendation: Make information
available for a discussion of targets
9
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Ex ante evaluation
• Some new elements, reflecting changes in
OP, possible support for meeting
conditionalities
• Do MS wish a special meeting of network?
• Should the Commission write up guidance?
10
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Evaluation plan
• A flexible planning tool for MS
• Less details likely for later years
• Attention: some baseline data cannot be
created ex post
• Commission encourages coordination
across programmes
11
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Electronic data transmission
• No legal provisions so far in proposed
regulation
• Planned for implementing regulation
• Key idea the same as for 2007-13
– Common indicators: electronic „structured“
transmission
– Programme-specific indicators: only in
annual report
12
European Union
Regional Policy – Employment, Social Affairs and Inclusion
Examples on result indicators
(annex 2)
• Mix of intervention areas (enterprise
support, infrastructure, innovation, crossborder cooperation)
• Mix of evaluation appproaches
(counterfactual, theory-based)
• Mix of data sources (statistics, surveys, tax
authority data)
• Is this more helpful?
13