Evaluation plans for 2014-2020
programming period in Poland
Experience and new arrangements
Ministry of Infrastructure and Development, Poland
Athens, 15th May, 2014
Joint Evaluation Network Meeting
Institutional context for evaluation
planning
European Commission
National
Evaluation Unit
20
persons
ESF Evaluation
Unit
Operational
Programmes
Evaluation Units
35
persons
Evaluation Units –
Intermediate Bodies
7
persons
16 Regional Operational
Programmes Evaluation
Units
(ESF + ERDF)
60
persons
43
persons
Evaluation Units –
Intermediate Bodies
Evaluation Units –
Intermediate Bodies
Evaluation Units –
Intermediate Bodies
Closer coordination as a main challenge for 2014-2010 period
Polish experience on
evaluation planning 2007-2013
 2007-2013 period: evaluation plans established for NSRF
and all operational programmes
 Strategic evaluation plans for the whole 2007-2013 period
 strategic goals for evaluation process
 organisational arrangements (thematic steering groups)
 main thematic areas
 main evaluation capacity building activities
 Annual evaluation plans
 list of evaluation studies + description
 evaluation capacity building activities
Polish experiences on
evaluation planning 2007-2013
 All plans examined by the National Evaluation Unit
 Shortcomings:
 difficult to coordinate
 no strategic integration of an evaluation process with
implementation and intervention logic → annually
planned evaluations mainly for current informational
needs
 no coherent evaluation agenda for the whole Cohesion
Policy
 Advantages:
 high degree of flexibility
Evaluation planning 2014-2020
in Poland
 Process of evaluation planning in Poland to be regulated by the
Guidelines on evaluation activities in the 2014-2020 period
(draft)
 In the field of the evaluation plans the Guidelines contains:
 general rules
 obligations for institutions, and role of the main
stakeholders
 structure and content of evaluation plans
 deadlines and logistic/organisational matters
 list of obligatory evaluation studies (e.g. ex-ante, mid-term,
ex-post, each priority evaluation, annual systematic review
of evaluation studies, gender equality evaluation)
Evaluation planning – general
principles
(1) Evaluation process closely related to the intervention logic
 evaluation plans (esp. scope and timing) reflect the intervention logic of
the programme
 intervention logic as a starting point for identification of the informational
needs (what kind of and when evaluation evidence will be needed)
 all planned evaluation activities should be justified in this context
(2) Planning process closely coordinated by NEU in order to ensure the
coherence of the evaluation system
 twofold aim of the planning: 1) effective evaluation of each programme 2)
effective evaluation of the whole Partnership Agreement intervention
 coherent evaluation agenda is needed → operational programme
evaluations + evaluations in the PA level (meta-evaluations, systematic
reviews, strategic and horizontal evaluations)
Evaluation planning – general
principles
(3) Monitoring of evaluation plans implementation
 whole evaluation process monitored by National Evaluation
Unit
 special emphasis on recommendation implementation
process → Recommendations Implementation System (RIS)
 annual reports on the progress of implementation of
evaluation plans (esp. recommendations implementation)
Evaluation planning in Poland –
proposed rules and procedures
 Evaluation plans prepared for every operational programme by the Evaluation
Units for the whole programming period
 National Evaluation Unit preparing the Evaluation Plan for Partnership
Agreement 2014-2020
 Annual update as a tool for changes and supplementation
Dilemmas:
 Evaluation plan for the whole period contains all evaluation studies planned
for each programme vs. contains only strategic and most important studies
 Annual update of one plan vs. annual plans as a supplement for the
strategic plan
 Evaluation Units are obligated to ensure:
 linkage between the planned evaluation studies and the intervention
logic of the programme
 high quality of evaluation studies
 identification of needed data
 resources needed for evaluation process
Evaluation planning in Poland –
proposed rules and procedures
 All evaluation plans and their annual updates accepted by
National Evaluation Unit
 NEU’s opinion based on the criteria of: adequacy, coherence,
feasibility, and efficiency in order to ensure the coherence of
the whole cohesion policy evaluation process
 After the NEU decision, evaluation plans are accepted by
Monitoring Committees
Main challenges for evaluation
planning
 strategic planning vs. current evaluation needs → annual update
or annual plans as a tool for flexibility
 effective integration of evaluation process with the intervention
logic (and then implementation process) within evaluation plans
→ high expert and evaluation potential needed → cooperation
with external experts + high quality trainings
 effective coordination in order to establish coherent evaluation
agenda for the whole Cohesion Policy → National Evaluation Unit
responsibility
 identification and access to the needed data (in particular for
counterfactual studies)
Preparation to evaluation
planning in Poland
 National Evaluation Unit prepared detailed Guidelines on evaluation
activities in the 2014-2020 period with section dedicated to evaluation
planning.
 After the discussion with the OP Evaluation Units, NEU decided to support
evaluation units, and to organise the evaluation planning process in closely
coordinated manner.
 NEU is going to launch the project: Process of preparing evaluation plans
for Cohesion Policy 2014-2020.
 In cooperation with experts on evaluation and sectoral matters
 Main goals of the project:
 Development of the evaluation planning methodology (taking into
account intervention logic)
 Preparation of evaluation plans for Operational Programmes and
Partnership Agreement
Preparation to evaluation
planning in Poland
 Main stages of the project:
1. Pilot project – preparation of an evaluation plan for one chosen
Polish region (Regional Operational Programme)
 intervention logic analysis → expert panel
 workshops with main Regional Operational Programme
stakeholders and partners
 preparation of the Regional Operational Programme
Evaluation Plan
2. Preparation of the Partnership Agreement Evaluation Plan
 presentation and discussion on a pilot project findings
 intervention logic analysis → expert panel
 workshops with main stakeholders and partners
 preparation of the Partnership Agreement Evaluation Plan
Preparation to evaluation
planning in Poland
3. Preparation of evaluation plans for all Operational
Programmes
 with the support of the NEU (help desk) and external
evaluation experts
 NEU opinion
4. Monitoring Committee approval
5. Sending plans to European Commission

Main products:
 evaluation plans for OP’s and PA
 methodology document on evaluation planning
EVALUATION PLAN OF KNOWLEDGE –
EDUCATION – DEVELOPMENT
OPERATIONAL PROGRAMME
– MAIN ASSUMPTIONS
A ROLE OF EVALUATION PLAN
1) Programme evaluation management
2) Predictable information needs fullfilment
3) Evaluation WWW - (What When Who)
4) Rising awarness about preparations to be done
5) Transparency
6) Stakeholder’s interest taken into account
MAIN PRINCIPLES OF THE
EVALUATION PLAN
1) 1 EP for 1 OP
2) Evaluation experts functionally independent
3) Evaluation capacity building
4) Social partners involvement
5) Long-term orientation = flexibility of provisions
FLEXIBILITY OF THE EVALUATION
PLAN
1) EP should be a strategic oriented document.
2) List of evaluation studies is open and
indicative.
3) The list covers only evaluation studies that
are predictable.
4) Detailed information about the studies will be
updated yearly.
ELEMENTS OF THE EVALUATION PLAN
1) Definition – Goals – Challanges and Principles
for evaluation process
2) Data access and provisions
3) Process of coordination
4) Capacity building and evaluation culture
5) Use and dissemination of findings
6) Indicative list of planned evaluation studies
EVALUATION STUDIES
CHARACTERISTICS
1) Subject /area
2) Purpouse and scope
3) Who
4) Evaluation approach planned
5) Tentative term – usually approximately
6) Budget – broadly estimated
7) Other information if necessary
LP.
SUBJECT/
AREA
ESF IMPACT
ASSESMENT
ON VET
1 TRAININGS
QUALITY
DEVELOPME
NT
GOAL/
RATIONALE
……………
EVAL.
UNIT
Ministry of
Education
EVALUATION
APPROACH
TBE
2015
2016
2017
2018
2019
2020
2021
2022
2023
TEMPLATE - EXAMPLE
x
COST OTHER
(pln)
INF.
Cooper
ation
with
Polish
Agency
200 000
for
Enterpr
eneurial
Develop
ment
TYPES OF EVALUATION STUDIES
1) Obligatory studies
2) Metaevalution, syntheses
3) Diagnoses, needs and barriers identification
4) Horizontal principles
5) Specific target groups oriented
6) Comparability studies
7) Capturing the effects
8) Ad hoc studies
MAIN SOURCES OF IDEAS
1) Research obligatory or necessary
2) Proposals from ex-ante evaluation
3) Proposals made by the IB’s
4) 2004-2006 and 2007-2013 experience
5) Policy makers needs – if possible to identify
Thank you for your attention!
[email protected]
Michał.Suł[email protected]
Ministry of Infrastructure & Development
POLAND