European Commission:
Example Sector Intervention Logics/
Theory of Change Models
Development Evaluation Day
Finnish Ministry of Foreign Affairs
17 December 2010
1
2
© Sidney Harris
Why this Approach?
1. Managing for Development Results (MfDR)
• Go beyond inputs and outputs
• Towards outcomes and impact
• Contribution of donor to country-led results
• Based on country-led reporting systems
3
Why this Approach?
2. Clear Macro Result Frameworks
• Don’t just focus on project level result frameworks
• There is also a need for macro level result
frameworks (e.g. Country Strategy Papers)
• Evaluations should avoid building retrospective
macro result frameworks
4
Why this Approach?
3. Intervention Logic/Theory of Change
• The need for a Hierarchy of Objectives &
Chain of Results
• Development of an Intervention Logic/
Theory of Change (school of thought)
• Designed to fill the 'missing middle’
between outputs and global impact
5
Objectives and Results
Global
Impacts
Global
objectives
Intermediate
Impacts
Intermediate
Objectives
Specific
Impacts
Specific
Objectives
Direct
Objectives
Outcomes
Outputs
Implementatio
n Objectives
Inputs
6
Intervention Logic/Theory of Change
Inputs
Outputs
Outcomes
Effect 1
Effect
EQ 4
Specific
Impacts
Intermediate
Impacts
Global
Impacts
Activities/
Funding 1
EQ 7
Effect
Activities/
Funding 2
Effect 2
Effect 11
Effect 5
Activities/
Funding 3
Effect 8
Effect 3
Effect 9
Effect 10
Effect
EQ 6
Activities/
Funding 4
7
The Missing Middle
The Good Wind
Evaluation!
Outputs
Outcomes
Specific
Impacts
Intermediate
Impacts
Global Impacts
8
Attention
1. This is a highly simplified tool, not a template.
Intervention logic must be developed for each country
based on its context and existing government strategies
2. Outcomes and Impacts are not fully attributable to
development interventions; indeed their contribution
diminishes and external factors become more and more
important as you move up the chain of results
3. Indictors are not standard/fixed but provide ideas
on how to set, monitor and evaluate objectives
4. Crosscutting issues should be mainstreamed into
relevant indicators (e.g. disaggregation of data by gender)
9
Education
10
Education Intervention Logic
Output
Clusters
Institutional
Management
Specific
Impact
Outcomes
Better
Allocation of
Educational
Resources
Increased
Affordability of
Education
Quality of
Education
Equitable
Access to
Education
MDG 2
Education
Facilities
Economic
Growth
Skills and
Learning
Enhancement
Health
Global
Impacts
Improved Family
Planning &
Health Awareness
Curricula &
Teaching
Materials
Teacher
Recruitment
& Training
Intermediate
Impacts
MDG 3
Improved
Participation in
Society
Poverty
Reduction
MDG 1
Social
Development
Greater Income
Opportunities
Optimal
Employment
11
Outcome Indicators (1)
Output
Clusters
Institutional
Management
Indicators
Outcomes
Better
Allocation of
Educational
Resources
1.
Transparent decision making
Increased
Affordability of
Education
2.
Cost for household
Curricula &
Teaching
Materials
Teacher
Recruitment
& Training
Health
Education
Facilities
Optimal
Employment
13. Employment: school/university
construction and maintenance
14. Employment: teachers and
government workers
12
Outcome Indicators (2)
Output
Clusters
Institutional
Management
Results
(Outcomes)
Better
Allocation of
Educational
Resources
Increased
Affordability of
Education
Quality of
Education
Curricula &
Teaching
Materials
Teacher
Recruitment
& Training
Equitable
Access to
Education
Indicators
3.
Absenteeism & Drop Out
4.
Teacher ratios/rates
5.
Qualification of teachers
6.
Teaching material ratios
7.
Transition & Repetition
8.
Equitable education participation
9.
Enrolment rate
10. Completion rate
11. Distance to school/university
Health
12. School and university places
Education
Facilities
Optimal
Employment
13
Specific Impact Indicators
Output
Clusters
Institutional
Management
Specific
Impact
Outcomes
Indicators
Better
Allocation of
Educational
Resources
Increased
Affordability of
Education
Quality of
Education
Skills and
Learning
Enhancement
Curricula &
Teaching
Materials
Teacher
Recruitment
& Training
Equitable
Access to
Education
15. Literacy &
Numeracy rates
16. Examination results
17. Social views
Health
Education
Facilities
Optimal
Employment
14
Intermediate Impact Indicators
Output
Clusters
Institutional
Management
Specific
Impact
Outcomes
Better
Allocation of
Educational
Resources
Increased
Affordability of
Education
Intermediate
Impacts
Indicator
Improved Family
Planning &
Health
Awareness
Quality of
Education
Skills and
Learning
Enhancement
Curricula &
Teaching
Materials
Teacher
Recruitment
& Training
Equitable
Access to
Education
Improved
Participation in
Society
Health
Greater Income
Opportunities 18. Employability
Education
Facilities
Optimal
Employment
15
List of Key Indicators (with examples)
Affordability Outcome Indicators
11. Distance to school/university
- x% of students within x kilometre of a school
1. Transparent decision making
- Implementation of transparent mechanisms to allocate resources between
12. School and university places
- Educational places as an x% of eligible students
sectors (e.g. using tools such as average cost ratios per student for different
educational levels)
2. Cost for Household
- Cost of education (i.e. fees, materials/equipment, lunches etc) as a % of
household expenditure; cost per student as a % of income per capita
Optimal Employment Outcome Indicators
13. Employment: school/university construction and maintenance
- Number of people directly and indirectly employed in school/university construction and
Quality Outcome Indicators
3. Absenteeism and drop out
- Teacher/student absenteeism rates; drop out rates
maintenance
14. Employment: teachers and government workers
- Number of people directly & indirectly employed as teachers and
government workers linked to education sector
4. Teacher ratios/rates
- Student teacher ratio; rate of double shifting
5. Qualification of teachers
Specific Impact Indicators
- Average qualification of teachers at each educational level
15. Literacy and numeracy rates
6. Teaching material ratios
- Literacy and numeracy rates by age, sex and social group
- Student textbook ratio
16. Examination results
7. Transition and repetition
- Examination results for various educational levels and subject areas by age, sex & social
- Progression rates; repetition rates
group
17. Social views
Access Outcome Indicators
- Support for violent conflict; prejudiced views towards other social groups
8. Equitable educational participation
- Ratio of girls to boys in primary, secondary and tertiary education; ratio of other
Intermediate Impact Indicators
disadvantaged groups compared to x% of population
9. Enrolment rate
- Gross and net enrolment rates
18. Employability
- % of school leavers/graduates gaining paid employment (split by type of employment)
10. Completion rate
- Gross and net enrolment rates
16
Mapping Evaluation Questions
Output
Clusters
Institutional
Management
Outcomes
Specific
Impact
Better
Allocation of
Educational
Resources
Quality of
Education
Economic
Growth
EQ 3
Skills and
Learning
Enhancement
Curricula &
Teaching
Materials
Teacher
Recruitment
& Training
Global
Impacts
Improved Family
Planning &
Health Awareness
EQ 1 EQ 2
Increased
Affordability of
Education
Intermediate
Impacts
Equitable
Access to
Education
EQ 4
EQ 5
Improved
Participation in
Society
Poverty
Reduction
Social
Development
Health
Greater Income
Opportunities
Education
Facilities
Optimal
Employment
EQ+
EQ’s also required for relevance
and aid effectiveness issues
17
Example Result EQs: Education Sector (1)
EQ 1: To what extend has EC support to education
helped to improve education service delivery and
resourcing?
EQ 2: To what extend has EC support to education
helped to strengthen transparency and accountability
in the management of education service delivery?
18
Example Result EQs: Education Sector (2)
EQ 3: To what extend has EC support to education
contributed to improving access and equity related
to education?
EQ 4: To what extend has EC support to education
contributed to improving quality of education?
EQ 5: To what extend has EC support to education
contributed to enhanced basic education skills,
especially literacy and numeracy?
19
Detailed Example EQ 3 – Quality
EQ 4: To what extend has EC support to education contributed to
improving quality of education?
Judgement Criteria
Indicators
4.1. Availability of strategies and resources to
enhance quality of learning and teaching
- % of non-salary to salary budget allocation
- Pupil teacher ratio
- Pupil classroom ratio (e.g. disaggregated by region)
- Pupil textbook ratio
- Strategies for improving quality of learning and teaching (e.g.
whole school development)
4.2. Quality related efficiency measures
-Decreasing drop out rates
-Decreasing % repeaters (e.g. primary school level)
4.3. Qualifications and competencies of teachers
and school leaders enhanced
-% of teachers certified to teach according to national standards
(teacher accreditation)
-In-service teacher upgrading institutionalised
-Training of school leaders on pedagogic and managerial skills in
place and implemented
4.4. Provisions made to ensure minimum quality
education for children in difficult circumstances
(e.g. poor, ethic minorities)
- Upgrading of school buildings and construction of additional
classrooms in remote areas
- Incentive schemes for teachers in remote and disadvantaged
areas operational
- Prevalence of bilingual teacher training, curricula-teacher
guides and textbooks for multi-lingual environments
20
Agriculture & Rural Development
21
Agriculture & Rural Development Intervention Logic
Output
Clusters
Specific
Impacts
Outcomes
Intermediate Global
Impacts
Impacts
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
MDG’s
2, 3, 4, 5, 6 & 7
Improved
Access
Markets
Enabling
Economic &
Regulatory
Environment
Increased
Profitability of
Rural Business
Increased
Productivity
MDG 1
Greater Food
Security
Increased
Trade
Improved
Quality of
Products
Poverty
Reduction
Increased
Affordability of &
Subsistence in Food
Increased
Production
Capital
Better
Management of
Natural
Resources
Economic
Growth
Agri-Labour
Farm Inputs
Information
Services
Higher Rural
Income
Non-Farm
MDG 1
Social
Development
Improved
Environmental
Sustainability
MDG 7
22
Agriculture Based Economies (1)
Identified by:
• Agriculture provides basis for growth
(i.e. contributes over 30% of GDP)
• 70% of poor live in rural areas
Typical areas:
• Sub-Saharan Africa
• Regions within India & Mexico
Intervention focus:
• Improve smallholder competitiveness
• Ensure livelihoods and food security
23
Agriculture Based Economies (2)
Output
Clusters
Outcomes
Specific
Impacts
Intermediate Global
Impacts
Impacts
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
Improved
Access
Markets
Agri-Labour
Enabling
Economic &
Regulatory
Environment
Increased
Productivity
Poverty
Reduction
Increased
Affordability of &
Subsistence in Food
Greater Food
Security
Increased
Production
Increased
Trade
Capital
Better
Management of
Natural
Resources
Economic
Growth
Increased
Profitability of
Rural Business
Farm Inputs
Information
Services
Higher Rural
Income
Non-Farm
Improved
Quality of
Products
Social
Development
Improved
Environmental
Sustainability
24
Transforming Economies (1)
Identified by:
• Agriculture no longer driver of growth (i.e. 7%
of GDP) with rising urban-rural poverty gap
• 82% of poor live in rural areas
Typical areas:
• South/East Asia (e.g. China, India)
• Middle East and North Africa (e.g. Morocco)
Intervention focus:
• Comprehensive approach and multiple pathways
out of poverty to reduce the urban-rural divide
- shifting to high-value agriculture
- decentralising non-farm economic activities to rural areas
25
Transforming Economies (2)
Output
Clusters
Specific
Impacts
Outcomes
Intermediate Global
Impacts
Impacts
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
Improved
Access
Markets
Agri-Labour
Enabling
Economic &
Regulatory
Environment
Increased
Productivity
Poverty
Reduction
Increased
Affordability of &
Subsistence in Food
Greater Food
Security
Increased
Production
Increased
Trade
Capital
Better
Management of
Natural
Resources
Economic
Growth
Increased
Profitability of
Rural Business
Farm Inputs
Information
Services
Higher Rural
Income
Non-Farm
Improved
Quality of
Products
Social
Development
Improved
Environmental
Sustainability
26
Urban-Based Economies (1)
Identified by:
• Small agriculture sector (i.e. less than 0.5% of
GDP) with high urban poverty
• 45% of poor live in rural areas
Typical areas:
• Latin America
• Central Asia
Intervention focus:
• Link farmers to modern food markets
• Improve quality of agri-business and food industry
• Introduce market for environmental services
27
Urban-Based Economies (2)
Output
Clusters
Outcomes
Specific
Impacts
Intermediate Global
Impacts
Impacts
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
Improved
Access
Markets
Agri-Labour
Enabling
Economic &
Regulatory
Environment
Increased
Productivity
Poverty
Reduction
Increased
Affordability of &
Subsistence in Food
Greater Food
Security
Increased
Production
Increased
Trade
Capital
Better
Management of
Natural
Resources
Economic
Growth
Increased
Profitability of
Rural Business
Farm Inputs
Information
Services
Higher Rural
Income
Non-Farm
Improved
Quality of
Products
Social
Development
Improved
Environmental
Sustainability
28
Outcome Indicators (1)
Output
Clusters
1.
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
Improved
Access
Markets
Farm Inputs
Information
Services
Enabling
Economic &
Regulatory
Environment
Indicators
Outcomes
Capital
Better
Management of
Natural
Resources
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Support for consumers vulnerable to short-term
food price changes
Support for farmers vulnerable to short-term
food price changes
Access to road transportation
Access to land
Access to water
Access to seeds and fertilisers
Access to improved breeds of animals/fish
Access to extension services
Access to information on market
Availability of credit
Investment in rural areas
Establishing a rural business
13. Preservation of natural resources
14. Sustainable management practices
15. Environmental services
29
Outcome Indicators (2)
Output
Clusters
Outcomes
Indicators
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Non-Farm
16. Non-farm employment
17. Agriculture labour employment
Agri-Labour
Provision of
Public Goods
Improved
Access
Markets
Increased
Productivity
18. Land productivity
19. Labour productivity
20. Capital productivity
Farm Inputs
Information
Services
Enabling
Economic &
Regulatory
Environment
Increased
Production
21. Production
22. Type of production/diversification
Improved
Quality of
Products
23. International marketing standards
Capital
Better
Management of
Natural
Resources
30
Specific Impact Indicators
Output
Clusters
Indicators
Outcomes
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
Improved
Access
Non-Farm
AgriLabour
Increased
Productivity
Markets
Increased
Affordability of &
Subsistence in Food
Farm Inputs
Information
Services
Enabling
Economic &
Regulatory
Environment
Increased
Production
Increased
Trade
Capital
Better
Management of
Natural
Resources
Increased
Profitability of
Rural Business
Improved
Quality of
Products
24. Agricultural prices
25. Growth in agricultural
businesses
26. Growth in rural non-farm
businesses
27. Affordability of food
28. Subsistence in food
29. Agricultural trade
30. Rural non-farm trade
31
Intermediate Impact Indicators
Output
Clusters
Outcomes
Indicators
Efficient
Employment
Mitigating
Agricultural
Price Volatility
Provision of
Public Goods
Improved
Access
AgriLabour
Greater Food
Security
32. Undernourishment
Increased
Production
Increased
Trade
Capital
Better
Management of
Natural
Resources
Increased
Profitability of
Rural Business
Increased
Affordability of
& Subsistence in
Food
Farm Inputs
Enabling
Economic &
Regulatory
Environment
31. Rural Income
Increased
Productivity
Markets
Information
Services
Higher Rural
Income
Non-Farm
Improved
Quality of
Products
Improved
Environmental
Sustainability
33. Land
sustainability
34. Sustainable
extraction of
water
35. Adapting to
climate change
32
List of Key Indicators (with examples)
Price Volatility Outcome Indicators
Production Outcome Indicators
1. Consumer vulnerability to short-term price changes
2. Farmer vulnerability to short-term price changes
21. Production
22. Type of production/diversification
Improved Access Outcome Indicators
Quality Outcome Indicators
3. Access to road transportation
4. Access to land
5. Access to water
6. Access to seeds and fertilisers
7. Access to improved breeds of animals/fish
8. Access to extension services
9. Access to information on markets
10. Availability of credit
11. Investment in rural areas
12. Establishing a rural business
23. International marketing standards
Natural Resource Outcome Indicators
13. Preservation of natural resources
14. Sustainable management practices
15. Environmental services
Profitability Specific Impact Indicators
24. Agricultural prices
25. Growth in agricultural businesses
26. Growth in rural non-farm businesses
Affordability/Subsistence Specific Impact Indicators
27. Affordability of food
28. Subsistence in food
Trade Specific Impact Indicators
29. Agricultural trade
30. Rural non-farm trade
Rural Income Intermediate Impact Indicators
Employment Outcome Indicators
16. Non-farm employment
17. Agricultural labour
31. Rural income
Food Security Intermediate Impact Indicators
32. Undernourishment
Productivity Outcome Indicators
18. Land productivity
19. Labour productivity
20. Capital productivity
Environment Sustainability Intermediate Impact Indicators
33. Land sustainability
34. Sustainable extraction of water
35. Adapting to climate change
33
Limitations & Issues with Approach
1. Common Results Frameworks
• More and better country-led result
frameworks (promotion of joint evaluations)
• Strengthened international agreement on
common objectives (i.e. MDGs+)
• Improved guidance and development of
relevant indicators
34
Limitations & Issues with Approach
2. DATA, DATA, DATA!!!!
• This is the problem for obtaining strong
evaluation results
• Data in many developing context is
extremely poor
• There is need to strengthen partner
countries in statistical capacity and M&E
35
Limitations & Issues with Approach
3. Challenge of Complexity
• Evaluation of outcomes and impact is very
complex (increasing number of external factors)
• Evaluation methods and tools required that can
deal with complexity (e.g. systems thinking)
• Get out of a project mindset. Look at synergies
between sectors
36
Thank you!
Any Comments/Questions?
37
The following 3 Intervention Logics will
not be presented in Helsinki.
They are only included in case they are
required during the discussion after the
presentation.
38
Road Sector
Output
Clusters
Outcomes
Specific
Impacts
Lower Road
Transport Time
Greater
Economic
Accessibility
Lower Road
Transport Costs
Greater
Accessibility to
Social Services
Infrastructure
(construction/
maintenance)
Institutional
Reform
Improved
Institutional
Management of
Road System
Economic
Increased
Employment
Environment
Social
Intermediate
Impacts
Global
Impacts
Improved
Conditions for
Economic Growth
Economic
Growth
Poverty
Reduction
Social
Development
Environmental
Impacts of Roads
Minimised
Enhanced Safety
& Health Linked
to Roads
Enhanced Human
Capacity
39
Health Sector
Output
Clusters
Essential
Medicines &
Equipment
Outcomes
Increased
Affordability of
Health Care
Regulation of
Private Sector
Institutional
Reform
Global
Impacts
Social
Development
Reduction in
Mortality
Poverty
Reduction
Improved Health
Behaviour
Improved Health
Awareness
Mitigation of
External Risks
Health Care
Worker
Recruitment &
Training
Increased Service
Utilization &
Intervention
Coverage
Intermediate
Impacts
Enhanced Service
Availability &
Quality
Infrastructure
Education
Specific
Impacts
Reduction in
Morbidity &
Disability
Reduced Health
Risks
Optimal
Employment
Economic
Growth
Stronger Labour
Markets
40
Water and Sanitation
Output
Clusters
Environment
Outcomes
Improved
Conservation &
Preservation of Water
Improved Use of
Water &
Sanitation
Economic
Increased
Employment
Education
Improved Quality
of Water &
Sanitation
Institutional
Reform
Infrastructure
(construction/
maintenance)
Specific
Impacts
Global
Impacts
Sustainable
National Water
Supply
Reduced
vulnerability of
Ecosystem to
climatic events
Improved
Conditions for
Economic Growth
Economic
Growth
Improved Water
& Sanitation
Health & Hygiene
Behavior
Increased Access
to Water
Greater Equity in
Allocation of
Water & Sanitation
Increased Access
to Basic Sanitation
Increased
Affordability of
Water & Sanitation
Intermediate
Impacts
Poverty
Reduction
Reduced Rate of
Water Related
Diseases
Social
Development
Improved
Health
41