Focusing on Development Outcomes
in Projects
SARAR M&E Workshop
20-21 June 2006
New Delhi, India
Outline
1. Management for results as an emerging
practice
2. Processes to manage quality
3. Methodology to manage for results –
outcome focused design and supervision
1 - Management for Results is an emerging
practice within the development community
 Compared to Financial Management and Procurement,
Management for Results is an emerging practice (within
the last 10 years), now rooted in a broader agenda of
the development community (Paris Declaration)
 Management for Results has been practiced in recent
years under several titles (M&E, management by
objectives, balanced scorecard, budget rationalization)
 The emphasis on Management for Results grows out of
a need to improve development effectiveness. It will
help the Bank and it’s development partners to tell their
results story in a more convincing way
2 - Quality Processes




Processes to manage quality
Country level
Portfolio level
Project level
The Bank’s view of what needs to be monitored
now includes the entire results hierarchy
Results Hierarchy
Impacts
Outcomes
Outputs
Activities
Inputs
Longer-term benefits
Effectiveness
Results
Deliverables
Efficiency
Procurement & Disbursements
Key Events in the Bank related to
Results Management
 Jan ‘96 - Performance indicators become mandatory in all
new projects;
 Feb ‘97 - Strategic Compact commits the Bank to 100%
quality-at-entry by FY2001;
 June ‘97 - Revised supervision reporting system introduced,
including indicators monitoring;
 Aug. ‘97 - Revised project documentation introduced for new
operations (including the Logical Framework);
 2002 - Results Secretariat unit formed within the Bank
 2003 - Logical Framework ‘simplified’ to the Results
Framework
 2004 - Task Force on Organizational Effectiveness created
 2005 – Paris Declaration on Aide effectiveness
Magnitude of the Results
Management Task Bank-wide
6 Bank regions (AFR, EAP, ECA, LAC,
MNA, SAR)
1282 new projects under preparation;
2372 existing projects under supervision;
US$ 20-25 billion in new lending per year;
the Bank also manages US$ 9 billion in
trust funds
Country level Results Cycle
National Development Strategy
Poverty Reduction Strategy
Results
Monitoring &
Learning
System
7
1
Results
Based
CAS
2
CAS
Completion Report
Portfolio Performance
(ARPP)
Identify Mix of Bank
Lending & AAA
aligned with CAS Outcomes
6
3
Results Based CAS
Progress
Reports
5
Country
Portfolio/
Program
Reviews
4
Design, prepare
& implement projects
& AAA
(PDO CAS)
SAR Portfolio level quality standard
Defines portfolio management as a
management function that seeks to
maximize the overall quality of the
portfolio by (i) carefully screening new
projects prior to entry, and (ii) identifying
projects that are performing poorly for
early closure or restructuring.
Project level standards for SAR work
 a. Standard for Quality-at-Entry (Design)
Project Concept Note (PCN)
Project Appraisal Document (PAD)
 b. Standard for Project Supervision
Project status report (PSR)
Project completion report (PCR)
a. Quality-at-Entry Standard
A fully satisfactory project contains a well
conceived intervention model (i.e., storyline)
which…
is designed to bring about specific,
worthwhile and economic outcomes;
embodies a focused and realistic strategy;
can be expected to document and achieve
results (outputs and outcomes) within the
time frame and resources allocated to the
project.
Typical Problems, Quality-at-Entry:
 Bringing about specific, worthwhile, and economic outcomes:
Many projects have difficulty bringing about identifiable
outcomes because they are focused on other things:
(i) the PDO is too high (i.e., CAS-level, long-term, sweeping,
non-specific, non-attributable, unattainable), or
(ii) the PDO is too low (i.e., output level, supply-driven,
internally controlled by the project, or trivial).
 Embodying a focused and realistic strategy: Many projects lack
sufficient focus, realism and clarity with respect to strategy;
 Documenting and achieving results within the time and
resources available: Many projects do not document results
adequately, and then have difficulty convincing observers that
the intended results were achieved.
b. Quality in Supervision Standard
For existing projects in the portfolio, high quality
supervision is characterized by…
 proactive management of emerging problems, and
 consistent, honest reporting focused on documenting
and achieving the expected development outcomes.
High quality supervision is accomplished without
extension of the time frame or expansion of the
funding available to the project.
Quality supervision also meets or exceeds the
requirements of the Bank’s current guidelines and
procedures for projects under supervision.
Typical problems in Supervision:
Supervision plans often lack a strategic
vision and a necessary focus on outcomes;
Many projects fail to use key performance
indicators as a management tool during
supervision;
Too many projects lack a focus on outcomes
in supervision reporting (ISR, MTR, ICR)
3 – Focus and methodology
 It is important to focus on ‘outcomes’
 What is a focus on outcomes?
Terminology:
“Results”
“Impacts”
“Outcomes”
“Outputs”
It is important to focus on ‘outcomes’
Results
Outcomes
 Output, outcome or
impact (intended or
unintended, positive
and/or negative) of a
development intervention.
 Specific concept:
expected demand-side
response needed to
validate the causal chain
 Success means
progressive movement
along the causal chain
 Provides useful focus to
the PDO
What is a focus on outcomes?
Provides a convincing storyline
(outcome level);
Provides a clear and meaningful vision
of success (objective statement);
Answers the question: What visible
changes in behavior can be expected
among end users as a result of the
project, thus validating the causal chain?
Outcomes appear on the demand side of the
project, validating the causal chain
Project implementation environment
Longer-term goals
Project Objective
Impacts
+ -- ICR
Outcomes
(External)
Demand
Effectiveness
Project Deliverables
Outputs
Supply
(Internal)
Efficiency Project Components
Project Funding ($)
Activities
Inputs
Implementing
Agency
The previous Logical Framework has been
‘simplified’ to the Result Framework
Level
Hierarchy of
Objectives
Key
Performance
Indicators
(KPIs)
Means of
Verification
Critical
Assumptions
Impact
[IM]
Outcome
[OC]
PDO
External
Result Framework
Output
[OP]
Input
[IP]
Internal
Outcome-focused projects utilize
participatory design
Output focus
Outcome Focus
Using a storyline to structure a design concept:
Crossing the River of Uncertainties
Results
Present
Unsatisfactory
Situation
Future
Vision
of Success
The parts of a storyline
Present
unsatisfactory
situation
Future vision of
success
A dysfunctional storyline fails to provide a
consistent approach for the delivery of results
Results
Present
Unsatisfactory
Situation
River of Uncertainties
Future
Satisfactory
Situation
A good storyline bridges key areas of
uncertainty with a outcome-level approach
Results
Identify
Problem
Country
Context
Determine
Strategy
Specify Vision
of Success
[OC]
Specify
indicators to
verify success
Sector
Context
Process
Environment
Reporting
Environment
Future
Satisfactory
Situation
Present
Unsatisfactory
Situation
Project Storyline of PCN/PAD
‘Crossing the River of Uncertainty’
Appendix:
Project examples
• Rural water
• Rural health
• Transport sector
Rural water example (causal chain)
–Increase in rural productivity and decrease in
incidents of poor health caused by water-related
diseases
–Significant increase in the adoption and use of
improved water supply, sanitation and hygiene
Demand
practices by participating households (objective
achieved)
–Participating households gain access to completed
water supply and sanitation facilities (output delivered)
Supply
–Target population selects among available options for
water supply and sanitation services to meet their
needs at affordable prices
– Awareness raising activities to promote new
integrated approach
–Integrated water supply, sanitation and hygiene
services are made available at the county level
–Capacity building for integrated delivery of water
supply, sanitation and hygiene services at the county
level
Problem: Poor water supply, sanitation and hygiene practices limit the
productivity of rural households and often adversely affect health
Rural water example storyline
Problems
being
addressed
Strategy to address the
problems
Poor water
supply,
sanitation
and hygiene
practices
[OC] limit the
productivity
of rural
households
[IM] and often
adversely
affect health
[IM].
The basic strategy is to
expand access to improved
water supply, sanitation and
hygiene services in
participating villages using
an integrated three-in-one
approach, participatory
processes, and county-line
agencies as service
providers.
Total project costs:
US$ 75 M
Vision of
Success
(objective
expressed as
expected
outcome)
Evidence of Success
(KPIs)
Significant
increase in the
adoption and
use of improved
water supply,
sanitation and
hygiene
practices by
participating
households.
-Evidence* that
participating
households have
adopted the improved
practices [OC], as
expected.
- Evidence* of
reduced time spent in
collection of water by
households [OC].
___
* Via independent
technical and social
audit teams.
Rural health example storyline
Problems
being
addressed
- Use of
health
services by
poor rural
households
is decreasing
in key
provinces
due to a
number of
factors [OC].
Strategy to address the
problems
The basic strategy is to try
to make health care more
affordable at the point of
use through specific
initiatives related to health
financing, the quality of
service available, and
public health more
generally.
Total project costs:
US$ 72.5 M
Vision of
Success
(objective
expressed as
expected
outcome)
Evidence of Success
(KPIs)
To increase the
use of health
services by
poor rural
households in
targeted
provinces [OC].
- Evidence* of reduce
percentage of poor
households that cite
the high out-of-pocket
cost of health care as
a reason for not
accessing health
services when needed
[OC].
___
* Via independent
technical and social
audit teams.
Transport sector example
storyline
Problem(s) being
addressed
Strategy to address the
problem(s)
Travel in the
municipality is
constrained by
hilly terrain,
traffic bottlenecks
[OC] and
associated slow
travel times [OC].
- The project is guided by
the municipal development
strategy, which aims to
emphasize the role of
transport in improving core
urban functions for the next
5 years.
- A total of five
infrastructure and TA
components are included in
the project.
- Total project costs:
US$ 200 M
Vision of
Success
(objective
expressed as
expected
outcome)
Evidence of Success
(KPIs)
Traffic
bottlenecks
removed [OC]
in order to
improve core
urban
transport
functions [IM]
in the
municipality.
Evidence* that
travel times have
been reduced and
associated traffic
bottlenecks
eliminated [OC], as
expected.
___
* Via independent
technical and
social audit teams.
The End
‘Focusing on Development Outcomes
in Projects’
presented by Charles G. Chandler
Assumption Analysis, Inc.
www.AssumptionAnalysis.com