Outcome Mapping Training
& Discussion
Kaia Ambrose, MEL Advisor, CARE Canada
Marnie Davidson, Health Advisor, CARE Canada
outcomemapping.ca
Introductions
•
•
•
•
•
Programme planning is a significant task in my work
Monitoring is a significant task in my work
Evaluation is a significant task in my work
I do this predominantly directly within a project
I do this predominantly indirectly with a number of projects
(providing advice / support)
• I have read about OM
• I have tried using at least some parts of OM
• I intend to try out OM after this workshop
Objectives of our time
together
Clarify what Outcome Mapping is and does
Introduce and apply Outcome Mapping
concepts and tools for M&E
Consider if and how Outcome Mapping could
be useful in our work
What are your M&E
challenges and assets?
What’s working well (or what
has worked well?)
What is not working so well?
Origins and principles
of Outcome Mapping
Source: A guide for project M&E: IFAD
Social change can be…
Beyond the control of the intervention
Independent of project duration
Non-linear
Cumulative, with tipping points
Dynamic, multi-directional
Emergent
Challenges in evaluating social
change interventions:
 establishing cause & effect in complex systems
 reporting on emerging outcomes
 timing – when to evaluate
 reconciling different perspectives
 encouraging adaptive learning
 measuring what did not happen
 managing vested interests & power differentials
 seeking ‘sustainable’ results
10
What we get funded to do:
Thanks to Ricardo Wilson-Grau
11
What we actually deal with:
OUTPUT
OUTPUT
Expected &
Unexpected + & Changes

This complexity arises when…
Stakeholders have differing roles, histories,
perspectives, relationships, etc. They influence
and are influenced by their interrelations with
each other and their contexts.
 Contributing to sustainable improvements in
human or ecological wellbeing involves
influencing and being influenced by
stakeholders, their contexts and
interrelationships.
As we engage with the
actors, behaviours and
dynamics can emerge
which are unpredictable,
unexpected, uncontrolled,
non-linear and which
further influence the
actors and their
interrelationships.
Thus, we are dealing with varying degrees of
UNCERTAINTY
Challenges in evaluating interventions
 establishing cause & effect in complex systems
 reporting on emerging outcomes
 timing – when to evaluate
 reconciling different perspectives
 encouraging adaptive learning
 measuring what did not happen
 managing vested interests & power differentials
 seeking ‘sustainable’ results
15
Definition of OM
• A participatory method
for planning, monitoring
and evaluation
• Focused on changes in
behaviour of those with
whom the project or
program works
• Oriented towards social &
organizational learning,
adaptive management
Outcome Mapping:
Is ‘actor-centered’ to complement funding-,
problem- or objectives-centered perspectives;
Can be applied before, during or after the
implementation of an intervention;
Can be adapted to different PM&E needs and
contexts;
Can be used to support other frameworks and
methods.
OM is helpful when:
 Intended results include changed behaviors or
relationships;
 Desirable results may evolve or emerge;
 Intervention is seeking ‘sustainable’ results;
 You intend to monitor and adjust the
intervention;
 You would like to focus on results as defined by
local actors or beneficiaries.
18
Three key concepts in OM:
1. Sphere of influence
2. Boundary Partners
3. Outcomes understood as
changes in behaviour
There are limits to our influence
Project or
Program
Partners
sphere of
influence
Beneficiaries
sphere of
concern
Changes in behavior are ‘outcomes’
o u t c o m e s
inputs
activities
outputs
sphere of
influence
changed
behavior
patterns
sphere of
concern
changes in
conditions,
well-being
OM for planning &
developmental evaluation
• How should the world be in your sphere of
concern?
• Who are / could be / were the significant actors
in that world?
• Which of those actors’ roles could you / did you
influence?
• How would you / did you influence them?
• What would / did progress look like?
5 key assumptions
• Sustainable ecosystems and human well-being
depends on human behaviour
• There are limits to the influence that any intervention
can expect to exert
• People contribute to their own well-being (agency),
there are no passive beneficiaries
• Differing, yet equally valid perspectives will always
co-exist
• Ecological, social and economic resilience depend on
inter-relationships
Step 1:
improved human, social, & environmental wellbeing
26
A vision statement..
 defines a ‘system’ in which you
intend to support change
 guides the intervention
 motivates and inspires the team
 is an ‘accountability-free zone’
27
Step 2:
Mission
The mission states how the intervention
intends to contribute to conditions described in
the vision statement.
28
A mission statement:
 Describes what you do, produce
 Identifies who you work with
 Tells how & why you work with them
29
Step 3: Boundary Partners
• Those individuals, groups, & organizations with
whom a program interacts directly to effect
change & with whom the program can
anticipate some opportunities for influence.
Who are our Boundary Partners?
Project
or
Program
Boundary Partners
Beneficiaries
other stakeholders
31
Boundary Partners
• For clarifying intent: who do we need to work
with to affect change? Who do we have direct
contact with and who do THEY have direct
contact with?
• For evaluative practice: clarify who we need to
collect evidence from; deepening
understanding of relationships
Step 4: Outcome Challenge
✓ Describes how one boundary partner is
contributing maximally to the vision.
✓ Defines one boundary partner’s ideal
actions, relationships & activities.
33
Step 5: Progress Markers
Love to see
(Deep transformation)
Like to see
(Active engagement)
Expect to see
(Early positive responses)
Love to
see
Like to see
Like to see
Like to see
Like to
see
Like to
see
Expect to
see
Expect to
see
Like to
see
Expect to
see
Why Graduated Progress
Markers?
• Articulate the complexity of the change process
• Allow negotiation of expectations between the program and its
partners
• Permit early assessment of progress
• Encourage the program to seek the most profound
transformation possible
• Help identify mid-course improvements
Progress Marker Checklist
Each Progress Marker:
 Describes a changed behaviour by the boundary
partner
 Can be monitored & observed
As a set, Progress Markers:
 Are graduated from preliminary to more profound
changes in behaviour
 Describe the change process of a single boundary
partner
Facilitation questions
• How can the programme know the boundary
partner is moving toward the outcome?
• What would they be doing?
• What milestones would be reached as the
boundary partner moves towards their
intended role in contributing to the vision?
Mind your language!
Greater awareness…
Empowered women…
Community ownership…
Reduced conflict…
Increased collaboration…
Governmental commitment…
Gender sensitivity…
Equal access…
Budgetary transparency…
Active participation…
Poverty alleviation…
Strengthened capacity…
?
Step 6:
Strategy Map
causal
persuasive supportive
I
E
40
6 kinds of strategies
causal
I
aimed at
individual
boundary
partner
E
aimed at
boundary
partner’s
environment
strong
influence
persuasive
supportive
arouse new
thinking;
build skills,
capacity
on-going
support
alter the
broad
physical,
information
create /
regulatory or dissemination; strengthen
information access to new peer networks
environment
info
41
Step 7: Organizational Practices
How does your team or organization stay
relevant, viable and effective?
keep learning
 foster creativity & innovation
 seek better ways to assist your partners
 maintain your niche
 maintain high level support
 build relationships
42
5 kinds of M&E Information
Contextual InformatIon
Program
Partner
Change in
economic/ social
well-being
Strategies
outcomes
relevance & viability
(behaviour changes in the
(actions of the program)
partners)
implementation
(interventions by the program)
Outcome Monitoring
OM and the logframe
Looking at our system: Boundary Partners
and their interactions
Progress Markers
Logic Model
Indicator
Journal for Monitoring Outcomes
Outcome Monitoring Journal
Period of monitoring and data collection:
Contributors to Monitoring Update:
Name of the person(s) who compiled the journal:
Outcome Challenge:
Based on your analysis of each progress marker, please indicate the number on the scale that best represents the boundary partner's progress in
achieving the progress marker (1 indicates no activity towards the progress marker while 7 represents full achievement of the progress marker)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
Progress markers
What happened (who, how, Date Strategy Used and its
Follow up/corrective measures
what, where)
(when) Efectiveness
Evidence
(documents
etc.)
1)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
2)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
3)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
4)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
5)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
48
Thanks to Kevin Kelpin
Evolving Lessons
Beyond outputs
Good Enough (and build)
Keep It Real
M&E – mande – evaluative thinking – - explicit
sense-making spaces
Challenges
• Qualitative data collection – informal
interviews, observation (including looking for
unexpected – positive and negative)
• Qualitative analysis – looking for patterns and
trends
• Critical analysis and sense-making – the need
for facilitated, well-constructed (agenda,
exercises) processes
• Usage of information!
In monitoring & evaluation:
1.There is are no “silver bullets”
2. Seek competence with passion & integrity
3. Recognize & help celebrate achievements of
your partners
4. Be “idealistic realists”
5. Learn, and teach upwards
54