Welcome
• Review the theory of outcome mapping
• Go through the stages of OM
• Play with some of the ideas and do a
few exercises
• Why, how, and what, of OM
Introductory Exercise
Share an example of where/when a person,
organization, system or network began behaving
in a way that moved the community toward a
vision of community change (community impact).
What was most noticeable about that work?
1
Outcome Mapping:
The Theory
Guiding Assumptions of OM
• Multiple players/partners lead to change
• Partners will need to be supported –
have the capacity to effect change
• Planning ,monitoring and evaluation are
cyclical – not discrete events
• Non-linearity and contribution – not
attribution and control
• Actively engages the team in the “work”
What is Outcome Mapping?
• An integrated approach planning, monitoring
and evaluation.
• An approach that focuses on changes in the
behaviour, relationships, or actions of partners
(as outcomes).
• A methodology that characterizes and assesses
the program’s contributions to the
achievement of outcomes.
6
2
What is an Outcome in OM?
• Change(s) in the behaviour,
relationships, activities, and/or actions
of an individual, group, or organization
that the program was helpful in bringing
about.
7
Why Behavioural Change?
• To stress that development is done by,
and for, people
• To illustrate that although a program
can influence the achievement of
outcomes, it cannot control them
because ultimate responsibility rests
with the people affected
8
Why Outcome vs Impact?
Focusing on changes in partners’ behaviour,
relationships, or actions allows a program to:
• measure results within its sphere of influence
• obtain feedback about its efforts in order to
improve its performance
• take credit for its contributions to the
achievement of outcomes
9
• show progress towards outcomes
3
Outcomes = Behavioral Changes
Behavioral
Changes
10
How can OM be used?
• Designing and articulating the program logic
• Recording internal and external monitoring
data
• Identifying positive performance and areas for
improvement
• Evaluating intended and unexpected results
• Gathering data on contribution to changes
sought
• Establishing evaluation priorities and an
evaluation plan
How is it different?
• Monitors and evaluates contribution
• It introduces learning into the monitoring
and evaluation exercise
• Combines process and outcome
evaluation
• Recognizes that the program will
change over time in relation to its
partners
4
Today
Today
Stage 1:
Intentional Design:
Identifying Boundary
Partners
Step 3: Boundary Partners
Those individuals, groups, & organizations
with whom the program interacts directly to
affect change & with whom the program can
anticipate some opportunities for influence.
15
5
Program`s Sphere of Influence
Program
= Program`s Boundary Partners
16
Boundary Partners (have boundary
partners)
Program
Program’s Boundary
Partners
Boundary Partners’
17
Boundary Partners
Typology of IDRC Boundary Partners
IDRC
Program
Applied Research Community
e.g. researchers, institutions,
NGOs, networks
System Gatekeepers / Dev`t
Agents e.g. business, policy
makers, NGOs, gov`t
Beneficiaries
e.g. communities, farmers,
18
women, youth
6
Question
« In which individuals, groups, or
organizations is your program trying to
encourage change so that they can
contribute to community change?
19
Step 4: Outcome Challenge
Description of:
• The changed behaviours (relationships,
activities, and/or actions) of a boundary
partner
• How they could contribute ideally to the
desired community outcomes?
21
7
Phrasing of an Outcome Challenge
“The program intends to see [boundary
partner] who [description of behaviours
in the active present tense]”
“E.g. The Employer Challenge intends to
see local employers who adopt various
workplace practices that improve the
quality of life of their employees.
22
Example Outcome Challenge Statement
The program intends to see researchers in developing countries
that have gained the expertise required to carry out HIV/AIDS
research in a multidisciplinary fashion using quantitative,
qualitative, and/or participatory methodologies. They are
conducting research on the legislative, economic, social, and
health aspects of HIV/AIDS with special emphasis on vulnerable
groups including youth and women. They are providing credible
evidence that allows research findings to be translated into
effective HIV/AIDS policies and programs. They are skillfully
disseminating research results in the appropriate format for the
audience (e.g., to the scientific community, the general public,
NGOs, government officials). They are successfully obtaining
funding from multiple sources to conduct HIV/AIDS research.
More experienced researchers are mentoring those new to the
field and HIV/AIDS researchers globally are engaged in
productive communications using various means especially
electronic networks.
8
Step 5: Progress Markers
• A graduated set of statements
describing a progression of changed
behaviours in the boundary partner
25
Progress Marker Checklist
Each Progress Marker:
 Describes a changed behaviour by the boundary
partner
 Can be monitored & observed
As a set, Progress Markers:
 Are graduated from easier to more difficult to achieve
changes in behaviour
 Describe the change process of a single boundary
partner
26
Levels of Progress Markers
The program sets out what it would:
•
•
•
Expect to see its boundary partners doing?
Like to see its boundary partners doing?
Love to see its boundary partners doing?
27
9
Example of Employer Challenge
Limit to a manageable amount of data to be
collected: e.g.
• x “expect to see” (e.g. employers sign up for
e-newsletter, regularly read e-newsletter)
• y “like to see” (e.g. employers complete a
progressive workplace audit of their
organization, brown bag lunches, share
practices with other organizations)
• z “love to see” (e.g. employers adopt a
practice)
28
Example Model Forests
The program Expects to See local communities:
•
Participating in regular model forest (MF) partnership
meetings
•
Establishing a structure for cooperation in the partnership
•
Acquiring new skills for involvement in the MF
•
Contributing resources necessary to get the MF operational
(minimum)
29
Example Model Forests
The program would Like to See local communities:
•
Articulating a vision for the MF that is locally relevant
•
Promoting their involvement with the MF nationally
•
Expanding the partnership
•
Calling upon external experts when necessary
•
Requesting new opportunities for training
•
Disseminating concrete examples of benefits arising from MF activities
•
Seeking out new partners for the MF
•
Obtaining funding from different national sources
30
10
Example Model Forests
The program would Love to See
local communities:
•
•
•
Helping other MFs establish themselves
Sharing lessons-learned internationally
Influencing national policy debates &
formulation on resource use and management
31
Questions
« How can the program know the boundary
partner is moving toward the vision (desired
community change)? What would they be
doing? »
« What milestones would be reached as the
boundary partner moves towards their
intended role in contributing to the vision
(desired community change)? »
32
Exercise
Prepare an Outcome Challenge Statement:
 Describe desired behavioural changes about
a single boundary partner
 Try and identify the corresponding progress
markers for your outcome challenge
 Try and identify possible strategies aimed at
individual/groups and the I/G environment
33
11
Stage 2:
Outcome &
Performance
Monitoring
Factors affecting readiness to
monitor
You must have:
• Acceptance of the
monitoring system by
funders managers
• A champion within the
program
• Adequate internal
resources
• A defined use for the
monitoring data
These can be major
barriers:
• Past failures –
frustrations
• Superficial or undefined
motives
• Low levels of skill –
capacity
• Negative incentives for
monitoring
12
What are
How far have partners
we doing
progressed towards goals? to support
them?
Boundary
Partner
Outcome
Journal
How well have we
performed?
Program
Strategy
Journal
Performance
Journal
Step 9: Outcome Journal
• Uses progress markers to chart
boundary partner`s progress
• Sets starting point or benchmark
• Captures details on changes in the
boundary partner, contributing factors &
actors, and sources of evidence
39
13
Monitoring
Process
• Regular face-to-face meetings to record
observed changes
• Interviews and/or focus groups and/or
surveys with boundary partners
Data
• Anecdotes, examples, performance stories.
40
Set Values for Low, Medium & High
• Can capture either quantity or quality of
change:
– Quantity (number of boundary partners)
– Quality (depth of change)
41
Example of Employer Challenge
• 25 “expect to see” stories (e.g. employers
sign up for e-newsletter, regularly read enewsletter).
• 32 “like to see” examples (e.g. employers
complete a progressive workplace audit of
their organization, brown bag lunches, share
practices with other organizations)
• 20 “love to see” examples (e.g. of employers
adopting a workplace practice) that affects x
employees.
42
14
Stage 3:
Evaluation
Planning
Evaluation Questions
Indicating cases of positive performance and areas
for improvement:
• What worked well? Why? Are all necessary
strategies included? Are we spreading ourselves to
thin by trying to use too many strategies?
• How can we maximize our contributions?
Evaluating intended and unexpected results:
• Who changed? How did they change?
• If they did not change as expected, do we need to do
something different or re orient our expectations?
15
Evaluation Questions
Gathering data on the contribution that a program made to
bringing about changes in its partners:
• What activities/strategies were used?
• How did the activities influence individuals, groups or institutions
to change?
Establishing evaluation priorities and a deeper evaluation
plan?
• What strategies, relationships or issues need to be studied in
depth?
• How, and from where, can we gather relevant data?
Conclusion
16
Ten Years of Outcome Mapping
• OM enables interventions to be better
focused, more realistic and sustainable
• The intentional design stage of OM is
used most often
• Those who adapt or modify OM
concepts and practices find it most
useful
When OM works best
• When there is complexity in the issue
and the interventions involved
• Recognition and willingness to act upon
the complexity
• The commitment of one or more
champions and the availability of
support
Back to Work…
What are your questions about
Outcome Mapping?
For what initiative(s) you are
working with – if any -- might
Outcome Mapping be
appropriate?
17
Thank you!
www.irdc.ca.evaluation
www.outcomemapping.ca
18