Using Outcome Harvesting
in a mixed methods evaluation
of the
African Centre for Biodiversity
Barbara Klugman
[email protected]
Panel:
Case studies of Outcome Harvesting
principles in practice around the world
AEA, Atlanta, October 2016
Evaluation Questions
• What outcomes has ACB contributed towards?
• ACB’s strategies and outputs: are ACB’s publications,
information, capacity development processes, and
analysis diffusing to targets?
• How are ACB’s reputation and effectiveness perceived
by field?
• Are ACB’s current monitoring & learning tools &
systems effective?
• Are ACB’s governance, organisational and management
systems / staff complement effective?
• Perspectives on ACB’s positioning, threats and
opportunities – looking forward?
Multiple methods to answer questions
•
•
•
•
•
•
Surveys: publications list (81 responses)
Consumer action alerts list (283 responses)
Media list (13 responses)
Primary partners (24) plus ACB
Interviews (32)
A series of meetings with staff
– Orientation & draft ideas for theory of change
– Finalising theory of change, reviewing baseline
– Reviewing evaluation data and creating outcome chains
• Outcomes harvesting (293) & substantiation (31)
1 survey question elicits outcomes
I have used ACB information or analysis for (please mark all that apply):
My own lifestyle choices
Our own research
Building understanding among
my peers/staff or org members
Mobilising the public (marches,
petitions, twitterthons)
Advocacy & lobbying of gov
Publicising issues in the media
Engaging legal & policy issues
Training
Social network survey to ACB partners answered the
evaluation question regarding its role in the field relative to
other partners
Which partners provide information
Which groups campaign together
Outcome Harvesting: establishing
plausible influence & contribution
in complex systems & processes
requires tracing strategies &
outcomes over time
ACB sent out a
consumer
petition;
2008
The
Agricultural
Research
Council
(ARC)
applied for
commercial
release of
its Spungta
G2 potato.
Over 2000
members of the
public signed
the petition
[Outcome]
Industry players
wrote letters
against release
of this potato
[Outcome]
ACB attached
these responses
plus studies &
sent these to
the GMO Act
Executive
Council
2009:
Executive
Council
rejected
ARC
application
[Outcome]
ARC lodged
an appeal
ACB lodged
request for
their docs
2012
2011
ACB
accessed
the docs
& sends
inputs on
the
appeal
the GMO
Act
Executive
Council
decides to
reject the
commerciali
zation of
GM potato
[Outcome]
2015
the
Minister
upholds
the EC
decision
[Outcome]
Reveal Patterns & Processes of Change
Grounded research builds capacities of
farmers who influence policy change
2012
ACB publishes on
commercialisation of African
agriculture.
[Output]
2013
Global Food Report publishes it
[Outcome]
ACB initiates research
relationships with farmer’s orgs
[Strategy]
2014
- ACB publishes with MVIWATA
[Output]
- ACB hosts Tanzania meeting
[Strategy]
- Government attends [Outcome]
- MVIWATA speaks on their
concerns using the published
findings [Outcome]
2015
new Tanzanian seed law
incorporates some issues
put forward by MVIWATA at
the ACB-hosted meeting
[Outcome]
Patterns in complex processes: mobilising to prevent
Africa seed harmonisation policy
ACB publishes Harmonisation of Seed laws in Africa – A recipe for
ACB press release alerts African CSOs re proposed
ACB & TWN host regional
disaster
SHL
expert meeting Uganda [Out
They circulate it & 20 orgs
ACB/TWN/Berne Declaration draft comments on ARIPO draft legal framework
They submit to ARIPO in the name of ‘African CSOs’
endorse [114]
2012 November
2013
March, ACB proposes AFSA seed working group; circulates
draft advocacy doc on UPOV to AFSA [123]
World Bank publishes response
to ACB’s publication
Throughout 2013/2014 ACB
builds CSO capacity &
comments on seed & PVP laws:
Mozambique, Malawi, Ethiopia,
Ghana, Uganda, Zimbabwe
July ARIPO invites CSOs to regional workshop on ARIPO legal framework for Protection of New Varieties of Pl
July AFSA establishes seed
working group led by ACB
[126]
Mozambican CSOs lobby
gov on seed laws [127]
German Ministry Development meets
ACB to discuss PVP [Outcome]
German gov commissions study on UPOV
impact [206] released July 2015: unsuitability
of UPOV 1991 for developing countries
2014
May/June ACB lobbies ARIPO to includes C
meetings especially Ministerial Meetin
ACB et al distribute comments
to member states
CSOs attend & lobby
government delegates
CSOs from AFSA attend with ACB-R
Ghana CSOs advocate on PVP laws[129]; ACB/TWN
assist FSG critique laws; global media coverage
TWN alerts Tanzanian CSOs re their government
decision to adopt PVP law aligned to UPOV 191
(Tanz is 1st LDC to join)
Tanzania CSOs invite gov to meeting
& urge not to revise PVP law
ARIPO officially responds
to CSO submissions on
proposed PVP protocol
[125]
Feb: FSG invites ACB to input at workshop on strategy to
block draft Ghana PVP law [148] CSOs major public campaign
November ARIPO
meeting: CSOs attend &
lobby government
delegates
ARIPO lodges Draft PVP Protocol
with UPOV to get its approval for
ARIPO to join UPOV 1991
March: ACB &TWN host regional
meeting on Seed Law in Cape Town
June: Zimbabwe ACB supports
June: Zambia ACB supports
Aug: Uganda ACB-RRM
attended by 52 CSOs & gov people
workshop CSOs proposes seed
workshop with CSOs & gov
supports
Food Rights Alliance
from Botswana, Ethiopia, Lesotho,
platform; gov & ARIPO are
officials
Kenya, Malawi, SA, Swaziland,
to
host
CSO
capacity dev on
present [162]
Tanzania, Uganda, Zambia,
seed/PVP [175]
ARIPO October
Sept: Tanzania National workshop
Nov: FSG & CIKOD successfully halt Plant
Zimbabwe
AFSA & 2 mem
attend ARIPO
workshop with
support
TWN & Berne Declarat
legal advice on legality &
complaint with UPOV
which discusses and di
July ACB-RRM has N
expert legal opinion o
of AIRPO Protocol in
international law; ACB
AFSA submits app
ARIPO, their membe
UNECA [170
stakeholder meeting
Breeders Bill – Parliament sends it back for 2nd
reading [225]
ACB-RRM supports Commons for EcoJustice,
Dec: Zimbabwe CSOs launch seed platform [163]
Malawi to attend for AFSA [184]
FAO takes account of AFSA recomms for National Seed Policy
Dec: ACB media release on ARIPO
Dec: APRODEV uses ACB media release to lobby EU [193]
Guidelines recognising informal seed systems issues [180]
Sept: ACB gets FAO policy guidelines
& does comments in AFSA’s name
CSOs urge gov not to adopt [O]
May: ACB hosts regional workshop in
2015 Addis on ARIPO/seed harmonisation
in Addis & how CSOs can lobby govs
Zimbabwe gov voices critique of
ARIPO Protocol [292]
May: Zambian CSOs write
to gov on ARIPO &
national laws [283]
Feb: Malawi gov voices
critique on ARIPO Protocol
[238]
Mobilising to prevent Africa seed policy
2012
ACB published its research on implications
of ‘seed harmonisation’ [Output]
Various media publish [Outcome]
ACB partners with 2 INGOs to draft
comments on ARIPO’s proposed legal
framework [Output]
20 organisations endorse the comments
[Outcome]
ACB & partners submit to ARIPO [Strategy]
2014
ACB/AFSA with TWN/Berne Declaration
continues research & advocacy [Strategies]
Government attends meeting that T CSOs
invited them to [Outcome]
CSOs lobby government not to change their
laws
FAO (UN) incorporates FSA recommendations
for National Seed Policy Guidelines [Outcome]
2013
ACB/TWN trains groups re the comments
including from T; alerts T groups that T gov
is about to adopt negative law [Strategies]
AFSA agrees to establish seed working
group; asks ACB to lead [Outcome]
ACB mobilises AFSA members to pressure
ARIPO for inclusion [Strategy]
ARIPO invites AFSA members to
participate in its meetings [Outcome]
2015
Media gives coverage to CSOs & ESAFF
messages against ARIPO [Outcome]
Similar processes in Uganda, Malawi,
Zimb, Zamb [Strategies]
M, Z & Z governments refuse to
consent to adopt ARIPO’s protocol &
influence pro-small farmer changes
[Outcome – partial victory]
Thank you!
[email protected]
See
ACBio.org.za
Using Outcome Harvesting in a
mixed methods evaluation
of the
African Centre for Biodiversity
Barbara Klugman
[email protected]
Case studies of Outcome Harvesting principles in practice
around the world
American Evaluation Association
Atlanta, Georgia, October 2016
Creation & implementation of
government programmes 0.34%
Diminished or blocked
corporate power
2,7%
Shifts in
mindset and
public
discourse
12%
Strengthened
collective public
action
9%
Increased
visibility in
media and
public
discourse
12%
Changes in gov & private sector
laws, policies & in jurisprudence
5.5%
Advocacy
Writing to Inputting
gov or on policy
corporations docs
Engaging
policy
spaces
Members of
the public
ACTORS
ACB
Communications
Media
(Website/FB &
press releases)
Campaigns
(Petitions, marches,
consumer actions)
Supporting
Litigating
partners’
participation in
policy spaces
Farmers’ & civil society
organisations &
networks
Strategies
Strengthened
alliances &
networks
31%
Organisation & movement building
Networking &
relationships
(AFSA, Food
Seed list)
Advocacy capacity
development –
workshops & indiv
support
KEY:
Outcomes
Strengthened
CSO Advocacy &
Litigation
27%
Research & Publications
Desk research Participatory Action
Research
Research
partnerships with
farmers’ associations