Dissemination and Use of Results from
OVC Program Evaluations
Florence Nyangara, PhD
MEASURE Evaluation/Futures Group
Dissemination Meeting, September 3rd, 2009
Washington, DC
Primary Objective of OVC Program Evaluations
■ Provide evidence to guide program decisions such as;
■ Scaling-up of best practices (models, strategies), and
■ Modify & improve interventions - to make them effective
Therefore, needed to collect quality and relevant data,
analyze, and use results to guide OVC programs
 Monitoring and Evaluation to Assess and Use Results
(MEASURE) Evaluation project
Overall Data Use Strategy
Employed a comprehensive data use strategy;
 Diverse stakeholders were involved throughout the study
 Ensured only relevant and useful data was collected by continuous
consultations between researchers & practitioners
 Data is packaged to meet the needs of target audiences
 Results are used to improve programs
Comprehensive Data Use Strategy
Stakeholder engagement: To ensure - support, ownership,
relevance and sustainability.
 Diverse stakeholders were involved (e.g. beneficiaries,
program)
 Capture different perspectives and information needs
 Stakeholders were involved throughout the study
 Get buy-in and promote ownership (consultation meetings)
 Continuous communication between researchers and various
stakeholders for updates (feedback).
Comprehensive Data Use Strategy
Ensured that only relevant data was collected- by
holding consultation meetings with donor, program
implementers, community, and beneficiaries. This
helped to;
 Identify key OVC program issues and information needs
for service, program, and policy decisions
 Identify program models for evaluation
 Inform questionnaire development
Examples
Example 1 - Dissemination of Case Studies:
 The 1st feedback – to share information on program descriptions,
implementation challenges, & opportunities
 Involved - program staff and in-country stakeholders
 Discussed and identified issues to consider for outcome evaluations
Example 2 - Dissemination of preliminary outcome evaluation results:
 Consultations with each program & key in-country stakeholders to
validate preliminary findings
 Presentations at international conferences and other forums helped
interpret findings
Packaging Data for and Reaching Various Audiences
Packaging information in various formats for diverse audiences
Publications: http://www.cpc.unc.edu/measure/ovc
 Six case study reports for each program evaluated1.
 Five briefing papers specific to each of the program evaluated
 Two summary papers - four CORE-funded programs
 Overarching paper on key findings
 Cost-effectiveness analysis paper
 One summary paper of key findings from the three studies in Tanzania
 Program-specific summaries of key findings (1-page) – TSA
Dissemination meetings
 Program sites
 In-country – national level
 International level
1 Jali Watoto – was a mini-case study
Use of Results Workshops: Tanzania Example
 Facilitated two workshops with OVC stakeholders in TZ
 TSA program staff (field-staff and managers)
 National OVC stakeholders (Implementing partners,
government, donors, bilateral agencies, etc)
Objectives of Results Use Workshops
 Present and discuss the key findings
 Develop actionable recommendations based on the results
 Develop a data use action plan to implement each of the
recommendations
 Develop and agree on a mechanism to monitor the data
use action plan
 Follow-up plan
Use of Results: Program Staff
 Findings were presented to the Salvation Army – Mama
Mkubwa program staff (field-supervisors, program
managers, M&E staff) from all regions
 Discussions of how TSA findings could be used to inform
program improvement and the well-being of OVC.
 Program-specific recommendations
 Developed a data use action plan
 Developed and agreed on how to monitor the plan
Use of Results: National Stakeholders
 Results were presented to National OVC stakeholders –
service providers, policy- makers, donors in TZ
 Discussions of how findings from three program
evaluations could be used for decision-making.
 National OVC program actionable recommendations
 Developed a data use action plan
 Developed and agreed on how to monitor the plan.
Example of a Recommendation
Researchers’ proposed recommendations were challenged &
participants came-up with their own, e.g.
 Researchers: Need to review & restructure Kids’ Clubs and
home-visit activities to make them more effective.
 TSA staff: volunteer motivation – through incentives.
 National stakeholders: more government involvement to
develop guidelines that will allow volunteer growth,
recognition, and ensure sustainability.
Information Use Bulletin
Contains:
 Reactions to the findings (surprises)
 Data use actionable recommendations plan (see National Tanzania)
 Responsibilities are assigned ( joint plans) which shows who will do
what - implicated people do something
 Follow-up plans to assess if the actionable recommendations are
implemented according to plan
 This formalizes the follow-up plans which is often forgotten after
disseminations
 Qualitatively (January/February, 2010)
Process Effect: Increased demand for more data for decision-making more programs want to conduct simple evaluations of these nature to
find out if their key program components (IGA) are making a difference.
MEASURE Evaluation is funded by the U.S. Agency for
International Development through Cooperative Agreement
GHA-A-00-08-00003-00 and is implemented by the Carolina
Population Center at the University of North Carolina at
Chapel Hill, in partnership with Futures Group International,
ICF Macro, John Snow, Inc., Management Sciences for
Health, and Tulane University. The views expressed in this
presentation do not necessarily reflect the views of USAID or the
United States government.
Visit us online at http://www.cpc.unc.edu/measure.