Annex 1
United States Agency for International Development
Bureau of Democracy, Conflict and Humanitarian
Assistance - Office of Food for Peace
Scope of Work for the Mid Term Evaluation of the
Northern Karamoja Growth, Health and Governance
(GHG) Program
Award No. FFP-A-12-00006
1
Scope of Work for Mercy Corps GHG Mid-Term Evaluation
Program to be Evaluated:
Donor:
Location(s):
Timeframe:
Northern Karamoja
Growth
Health
and
Governance Program (GHG)
USAID – Food For Peace (Title II Development
Food Assistance Program)
Uganda, Northern Karamoja - Abim, Kotido and
Kaabong Districts
June – August, 2015
Introduction
The 1.2 million inhabitants of the Karamoja sub-region of northeastern Uganda face a slow crisis
produced by the breakdown of their traditional agro-pastoralist livelihood strategy, repeated climactic
shocks and ongoing rural insecurity. The decay of social norms and institutions, such as elder authority
and the meaningful, active role of young men and women in tribal culture, inevitably accompany the
crisis, as does malnutrition for children, high rates of gender based violence, and gender inequality. It is
difficult to overstate the challenges facing the Karamojong.
Those challenges are, however, matched by
opportunities for long-term development with the
potential for significant impact on the food
insecure. As of early calendar 2014, four
successive seasons of good rains have filled
many village granaries, although pockets of
extreme food security persist (especially in
Kaabong district), and the security situation is
improving thanks to successful military
disarmament
and
non-governmental
peacebuilding efforts. In recognition of the subregion’s development, emergency humanitarian
efforts are scaling down, leaving a gap for more
development-focused interventions to fill. There
is a peace dividend waiting to be realized in
Karamoja.
The Northern Karamoja Growth, Health and
Governance (GHG) program was designed to
capitalize on this pivotal moment through a range
of integrated economic, health, and governance
initiatives that will cement the gains from
increased security and build a foundation for
broader self-sufficiency and improved health,
while well targeted food aid for pregnant and
lactating women and children under the age of two hastens the process of transition from decades of food
aid by filling nutrition deficits in highly food insecure households. GHG’s geographical focus includes
the northernmost three Karamojong districts of Kaabong, Kotido and Abim, home to approximately
540,000 individuals. It has three broad integrated strategic objectives (SOs):
2
Strategic Objectives
Intermediate Results
SO1: Livelihoods Strengthened
IR 1.1: Improved Productivity among Male and Female
Agriculturalists, Agro-pastoralists and Pastoralists
IR 1.2: Market Access and Marketing Behaviors Improved
IR 1.3: Business Environment Improved
SO2: Nutrition among Children under
Two Improved
IR 2.1: Access to Quality Maternal and Child Health and
Nutrition Services Improved
IR 2.2: Household Maternal and Child Health and Nutrition
Practices Improved
IR 2.3: Sustainable Access and Appropriate Use of Safe Water
and Sanitation Facilities Improved
SO3: Governance and Local Capacity
for Conflict Mitigation Improved
IR 3.1: Local Conflict Management Capacity Strengthened
IR 3.2: Cooperation Between Formal and Informal Governance
Structures Increased
IR 3.3: Constructive Male and Female Youth Engagement in
Peace and Development Initiatives Enhanced
Attached as an annex is the approved IPTT for the GHG program
The GHG consortium pulls several complementary capacities into one coherent, unified approach,
embodied in the facilitative strategy that pushes local actors out front to sustainably provide the products
(e.g. seeds and energy) and services (e.g. land opening, animal husbandry, transport, security, finance and
healthcare) that make life productive, healthy and meaningful in the 21st century. As the consortium lead,
Mercy Corps takes overall responsibility for the development of GHG’s facilitative strategy as well as the
quality of all programming implemented by the project. It also implements all economic programming
and employs a Gender Advisor to ensure that an understanding of Karamoja’s complicated gender
dynamics are incorporated into all activities. World Vision Inc. (WV), with its robust commodity
management and public health qualifications, is leading supplementary feeding activities, communitylevel public health initiatives, and water, sanitation and hygiene programming. Two extraordinarily
knowledgeable local partners, Kaabong Peace and Development Agency (KAPDA) and Abim Women
Together in Development (AWOTID), constitute the spear-tip for GHG’s conflict management and
governance activities, working primarily through local formal and informal authority structures to bolster
local systems for conflict reduction and government service improvement. Lastly, Tufts University’s
Feinstein International Center (FIC) is charged with performing annual impact evaluations using its
community-focused Participatory Impact Assessment methodology that will help the rest of the
consortium partners understand the impact (or lack thereof) of their work.
Foster Pro-Poor Growth to Increase Food Security: Traditionally, growth-focused interventions have
skewed benefits toward wealthy segments of the population with the misguided hope that prosperity
would “trickle down” to smallholders, while community-focused programs tended to focus too heavily on
the symptoms of poverty without addressing structural causes of food insecurity for large portions of
society. The GHG approach to livelihoods enhancement combines these two schools of thought by
generating economic growth in Karamoja with interventions that verifiably impact the economically
vulnerable with specific attention to women. After five years, GHG will leave behind a system where
poor households in Kotido, Kaabong, and Abim are empowered to regularly access quality inputs and
services to improve production, sell their commodities at fair prices and access financial services to
improve income, and advocate for improved government policy. In addition, GHG aims to improve
3
accountability and management within the public health system through the support of HMIS mentorship
and supervision and the reconstitution of the HUMCs.
Many of the root causes of food insecurity in Karamoja, such as reduced mobility for pastoralists; and
reduced access to milk animals, result from both a lack of access to good information and inputs as well
as a continuing level of some insecurity. Although it should be noted that the conflict dynamics in the
region are rapidly changing; and hard-won food security gains can be quickly undermined by conflict.
Therefore addressing the underlying causes of conflict is essential to create and sustain improvements in
income, health and food security. Working through Karamojong civil society organizations (CSOs)
Pastoralism and Poverty Frontiers (PPF) and Kaabong Peace and Development Agency (KAPDA),
SUSTAIN will address the conflict dynamics in Karamoja by strengthening the capacity of local
government and communities to address conflict-related causes of food insecurity. This will lead to
greater mobility by vulnerable populations, increasing their access to both quality and quantity of food, as
well as access to health services (Strategic objective 2)
Evaluation Purpose & Description
The primary aims of the mid-term evaluation are to assess the strengths and weaknesses of program
implementation, identify factors associated with activities’ greater or lesser performance, to identify early
evidence of whether the activities are leading to change (positive or negative) and factors that may be
impeding or promoting positive change, in order to recommend adjustments to activities that would help
maximize intended results. In order to accomplish this aim, Mercy Corps is searching for a dynamic and
organized mid-term evaluation consultant team with proven experience in mixed research methods in
semi-arid lands of Africa. The mid-term evaluation consultant team will produce a comprehensive report
based on qualitative methods (and review of former quantitative research) critically evaluate program
relevance, connectedness, coherence, coverage, efficiency, effectiveness, outputs, and any early
outcomes; and along with a team of program leadership staff to present a series of recommendations and
considerations to maximize impact and capitalize on key synergies for the second half of the program.
Evaluation Objectives
The objectives of the evaluation will be:
1. To evaluate the strengths and weaknesses of program implementation and the quality of outputs,
in terms of adherence to terms agreed by FFP and their acceptability and perceived value to target
communities, identifying factors that appear to enhance or detract from the quality, acceptability
and usefulness of implementation and outputs. There should be an explicit focus on looking at the
market facilitation and facilitative strategy as the foundation of program implementation and
design. There should also be attention paid to look at probability of the sustainability beyond the
life of the program as a result of using the facilitative approach.
2. To seek evidence of changes (intended and unintended) associated with program activities,
evaluate how the changes relate to progress toward program objectives, and identify factors that
appear to promote or hinder the program’s progress toward desired objectives.
3. Based on the findings from 1-3, to recommend adjustments to program implementation or design
and explain how these changes would improve program outcomes and sustained impact.
4. To facilitate a workshop to engage various stakeholders (program staff, donors, local and national
government, beneficiary communities) in an analysis and reflection on the evaluation results to
determine potential program adjustments and strategic planning.
4
Key evaluation questions
The primary key evaluation questions around which data collection, analysis and reporting should be
organized are:
1. How well have the project’s activities’ implementations achieved planned schedules, beneficiary
numbers and outputs? What factors promoted or inhibited adherence to schedules? How were
problems and deterrents managed?
2. What are the strengths and weaknesses of the basic framework of the project design and
implementation so far? What factors of the implementation or context appear to advance or deter the
various activities’ acceptance in target communities and efficient generation of planned outputs of
high quality?
3. In each technical sector what are the strengths and weaknesses in the implementation design and
processes and the quality of outputs? What factors in the implementation and context are associated
with greater/lesser efficiency in producing outputs of higher/lower quality? Which activities and
implementation processes are more/less acceptable to members of the target communities and why?
4. What signs/reports are there of changes associated with project activities, e.g., what changes do
community members and other stakeholders attribute to the activities? What factors appear to
promote the apparent changes or deter intended change?
5. How could the project be modified to improve its acceptability to targeted communities and the
efficiency and effectiveness of the implementation?
The following table highlights aspects of the project implementation and design to consider in the
evaluation and suggests illustrative methods of investigation and evaluation to further describe the type of
evaluation design expected.
Evaluation questions will be refined and reviewed using a participatory process including the feedback of
the mid-term evaluation consultant team and key technical experts, with input sought from relevant
program staff. Findings, conclusions and recommendations for these questions may be presented
according to strategic objective for the technical components, cross-cutting areas and implementation
processes.
Area of Focus
Aspects to consider
Illustrative methods of
investigation/evaluation
How well have activities’ implementations achieved planned schedules, numbers of beneficiaries, and
outputs? What factors promoted or inhibited adherence to schedules? How have problems/ deterrents
been managed?
 Start dates and rates of
 Use secondary data from routine
Adherence to activity
expansion of coverage,
monitoring, ARRs and other reports to
schedules
beneficiary numbers, and
compare planned and actual start
outputs for each activity type
dates, numbers of outputs and other
 Numbers and timeliness of
targets, noting differences in
planned beneficiaries and
achievements according to location,
outputs, e.g.,:
implementing partner, or sector
 formative research/barrier
 Compare across locations, beneficiary
analyses
groups, project administrative unites,
 various types of direct or
etc. to identify factors associated with
embedded trainings
differing degrees of achievement.
 indirect beneficiaries
 Interview members of project staff at
following training of
various levels about factors that
trainers, farmer to farmer,
delayed or interrupted activities and
1.
5
Area of Focus
Aspects to consider






cascades, market linkages,
etc.
formation of or
connections with
community groups,
business associations, etc.
various types of
community and household
infrastructure constructed
or rehabilitated
community action plans
distributed cash and goods
loans facilitated
asset rehabilitation or
creation
linkages between
beneficiaries and markets,
private sector actors,
financial institutions, etc
facilitated.
Illustrative methods of
investigation/evaluation

outputs, and how problems were
identified and overcome
Ask groups and individuals from
different stakeholder groups at
locations of greater and lesser
achievements about factors they
believed inhibited or promoted
efficiency and efforts have been made
to overcome barriers
2.
What are the strengths and weaknesses of the basic framework of the project design and implementation
so far? What factors of the implementation or context appear to advance or deter the various activities’
acceptance in target communities and efficient generation of planned outputs of high quality?
 Strengths/weaknesses of the
 Review the project work plan and
Project Management
project work plan and schedule
schedule to assess how completely and
clearly it defines the work needed to
 Evidence that management has
meet objectives, when and by whom.
explored and implemented
Is the schedule feasible?
new/innovative ideas and
 Examine the roles of the different
approaches
implementing partners and how the
 Evidence that management has
plan promotes good collaboration
adapted workplans and
among them and leverages partners’
program approaches based on
relative advantages.
information gathered as part of

Interview members of management
program implementation.
about outcomes of work plan reviews
 Changes and challenges in the
and how they handled changes and
operating context and how
challenges that presented.
management responded
 Ask implementing staff in different
roles how feedback and ideas are
solicited and shared within and among
partners, especially how field staff.
 Ask implementing staff who
participates in program monitoring and
review and how participatory that
process is.
 Adequacy of numbers and
 Review the characteristics and
Staffing
capacities (knowledge,
capacities of staff at all levels in all
experience) from beginning of
sectors, and assess their confidence
activity until present
and capacities to perform assigned
tasks
 Strengths/weaknesses of

Review training and supervision
supervision and support to
6
Area of Focus
Aspects to consider


Engagement with beneficiary
communities




assure accountability,
performance, and confidence.
In/adequacy of resources
(tools, work space,
transportation, communication,
information, work aids) to
support activities’ efficient
performance at all times from
start to current time
Gender sensitivity and balance
at various levels of authority
Strengths/weaknesses of the
ways the project has included
community members,
including vulnerable or
marginalized members, in
decisions about activity
choices, design,
implementation, and
monitoring
Community members’
perceptions about their
participation in the project and
the degree and nature of their
engagement with project staff
Use of incentives of different
types
Safeguards against exploitation
and discrimination
Illustrative methods of
investigation/evaluation




Cultural acceptability





Implementation methods: type,
timing, style
Interactions between
implementing staff and
community members
Interactions between
implementing staff and local
partners (community groups,
private sector actors, etc.)
Interactions between private
sector participants and
intended beneficiaries of their
products & services
Outputs

schedules for monitoring and
supporting activity staff , including an
assessment of the numbers of people
and sites per supervisor
Interview a sample of field staff and
supervisors in different
sectors/activities and at different levels
about
 factors that affect their performance
and motivation,
 sources of dis/satisfaction with
resources, training, supervision,
support from technical experts,
 their roles in decisions about
activity design and implementation
 training they received
Interview groups and individuals from
different stakeholder groups about
 Who has been involved in the
project and how?
 Who else would have liked to be
involved, and how?
 Dis/satisfactions with the way
project staff and community
members engaged
 Reports of or potential for
exploitation or discrimination by
implementing staff or with project
resources
Interview volunteers who contribute
significant amounts of unpaid time to
project-supported activities to learn
what motivates their participation and
performance.
Interview program partners
(community groups, private sector
actors, etc.) to learn what motivates
their participation.
Interview members of target
communities, government counterpart
agencies, other local partners and field
staff to assess perceptions and
attitudes about the choice,
implementation and outputs of
activities.
 How well do the activities address
perceived needs?
 What aspects do they dis/like?
 How would they prefer things to be
done?
7
Area of Focus
Aspects to consider
Illustrative methods of
investigation/evaluation
Communication




Partnerships and Linkages



Financial management



Branding

Interview implementing staff,
government counterparts, and staff of
linked projects about
 the nature and sources of
dis/satisfaction with their
collaboration and ways that it could
be improved
 how they feel their cooperation
benefits the implementation and
results on both sides
 other projects, agencies, groups that
are doing similar or complementary
work to which the project is not
linked
 Review samples of project’s MOUs
with collaborators



Completeness and clarity of the
documented M&E plan
 Ways the system:
 collects data useful to
monitor the quality and
outputs of processes


Interview members of implementing
partners, communities, government
counterparts, and other stakeholder
groups to assess:
 Knowledge of objectives, activities
and implementation, activity
duration, eligibility, outputs, and
entitlement transfers
 How/when they learned about
project objectives and activities
 Frequency and content of
communications with other types of
stakeholders

Compliance with USAID
policy
Knowledge and attitudes
toward donor and
implementers within target
communities

Monitoring & Evaluation
Quality and timeliness of
communications of vision,
objectives, plans,
implementation guidelines, and
other project information
among project staff, partners,
government counterparts and
communities
Knowledge in various
stakeholder groups about the
project
Strengths/weaknesses of the
ways the project encourages
and handles feedback from
staff and partners
Consider collaboration/links
with:
 Other USAID projects
 Government activities
 Other complementary
activities in the project area
 Private sector initiatives in
the area.
Strengths/weaknesses of
coordination within the project
and between the project and
other projects and agencies
Factors that make partnerships
more/less beneficial to the
project implementation
Financial accountability
Sufficiency of finances to
assure good project
implementation
Flexibility of the budget to
respond to changing conditions


Review financial records
Interview managers about
 the adequacy of finances and effects
of financial constraints on project
implementation
 perceived limits of financial
flexibility to respond to change
Assess how well planned and actual
actions do/do not comply with USAID
branding requirements
Interview beneficiaries and
community leaders about their
knowledge of and attitudes toward
USAID and implementing partner
agencies
Critically review the M&E plan and
systems: staffing, processes, and
outputs.
Interview staff in various roles in the
collection, analysis and reporting of
routine monitoring about their
8
Area of Focus
Aspects to consider
Illustrative methods of
investigation/evaluation





Environmental Compliance



Commodity management



solicits and reports opinions,
ideas and concerns from
field staff
 provides constructive
feedback to implementing
staff to inform, assist, assure
accountability and motivate
good performance
 assures accurate reporting to
USAID
 supports timely problemsolving and decision making
for all stakeholders
 assures data quality:
validity, reliability,
timeliness, integrity and
precision
 has been used to adjust
implementation
 is or is not supported by the
institutional structures
 monitors environmental
impact
 monitors gender equity
The strengths/weaknesses of
data collection methods
The roles of data bases in
monitoring, analysis and report
generation
Challenges the M&E team
faces
How non M&E implementing
program staff are involved in
monitoring.
Adherence to the
Environmental Mitigation and
Management Plan (EMMP)
Adequacy of the EMMP
Recognition/ avoidance of
unforeseen damage
Division of roles and
cooperation among project
staff, external partners and
community members
Common causes of delivery
delays and commodity losses
Adequacy of mechanisms to
safeguard against loss or abuse






activities and roles, to determine their
understanding and confidence in the
data collected, and challenges they
face getting or using the data.
Interview recipients of reports and
other outputs about how they use the
information they receive, which
information is most useful, the
timeliness of the information, and any
other information they would like to
have
Interview key decision makers about
the timeliness and usefulness of the
data from the M&E system
Examine how well planned and actual
actions and outputs do/do not comply
with the project’s EMMP
Interview technical experts,
implementing staff and other key
informants about project activities’
apparent or potential threats to the
environment and identify those not
addressed by the EMMP and how well
the project implementation has
addressed these threats
Review processes and records related
to tracking, management, and delivery
of commodities to points of use or
distribution: identify challenges and
measures taken to address them
Inspect warehouses and storerooms of
various sizes and managed by
different entities
9
Area of Focus
Aspects to consider
Illustrative methods of
investigation/evaluation

Quality and cleanliness of
 Interview people involved in the
storage facilities
resource management about what is
 Completeness and clarity of
and is not working well, why, and
commodity records and reports
how problems have been addressed
 The comprehensiveness of the  Critically review the exit/sustainability
Exit Strategy/ Sustainability
exit/sustainability strategy
strategy and progress in its
 The identification of specific
implementation in light of the findings
sustainability partners
related to the challenges to practices
 Factors that threaten the
promoted by and threats to
continuation of targeted
infrastructure developed by the project
practices and services and the
 Interview key informants,
maintenance of new
sustainability partners, and
infrastructure
beneficiaries about threats and
 Progress in implementing the
promoters of targeted practices and
strategy
infrastructure
 Ways the project is
strengthening or establishing
links between communities and
private or public financial,
market, or technical resources
3. In each technical sector what are the strengths and weaknesses in the implementation design and
processes and the quality of outputs? What factors in the implementation and context are associated
with greater/lesser efficiency in producing outputs of higher/lower quality? Which activities and
implementation processes are more/less acceptable to members of the target communities and why?
 Agricultural/livestock
 Application of findings from
 Review formative research and
production
formative research to
evaluate how well implementation
 Food/Cash for Work
implementation
does or does not apply the findings.
 Income Generation
 Technical quality of project
 Observe activities and talk with
 Livelihood diversification
inputs and outputs
implementing staff, program partners,
 Health & Nutrition
 Strengths/weaknesses of how
and direct beneficiaries about:
 Behavior Change
the various activities engage
o what activities are more/less
Communication
target groups and protect
effective
 WASH
against unintentional harm
o what/how could activities be
 Direct & Indirect training
 Selection of direct
improved
 Women’s Empowerment
beneficiaries; coverage of
o which activities are more/less
& Promotion of Gender
target groups
interesting or useful
Equity
 Selection of program partners
o who benefits; who should benefit;
 Prevention of Gender
 Perceptions of quality,
how beneficiaries are selected
Based Violence
appropriateness and use of
o opportunity costs of participation in
 Natural Resource
distributed goods and
activities
Management
promoted services
o their knowledge and understanding
 Early Warning Systems
 Composition, activities and
of key project messages
and Disaster Risk
governance of groups created
 Compare/contrast men’s and women’s
Reduction
or promoted by the project
participation and perceptions
 Networks and connections
 Inspect the technical quality of
facilitated by the project
community and household
 Collaboration with and support
infrastructure to which the project
to relevant government service
contributed
providers
 Talk with non-beneficiaries from the
 Cultural acceptability and
same communities about: availability
relevance of project methods
and access to goods and services as a
and messages
result of the programs interventions
10
Area of Focus
Aspects to consider
Illustrative methods of
investigation/evaluation

 Examine the composition of the
various groups created or supported by
the project: who in the community did
or did not join. Why or why not?
 Talk with members of groups formed
or promoted by the project about:
 How the group was formed ; level
of satisfaction with group
composition
 Challenges/successes working as a
group
 Nature and adequacy of support
from program
 How and why the group chooses
activities
 How members’ other roles in the
community affect their participation
in the group
 Examine beneficiary records to assess
the proportion of households and
communities benefit from multiple
sectors in different combinations.
 Talk with program staff to
understand who was targeted for
multiple sectors and why
 Interview members of households
benefiting from single and multiple
program activity sectors about their
participation; Compare characteristics
of those who benefit from one vs
multiple sectors
 Review messages on similar topics
transmitted through different pathways
for consistency and clarity
 Compare the understanding of the key
messages of trainers and direct and
indirect trainees
 Interview staff members about their
interactions with staff working in other
sectors, especially regarding site and
beneficiary selection and developing
messages to beneficiaries
 Interview members of various types of
groups initiated by the project about:
e.g., making decisions, managing joint
resources, sharing information &
experiences.
 Interview private sector partners about
level of satisfaction with their
participation and the program, intention
to continue activities with



Consistency of content and
recipients’ understanding of
similar messages received via
different pathways
Strengths/weaknesses of
measures taken to assure
gender equity with regard to
access to, participation in, and
benefit from project activities
Strengths/weaknesses of
linkages, coordination and
integration among the different
sectorial and cross-cutting
components
The validity and
comprehensiveness of
assumptions in the project’s
Theory of Change that are
critical to activity
implementation and outputs
11
Area of Focus
Aspects to consider
Illustrative methods of
investigation/evaluation
beneficiaries, expansion plans.
4.
What signs/reports are there of changes associated with project activities, e.g., what changes do
community members and other stakeholders attribute to the activities? What factors appear to promote
the apparent changes or deter intended change?
 The project’s Theory of
 Interview community members
Changes observed or reported
Change or Results Chains
(beneficiaries of one or more sectors
 Intended and unintended
and non-beneficiaries) and project
change
staff to gain perspectives about
 Positive and negative change
 changes they have made
 Differential change among
themselves, observed in others, or
beneficiaries (individual,
observe in the social, economic or
community) of one sector,
physical environment
beneficiaries of multiple
 factors that promoted the changes
sectors, and non-beneficiaries
 barriers to changes intended by the
 Perceived benefits of
project
participation in activities from
 conditions that promote or threaten
multiple sectors vs a single
sustained change
sector
 Technically evaluate how strategically
 Perceived trajectory of change
selected infrastructural outputs affect
and conditions that threaten or
or can affect livelihoods, well-being,
promote sustained change
maternal child health or environmental
conditions
5. How could the project be modified to improve its acceptability to targeted communities and the efficiency
and effectiveness of the implementation?
 Observed/ perceived strengths
 Use the results of inquiries to the
Based on findings from 1-4
and
weaknesses
of
the
questions above to form conclusions
above
implementation so far
and recommend concrete actions to
 Factors in the design,
help improve project performance and
implementation and context
final results
that affect the efficiency or
 Prioritize the recommendations and
acceptability of the processes,
identify the actor(s), the purpose for
outputs and immediate
change and anticipated benefits. All
outcomes
recommendations should be directly
 Targeted communities’ and
related to stated conclusions and based
individuals’ perceptions and
on evidence presented as findings.
priorities
 Relative cost/feasibility and
anticipated value of acting and
benefiting within the life of the
project
 Potential to advance the
project’s ultimate objectives
and goal.
 Potential for program outputs
to continue beyond the end of
the program.
As the evaluation team members investigate the key evaluation questions, the GHG project team asks that
the evaluation team consider the following detailed questions as they evaluate the technical components
and project areas.
12
1. Technical components
A. Livelihoods Strengthened
The assessment of the outputs and initial results of the strategic objective 1 on beneficiary households
should look at the variety of sectors and interventions including access to services such as inputs,
livestock health services and financial services, commodity traders and improved performance of
agribusinesses and MSMEs.
I.
Are the market systems and private sector partners being supported by the program reaching
the target beneficiaries? Which interventions and actions are most appealing to targeted
beneficiaries? What factors make the various promoted technologies and practices more or
less appealing to the beneficiaries?
II.
To what extent do the beneficiary farmers implement the techniques or access the goods and
services promoted by the program in their own fields? What are the factors that challenge or
promote their trial of those techniques?
III.
How does the program address gender equity in promoting market systems development –
both crop and livestock? What else could be done to improve equitable participation,
workload distribution and the benefits from interventions?
IV.
Which groups/individuals within the target population are better/less able to access loans and
or financial services? Why? How should the program be adjusted to benefit those with poor
access?
V.
What are the strengths and weaknesses of the ways GHG engages farmers (agro pastoralists)
and livestock owners, especially women, in adopting new varieties of crops or accessing
improved livestock health services, post-harvest handling and storage, and market access
activities?
VI.
What are the strengths and weaknesses of the input agent, community animal health workers
(CAHWs) and animal drug vendor implementations? Based on feedback from target
communities and implementing staff, which methods have been most and least successfully
presented using this approach?
VII.
What market-led/market facilitation approaches has GHG taken to help women access land,
inputs, financial services and technical advice? What are the strengths and weaknesses of
these approaches?
I.
II.
III.
IV.
V.
B. Nutrition among Children under Two Improved
Are the approaches the program is using to promote behavior change and to prevent malnutrition
appropriate to the context and the local health and nutrition priorities? Provide examples of
program approaches that appear to promote or hinder men’s and women’s uptake of behavior
change. Does the community have a clear understanding of the health- and nutrition-related
services offered by the program, their benefits, and who is eligible to receive them? In what ways
do members of the community believe GHG has influenced nutrition-related behavior?
Do beneficiaries of food rations understand why they are receiving them and for how long they
will receive them? How are beneficiaries using the food and benefiting from the ration? What do
they expect to do when they do not receive food rations? Is GHG adequately addressing ration
use challenges?
How has the performance based financing changed service delivery at facilities from the district’s
management responsibilities to the release of select equipment and infrastructure? What are the
strengths and weaknesses of this approach?
How have GHG’s HMIS mentorship and supervision and the reconstitution of the HUMCs
interventions impacted facility management? In what ways do facility staff and local government
partners believe GHG has changed system functionality?
How have student placements changed service delivery at placement facilities? How do local
13
VI.
I.
II.
governments understand and perceive the student placements? Are students accepted by the
population?
How do members of targeted communities (beneficiaries and non-beneficiaries) perceive the
distribution timing and processes and the value of food aid commodities? What has been the
effect of providing the protective rations? Should the protective ration be distributed year round
or only during the lean season?
C. WASH
Which WASH activities are effective and will likely lead to significant change within the water
interventions? What could be done differently or better? How well are the community and
government engaged and involved in regards to operations and maintenance?
Sanitation and Community Led Total Sanitation (CLTS) – Is this an effective intervention in the
context of Karamoja? What is working and what is not working? How could the program evolve
to better attain targets for sanitation?
D. Governance and Local Capacity for Conflict Mitigation Improved
I.
II.
III.
What role did communities and individuals have in the revitalization of district and sub-county
peace structures (peace committees, traditional authorities and women’s CSOs)? What measures
were taken to ensure women’s representation and contributions to these fora? How do structures
engage with other peace actors and communities to identify and respond to security concerns?
How have these relationships changed or improved? Is this approach effective?
What factors threaten and promote the likelihood of the peace structures to continue working after
the end of the project? What are measures the project can take in order to improve this
likelihood? What are the alternatives to sustain this work post-program?
What are the tangible outputs from the interventions and actions with youth groups and women’s
representatives? Is the strategy effective and working towards achieving intended outputs and
outcomes? Should this be scaled up further?
2. Program quality and cross-cutting areas
I.
II.
III.
IV.
I.
II.
III.
E. General
What are the strengths and weaknesses of linkages and coordination among program components
(including each SO as well as cross-cutting components)?
Which strategies have worked in encouraging partner participation and involvement, and which
did not?
Who has been adversely affected by program activities? How have they been affected?
Is GHG staff in all sectors qualified and adequately trained and equipped to perform their
assigned tasks effectively? Where are there weaknesses and needs for further training or capacity
building?
F. Gender
Are messages and services being delivered in a gender-appropriate manner in order to maximize
effectiveness?
Which dimensions of empowerment have been the most affected as a result? What can the
program do differently in order to better facilitate the possibilities for female empowerment in
these domains?
How are men being engaged in project learning opportunities related to gender equity, and how
14
has this changed their roles in supporting the health, nutritional status, and food security of their
families?
I.
II.
III.
IV.
V.
VI.
G. Behavior Change
Are the program’s social behavior change methods appropriate to effect desired changes in
behavior especially in regards to maternal child health and nutrition and WASH behaviors (e.g.
CLTS)? How could program approaches be improved?
How does the program measure number of beneficiaries reached with behavior change messages
and how does the program measure the ability of key behavior change actor’s efforts and abilities
to pass on key messages to other beneficiaries? Is this valid?
What is currently being done in the program to reach a high proportion of target groups with
high-quality, effective SBC methods? What appears to be working and what is not? Are there
ways the approach could be modified to expand coverage?
What tools and methods are being used by program staff to measure and improve the quality of
key SBC processes (e.g., group facilitation, individual counseling)? What could be done to ensure
even higher quality SBC processes?
What are the strengths and weaknesses of the methods GHG used to develop the SBC strategy
and messages?
How consistently are messages transmitted across the various methods?
H. Sustainability
What systems or activities have been put in place to ensure sustainability of behavior change and
other program outcomes beyond the life of the program?
Which services being established or strengthened by the program need to continue beyond the life
of the program to sustain benefits? What are the main threats to these services' continuation?
What will likely happen to these services after the program ends? What strategies are in place to
address threats to their continuation?
What physical structures being constructed or rehabilitated by the program are intended to be
continued beyond the life of the program? What are the main threats to maintaining these
structures? What likely will happen to these structures after the program ends? What strategies
are in place to address these threats?
What relationships and or linkages that are being established or strengthened by the program
under all strategic objectives are intended to be continued beyond the life of the program? What
are the main threats to enabling these relationships/linkages to survive? What likely will happen
to these relationships after the program ends? What strategies are in place to address these
threats?
I.
II.
III.
IV.
3. Implementation processes
I.
Facilitation approach

Is the program defining the poor and vulnerable in an adequate manner and is their some rigor to
the process of ensuring that the relevant target group are known and identified.

Within the context of the target geography how are the poor and vulnerable participating, being
targeted and or reached by the facilitative approach? This is in particular relevant for Strategic
Objective 1 and Objective 3. The GHG program has set ambitious targets for overall
beneficiaries, however it is important to understand the dynamics of the region and whether the
poor and vulnerable are able to participate and benefit from program interventions.
15
I.
II.
I.
II.
III.
I.
II.
III.
IV.
V.
I.
II.
III.
IV.
V.
I.
J. Commodities
Were there any issues with the beneficiary selection process and how were those managed?
What problems has the program faced in importing, delivering and distributing commodities to
program beneficiaries, and how were those managed? How does the food pipeline work? Please
elaborate on the processes, from ordering to receiving at the port, inland transport, warehousing
and distribution.
What problems has the program faced with end-use monitoring, and how were those managed?
What are the positive and negative aspects of the dissemination and use of the outputs of this
monitoring?
So far, what are the lessons learned about commodity management? Which processes work well
in the entire food pipeline, and which need adjustments?
How successfully has GHG distributed the correct amount of food to beneficiaries on schedule?
Based on distribution records, what proportion of food was distributed on time? Did any
beneficiaries not receive food for long intervals of time during a period of entitlement? What
most commonly delayed or prevented distributions or reduced quantities, and what has been done
to improve timely distribution?
K. M&E
How well have M&E systems been specifically tailored to the different components of the
program, while maintaining a cohesive vision of GHG as a food security program?
How responsive are the M&E systems? How has the information collected and analyzed by the
system contributed to programmatic decisions? Is the right information collected in a timeframe
appropriate to decision makers’ needs?
What is the quality of data collected and reported through the M&E system?
Are the program’s performance targets reasonable? Are they ambitious enough to make a
difference to the program beneficiaries? Are the targets justifiable, i.e., based on the consortium’s
past experience with similar programs? Are the targets comparable to the targets for similar
interventions in the country?
How effective are the methods for collecting and reporting indicators that are valid and reliable?
How can threats to the measurements’ validity and reliability be avoided?
L. Knowledge management (KM)
How well has the vision for the program been articulated? How effectively has this vision been
imparted to staff within the different consortium partner agencies and other stakeholders,
especially beneficiaries and local government?
In what respects do field staff feel they have been heard? What system is in place to encourage
staff to bring new ideas? How effective is the system? How has the program used the
assessments --sector specific gender audits, social network maps, population level household
studies in designing activities?
Do internal reporting systems, including staff meetings and discussions, allow for examination of
what did not go as planned, where adaptation is needed or where a new idea could be applied? Do
internal reporting systems have feedback systems that extend to the beneficiary level?
Does the program have any systems in place whereby lessons can be shared across consortium
partners or across technical areas? How does the program identify, capture and document lessons
learned? How are lessons learned shared, and how often?
What evidence does the program have showing that it successfully used lessons learned to
improve program quality or avoid mistakes?
M. Partnerships
What has been the nature and benefits of GHG partnerships with other actors in the project areas
16
II.
III.
(government, NGO, youth, private)? What factors appear to make such partnerships more or less
beneficial to the GHG implementation and beneficiaries? How have partnerships strengthened
provision of goods and services and their operations as well as led to peace and conflict
resolution? What additional partnerships should GHG consider and why? Which existing
partnerships appear to not add value?
How do KAPDA and the youth group organizations describe the nature of and degree of
satisfaction with their collaboration with GHG? How do they think collaboration could improve?
In what ways have private sector partners and target beneficiaries that do not have key leadership
roles (or are not members of organizations relevant to program activities) been included in the
design and implementation of the various GHG activities? How have the different types of
involvement appeared to benefit or detract from the quality and acceptance of the activities and
their outputs?
Evaluation methodology
The mid-term evaluation team will design the evaluation methodology and conduct the evaluation. They
will be expected to organize a pre and post-evaluation workshop to share initial design with project and
FFP representatives before beginning data collection as well as early findings prior to finalizing the
overall report.
The evaluation team will be expected to employ qualitative and participatory methods to conduct this
evaluation and will hire, train and use external enumerators were necessary, methods used should include
o
o
o
o
o
o
o
o
o
Review of program monitoring data and records
Direct observations of activities through site visits
Focus group discussions
Participatory impact assessment or other participatory rural appraisal-type techniques
Most significant change methodology
Semi-structured key informant interviews
Individual interviews with project staff, direct beneficiaries, and other individuals living
in target communities
Market mapping and market studies
Literature review
The proposed methodologies and tools will be presented to Mercy Corps for validation and approval prior
to field work.
While the evaluation will be designed in a participatory fashion with GHG and FFP staff and not directly
with beneficiaries, it should be noted that actual fieldwork will be conducted using participatory
methodologies, engaging communities and different stakeholders, especially members and service
providers in target communities, to better understand their perceptions of the implementation and outputs,
and to discern their beliefs regarding how GHG activities have affected their lives.
When consulting beneficiaries and other community members, evaluators will be encouraged to look for
and report signs of apparent early outcomes, both intended and unintended, and consider whether these
are signs of movement in the direction of the program objectives, i.e. do they fit the theory of change that
defined the results framework, or does either the theory or the activities need to be adjusted. This analysis
should involve two-way communication between all stakeholders.
17
Furthermore, the evaluation team should use existing data from reports and activity records to quantify
progress towards objectives among direct beneficiaries.
Composition of the MTE Team
The evaluation team will be led by an evaluation specialist with proven experience, who will act as team
leader and organizer for the MTE, and will have primary responsibility for the design. The evaluation
team will consist of a group of experts who must have skills in at least the following key technical areas:
Nutrition, Agriculture/Livelihoods, and BCC/Communication specialists in general – to assess the flow of
communication through the facilitation process. In cases where participation of the MEL Officer, the
GHG Chief of Party, or other program staff might bias the evaluation design, these individuals will serve
as informants, and it should be stressed that none of these individuals will have an evaluator role. The
GHG team will play a key role in capacity facilitation regarding practicalities of the evaluation (e.g.
logistics, sharing contact information and providing directions to specific program sites), but will not be
directly involved in data collection or primary analysis. The MEL team will also support with identifying
high quality external surveyors, survey supervisors, translation services into local languages and logistics
planning for data collection in accordance with agreed requirements and conditions from evaluation team
leader. The evaluation team leader will be expected to finalize Scopes of Work for any necessary
surveyors.
The evaluation team will be expected to have proven experience in workshop design, strategic planning,
and intercultural communication skills, and is expected to have substantial experience working with food
security, prevention of malnutrition, good governance and Social Behavior Change programs in SubSaharan Africa. The consultant firm should have previous experience evaluating market facilitation
programming, and specific familiarity with the M4P approach is a must. The evaluation team will have
demonstrated experience in ‘highly’ participative evaluations, assessments and appraisals approaches in
intercultural context.
As the evaluation will be mostly conducted in English (with interpretation into Karamojong) and the final
report will be in English, knowledge of Karamoja would be an advantage. The evaluation team should
have significant experience in using mixed methods for evaluations. Experience with Title II development
programs is a strong advantage.
Specific expectations for the proposed evaluation team:
Every team member’s resume must show substantial application of strong qualitative research and
evaluation skills.
The Team Leader should have significant formal education at the post graduate level (Applicants that do
not hold a graduate degree in a field should document relevant formal education in the field.) in a field
relevant to evaluation (e.g., program evaluation, statistics, anthropology, applied research, organizational
development, sociology, organizational change, etc.) and extensive experience using mixed methods of
investigation (qualitative and quantitative) in developing countries. Knowledge in the conceptual
framework of food security and nutrition and experience in food security programming is highly
desirable.
Each Technical Specialist should have a post graduate degree (M.S., M.A., or Ph.D.) in a field related to
at least one of the technical sectors of the project, plus extensive practical experience in developing
countries.
18
Team members’ roles
The Team Leader’s roles include to:






organize and lead the overall evaluation;
assure a thorough review and analysis of available secondary data by the appropriate team
member(s),
lead the selection of a purposely selected sample of activity sites and outputs for primary data
collection and assure adequate triangulation and validation of findings;
lead the collection and analyses of primary and secondary data to evaluate the program’s M&E
processes and the integration of program sectors and activities;
assure that 1) final report presentation is logical and presented in a way that clearly separates
findings, conclusions, and recommendations, and 2) all findings, conclusions and
recommendations are based on evidence presented in the report
to liaise, on the part of the evaluation team, with the awardee and USAID
The Technical Specialists would be responsible to:


lead the collection and analyses of primary and secondary technical data related to his/her field(s)
of expertise and form recommendations.
consider all general aspects of the implementation of all activities related to his/her sector, i.e.,
resource management, staffing, linkages/partnerships, branding, community involvement, cultural
acceptability, gender, exit/sustainability measures, environmental protection, adherence to
schedules, and integration with other sectors.
Program responsibilities
Mercy Corps will provide office space in Kotido and or Kaabong, logistics for field trips including up to
two (2) vehicles. The Mercy Corps team will provide support as requested by the evaluation team leader
in regards to identifying service providers for translation into local languages, high quality surveyors,
survey supervisors, date entry, etc. The evaluation team will be responsible to provide their own
computers as required.
All data collection materials, databases and analysis material will remain the sole property of Mercy
Corps and will have to be handed over at the end of the assignment in a way that they can be easily
consulted if required.
Mercy Corps is responsible for providing the names, contact details and phone numbers for donors and
other key stakeholders, preparing and providing background and reporting documents, data and other
material. The consultant team will also be provided with program mapping documents, including a list of
beneficiary sites. Mercy Corps also takes the responsibility of sharing the draft evaluation report with key
stakeholders, gathering feedback, and transferring a consolidated feedback document to the consultant in
order to generate a final version of the Mid-Term Evaluation Report.
Resources that will provided to the evaluation team approximately 6-8 weeks before arriving in Uganda:

Lists of activity locations, identifying the type(s) of activities at each location, and the dates of
implementation, numbers of direct and indirect beneficiaries, quantities of commodities
19



















distributed, etc., for each activity type. This list should include locations of all community assets
developed or rehabilitated using project resources, including those still in process of development
or rehabilitation.
Food, voucher, cash, and non-food item distribution reports that include locations of distribution,
type of distribution, planned quantities and timing and actual quantities and timing of
distributions.
Locations of all warehouses, with uses and capacities.
Project commodity management tools and reports
Maps showing the project area with administrative boundaries, roads, markets, food distribution
points, activity sites, partner offices, lodging, livelihood or ecologic zones, etc.
Approved proposal narrative and relevant attachments and documentation of approved
modifications
All Annual Results Reports and attachments (ARRs)
Inception reports and initial year 1 assessment and sector reports
Pipeline and Resource Estimate Proposals (PREPs)
Baseline report and reports from all research conducted for the project’s benefit (e.g., formative
research, barrier analyses, gender analyses, market analyses…)
Project monitoring tools and manuals
Complete M&E plan
Examples and lists of recipients of all types of routine monitoring reports\
A current organogram of project staff (with names and phone numbers for incumbents and
notation of vacancies) showing supervision/management lines
Descriptions, dates and numbers of beneficiaries of capacity building activities for project staff
and project beneficiaries (individuals, groups, and communities)
Project beneficiary and activity data bases and descriptions
Environmental Mitigation and Monitoring Plan (EMMP)
Exit strategy and sustainability plan
Activity implementation protocols and guidelines
Descriptions of the nature (e.g., format, location) and contents (e.g., type of data, period of
collection) of the various data sets that the awardee will provide for the evaluation.
Deliverables:
The evaluation team will provide the following:



An evaluation plan, including:
o Evaluation methodology
o Roles/responsibilities of each team member
o Protocols and instruments for data collection and analyses
o Evaluation itinerary
o Statement of limitations of the methods and potential effects on results
De-briefing and validation of findings with project staff in the field
In-country presentation of preliminary results to project leadership and donor representative in
Kampala
20



Mid-term evaluation report written in English of a maximum of 75 pages excluding title page,
table of contents, glossary of terms and acronyms, acknowledgements, executive summary,
photos and annexes, and including the following components:
o Background
o Evaluation objectives
o Evaluation methods (details may be put in annex)
o Findings related to each key evaluation question
o Discussion and conclusions
o Recommendations, with prioritization
o Annexes
Executive summary of up to 4 pages
Electronic files of the clean (final) qualitative (and potentially some quantitative) data collected.
Methodological strengths and limitations
As a qualitative methods approach, this mid-term evaluation aims to maximize the amount of actionable
information acquired from the field, but it should be noted that none of the results acquired from this
research will be generalizable to the population in a rigorous statistically significant manner. While
qualitative research are appropriate for recognizing trends in programming and gathering in-depth
information on program processes, any results will not be comparable to indicators from baseline or final
evaluation.
Presentations & Coordination
USAID/FFP funds a similar program in Karamoja implemented by ACDI/Voca. The ACDI/Voca
program is also being evaluated. The GHG mid-term evaluation consultant must coordinate with the
consultant conducting the mid-term evaluation for ACDI/Voca, in order to maximize learning and sharing
of lessons learned. At a minimum, the GHG consultant will meet with the ACDI/Voca consultant at the
beginning of the consultancy to coordinate and compare field implementation tools. The consultant will
also arrange, with the ACDI/Voca consultant, to provide a joint presentation of findings at the end of the
consultancy.
In addition to the required collaboration with the ACDI/Voca consultant, the GHG consultant is
responsible for three presentations. One will be required at the beginning of the work in-country to
present the methodology of the evaluation to the program team to consist of the Mercy Corps Country
Director, the COP, the DCOP (both MC and WV DCOPs) and all program managers and team leaders
working under GHG and one representative from USAID/FFP, and two at the end to present preliminary
results. The first of the results presentations will be made to the GHG program team in Karamoja to
discuss results and receive meaningful feedback. The final presentation will be a shorter joint presentation
with ACDI/Voca, presented to USAID/FFP representatives, Mercy Corps Uganda Country Director,
GHG COP and DCOP, government partners, and other stakeholders.
Timeframe / Schedule
Mercy Corps Uganda estimates approximately 122 - 125 days for the assignment with approximately 65
days of in-country work and 60 days of out-of-country work and travel days, subject to discussion and
agreement with the evaluation team.
The evaluation team will be responsible to propose a final schedule for the work.
21
The firm will be responsible to propose a schedule of work, however, it is mandatory that the data
collection is accomplished during the period July 2015 – August, 2015. Illustrative activities and levels of
effort are:
Activities
Illustrative level of effort
Literature review of resources (during a period of
approximately 1 months prior to arrival in
country)
20 working days between 1st June and 15th July,
2015
Travel to Kampala
1-2 days
Introductory meetings between MTE team and
project and donor stakeholders in Kampala
2 days
Travel to Karamoja
1 day
Finalize evaluation work plan (site selection, data
collection travel plan, validation and analysis
plan)
4 days
Train enumerators and/or qualitative interviewers,
pre-test interview guides, as applicable
5 days
Data collection in Karamoja
35 days (5 weeks working 7 days)
Preliminary analysis
10 days
Presentation/validation of preliminary findings
with selected project staff in the field
2 days
Travel to Kampala
1 day
Discussion with ACDO Voca evaluation team – if
there are two separate evaluation teams working
on the FFP Title II programs in Karamoja
2 days
Preparation and presentation of preliminary
findings to project leadership and donor
representative(s) in Kampala
2 days
Return travel to base
1-2 days
Preparation of draft evaluation report
22 working days (one month)
Revision and preparation of final evaluation report
(after 2-3 week project/donor review period)
14 days
22