Introduction to Monitoring and
Evaluation
Learning Objectives
By the end of the session, participants will be able
to:
• Define program components
• Define key concepts in monitoring and
evaluation (M&E)
• Know the basic purpose of M&E
2
Program Components
Inputs Processes OutputsOutcome Impact
3
Program Components: Inputs
• Inputs: Resources used in a program/project, such
as money, staff, curricula, and materials
• Examples:
– Funding: Government, World bank, Global Fund, PMI
& other donors
– Commodities: ACTs, ITNs, RDTs, laboratory supplies
4
Program Components: Processes
• Processes: Activities/services that the
program/project undertakes/provides to
accomplish its objectives, such as outreach,
materials distribution, workshops, and training
– Ex. Training of health personnel and program staff
5
Program Components: Outputs
• Outputs: Direct products or deliverables of the
program, such as intervention sessions
completed, people reached, and materials
distributed; direct deliverables of the service
delivery environment, such as access, quality, and
client satisfaction.
• Examples:
– Number of ITNs distributed
– Number of houses sprayed with IRS
– Number of ACTs distributed
6
Program Components: Outcomes
• Outcomes: Program results that occur at the
population level both immediately and some time
after the activities are completed, such as
changes in knowledge, attitudes, beliefs, skills,
behaviors
• Examples:
– ITN coverage and use
– Knowledge about ACTs
– Adherence to treatment strategy (usage of combination
therapies rather than monotherapies)
7
Program Components: Impacts
• Impacts: Long-term results of one or
more programs over time, such as changes
in disease incidence, fertility, morbidity,
and mortality rates
– Changes in malaria morbidity and mortality
8
Example from an Malaria Program
Population
Level
Project / Program Level
Inputs
Resources
Staff
Drugs, LLINs, RDTS
Supplies
Equipment
Processes/
Activities
Activities:
Trainings, events
Distribution of
supplies
Outputs
Services:
ACTs and LLINs
distributed,
Staff trained
Outcomes
Intermediate
Coverage of
interventions
Impact
Long-term
Morbidity
Mortality
9
Brainstorming Activity
• What is monitoring?
• What is evaluation?
10
Definition: Monitoring and Evaluation
MONITORING is a process of measuring progress towards
program/project objectives through tracking activities
conducted, resource utilization, and the outputs generated
EVALUATION is a process of determining systematically and
objectively the relevance, effectiveness and impact of
interventions in relation to their objectives
11
Monitoring vs. Evaluation
• Monitoring is
tracking program/
project deliverables
• Evaluation is assessing
the effectiveness of the
program/project
• Focus on
performance of the
implementing
agencies
• Focus on the results
expected to be achieved
in the target population
12
Changes occur with adequate intervention efforts
and time
Increased complexity of study design
Status at
the start of
program
Time since intervention began: more time …bigger changes
13
Illustration of Program
Monitoring
Program
Indicator:
# of ITNs
Distributed
Program
start
Time
Program
end
14
Outcome Evaluation
With program
Program
Indicator:
ITN
Ownership
Program
Start
Time
Program End
15
Impact Evaluation
Without program
Program
Indicator:
Attribution
“Impact”
Inpatient
malaria
deaths
With program
Program
Start
Time
Program End
16
Differentiating Actual Impact from Bias
Actual program effect
Outcome With program
Overall
change
Outcome
(Impact)
Bias
Outcome Without program
Intervention
start
TIME->
TIME->
End
17
Monitoring & Evaluation Pipeline
Inputs
Outputs
Number of Projects Measuring
each Component
Outcomes
Impact
All
Most
Some
Few
Long-term effects
Short-term and
intermediate effects
•
•
•
•
•
•
Resources
Staff
Funds
Materials
Facilities
Supplies
• ITNs distributed
• Houses sprayed
• Staff trained
• ITN coverage and use
• IRS coverage
• Diagnostic and
Treatment coverage
Levels of Monitoring & Evaluation Efforts
• Morbidity
• Mortality
18
Role of M&E in Program Life Cycle
19
Monitoring Questions
• Were inputs made available to program/ project in
the quantities and at the time specified by the
program/project work plan?
• Were the scheduled activities carried out as
planned?
• How well were they carried out? Did the expected
changes occur at the program/project level, in
terms of people reached, materials distributed?
20
Evaluation Questions
• Did the expected change occur at the population
level (not necessarily attributable to
program/project)? How much change occurred?
• Did the target population benefit from the program
and at what cost?
Impact Evaluation:
• Can improved health outcomes be attributed to
program efforts?
21
When should evaluations be
planned?
Before programs are implemented
22
Class Activity: Is it Monitoring or Is it
Evaluation?
1. The MOH wants to know if the programs
being carried out in Region A are increasing
ITN use among pregnant women and
children under five in that region
2. The Government wants to know how many
RDT have been used in health facilities for a
year
3. The NMCP would like to see if there have
been changes in the under five mortality
rate since the implementation of their LLIN
campaign 3 years ago
23
Class Discussion: Is it Impact Evaluation?
1. The Government wants to know if its
campaign to distribute LLINs has reduced the
prevalence of malaria in the distribution
districts
2. The WHO wants to know if the all cause
under-five mortality rate has increased or
decreased in Country X since the year 2000
24
Summary
Monitoring
• Routine & continuous
• Internal to program
• Regular
• Measures actual
performance
• Tracks cost
• Done by those in
program
Evaluation
• Time bound
• External or internal
• Periodic assessement
• Impact Evaluation
• Evidence of changes
due to program
• Rigorous and requires a
design
25
MEASURE Evaluation is a MEASURE program project funded
by the U.S. Agency for International Development (USAID)
through Cooperative Agreement GHA-A-00-08-00003-00 and
is implemented by the Carolina Population Center at the
University of North Carolina at Chapel Hill, in partnership
with Futures Group International, John Snow, Inc., ICF Macro,
Management Sciences for Health, and Tulane University.
Visit us online at http://www.cpc.unc.edu/measure.
27