STRENGTHENING
MONITORING and EVALUATION of
NATIONAL AIDS PROGRAMMES
in the CONTEXT of the
EXPANDED RESPONSE
February 4-6 2002
Dakar, Senegal
Evaluation
Monitoring AND Evaluation
Monitoring: What are we doing?
Tracking inputs and outputs to assess whether
program are performing according to plans
(e.g. people trained, condoms distributed)
Evaluation: What have we achieved?
Assessment of impact of the programme on
behaviour or health outcome
(e.g. condom use at last risky sex, HIV incidence)
Surveillance: monitoring disease
Spread of HIV/STD
(e.g. HIV prevalence among pregnant women)
A FRAMEWORK for Monitoring and Evaluation
Input
People
money
equipment
policies
etc.
Process
Training
Logistics
Management
IEC/BCC
etc.
Output
Outcome
Services
Service use
Knowledge
Behaviour;
Safer
practices
(population
level)
Impact
HIV/STI
transmission
Reduced
HIV impact
DATA COLLECTION for Monitoring and Evaluation
Input
Process
Output
Outcome
Impact
HIV/STI
surveillance
Household
Surveys
Programme Monitoring
Facility
surveys
INITIATIVES for Monitoring and Evaluation
Input
Process
Output
Programme Monitoring
Outcome
Household Surveys
Facility surveys
Multi-sectoral AIDS (MAP) program
Monitoring and Evaluation
Impact
HIV/STI
surveillance
Second generation
surveillance
USAID / CDC Expanded response
Monitoring and Evaluation
United Nations General Assembly Special Session (UNGASS)
Monitoring and Evaluation
MONITORING and EVALUATION at the NATIONAL LEVEL:
NATIONAL SUCCESS STORIES
Jamaica
Senegal
Uganda
Thailand
Did the National Response Make the Difference?
Input
Process
Output
Outcome
Impact
1
3
Can the changes in
behaviour be attributed to
interventions / programs?
2
Can the changes
in HIV prevalence
be attributed
to changes in
behaviour??
HIV
prevalence
changing!
The components of AIDS programmes
Input
Process
Output
Outcome
IEC programs: knowledge, attitudes
Condom promotion and distribution
School programs: adolescent KAP
Targeted interventions (e.g. CSW, IDU)
Control of STDs
Voluntary counselling and testing
Reduction of mother-to-child transmission
Blood safety, prevention nosocomial transmission
Care & support programs (including ARV)
Impact
Global Initiative to Improve M&E of AIDS
programs at the National Level
• Goal: To help countries develop sustainable and
effective M&E systems
• Coordination: UNAIDS, WHO, USAID, CDC in
collaboration with World Bank, European
Commission, FHI, MEASURE and many others
Global Initiative to Improve M&E of AIDS
programs at the National Level (2)
• Process:
1998 Review of existing practices, systems and instruments (13
countries);
1999-2000: four consensus-building / technical meetings with
stakeholders
2000: Publication of UNAIDS guide (English)
2001: Publication of UNAIDS guide (French)
• Country workshops (Africa - 25 countries)
–
–
–
–
April 2001 - Entebbe
Feb 2002 - Senegal
Apr 2002 - Kampala (training in M&E)
Apr 2003 - Dakar
• Coordination: MERG, GAMET
Lesson Learned: 5 Elements of a Good
Monitoring and Evaluation System
1 Presence Monitoring and Evaluation unit
2 Clear goals and objectives of the program
3 A core set of indicators and targets
4 A plan for data collection and analysis
5 A plan for data dissemination
Monitoring and Evaluation Unit
Not so
good
• No functioning unit for M&E
• 1 or 2 persons responsible
for the whole country
• Very limited resources for
M&E
• No formalised links with
technical and other
resources
GOOD
• Established M&E unit within
the NAC and MoH
• Specific expertise in or
affiliated with the unit: (M&E,
epi, behavioural, statistics, data
dissemination)
• Budget (10% of the national
AIDS budget with national
contribution)
• Formalised links with the
research institutions, leading
NGOs and donors
Clear goals and objectives
Not so
good
• National strategic plan has
no specific goals and
objectives
• No system of ongoing
assessment with programs
reviews and built-in
evaluation
• Limited coordination with
districts and regions
• Limited coordination
between sectors
• Donor-driven M&E system
GOOD
• Well-defined national
programme goals and
targets - M&E plan
• Regular reviews/evaluations
of the progress of the
implementation of the
national programme plans
• Guidelines and guidance to
districts and regions or
provinces for M&E
• Guidelines for linking M&E to
multiple sectors
• Co-ordination of national and
donor M&E needs
A set of indicators (and targets)
Not so
good
• No indicators or indicators
that cannot be measured
• Indicators that cannot be
compared with past
indicators or with other
countries
• Indicators are only used for
donors and each donor has
its own set of indicators
• Indicators are irrelevant to
those who collect the data
• Each district or sector uses
its own indicator
GOOD
•
A set of priority indicators and
additional indicators that cover
programme monitoring,
programme outcomes and
impact - M&E plan
•
Selection of indicators through
process of involving multiple
stakeholders and maintaining
relevance and comparability
•
Utilization of past and existing
data collection efforts to assess
national trends (e.g. DHS)
Data collection and analysis plan
Not so
good
• M&E is an ad hoc activity
without a plan, mostly driven
by donors
• Data are collected but not
analysed sufficiently / utilized
• There is no systematic
monitoring of programme
inputs and outputs
GOOD
• An overall national level data
collection and analysis plan,
linked to the national
strategic plan
•
• A plan to collect data and
analyse indicators at
different levels of M&E
(programme monitoring)
• Second generation
surveillance, where
behavioural data are linked
to HIV/STI surveillance data
Data dissemination plan
Not so
good
• Dissemination is ad hoc and
not planned or coordinated
• Annual surveillance report is
much delayed not user
friendly and not well
disseminated
• Dissemination to the districts
and regions is not done
• Dissemination activities are
donor driven
GOOD
• Overall national level data
dissemination plan
• Well-disseminated
informative annual report of
the M&E unit
• Annual meetings to
disseminate and discuss
M&E and research findings
with policy-makers and
planners
• Clearinghouse / Resource
centre at national level