Data Triangulation
Objectives:
 At the end of the session, participants will be
able to:
 Describe the role of data triangulation in program
evaluation
 List data sources and approaches that can be
used for triangulated analysis
Data Triangulation for M&E
 Linking different information sources involving persons, place
and time
 Analysis of data from multiple sources can increase the validity
and reliability of findings; it can corroborate findings and
weakness of any one data source can be compensated for by
the strengths of another
 Analysis of program level data with outcome/impact level data
can help substantiate the linkage between program
interventions and population-level outcomes/impacts
Questions that data triangulation can
help address:
 Are interventions working and are they making a
difference?
 What changes in population-level outcome and
impact indicators have been observed and what do
they mean?
 Can the observed changes in outcomes and impact
indicators be attributed to program outputs?
 Are the collective efforts being implemented on a
large enough scale to impact the course of the
epidemic?
Source: A framework for monitoring and evaluating HIV prevention programmes for most-at-risk populations.
UNAIDS 2007
Source of data
 Repeated HIV and/or STI prevalence
surveys/surveillance
 Repeated population surveys with behavioral data
 Routine program or service delivery data
 Quality of service assurance and quality
improvement assessments
 Qualitative studies (in-depth interviews, focus group
discussions, key informant interviews, etc)
Source: A framework for monitoring and evaluating HIV prevention programmes for most-at-risk populations.
UNAIDS 2007
When to do Data Triangulation
 When interventions have been in place for a sufficient duration
of time to reasonably expect that changes at the population level
may be attributable to program interventions
 When interventions have been implemented with sufficient
intensity and with high enough coverage to reasonably expect
effects to be observed in the target population.
 When good program-level data (i.e outputs, coverage, quality of
implementation are available) are available
 When process evaluation indicates that program activities are
being realized as planned.
Source: A framework for monitoring and evaluating
HIV prevention programmes for most-at-risk
populations. UNAIDS 2007
Considerations
 Is there a culture of data sharing?
 Are data from different sources representative of
te population ofinterest?
 Do the time frames of te data points match?
Example 1:
Female Sex Worker Program in Nepal
Family Health International
Logic of the program in Nepal
 Reach people with information to increase their
knowledge and awareness
 Provide services to allow them to act on that
knowledge
 Expand coverage of information and services so that
so that changes are observable at the populationlevel
 Improve the quality of services while expanding
coverage (e.g. STI treatment)
 Information and good quality services for enough
target group members will lead to service use, which
will lead to safer behaviour and lower risk of
exposure to HIV
Source: FHI Regional Office, Bangkok.
Program records indicate the program is being
delivered and reaching more of the target
population through more channels over time
Num ber of sex w orkers reached and contacts by type of staff
16,000
14,000
12,000
10,000
8,000
6,000
4,000
2,000
0
Total Number Reached
2000
2001
2002
2003
2004
Total Number of Contacts Outreach Workers
Total Number of Contacts Peer Educators
Year
Source: FHI Regional Office, Bangkok.
FHI Nepal, 2005
This increasing coverage is confirmed by FSWs in
target areas
(exposure to NGO-related activities among FSWs, 1998 – 2002)
100
Percentage (%)
90
74
80
70
63
52
60
48
50
40
68
61.8
38.5
29
32.3
44.3
47.8
64.8
56.4
44.3
39.3
31 30
68.3
30
20
21.5
19.3
10
0
1998
1999
2000
2001
Received condoms
Received brochures/booklets/pamphlets
Received other information about HIV/AIDS
Received items/information from all three sources
Source: FHI Regional Office, Bangkok.
2002
There were signs of improved quality of
services e.g. STI diagnosis and treatment
Quality Criteria
Situation Prior to Dec. 2003
Current Situation
Treatment
All STI patients receiving treatment as per the STI
management guidelines. Drug quality needs to
be standardized. Follow up visits are
infrequent.
Compliance of the drug therapy cannot be assured
with multi-dose drugs.
Drugs supply is of better quality. Drugs are mostly single
dose and taken under direct observation, improving
compliance
Clinic location &
accessibility
Most static clinics placed centrally in the town and
are easily accessible but mobile clinics are kept
at DIC or government owned offices and
changed from time to time
Static clinics which were not appropriately located have
been changed and improved
Clinic setup
Clinics have mostly 3 rooms one each for
registration, consultation and laboratory test.
Only some of the clinics have private rooms for
consultation and examination
VCT now integrated into STI clinics, and a separate room
for counseling has been added in most static clinics
Staffing
Clinical teams consist of 3 persons: one medical
officer, a staff nurse and a laboratory
technician. Frequent turnover of the trained
staff.
Addition of one trained VCT counselor in static clinics.
Staff turnover rate has decreased.
Clinical Skill
General history taking, general and genital
examination are performed confidently, oral
and anal examinations are not routinely
conducted
Improved sexual history taking skills and clinical skills,
but regular guidance still needed.
Source: FHI Regional Office, Bangkok.
Trends in desirable outcomes are evident –
Percentage (%)
Knowledge of condoms is high among FSWs and condom carrying
behavior is increasing over time, 1998 - 2002
100
90
80
70
60
50
40
30
20
10
0
94.5
92.8 92.2
98 95.8
98.3 97.9
99.3 96.9
85
54.8
34.7
58.4
39.3
28.3
1998
1999
2000
2001
Ever heard of condoms
Know using condom can prevent HIV transmission
Usually carry condoms with them
Source: FHI Regional Office, Bangkok.
2002
Trends in Desirable Outcomes are evident –
Percentage (%)
Condom use with clients among FSWs is increasing steadily over
time, 1998 - 2002
100
90
80
70
60
50
40
30
20
10
0
82.3
77.5
93.3
86
94
87.3
51
44.8
36.3
38.7
95
90.3
67
61.8
40.3
33
54.3
40.2
26.5
19.8
1998
1999
2000
2001
Ever use of condoms
Use of condoms with last client
Consistent use of condoms with clients in the past year
Gave condom to client at last sex
Source: FHI Regional Office, Bangkok.
2002
Those Reached by the Program Have Safer Behaviors:
Exposure to various NGO-related activities in the past year impacts on
consistent condom use with clients among FSWs - 2002
Exposed
100
Unexposed
Consistent Condom Use (%)
90
80
70
65.7
64.8
61.5
60
50
40
36.7
37.3
38.4
30
20
10
0
Received condoms
Source: FHI Regional Office, Bangkok.
Received brochures &
materials
Received IPC information
about HIV/AIDS
There is evidence of a favourable Dose-Response
relationship:
Greater exposure to various NGO-related activities results in better condom
100
91.9
use behavior - 2002
90
83.9
76.2
Percentage (%)
80
69.2
70
68
60
48.2
50
40
30
75.8
39.7
42.1
Any one*
Any two*
33.3
24.6
28.2
20
10
0
None
Carrying condoms
All three*
Consistent condom use with clients
Consistent condom use with regular clients
*1) Received condoms, 2) received brochures/materials, 3) received IPC information about HIV/AIDS
Source: FHI Regional Office, Bangkok.
Summary of Key Findings among FSWs
 Program activities have expanded in scale and improved in
quality
 HIV and STI prevalence are decreasing over time
 Knowledge of condoms is high and consistent condom use is
increasing and high among clients and regular clients
 Exposure to NGO-related information sources about HIV/AIDS
and condoms is increasing and high
 Exposure to various NGO-related activities is increasing and
high
 Exposure to NGO-related activities is strongly linked with
condom carrying & consistent condom use in a dose-response
manner
Source: FHI Regional Office, Bangkok.
Example 2: Botswana
Antiretroviral scale-up in Botswana
National AIDS Committee Botswana,
WHO, UCSF and UNAIDS (2006)
Overview of Botswana Triangulation
 Objective:
 To develop a model to measure the impacts of
ART and PMTCT programs on adult and child
mortality
 Application:
 Provide timely information on the impact of
national scale-up of ART for policy and
programmatic decision making
Source: Case Study. Country-enhanced monitoring
and evaluation for antiretroviral therapy scale-up:
analysis and use of strategic information in
Botswana. WHO 2006.
Approach
 Meetings with stakeholders to identify objectives of
analysis
 Identification and compilation of data from many sources
 Vital statistics (morbidity and mortality)
 Population survey data
 Patient Management systems (HMIS)
 Program data (i.e. HIV testing, ART)
 Examination of trends in HIV prevalence and mortality in
relation to ART and PMTCT availability and service
uptake
Source: Case Study. Country-enhanced monitoring
and evaluation for antiretroviral therapy scale-up:
analysis and use of strategic information in
Botswana. WHO 2006.
Source: Case Study. Country-enhanced monitoring and
evaluation for antiretroviral therapy scale-up: analysis
and use of strategic information in Botswana. WHO
2006.
Source: Case Study. Country-enhanced monitoring and
evaluation for antiretroviral therapy scale-up: analysis
and use of strategic information in Botswana. WHO
2006.
Source: Case Study. Country-enhanced monitoring and
evaluation for antiretroviral therapy scale-up: analysis
and use of strategic information in Botswana. WHO
2006.
Conclusions of Botswana
Analysis
 Since the inception the ART programme,
Botswana has achieved reductions in mortality of
adults aged 25–54 years
 Reduced mortality is associated with early
initiation of district ART programmes and with the
overall rate of ART uptake in the district.
Source: Case Study. Country-enhanced monitoring and
evaluation for antiretroviral therapy scale-up: analysis
and use of strategic information in Botswana. WHO
2006. Source: Case Study. Country-enhanced monitoring and evaluation for antiretroviral therapy scale-up: analysis and use of
strategic information in Botswana. WHO 2006.
Strengths and Limitations of
Triangulation
 Strengths
 Pre-existing data sources are used
 Can provide relatively rapid results
 Limitations
 Existing data may be insufficient
 Institutional barriers to data sharing
Triangulation Resources
 HIV Triangulation Resource Guide: Synthesis of
Results from Multiple Data Sources for Evaluation
and Decision-making (WHO 2009)
 Data Triangulation for HIV Prevention Program
Evaluation in Low and Conncetrated Epidemics (FHI
2010)
MEASURE Evaluation is a MEASURE project funded by the
U.S. Agency for International Development and implemented by
the Carolina Population Center at the University of North Carolina
at Chapel Hill in partnership with Futures Group International,
ICF Macro, John Snow, Inc., Management Sciences for Health,
and Tulane University. Views expressed in this presentation do not
necessarily reflect the views of USAID or the U.S. Government.
MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.