Indicators, Data
Sources, and Data
Quality for TB M&E
Criteria of good indicators





Valid: indicator actually measures the
phenomenon it is intended to measure
Reliable: indicator produces the same
results when used repeatedly, so it always
measures the same phenomenon
Specific: indicator measures only the
phenomenon it is intended to measure
Sensitive: indicator reflects changes in the
phenomenon under study
Operational: indicator is measured with
developed and tested definitions and
reference standards
Qualitative vs. Quantitative
Qualitative: indicators answer questions
about how well the program elements are
being carried out
 Quantitative: indicators measure how
much and how many

Factors in Indicator Selection
What national, district and local levels
need to know
 Availability of the data


Availability of human and financial
resources to manage the data

Program needs

Lender requirements
TB data collection methods
and sources
Routinely collected data
 Process monitoring and evaluation
 Program evaluation/reviews
 Global TB reporting
 Special surveys

Routine Recording
 TB
 TB
Register
Treatment Card
 Laboratory Register
 Cough register
Routine Reporting
 District
TB Register
 Quarterly report of new cases and
relapses of TB
 Quarterly report on results of treatment of
pulmonary TB patients registered 12-15
months earlier
Process Monitoring and
Evaluation
Analysis of recording and reporting
 Supervision
 Records of trainings held, meetings held,
events, etc…

Program Evaluation/Review
Comprehensive review of the entire
program
 Conducted every 2-5 years
 External and internal experts break up into
groups and cover a representative sample
of the country
 Usually provides input for developing or
revising the medium-term development
plan

Special studies
Prevalence survey
 Population-based survey
 Facility surveys
 Vital-registration surveys
 Tuberculin surveys
 Drug-resistance surveys

Example of a national level
data-collection system
Prevalence
Survey
Prevalence
Survey
DRS
Facility
survey
Facility
Survey
DRS
Facility
survey
External Monitoring Visits
Routine information system & surveillance
2000
2002
2004
2006
Why is data quality important?
The primary function of health-information
systems is to provide data that enhance
decision-making in the provision of health
services.
 By ensuring high-quality data, the health
information system attempts to guarantee
that decision-makers have access to
unbiased and complete information

Standards for good quality data





Validity: Do the data clearly and directly measure
what was intended to measure?
Integrity: Are mechanisms in place to reduce the
chance that data are intentionally manipulated?
Precision: Are the data at the appropriate level of
detail?
Reliability: Would you come to the same finding
if the data collection and analysis were
repeated?
Timeliness: Are data available frequently enough
to inform decisions?
Impediments to good data
quality
Inappropriate data-collection instruments
and procedures
 Poor reporting and recording
 Errors in processing data (editing, coding,
data entry, tabulating)

What can be done to improve and
ensure data quality?








Keep the design of the information system as simple as
possible
Involve users in the design of the system
Standardize procedures and definitions
Pre-test data-collection instruments to make sure they
are useful and user-friendly
Ensure that data collected are useful to the data collector
Regular supervision and feedback from supervisors
Plan for effective checking procedures (such as crosschecking)
Training (data collection instruments, data processing,
analysis and decision-making based on evidence)
Data-quality assessments
Example at district level:
Step 1: Interview appropriate individual to
obtain understanding of data collection,
analysis, and maintenance process
 Step 2: Review reports to determine
whether they are consistent

Data-quality assessments (con’t)

Step 3: Periodically sample and review data for
completeness, accuracy, and consistency
 Indicator
definitions are consistent with NTP
guidelines
 Data collection is consistent from year to year
 Data are complete in coverage
 Formula used to calculate indicator (if any) is applied
correctly

Step 4: Compare central-office records with
district or district with facility for consistency and
accuracy
Data-quality assessments (con’t)
Possible data-quality limitations:
 Validity: The reported data do not accurately represent
the population. For example, records may over-report or
underreport certain parts of the population
 Integrity: The data could be manipulated for a variety of
reasons
 Timeliness: If reporting is not up to date, than decisions
may not based on most recent evidence
 Reliability: Implementation of data collection may be
irregular or mistimed
Conclusion
needed based on context/region
and what the biggest issue is