SIGHT AND L IFE | VOL. 28 (2) | 2014
VITAMIN A SUPPLEMENTATION (VAS) COVERAGE DATA IN AFRICA
Vitamin A Supplementation
(VAS) Coverage Data in Africa
Modernizing data collection to inform
distribution strategies
Heather Katcher
Helen Keller International, Eastern,
Central and Southern Africa Regional Office,
Nairobi, Kenya
Jessica Blankenship
Independent Consultant
Martin Nankap
Helen Keller International Cameroon,
Yaoundé, Cameroon
Temina Mkumbwa
Helen Keller International Tanzania,
Dar es Salaam, Tanzania
Fatmata Sesay,
Mohamed Turay
Helen Keller International Sierra Leone,
Freetown, Sierra Leone
Aimerance Kabena
Helen Keller International Democratic Republic
of the Congo, Kinshasa, Democratic Republic
of the Congo
Mohamed L Yattara
Helen Keller International Guinea,
Conakry, Guinea
Rolf DW Klemm
Helen Keller International Headquarters,
New York, NY, USA
Improving VAS program performance
Vitamin A interventions have contributed to the reduction in
under-five mortality rates and to progress toward the child survival Millennium Development Goal. Globally, twice-yearly vitamin A supplementation (VAS) remains the most prominent public health strategy for controlling vitamin A deficiency-related
mortality and xerophthalmia.1 In 2010, there were limited data
to assess VAS programs in sub-Saharan Africa other than administrative tally sheets, which often reported coverage over 100%.
Neither the true coverage nor the implementation quality of the
programs was known. To fill this information gap, Helen Keller
International (HKI), in collaboration with host governments, began implementing post-event coverage surveys (PECS) to validate coverage estimates and generate data about the quality of
program implementation. This article presents the rationale for
conducting PECS, the survey content and methodology, HKI’s innovations using mobile devices and experiences applying findings to improve VAS program performance.
Tally sheet VAS coverage estimates
The conventional way to estimate VAS coverage achieved by
Child Health Days and other campaign-like events is by tally
sheets, which compare records of capsules distributed to estimates of the target population based on census data. Tally sheets
are simple and fast, but prone to error due to miscalculations in
aggregating data, delayed or incomplete reports, and inaccurate
population estimates. Because no information is collected about
the children supplemented other than their age group, it is not
possible to identify factors associated with receipt of services or
reasons that children are missed.
Post-Event Coverage Surveys (PECS):
a strategic complement to tally sheet data
By conducting brief household surveys among caregivers of children 6–59 months of age following a distribution event, valuable
data can be collected on coverage rates, the effectiveness of distribution and social mobilization strategies, and barriers to attendance. Since 2010, HKI has conducted 49 PEC surveys using
103
104
VITAMIN A SUPPLEMENTATION (VAS) COVERAGE DATA IN AFRICA
standard WHO Expanded Program on Immunization (EPI) cluster sampling methodology across 13 countries in sub-Saharan
Africa. The work is supported by the Canadian Government. PECS
are designed to assess the proportion of children who received
VAS during a given distribution event and to identify facilitators
and barriers to household participation, sources of information
about VAS events and knowledge of the services received. PECS
also collect demographic and socioeconomic information from
the caretakers surveyed (Table 1). Results of PECS are reviewed
with government and other partners to draw conclusions about
the findings and to apply lessons learned.
“PEC surveys are designed to
assess the proportion of children
who received VAS during a given
distribution event”
To minimize recall bias, PECS are conducted within six weeks
of VAS distribution events, and, during interviews, enumerators
show a sample of a vitamin A capsule to caretakers to help them
differentiate VAS from other interventions. Household surveys
are complemented by interviews with key informants such as
facility-based health workers, community health workers and
village leaders to evaluate other aspects of delivery including
supplies of commodities, transportation, access and staff skills,
and to collect recommendations for improving future rounds
of VAS distribution. Although HKI has used PECS primarily to
evaluate VAS and deworming events, questions can be added to
the survey to simultaneously monitor the delivery of other nutrition and health programs such as insecticide-treated bed nets,
micronutrient powers, and fortified foods, as long as the survey
is kept at a manageable length.
table 1: Data collected during PECS.
Caretakers
“Mobile-izing” PECS
In order to apply findings in a timely manner, data must be processed, analyzed and reported rapidly. To increase the speed of
reporting, HKI migrated PECS data collection from paper to a
mobile phone-based platform. Surveys are first designed offline
in Excel format, then uploaded to mobile phones enabled with
the Open Data Kit (ODK) platform. Data collected this way can be
immediately transmitted to a centralized server where they are
statistically and geo-spatially analyzed (Figure 1 and Figure 2).
Mobile phone data collection has reduced both error and time
between data collection and interpretation. Moreover, GPS coordinates have allowed geo-spatial analyses of physical barriers to
high and equitable coverage.
HKI PECS methodology includes quality assurance procedures for the data collection process. PECS enumerators and
team leaders undergo a three-day training course that includes
administration of questionnaires, selection of households, data
entry and review of submitted data. The training involves a fieldbased practice survey during which enumerators enter data in
pairs to ensure their responses are consistent. During the survey,
enumerators are observed by team leaders or supervisors on a
daily basis to confirm that the survey methodology is adhered to,
that questions are asked appropriately, and that responses are
correctly entered.
Comparison of tally sheet vs PECS coverage estimates
To determine the reliability of tally sheet coverage estimates,
coverage data measured by tally sheets was compared with coverage data measured by PECS. In 94% (46/49) of the locations
where PECS were conducted, coverage estimates measured by
tally sheets were higher than PECS estimates.2 Figure 3 shows
the percentage difference in VAS coverage estimates from tally
sheets and PECS conducted from 2010 to 2014, with the percentage difference in coverage rates ranging from 0.2% to 170%.
Only 12% (6/49) of the tally sheet coverage estimates were
within 5% of the PECS estimates, while 20% (10/49) differed
by 5%–10% and 43% (21/49) were more than 20% off. These
Health workers, CHW, village leaders
If services (VAS, deworming, etc.) were received
Information about the event (if it took place, # of teams employed,
Sources of information about the event
If adequate stocks were provided
Reasons for not receiving services
If there are groups frequently left out
Knowledge of VAS and other services received
Knowledge of VAS and other services
Demographic information
How the event can be improved
Socioeconomic information
Professional information (position, level of experience)
training offered, payment received, etc.)
VITAMIN A SUPPLEMENTATION (VAS) COVERAGE DATA IN AFRICA
SIGHT AND L IFE | VOL. 28 (2) | 2014
figure 1: “Mobile-izing” PECS: How it works.
How it works
xls
Design
Collect
Analyze
Author surveys quickly and easily in Excel and
have instant access on your Android phone.
No advanced technical degree required.
Easily distribute your survey on
an Android device or on the web.
Data connection not needed
Visualize your data as it is collected,
and gain unerstanding with our
powerful insights tool.
data indicate large discrepancies between tally sheet and PECS
coverage estimates and suggest that the tally sheet method may
vastly overestimate VAS coverage in some settings.
figure 2: PECS administered using a mobile phone
in Dar es Salaam, Tanzania.
“The tally sheet method may
vastly overestimate VAS coverage
in some settings”
How PECS have strengthened VAS programs
In addition to providing more valid coverage estimates, PECS
have provided essential information to improve VAS delivery. The
rapid aggregation of PECS data provides timely feedback about
program operations such as planning, implementation, and supply chain management. For example, data on how respondents
learned about the most recent distribution round are used to revise and rework social mobilization strategies before and during
distribution events. Where VAS coverage is low, PECS data help
identify distribution strategies that are not working and inform
new approaches. Survey results also help characterize population groups chronically missed during the VAS distribution rounds
(also known as the “hard to reach” groups) by defining geographic
or socio-demographic features that contribute to their not receiving services. This information is then used to design special outreach and/or communication strategies targeting these groups.
105
VITAMIN A SUPPLEMENTATION (VAS) COVERAGE DATA IN AFRICA
figure 3: Comparison of tally sheet coverage and PECS estimates from 49 surveys conducted from 2010 to 2014.
180,0
160,0
140,0
Percent Difference
120,0
100,0
80,0
60,0
40,0
20,0
rk
in
aF
as
Gu o –
S
Gu inea ahe
l
in
ea – B
Si Gu – ok
er
é
ra ine Kan
Le a – ka
on M n
a
e
Gu – N mo
u
Bu ine ati
o
a
rk
– n
Bu ina Ka al
n
Si rkin Fas ka
n
er a o
ra Fa – E
Le so as
on –
t
e – Sa
DR Na hel
C tio
N – N na
Se iger atio l
ne – na
ga Ta l
l – ho
N u
Bu Nig ati a
rk er on
in ia al
aF –
Gu a FC
in so
– T
G e
Cô uin a – Ea
te ea Fa st
r
d’
Iv – K ana
oi er h
re ou
Ta
nz N – N an
an ige a e
ia r – tio
n
–
Ro Na al
un tio
Si
d n
Ca err Ni 1 al
m a L ge 20
er eo r – 12
oo n
n e – Zin
d
–
DR Bu Litt Nat er
C rki ora ion
–
n
Ba a F l Re al
s C as gi
o
o o
Bu ngo – S n
rk , K ah
e
in
a F insa l
Ni aso sha
ge –
r – Ea
s
Ni Niam t
e
M ge
Ke Ta ali r – D y
ny nz –
a – an Na iffa
No ia – tion
Ca M n-A Zan al
m oz SA zib
er am L
r ar
Ca oo
m n – biqu egio
er
oo Lit e – ns
n tor Ga
–
a
z
M Litt l Re a
M oza ora gio
oz m l R n
am bi
e
bi que gio
qu – n
Ca
e– G
m
er D Na aza
on RC ti
–
– on
M Li Ka al
Ke oz ttor tan
ny am al ga
a – bi Re
Ta
M que gion
u
nz
an Tan mia – Ga
z
s
i
Ca a – ani , Mu za
m Ta a – ta
er bo N ga
oo ra a
M n – , Sh tion
oz
am Litt iny al
bi ora ang
qu l R a
e
e
Ni – N gion
ge am
Ni ria pul
Bu ge – B a
rk ria en
i
N na – ue
Ta ige Fa Ebo
nz ria so ny
an
–
i
ia – A Sa
– kw he
Da a l
r e Ibo
sS m
al
aa
m
0,0
Bu
106
Post Event Coverage Survey
Country-specific examples illustrating the utility of PECS data for
improving VAS programs are provided above. In Cameroon, PECS data indicated that children were not being reached mainly because distribution teams did not visit their
households or because parents or children were not at home
when the team passed by.3 Using this information, large catchment areas were subdivided so that distribution teams could
reach all their assigned locations. Advocacy was also done at
the national level for the events to take place from Friday to Sunday instead of during the week to increase the likelihood that
children and parents would be at home. Communications about
the event were expanded to include announcements at markets
and houses of worship, and flyers were distributed at schools,
churches, markets, health centers, women’s groups and along
the roadside to raise awareness. Subsequent PECS verified that
coverage in the target area increased from 53% in 2011 to near
80% in 2013.
In Dar es Salaam, Tanzania, tally sheet-based coverage for
the January 2014 VAS distribution was 100.7%, while a PECS
indicated coverage to be only 36.4%. Health teams were alerted
to these results and quickly convened to review possible rea-
sons for the low coverage and identify alternative strategies. In
July 2014, mobile health teams used vehicles branded with VAS
posters and a loudspeaker to announce the arrival of services
and raise awareness of the distribution event. They also distributed VAS and deworming in places with high visibility and
large crowds, such as open markets and busy streets (Figure 4).
The mobile health teams supplemented 3,700 children over the
course of two days in areas where most children would not have
been taken to health facilities for VAS and deworming.
In Sierra Leone, three kinds of mobile phone applications
were tested for collecting data to monitor delivery of VAS, as
well as drugs for the control of lymphatic filariasis and supplementary measles immunization.4–7 The technology made coverage data available within 48 hours, enhanced data quality, and
enabled rapid reporting. Data collected and experiences in programming surveys, administering questionnaires and viewing
data were reviewed to identify which of the three applications
performed best.
In Guinea, the use of GPS during their most recent PECS enabled program staff to locate large areas missed by distributors,
providing visual evidence of the oversight to government health
SIGHT AND L IFE | VOL. 28 (2) | 2014
VITAMIN A SUPPLEMENTATION (VAS) COVERAGE DATA IN AFRICA
In the Democratic Republic of the Congo, PECS were conducted in six health zones that had been characterized as “low
performing” to determine if a new communication strategy was
effective at raising awareness and coverage (Figure 6).9 PECS results showed that use of town criers and TV and radio spots that
included a pastor, doctor, husband, and mention of the divine
broadcast in the local language were effective in raising awareness of the campaign. PECS results enabled the planning team
to confirm that the new communication strategy was effective.
In Sierra Leone, Côte d’Ivoire, Senegal, and Burkina Faso,
PECS have validated tally sheet estimates, providing the government and partners with confidence that their campaigns are
consistently reaching at least 90% of eligible children.
figure 4: A mobile health team distributes VAS to children
in Dar es Salaam, Tanzania.
officials (Figure 5).8 The survey results also provided data to
central decision-makers, revealing that, despite adequate
stocks, deworming capsules were not always being delivered,
and that distribution teams included fewer members than had
been budgeted for.
Conclusion
Having valid coverage estimates is critical for monitoring VAS
programs and determining whether programs are reaching their
objectives. Although measuring coverage using tally sheets is
simple and fast, it is highly vulnerable to errors from miscalculations in aggregating data, delayed or incomplete reports,
and inaccurate population estimates. PECS offer a flexible and
reliable alternative that can be used not only to provide accurate VAS coverage estimates, but also to generate timely information to improve delivery of VAS and related interventions.
figure 5: Visualization of GPS data helped to identify regions of Guinea that were missed during the distribution and to inform government
health officials about the situation. Green indicates that the caretaker reported VAS was not received; red indicates that VAS was received.
107
108
VITAMIN A SUPPLEMENTATION (VAS) COVERAGE DATA IN AFRICA
Correspondence: Heather Katcher,
Helen Keller International Eastern, Central and
Southern Regional Office, PO Box 13904–00800, Nairobi, Kenya
E-mail: [email protected]
References
01.Dalmiya N, Palmer A, Darnton-Hill I.
Sustaining vitamin A supplementation requires a new vision.
Lancet 2006;368:1052–1054.
02.Katcher HI, Jaschke LE, Oruru G et al. Validation of vitamin A
supplementation coverage using post-event coverage surveys in
11 sub-Saharan African countries. Micronutrient Forum;
2014 Apr 2–6; Addis Ababa, Ethiopia.
03.Nankap M, Ndjebayi A, Guintang J et al. Factors affecting the
performance of community volunteers during Child Health Days
in Douala, Cameroon. Micronutrient Forum; 2014 Apr 2–6; Addis
Ababa, Ethiopia.
04.Hodges MH, Turay M, Sesay FF et al. Experiences using mobile
phone surveys to monitor and evaluate the distribution of health
services in Sierra Leone. Micronutrient Forum; 2014 Apr 2–6;
Addis Ababa, Ethiopia.
figure 6: PECS made it possible to determine if a new
communication strategy was effective in raising awareness of
the VAS distribution event in low-performing health zones
in the Democratic Republic of Congo.
05.Hodges M, Sesay F, Kamara H, et al. High and equitable mass
vitamin A supplementation coverage in Sierra Leone: a post-event
coverage survey. Glob Health Sci Pract. 2013;1(2):172-179.
06.Sesay FF, Kamara HI, Hodges MH. High coverage of vitamin A
supplementation and measles vaccination during an integrated
Questionnaires can even be expanded to collect information on
other maternal and child health interventions. Moving PEC data
collection to a mobile platform substantially compressed the time
between data collection and interpretation, reduced errors, and
enabled geo-spatial analysis to identify barriers to high and equitable coverage. Periodic PECS thus offer a practical and field-tested method for validating VAS coverage estimates and generate
data that can be used to improve VAS program implementation.
maternal and child health week in Sierra Leone. International
Health 2014; pii: ihu073.
07.Sonnie M, Bah MS, Paye J et al. Episurveyor enhances monitoring
of mass drug administration for lymphatic filariasis to maintain
effective coverage in Sierra Leone. The American Society of Tropical
Medicine and Hygiene Annual Meeting; 2012 Nov 11–15;
Atlanta, Georgia.
08.Yattara M, Massandouno L, Fofana M et al. Validation of administrative coverage for vitamin A supplementation and deworming
“Periodic PECS offer a practical
and field-tested method for
validating VAS coverage estimates”
through Integrated National Immunization Days in Guinea.
Micronutrient Forum; 2014 Apr 2–6; Addis Ababa, Ethiopia.
09.Kabena A, Nahimana D, Kabavulu P et al. Reaching the hard to
reach with vitamin A supplementation in low-performing health
zones of DR Congo. Micronutrient Forum; 2014 Apr 2–6;
Addis Ababa, Ethiopia.
Acknowledgments
The authors gratefully acknowledge financial support from the
Department of Foreign Affairs, Trade and Development (DFATD)
of Canada and the collaboration of our partners within the Ministries of Health in Cameroon, DRC, Sierra Leone, Guinea and
Tanzania. We also acknowledge the editorial and technical contributions by Leigh Jaschke, Jennifer Nielsen and David Doledec.