Under-Five Mortality Estimation
Assessing Summary Birth History Methods
with Microsimulation
Resumen ampliado para el VI Congreso ALAP
Andrea Verhulst1 and Bruno Masquelier2
Université catholique de Louvain
Abstract
In order to track progress on the under-five mortality rate for monitoring Millennium
Development Goal 4, summary birth histories collected in surveys and censuses are a key
source of data in many developing countries. Two kinds of methods for analyzing summary
birth histories are now available to users: first, model-based methods derived from the pioneer
work of Brass, second, empirically-based methods developed more recently by researchers at
the Institute for Health Metrics and Evaluation. The performance of both approaches has not
been evaluated extensively and needs further scrutiny. In this paper, we assess them
comparatively against gold standard estimates generated by demographic microsimulations.
This allows controlling for the data quality in order to focus on the modeling assumptions of
the summary birth history methods.
1
2
F.R.S.-FNRS Research Fellow, contact: [email protected].
F.R.S.-FNRS Postdoctoral Researcher.
Introduction
Millennium Development Goal 4 aims to reduce the under-five mortality rate (U5MR)
by two thirds between 1990 and 2015. However, in many developing countries,
tracking progress on the U5MR is impaired by the lack of comprehensive vital
registration systems.
Most countries overcome this lack of well-functioning vital registration systems
with two types of alternative data: full birth histories (FBHs) collected in surveys, and
summary birth histories (SBHs) collected in surveys and censuses, both from women
in reproductive age. In FBHs, mothers list all their live-born children and report their
date of birth and, for those who died, their age at death. This information allows
estimating the U5MR directly through an occurrence/exposure procedure. By contrast,
SBHs only contain the number of children ever born (CEB) and children dead (CD)
without location of births and deaths in time. An indirect procedure has to be used to
approximate the length of exposure to the risks of dying and to derive estimates of
U5MR.
Although direct estimates derived from FBHs are deemed more reliable (Silva,
2012), SBHs are easier and cheaper to collect, and they are still the only source of
information in several countries for which census data exist but no recent survey has
been conducted. In addition, surveys are of limited utility for making subnational
estimates or estimates across many socio-economic strata (Dwyer-Lindgren et al.,
2013). Thus, SBHs collected in censuses remain indispensable for this purpose.
Two kinds of indirect methods for analyzing SBHs are now available to users: first,
model-based methods derived from the pioneer work of Brass (Brass and Coale,
1968), second, empirically-based methods developed more recently by researchers at
the Institute for Health Metrics and Evaluation (IHME) (Rajaratnam et al., 2010). The
performance of both approaches has not been evaluated extensively and needs further
scrutiny. In this paper, we assess them comparatively against gold standard estimates
generated by demographic microsimulations. This allows controlling for the data
quality in order to focus on the modeling assumptions of the SBH methods.
Two approaches to derive U5MRs from SBHs
The Brass indirect method makes use of the age of mothers, or of another cohort
duration function such as the time since first birth, as a proxy for the length of
exposure to the risks of dying of their children. On this basis, the proportions of
children who died, classified by 5-year duration group, are transformed in U5MR by
using model age patterns of fertility and child mortality.
The standard variant of the Brass method uses either the Coale and Demeny or the
UN mortality models (Hill, 2013a). This standard method includes a time location
procedure permitting to estimate U5MR trends for the period approximately 15-20
years before data collection.
The method is known to be vulnerable to errors and biases related to (1) the data
quality, (2) the violation of the modeling assumptions and (3) the selection of respondents. The first type includes principally recall errors in the reporting of CEB/CD. The
second type concerns biases resulting from potential violations of the following
modeling assumptions: no variation of child mortality by 5-year classifying group,
accuracy of the fertility and mortality models, constant fertility and gradual and
unidirectional change of child mortality in the recent past. The third type proceeds
from the association between the non-response of some mothers and the risk of dying
of their children.
These errors and biases have already been explored in the literature (Ewbank,
1982; Arthur and Stoto, 1983; Garenne, 1984; Hill, 1984; Silva, 2012; Ward and Zaba,
2008). Estimates derived for the more recent time period, based on SBHs obtained
from women aged 15-19 and 20-24, are known to be severely biased upwards by an
“age effect”, that is the over-mortality related to the first births and the lower
socio-economic background of younger mothers. The variant based on the time since
first birth (Hill and Figueroa, 2001) which aims to reduce this kind of bias still needs to
be evaluated. In contrast, the estimates associated with women aged 30-34, and
referring to the period around 6 years before the data collection, is considered robust
(Hill et al., 2012). For the estimates located further in the past, omissions of CEB/CD
usually increase with the age of mothers, and their effects interact with increasing
violations of assumptions about fertility and mortality trends. Selection biases have
only been explored in the context of HIV/AIDS epidemic where the survival of mothers
is highly correlated with the mortality risk of their children, and were shown to be
severe (Hill, 2013b).
More recently, Rajaratnam et al. (2010) developed a new set of empirically-based
summary birth history methods. Using cohort and period measures of CEB/CD, they
used regression models to capture the country and regional variations in the age
patterns of fertility and mortality from the available Demographic and Health Surveys
(DHS). Their cohort-based method uses the same 5-year group inputs and presents a
similar time location procedure than the standard method. The main innovation
concerns their period-derived method that generates U5MRs for each year before the
data collection up to 25 years. The period measures of CEB/CD are obtained from
observed frequency distributions of births and deaths averaged by region, age of
mother and number of CEB/CD.
Both cohort and period-derived methods have a maternal age and a time since first
birth variants, thus totaling 4 different options. In addition, a fifth method combines
the four options through a Loess local regression, and is considered to perform best.
In comparison with the Brass approach, these new methods do not require
choosing model patterns, and are better able to capture changes in fertility and
mortality trends. In addition, they aim to produce more reliable estimates for the
recent period: the cohort-derived method by capturing the observed higher mortality
of younger mothers in DHS surveys (the more recent estimates associated with
women aged 15-19 are excluded however due to sampling errors), and the
period-derived method by using all mother age groups to generate these recent
estimates. Nonetheless, these empirically-based methods are still based on implicit
modeling assumptions related to the representativeness and data quality of the DHS
surveys. The authors showed that their new methods performed better than the
standard method. But they only validated them on average against direct estimates
from all DHS FBHs. Therefore, more research is needed on their performance for
specific demographic contexts.
Method and data
We resort to demographic microsimulations to produce gold standard validation
estimates. We use the Socsim program3 which generates fictitious populations at the
individual level. The program uses a Monte Carlo algorithm that transforms monthly
demographics rates into individual waiting time before the occurrence of competitive
events (births and deaths in this case). The resulting individual records include links
between mothers and children, and allow applying both direct and indirect estimation
methods. We use direct estimates as our gold standard to measure the relative
deviation4 of the SBH methods.
The 2012 Revision of the World Population Prospects (UN, 2013) provides the
input rates to simulate a great variety of contrasted mortality and fertility regimes. We
simulate fictitious populations from 1950 to 2010 so as to obtain final population of
minimum 500,000 individuals. We then apply the estimations methods as if a survey
or census had been taken at the mid-year 2010.
Preliminary results
Our first results explore 18 Latin American and Caribbean countries. All the selected
countries are characterized by smooth but large fertility and mortality declines in
recent decades. These preliminary simulations do not include any age or selection
effects.
We compare three SBH methods to our gold standard estimates: (1) the standard
method using the West Coale and Demeny model, (2) the new maternal age
cohort-derived (MAC) and (3) period-derived methods (MAP) from the IHME team.
Due to the noisiness of MAP estimates, we smooth trends in mortality with Loess local
regression as recommended by the authors. The country-specific plots are presented
in Figure 3.
Figure 1 shows the relative deviation associated with the standard and MAC
methods by age group of mother, excluding the women aged 15-19. In the absence of
age effect, the standard method is relatively reliable for the two most recent estimates,
particularly for the 25-29 age group, whereas the MAC method is characterized by
systematic underestimation of mortality (by 19% and 9% on average). This most
likely reflects the absence of age effect in our simulated environment. In conditions of
higher mortality of children born to young women, the MAC method could be closer to
the reference. Going further back in time through older age groups of mothers, the
standard method suffers from a growing upward bias, reaching an average of 31%
deviation with women aged 45-49. In contrast, the MAC method is remarkably robust
for these older estimates.
Figure 2 shows the relative deviation of the MAP method for the five 5-year
covered periods, with the right-hand plot offering a closer look at the five more recent
years. Estimates are on average too high (around 10% in average) for the periods
prior to 2005 and too low for the most recent years (12% for the first year). This is
because the declining trends tend to be too strong, which is particularly visible in
country-specific plots presented in Figure 3.
3
4
http://lab.demog.berkeley.edu/socsim/.
Relative deviation = (indirect estimate -direct estimate) / direct estimate.
Discussion and further work
Our preliminary results showed first that in the absence of age effect, the standard
method performed well for the more recent estimates, whereas the MAC estimates
were too low. In our future work, we will introduce different levels of child mortality
by age group of mother and birth order so as to explore how the MAC method can
handle them. We will also evaluate to what extent the since first birth variants permit
to reduce this kind of effects.
Secondly, our results showed that for older estimates, the standard method
performed very poorly in contexts of strong fertility and mortality decline by
overestimating strongly the rate of decline in U5MR, whilst on the contrary, the MAC
method produced rather robust estimates. We will expand this comparison to other
regional contexts and use other model patterns.
Thirdly, the MAP method was shown to have a tendency to reproduce too rapid
declines in child mortality. This is because the method is based on birth and death
distributions averaged across all past DHS surveys conducted in the Latin American
and Caribbean region. Therefore, although it provides very recent estimates, the MAP
method should be used with caution. In the full paper, we will assess the method for
the other regions, and also examine the performance of the combined method.
Other complementary research avenues include new methods to assess the SBH
data quality, and the evaluation of the selection biases related to the correlation
between child and mother survival on FBH and SBH estimates, which is crucial for the
study of mortality in crisis contexts (Bergouignan, 2010).
Figure 1: Relative deviation of U5MR estimates generated from standard and MAC
methods by age group of mother.
Figure 2: Relative deviation of U5MR estimates generated from MAP method by 5 and
1-year period.
Figure 3: Comparison of U5MR estimates generated from direct and SBH methods for
18 Latin American and Caribbean countries.
Figure 3: (Second part) Comparison of U5MR estimates generated from direct and SBH
methods for 18 Latin American and Caribbean countries.
References
W. B. Arthur and M. A. Stoto (1983), “An analysis of indirect mortality estimation”,
Population Studies, 37(2):301–314.
C. Bergouignan (2010), “Recourir aux microsimulations pour étudier la mortalité de
crise : illustration par la mortalité au Burundi en 1993”, Cahiers québécois de
démographie, 40:299–329.
W. Brass and A. Coale (1968), “Methods of analysis and estimation”, In W. Brass, A.
Coale, P. Demeny, D. Heisel, F. Lorimer, A. Romaniuk, and E. Van de Walle (eds.), The
Demography of Tropical Africa, Princeton University Press, Princeton, pp. 88–150.
L. Dwyer-Lindgren, E. Gakidou, A. Flaxman, and H. Wang (2013), “Error and bias in
under-5 mortality estimates derived from birth histories with small sample sizes”,
Population Health Metrics, 11(13).
D. Ewbank (1982), “The sources of error in Brass’s method for estimating child
survival: the case of Bangladesh”, Population Studies, 36(3):459–474.
M. Garenne (1984), “Problems in applying the Brass method in tropical Africa: a case
study in rural Senegal”, Genus, 40:119–133.
K. Hill (1984), “An evaluation of indirect methods for estimating mortality”, In J. Vallin,
Pollard J., and L. Heligman (eds.), Methodologies for the Collection and Analysis of
Mortality Data, Odina Editions, Liège, pp. 145-176.
K. Hill (2013a), “Indirect estimation of child mortality”, In T. Moutrie, R. Dorrington, A.
Hill, K. Hill, I. Timaeus, and B. Zaba (eds.), Tools for Demographic Estimation, International Union for the Scientific Study of Population, Paris, pp. 148-164.
K. Hill (2013b), “Introduction to child mortality analysis”, In T. Moutrie, R. Dorrington,
A. Hill, K. Hill, I. Timaeus, and B. Zaba (eds.), Tools for Demographic Estimation,
International Union for the Scientific Study of Population, Paris, pp. 148-164.
K. Hill and M.-E. Figueroa (2001), “Child mortality by time since first birth”, In B. Zaba
and J. Blacker (eds.), Brass Tracks. Essays in Medical Demography, The Athlone Press,
London, pp. 9-19.
K. Hill, D. You, M. Inoue, M. Oestergaard, and TAG (2012), “Child mortality estimation:
Accelerated progress in reducing global child mortality, 1990-2010”, PLoS Med, 9(8):
e1001303.
J. Rajaratnam, L. Tran, A. Lopez, and C. Murray (2010), “Measuring under-five
mortality: Validation of new low-cost methods”, PLoS Med, 7(4):e1000253.
R. Silva (2012), “Child mortality estimation: Consistency of under-five mortality rate
estimates using full birth histories and summary birth histories, PLoS Med,
9(8):e1001296.
UN (2013), World Population Prospects: The 2010 Revision. Extended Dataset, United
Nations Publication. URL: http://esa.un.org/wpp/.
P. Ward and B. Zaba (2008), “The effect of HIV on the estimation of child mortality
using the children surviving/children ever born technique”, Southern African Journal
of Demography, 11 (1).