The Consequences of Measurement Error when Estimating the
Impact of BMI on Labour Market Outcomes *
Donal O’Neill and Olive Sweetman **
National University of Ireland Maynooth
Revised 13/11/2012
Abstract. This paper uses data on both self-reported and true measures of individual
Body Mass Index (BMI) to examine the nature of measurement error in self-reported
BMI and to look at the consequences of using self-reported measures when estimating
the effect of BMI on economic outcomes. In keeping with previous studies we find
that self-reported BMI is subject to significant measurement error and this error is
negatively correlated with the true measure of BMI. In our analysis this non-classical
measurement error causes the traditional approach to overestimate the relationship
between BMI and both income and education. Furthermore we show that popular
alternatives estimators that have been adopted to address problems of measurement
error in BMI, such as the conditional expectation approach and the instrumental
variables approach, also exhibit significant biases.
JEL Classification: L14, C13
Key Words: Obesity, Non-Classical Measurement Error, Auxiliary Data, Instrumental
Variables
*
We would like to thank Aedin Doris and Dave Madden for helpful comments on an earlier draft of
this paper.
**
Corresponding author: e-mail: [email protected], tel:353-1-7083555.
2
1. Introduction
Obesity is a medical condition described as excess body weight in the form of fat. The
International Obesity Task Force (2010) estimates that approximately 1.0 billion
adults are currently overweight and a further 475 million are obese. In the European
Union 27 member states, approximately 60% of adults and over 20% of school-age
children are overweight or obese. Obesity is an important cause of morbidity,
disability and premature death (WHO, 2004) and increases the risk for a wide range
of chronic diseases. As result there are substantial direct and indirect costs associated
with obesity that put a strain on healthcare and social resources.
As well as the studies examining the cost of obesity to the state, in recent years
there have also been a number of studies that examine the impact of obesity on
individual outcomes such as wages (Cawley 2004, Brunello and d’Hombres 2007),
labour force participation and employment (de Sousa 2012) and educational
achievement (Kaestner and Grossman 2009, von Hinke et al. 2012). Many of these
studies find a significant negative association between body weight and individual
economic success. The costs of obesity are therefore borne at the individual as well as
the national level.
The most widely-used method of measuring and identifying obesity is Body
Mass Index (BMI), where BMI = weight in kg/height in m2. 1 In some cases
researchers use field experiments to examine the impact of obesity on labour market
outcomes (Rooth 2009), however, the majority of these studies rely on self-reported
measures of BMI from survey data sets such as the National Longitudinal Study of
Youth, the European Community Household Panel and the National Child
Development Survey. In this case researchers have to address the possibility that selfreported BMI is measured with error. There is a large body of evidence that suggests
that self-reported BMI tends to underestimate true BMI; this occurs both because
people underreport their weight and because people overstate their height. Most
1
Recently Burkhauser and Cawley (2008) compared multiple measures of fatness and found that many
important patterns, such as who is classified as obese, group rates of obesity, and correlations of
obesity with social science outcomes, are all sensitive to the measure of fatness and obesity used (see
also Johansson et al. 2009 and Wada and Tekin 2010). We do not address this issue in our paper.
3
authors recognize this and as result typically adopt a range of approaches to deal with
the problem of mismeasured obesity.
In this paper we use a unique data set that contains both self-reported and
recorded measures of height and weight for a sample of 7,522 females. The
availability of both the self-reported and recorded measures allows us to examine in
detail the nature of measurement error in BMI. In keeping with previous work we
show that measurement error in BMI is non-classical. This means that the approaches
adopted in earlier papers to correct for measurement error, which rely on the
assumption of classical measurement error, are unlikely to provide consistent
estimates. As well as measures of obesity our data also contain detailed information
on individual characteristics, their labour market experiences and their family
circumstances. This allows us to determine the direction of the bias that arises in
simple regressions of outcomes on obesity when measurement error is non-classical
and also to assess the performance of many of the alternative approaches that have
been used to deal with the problem of measurement error.
Section 2 summarises the statistical literature on measurement error. We focus
on the biases that arise when the assumptions of classical measurement error are
relaxed. Section 3, discusses our data and examines the nature of measurement error
in our self-reported measures of BMI. Section 4 considers the implications of this
measurement error when examining the impact of obesity on individual outcomes. We
focus on two outcomes previously examined in the literature, a continuous measure
namely income, for which we use traditional linear regression analysis and a binary
variable measuring educational attainment for which we rely on a probit model.
2. Measurement Error in Economic Analysis
Econometric analysis involves examining the outcomes of random experiments in
order to make inferences about the distribution function underlying the true data
generating process. Measurement error in the observed data, however, may lead
researchers to draw incorrect inferences. The impact of measurement error on the
mean of a random variable has been studied extensively (e.g. Fuller (1987), Carroll et
al (1994) and Bound et al (2001)). For the most part, studies of measurement error
focus on the typical textbook model of classical measurement error. However in their
4
study of measurement error in labour market data, Bound et al. (1994) argued that in
most analyses of labour markets the assumption of classical measurement reflected
convenience rather than conviction. Bound et al (2001, p. 3709) conclude their survey
by calling for researchers to pay greater attention to the possibility of non-classical
measurement error, both in assessing the likely biases in the analyses that take no
account of measurement error and in devising procedures that correct for such error.
Non-classical measurement error can arise in many circumstances. For
example if the variable in question is binary (say a 0/1 indicator variable for example)
then any measurement error must be correlated with the truth (Aigner 1973). The
reason for this is that with a dummy variable, an observation can only be misclassified
on one of two ways; a true 0 can be misclassified as a 1 or a true 1 can be
misclassified as a zero. Clearly the value of the error is determined by the true value
of the binary variable, thereby violating the classical assumption. Hyslop and Imbens
(2001) consider a situation in which the individual is fully aware of his/her ignorance
and actively seeks to provide an optimal response given his/her information set. This
leads to measurement error that is uncorrelated with variables in the information set
and therefore, by necessity, correlated with the true value of interest.
In recent years a number of papers have examined the consequence of nonclassical measurement error in labour economics. Pischke (1995), O’Neill et al.
(2007) and Gottschalk and Huynh (2010) all show that non-classical measurement
error of the type typically found in income data attenuates the role of white noise
measurement error in models of earnings dynamics, while Kim and Solon (2005)
suggest that real-wages may be even more procyclical than recent studies suggest
once one accounts for mean-reverting measurement error.
Several approaches have been suggested for dealing with measurement error,
some of which are summarised in Bound et al. (2001). One popular approach relies on
the availability of auxiliary data which can be used to correct estimates for
measurement error (Lee and Sepanski 1995). While the auxiliary data allow
researchers to examine the nature of measurement error, these data typically do not
contain information on the dependent variable of interest. As a result the information
gained from the auxiliary data must be “transported” into the main survey data. For
instance Cawley (2004) corrects for measurement error in reported BMI by predicting
5
true height and weight in the NLSY using information on the relationship between
true and reported values in the Third National Health and Nutrition Examination
survey (NHANES III). In particular he uses the NHANES III to run separate
regressions by race and gender of actual weight on reported weight. The process is
repeated for height. Then self-reported height and weight in the NLSY are adjusted by
the coefficients on the reported values in the NHANES III. These fitted values are
then treated as, error free measures, and used throughout the remainder of the paper.
This approach is known as the “conditional expectation approach” (Lyles and Kupper
1997).
A second favoured approach uses the instrumental variables method to obtain
consistent estimates. This approach is popular among econometricians (for an
overview see Angrist and Krueger 2001) and requires a valid instrument. When
looking at the effect of obesity on labour market outcomes, this would require finding
a variable that is correlated with the true unobserved measure of obesity but
uncorrelated with either the measurement error or the stochastic component of the
outcome equation. The variation in the outcome variable induced by the instrument
can then be used to identify the true relationship between say BMI and the outcome of
interest. In BMI studies researchers often use the BMI of a sibling or other relative
such as a child as an instrument for reported or measured BMI (Cawley 2000, Cawley
et al. 2004, , Brunello and D’Hombres 2007, Kline and Tobias 2008, Kortt and Leigh
2010, Lindeboom et al. 2010, Cawley and Meyerhoefer 2012). 2
With classical measurement error, both of these approaches provide consistent
estimates but it is easy to show that neither of these standard approaches will work if
measurement error is non-classical. In order to focus attention on the problems of
measurement error, we consider estimating a conditional mean function, E[Y|X*],
where Y is the outcome of interest (assumed measured without error) and X* is the
true value of the variable of interest 3. Assuming E[Y|X*]=βX* we write our regression
equation as :
2
In many studies the instrumental variables approach was used to overcome endogeneity of BMI.
However, our findings will have implications for these studies as well in so far as the measure of BMI
being instrumented is also measured with error. We discuss this in more detail later in the paper.
3
This approach is in keeping with Bound et al. (1994). However, unlike that study we do not require
that the measurement error in BMI be uncorrelated with the stochastic component of the income
generating function.
6
.4
(1)
By assumption E[ X*]=0 which allows us to ignore potential problems associated
with endogeneity of X*, although we will return to this issue later in the paper. In
addition the observed value of X*, which we denote by X, is given by
(2)
where u is the measurement error. Classical measurement error refers to the case
where the error for any individual, ui, is unrelated to the true value Xi*; this in turn
implies E(X*u)=0. Non-classical measurement error can arise in two cases; firstly
there may be a relationship between the reported measurement error and the true
value of the variable so that E(X*u)≠0; secondly there may be a relationship between
the reported measurement error and the residual in equation (1) so that E(єu)≠0. The
latter situation is sometimes referred to as differential measurement error; in this case
X contains information about Y even after we condition on X*. 5
It is easy to show that the simple OLS estimator from a regression of Y on the
observed X is inconsistent in the presence of measurement error of this type. Allowing
for correlation between the measurement error and the true value of X* and for
correlation between є and u we can show that:
4
Although the impact of measurement error on the estimated coefficients in generalized linear models
such as logistic and probit models depends on depends on the derivative of the regression function with
respect to the mismeasured and the curvature of the likelihood function, in many cases the bias is
similar to that in the linear regression model (see for example Stefanski and Carroll (1985) and Buzas
et al. 2005).
5
Measurement error is said to be non-differential when the conditional distribution of y given X and X*
is the same as that of y given X*. In this case X is said to be a surrogate for X*. Black et al (2000) derive
bounds for the parameter of a univariate regression for the case of non-differential, nonclassical
measurement error.
7
If Cov(u,X*)= Cov(u,ε)=0 this simplifies to the textbook attenuation bias
associated with classical measurement error. However, violation of either of these
conditions will alter the probability limit of the OLS estimator such that the direction
of the inconsistency cannot be established a-priori.
The problems posed by non-classical measurement error for both the
conditional expectation approach and the IV approach to measurement error are also
immediate. The conditional expectation approach regresses true earnings on observed
earnings using the auxiliary data and uses these coefficients to predict true BMI in the
survey data. To see how this approach works consider taking conditional expectations
in equation (1):
Clearly if
However, if
then a regression of Y on
then a second regression of Y on
will consistently estimate β.
alone may result in
biased and inconsistent estimates of β.
Likewise the standard IV approach requires an instrument Z such that
E(ZX*)≠0 but E(Zu)=0. However, the correlation between X* and u will mean that
instruments that are strongly correlated with X* are also likely to exhibit a correlation
with the measurement error thus violating the condition for consistency. 6 Formally the
IV estimator converges to
6
Cawley (2004) acknowledges this point and for this reason argues that it is important to correct for
measurement error prior to using IV estimation (to control for endogeneity). However, most papers in
the literature do not adopt this approach and simply instrument the self-reported measure. Furthermore,
as shown above, the approach used by Cawley (2000) to correct for measurement does not overcome
the problem of differential measurement error so that the problems with IV we discuss still apply to his
“corrected” measure.
8
Clearly the second term in the denominator of this expression will lead to
inconsistencies in the IV estimates, and if this term is negative then the IV estimator
may overestimate the true coefficient of interest, β.
3. Data
The Growing up in Ireland (GUI) survey tracks the development of a cohort of Irish
children born between November 1997 and October 1998. The data used for our
analysis are from the first wave of interviews, which were carried out between August
2007 and May 2008. 7. Information was collected from the children, their parents, their
teachers, the school principals, and their childminders (where relevant). Dor et al.
(2010) find that in the U.S the incremental costs of obesity are significantly higher for
obese women than for obese men. Consequently to illustrate our results we focus on the
impact of obesity on economic outcomes for the mothers in our sample. At the time of
the survey the average age of the mothers was 39 years of age.
The mother provides information on variables such as household composition,
income, occupation, parental education, and the child’s health, lifestyle and education.
In addition she was asked to report her measured height in cms and her weight in kgs.
The key feature of the data for our purposes is the fact that in addition to these selfreported measures of height and weight the interviewer also measured the
respondent’s height and weight. We refer to the latter as recorded measures and treat
them as the true height and weight of the respondents. We compare these to the selfreported measures to determine the extent and the nature of measurement error in
BMI for all the mothers in our sample.
Summary statistics for the self-reported and recorded measures of BMI are
given in Table 1. Figures 1-2 show the distributions of the BMI index using both the
self-reported and the true measure. The lines on each graph correspond to a BMI of 25
and 30 respectively. Within this range individuals are deemed overweight, beyond
this individuals are deemed obese. Using these data we find that 42.65% (13.87%) of
the mothers in our sample are overweight (obese) on the basis of self-reported data.
However, the true numbers are 49.85% and 17.28%. The tendency for respondents in
7
Only the first wave is available to date.
9
our sample to underestimate their BMI in self-reported data is consistent with
previous findings. 8
To examine the extent of measurement error in the self-reported measures of
BMI in the GUI survey, we calculate the error in the self-reported data by subtracting
the true BMI measure from self-reported measure. The density of the measurement
error is given in Figure 3 and summary statistics for the measurement error are given
in Table 2. The results in Table 2 show that the mean error is negative. In addition the
magnitude of the measurement error is sizeable, accounting for 34.7% the standard
deviation of the true BMI variable. Figure 3 shows that the error is not normally
distributed. The distribution is more concentrated around the mean than would be the
case with a normal distribution. The remainder of the paper explores the discrepancies
between true and reported BMI in more detail and considers the implications of these
differences for the relationship between BMI and economic outcomes.
4. Analysis and Results
In the previous section we showed that mothers in the Growing up in Ireland survey
are more likely to underreport their BMI. In the notation of section 2, this implies
E[u]<0. However, as noted in section 2, the implications of measurement error for
economic analysis will differ depending on whether the error is classical or nonclassical in nature. To determine this we examine the relationship between the error
and the true measure of BMI. Figure 4 graphs the relationship between the
measurement error and the true measure of BMI. From this we see a negative
relationship between the level of measurement error and the true value of BMI.
People with higher BMI’s are more likely to underreport their BMI. 9 The correlation
is -.34. This negative correlation illustrates the non-classical nature of measurement
error in BMI in our data. As noted in section 2 this has implications for the direction
of the bias when using self-reported BMI data to examine the relationship between
BMI and economic outcomes, as well as for the consistency of proposed alternative
estimators. We consider these issues in the remainder of this section.
8
See for example Morgan et al (2008) and Shiely et al. (2010) for Ireland, Elgar and Steward (2008)
for Canada, Villanueva (2001) for the United States and Spencer et al. (2002) for the U.K.
9
The negative correlation between measurement error and the true value of BMI is consistent with
international findings (see for example Shiely et al. (2010), Elgar et al. (2005), Villanueva (2001),
Spencer et al. (2002)).
10
To examine the consequences of non-classical measurement error in BMI we
consider the relationship between BMI and two outcomes measures, income and
education. Although ideally we would like to look at the relationship between
individual income data and a mother’s BMI, this is not available in our data. Instead
we are restricted to examining the relationship between mother’s BMI and total
household income. While this question is somewhat different to that examined in
previous work, where the focus was individual wages, the use of household income as
the dependent variable nevertheless provides a valid framework for illustrating the
consequences of non-classical measurement error in the linear regression framework.
In addition to the income results we also illustrate the consequence of measurement
error when examining the relationship between a mother’s BMI and her highest level
of educational attainment. The relationship between individual BMI and education
has been studied previously in the literature and so is of independent interest. In
addition this analysis illustrates the consequences of non-classical differential
measurement error in non-linear estimation models.
4a. Mother’s BMI and Household Income
To begin, we estimate the relationship between true BMI and household income.
Apart from sampling error this coefficient gives the “true” relationship between BMI
and income. This establishes the benchmark for the true parameter of interest. The
results are given in the first column of Table 3. The estimated coefficient on true BMI
shows a significant negative relationship between BMI and income, which is
consistent with previous work in the literature. The true parameter estimate is -.85,
which indicates that a 5 point increase in BMI reduces income by €4250.
To examine the impact of measurement error we estimate the same regression
only this time using self-reported BMI. The results are given in the second column of
Table 3. In contrast to what we would expect with classical measurement error, we
see that the observed measurement error in BMI overstates rather than attenuates the
relationship between BMI and income. Using self-reported BMI we estimate that a 5
point increase in BMI would result in a €4728 reduction in income. As a result the use
of self-reported BMI overstates the loss of income by approximately 11%. We now
11
examine whether popular approaches to tackling measurement error can overcome
this bias.
We first consider the use of auxiliary data. Following Cawley (2004) and
others 10 we regress true BMI on self-reported BMI. The coefficients from this first
stage regression are then used to adjust the self-reported data for measurement error.
Similar to Cawley (2004), we use the square of self-reported BMI in addition to the
level of BMI. The predicted measure is then used in place of the self-reported
measure in the regression analysis. The results from the second stage are presented in
Table 4. For comparison the true estimated coefficient is given in the first column and
the corrected estimate in the second column. The results show that in this case the
auxiliary regression approach remains biased with a bias that is almost identical to
that with the original self-reported data. This finding warns against using the
similarity of corrected and uncorrected estimates to infer the absence of measurement
error bias.
As noted earlier the conditional expectation approach fails when the
measurement error is differential. The availability of internal validation in our study
allows us to examine this in more detail. We obtain consistent estimates of
in
equation (1) by regressing income on true recorded BMI, and then generating the
residuals. 11 Under the assumption of non-differential measurement error the residuals
should be uncorrelated with the observed measurement error u. To examine this we
regress the predicted residuals on the measurement error. The coefficient on the
measurement error is statistically significantly negative, with a point estimate of .754. the null-hypothesis of no relationship is rejected with a p-value=.004. The
differential nature of the error in our BMI data is thus clearly evident in these data. It
is this failure of classical measurement error that biases the estimates obtained using
the conditional expectation approach. 12,13
10
Lindeboom et al. (2011) also report using this procedure to correct for measurement error in selfreported BMI in their study of the impact of obesity on labor market outcomes.
11
This requires that we have correctly specified the functional form of the conditional mean function.
However we have repeated the analysis above for the range of polynomials in income up to and
including degree 5. The results we report are robust to these alternative specifications of the conditional
mean.
12
Clearly whether measurement error is differential or not can depend on the variables included as
controls in the regression of interest. To consider this we re-estimated our models using a set of
additional controls typically used when estimating income equations in labour economics. These
12
A second approach typically used to correct for measurement error in these
data is to use an instrumental variable approach. A common instrument in this type of
analysis is to use the BMI of a sibling or other relative to instrument for the
respondents BMI on the assumption that this should pick up genetic and
environmental factors but may not be related to the measurement error. We follow
Cawley (2000) and use the weight of the study child to instrument for their parent’s
BMI. 14 The F-statistic from our first stage regression of mother’s BMI on child’s
weight is 400, well in excess of the value 10 suggested by Bound et al. (1995); as
such our analysis is unlikely to be affected by problems associated with weak
instruments. The estimates from using child’s weight as an instrument for selfreported mother’s BMI are given in Table 5, where again the first column shows the
true OLS estimate for comparison. In this case we that the IV results predict that a 5
point increase in BMI reduces income by €6950, compared to a reduction of €4250
based on the true conditional mean.
In the comparisons so far we have assumed away the problem of endogeneity
by imposing E[ X*]=0, in which case the slope of the conditional mean corresponds
to the causal effect of BMI. However, in the event that this assumption is false the
OLS estimate and the IV estimate in Table 5 are not strictly comparable since they are
likely to be estimating different objects; the first a conditional mean and the second a
causal effect. A more appropriate comparison would compare the OLS estimate with
an IV estimate of the conditional mean. To do this we consider the use of an
instrument aimed at removing only the bias associated with measurement error. In the
econometrics literature repeated measures are often used as instruments for
measurement error. Unfortunately such instruments are not available in our data.
However we can nevertheless illustrate the consequences of using IV in the presence
include parent’s age, education, health status, martial status and a control for English language
proficiency. Our results indicate the error in self-reported BMI is still correlated with the residual from
the income equation even when this extended set of controls is used.
13
Bound et al. (2001) page 3738, discuss an extended correction that uses the internal validation data to
correct for differential measurement error. This approach uses internal validation data to control for the
omitted variable that arises when the standard conditional expectation approach is used with
differential measurement error. We have verified that this approach works in our sample but do not
focus on this adjustment here. If one has all the data required to implement this extended approach in
practice then adjusting for measurement error is a matter of efficiency and not bias. We focus on the
latter case which is more serious and seems to arise more often in practice.
14
Other studies that use child’s weight an instrument include Davey Smith et al 2009 and Cawley and
Meyerhoefer 2012). Kline and Tobias 2008 and Lindeboom et al (2010) use the BMI of a parent to
instrument for that of the child, while Brunello and D’Hombres (2007) and Kortt and Leigh (2010) use
the BMI of other biological family members as instruments.
13
of measurement error by artificially creating appropriate instruments. To do this we
create three new variables from our data set, all of which are mismeasured values of
the true BMI and which we call BMI1-BMI3. When simulating the measurement
error for each of these three additional measures we take samples of errors that have
mean zero and are independent of both the true value and income. This error is then
added to the true BMI to create a new measure. In this way we have three new
mismeasured values of BMI that are all subject to classical measurement error. We
will treat one of these measures (BMI1) as a new reported measure of BMI, while the
other two will be used as instruments. These latter two variables should be “valid”
instruments for the measurement error problem but will not adjust for any endogenity
problems. Because of this we argue that a comparison of the OLS estimate with the
IV estimates using these instruments will provide a better indicator of the bias in IV
estimates of the conditional mean when measurement error is nonclassical.
With these additional simulated variables we run three new regressions.
Firstly, we regress income on BMI1 to illustrate the standard effects of classical
measurement error. We then use our two additional measures (BMI2 and BMI3) as
instruments for BMI1; by construction these repeated measures are valid instruments
for measurement error and should return the conditional mean. We then use these
same additional measures as instruments for our real self-reported BMI measure that
is subject to non-classical measurement error and compare the findings. The results
are given in Table 9. The first panel repeats the results for the true specification for
convenience. The results in the second panel illustrate the textbook attenuation bias
associated with classical measurement error. In our example the bias is of the order of
8%. The third panel shows how the availability of “valid” instruments eliminates the
error bias when the measurement error is classical. In addition a Hansen test of
instrument validity fails to reject the over identifying restrictions that result from
having two instruments. However, the results in the fourth panel show how the use of
these “valid” instruments do not result in consistent estimates when used to
instrument the self-reported measure of BMI found in our data. The IV estimate
overestimates the size of the true effect by 13.7%, which is slightly larger than the
bias in the raw OLS estimate. Furthermore the results of the Hansen test in this case
show that we cannot rely on the over identifying test to detect the problems with the
14
instruments. This reflects the low power of this test when none of the instruments are
valid. 15
4b. Mother’s BMI and Mother’s Education
As part of the Growing up in Ireland Survey mother’s were asked to report their
highest level of education in 5 ranges from at most a primary education at the lowest
end to a postgraduate education at the highest level. Using this we create an indicator
variable equal to 1 if the mother has a third level degree or higher and zero otherwise.
We then estimate the relationship between this measure of education and BMI, using
both self-reported and true BMI. Since the dependent variable is binary in this case
we use a probit model rather than a linear regression model. In addition we consider
both the conditional expectation approach and the instrumental variables approach in
the context of this probit specification of educational attainment.
The results are presented in Table 7. Looking at the first column we find a
statically significant negative relationship between true BMI and education. The point
estimates imply that a 5 point increase in BMI reduces the probability of having a
third level degree by 4.5% percentage points. The second column shows the results
when the self-reported measure is used. As was the case when income was used as the
dependent variable we see that using the self-reported measure of BMI overstates the
relationship between BMI and education. The results based on self-reported BMI
imply that a 5 point increase in BMI would reduce the probability of receiving a
college degree by 5.45 percentage points (a bias of over 20% relative to the true
effect). As before the results from the conditional expectation approach presented in
the third column are very similar to those obtained from using the self-reported
measures. Finally the fourth column reports the IV estimate when our artificial
15
An alternative approach would be to use children’s weight as an instrument and compare the IV
estimate of the causal effect when true recorded BMI is instrumented with the IV estimate obtained
from instrumenting the self-reported BMI. Again a comparison of these estimates (which we might
now think of as causal effects) should also provide an indication of the consequences of measurement
error for IV estimation. When we do this we obtain an IV estimate using the recoded BMI equal to 1.22 compared to an estimate of -1.40 using the self-reported BMI. This produces a bias of 13.9% in
the causal effects which is very similar to the bias of 13.7% reported above for the conditional means.
These findings also suggest that much of the increase in the point estimates that result from the use of
instrumental variables should be attributed to issues associated with endogeneity and not classical
measurement error as has been previously suggested.
15
repeated measure is used to instrument self-reported BMI. 16 As before we find that
the order of the bias from the IV approach is similar to that from the raw probit model
using self-reported BMI.
It is clear from this that the results from our probit analysis of educational
attainment are in line with our least squares analysis of income. Firstly, using selfreported BMI overstates the negative relationship between BMI and the outcome
variable; secondly using the conditional expectation approach results in a bias that is
very similar in magnitude to that obtained with the self-reported measure itself; finally
the instrumental variable approach leads to a greater bias than that obtained using the
self-reported measure alone.
5. Conclusion
Obesity imposes very large costs on both governments and individuals. As a result
there is growing concern over measured levels of obesity throughout the world.
However, studies examining the individual costs of obesity typically rely on selfreported data to measure BMI. The use of self-reported BMI gives rise to potential
problems of measurement error which could bias any estimated relationships. This
paper uses a unique data set that contains both self-reported and true measures of an
individual’s Body Mass Index (BMI) for a large sample of adults to examine the
nature of measurement error in BMI and the consequences of this error for estimating
the relationship between BMI and economic outcomes. We find that self-reported
BMI is subject to significant measurement error and importantly this error deviates
from classical measurement error in two important ways. Firstly the error exhibits a
pronounced negative correlation with the true measure of BMI; secondly self-reported
BMI contains information about outcomes even after conditioning on true BMI. In
our analysis we show that these departures from classical measurement error cause the
traditional estimators to overstate the relationship between BMI and outcomes. We
illustrate this using a linear regression model relating BMI and income and a probit
model relating BMI and education. Furthermore we show that popular alternatives
estimators that have been adopted to address problems of measurement error in BMI,
16
The IV results reported here are estimated using maximum likelihood assuming bivariate normality
between the errors in the BMI equation and the latent education equation. We have also estimated the
model using Newey’s (1987) minimum chi-squared estimator and obtained similar results.
16
such as the conditional expectation approach and the instrumental variables approach,
also exhibit significant biases. The estimated biases are of the order of 10% to 15%
depending on the procedure used.
17
References
Aigner, D. (1973), Regression with a Binary Independent Variable Subject to Errors
of Observation,” Journal of Econometrics, Vol. 1, pp. 49-60.
Angrist, J. and A. Krueger (1999), “ Empirical Strategies in Labor Economics,” in
Handbook of Labor Economics, Vol. 3A, O. Ashenfelter and D.Card (eds),
Amsterdam and New York, Elsevier Science.
Angrist, j. and A. Krueger (2001), “Instrumental Variables and the Search for
Identification: From Supply and Demand to Natural ~Experiments,” The Journal of
Economic Perspectives, Vol. 15, no. 4, pp. 69-86.
Black, D., M. Berger and F.Scott (2000), “Bounding Parameter Estimates with
Nonclassical Measurement Error,” Journal of the American Statistical Association,
Vol. 95, No. 451, pp. 739-748.
Bolin, K. and J. Cawley (2007), The Economics of Obesity, Advances in Health
Economics and Health Services Research, Vol. 17.
Bound, J. and A. Krueger (1991), "The Extent of Measurement Error in Longitudinal
Earnings Data: Do Two Wrongs Make a Right?" Journal of Labor Economics 9, 1-24.
Bound, J., C. Brown, and G. Duncan (1994), “Evidence on the Validity of CrossSectional and Longitudinal Labor Market Data,” Journal of Labor Economics, 12,
345-368.
Bound, J., D.A. Jaeger, and R.M. Baker (1995): “Problems with Instrumental
Variables Estimation when the Correlation between the Instruments and the
Endogenous Explanatory Variable is Weak,” Journal of the American Statistical
Association, 90, 443 – 450.
Bound, J, C. Brown, and N. Mathiowetz (2001), “Measurement Error in Survey
Data,” in: J.J. Heckman and E.E. Leamer, eds. Handbook of Econometrics Vol. 5
(North-Holland, Amsterdam) pp. 3707-3843.
Brunello, G. and B. d’Hombres (2007), “Does Body Weight Affect Wages? Evidence
from Europe,” Economics and Human Biology, Elsevier, vol. 5(1), pages 1-19.
Burkhauser, R. and J. Cawley (2008), “Beyond BMI: The Value of More Accurate
Measures of Fatness and Obesity in Social Science Research,” Journal of Health
Economics, Vol. 27(2), pp. 519-529.
Buzas, J. T. Tosteson and L. Stefanski (2003), “Measurement Error,” Handbook of
Epidemiology, pp 729-765.
Carroll, R.J. , D. Ruppert, D. and L. Stefanski. (1994), Measurement Error in
Nonlinear Models (Chapman and Hall, London).
Cawley J. (2000), “An Instrumental Variables Approach to Measuring the Effect of
Body Weight on Employment Disability,” Health Services Research 35(5): 1159–
1179.
18
Cawley, J., Markowitz, S., Tauras, J., (2004), “Lighting up and Slimming Down: the
Effects of Body Weight and Cigarette Prices on Adolescent Smoking Initiation,”
Journal of Health Economics 23 (2), 293–311.
Cawley, J. (2004): “The Impact of Obesity on Wages,” The Journal of Human
Resources, Vol. 39, no. 2, pp. 451-474.
Cawley, J. and C. Meyerhoefer (2012), “The Medical Care Costs of Obesity: An
instrumental Variables Approach,” Journal of Health Economics, 31, pp. 219-230.
Davey Smith, G. J. Sterne, A. Fraser, P. Tynelius, D. Lawlor, (2009), “The
association between BMI and mortality using offspring BMI as an indicator of own
BMI: large intergenerational mortality study,” British Medical Journal 339, b5043.
De Sousa, S. (2012), “ Does Size Matter? A Propensity Score Approach to the Effect
of BMI on Labour Market Outcomes,” EUI Florence.
Dor, A. C. Ferguson, C. Langwith, E. Tan, (2010) “A Heavy Burden: The Individual
Costs of Being Overweight and Obese in the United States,” The George Washington
University School of Public Health and Health Services Department of Health Policy
Elgar F.J. and J.M. Stewart (2008). “Validity of Self-Report Screening for
Overweight and Obesity. Evidence from the Canadian Community Health Survey,”
Canadian Journal of Public Health, .Sep-Oct;99(5):423-7.
Elgar F.J, C. Roberts, C. Tudor-Smith and L. Moore (2005), “Validity of SelfReported Height and Weight and Predictors of Bias in Adolescents,” Journal of
Adolescent Health, 2005 Nov;37(5):371-5.
Fuller, W (1987), Measurement Error Models (Wiley and sons, New York).
Gottschalk, P. and M. Huynh (2010), "Are Earnings Inequality and Mobility
Overstated? The Impact of Nonclassical Measurement Error," The Review of
Economics and Statistics, MIT Press, vol. 92(2), pages 302-315, May.
Haider, S. and G. Solon (2006), Life-Cycle Variation in the Association between
Current and Lifetime Earnings. American Economic Review, 96(4): 1308–1320
Hyslop, D. and G. Imbens (2001), “Bias from Classical and Other Forms of
Measurement Error,” Journal of Business And Economic Statistics, 19(4), pp. 475481.
Johansson, E., P. Bockerman, U. Kiiskinen and M. Heliovaara (2009), “Obesity and
Labour Market success in Finland: The difference between having a high BMI and
being Fat,” Economics and Human Biology, 7, pp. 36-45.
Kaestner, R. and M. Grossman (2009), “Effects of Weight on Children’s Educational
Achievement,” Economics of Education Review, 28, pp. 651-661.
Kim, B. and G. Solon (2005), “Implications of Mean-Reverting Measurement Error
for Longitudinal Studies of Wages and Employment,” Review of Economics and
Statistics. 87, 193-196.
19
Kline, B. and J. tobias (2008), “The wages of BMI: Bayesian analysis of a Skewed
Treatment Response model with nonparametric Endogeneity,” Journal of Applied
Econometrics, 23, 767-793.
Kortt and A. Leigh (2010), “Does size Matter in Australia?,” The Economic Record,
vol. 86, no. 272, 71–83
Lee, L. and J. Sepanski (1995), “Estimation of Linear and Nonlinear Errors-inVariables Models using Validation Data,” Journal of the American Statistical
Association, 90(429), 130-140.
Lindeboom, M., P. Lundborg and B. van der Klaauw (2010) “Assessing the Impact
of Obesity on Labor Market Outcomes,” Economics and Human Biology 8 (2010)
309–319
Lyles, R. and L. Kupper (1997), “A Detailed Evaluation of Adjustment Methods for
Multiplicative Measurement Error in Linear Regression with Applications in
Occupational Epidemiology,” Biometrics, Vol. 53, pp. 1008-1025.
Madden, D (2012), “A Profile of Obesity in Ireland,” Journal of the Royal Statistical
Society A, 175, 893-914.
Morgan K, H. McGee, D. Watson, I. Perry, M. Barry, E. Shelley, J. Harrington, M.
Molcho, R. Layte, N. Tully, E. van Lente, M. Ward, J. Lutomski, R. Conroy, R.
Brugha (2008), SLAN 2007: Survey of Lifestyle, Attitudes & Nutrition in Ireland:
Main Report. Dublin: Department of Health and Children.
Musingarimi, P. (2008), “Obesity in the UK: A Review and Comparative Analysis of
Policies within the Devolved Regions,” International Longevity Centre.
Newey, W. K. (1987) “Efficient Estimation of Limited Dependent Variable Models
with Endogenous Explanatory Variables,” Journal of Econometrics 36: 231–250.
O’Neill, D., O.Sweetman and D. Van de gaer (2007) “The Effects of Measurement
Error and Omitted Variables when using Transition Matrices to Measure
Intergenerational Mobility,” The Journal of Economic Inequality, Vol. 5, no. 2, pp.
159-178.
Pischke, J (1995), “Measurement Error and Earnings Dynamics: Some Estimates from
the PSID Validation Study,” Journal of Business and Economic Statistics, 13(3), pp.
305Rooth, D. (2009) “Obesity, Attractiveness, and Differential Treatment in Hiring: A
Field Experiment,” Journal of Human Resources, 44 (3), pp. 710–735
Scholtens, S., B. Brunekreef, T. Visscher, H. Smit, M. Kerkhof, J. de Jongste, J.
Gerritsen, A.Wijga (2007) “Reported versus Measured Body Weight and Height of 4year-old Children and the Prevalence of Overweight,” European Journal of Public
Health, Vol. 17, No. 4, 369–374
Serneels, P. (2002), Explaining Non-Negative Duration Dependence Among the
Unemployed, The Centre for the Study of African Economies Working Paper Series.
Working Paper 172.
20
Shiely F., I. Perry, J. Lutomski, J. Harrington, C. Kelleher, H. McGee, K. Hayes
(2010), “Temporal trends in Misclassification Patterns of Measured and Self-Report
Based Body Mass Index Categories - Findings from Three Population Surveys in
Ireland,” BMC Public Health, 10:560
Solon, G (1992), “Intergenerational Income Mobility in the United States,” American
Economic Review, 82(3), 398-408.
Spencer E.A, P.N Appleby, G.K Davey and T.J Key (2002), “Validity of SelfReported Height and Weight in 4808 EPIC-Oxford participants,” Public Health
Nutrition Aug; 5(4):561-5.
Stefanski, L. and R. Carroll (1985), “Covariate Measurement Error in Logistic
Regression,” The Annals of Statistics, 13(4), 1335-1351.
Villanueva, E. (2001), “The Validity of Self-reported Weight in US adults: a
Population Based Cross-Sectional Study,” BMC Public Health, 1:11
von Hinke, S. G. Smith, D. Lawlor, C. Propper and F. Windemeijer (2012), “The
Effect of Fat Mass on Educational Attainment: Examining the sensitivity to different
identification Strategies,” Economics and Human Biology, 10, pp. 415-418.
Wada, R. and E. Tekin (2010) “Body Composition and Wages,” Economics and
Human Biology, 8, 242-254.
21
Table 1.
Summary Statistics on Recorded and Self-Reported BMI
Variable
Observations
Mean
Recorded
BMI
SelfReported
BMI
7522
25.97
Standard
Deviation
4.89
7522
25.30
4.59
Minimum Maximum
15.05
64.15
12.7
69.43
Note the maximum srBMI corresponds to someone who was 177Kg(27.85 stone) and 1.5m
(5.25 feet)
Table 2.
Summary Statistics on Self-Reported Measurement Error in BMI
Variable
Observations
Mean
SelfReported
BMI Error
7522
-.6671
Standard
Deviation
1.697
Minimum Maximum
-12.33
9.5
Table 3
Estimated Coefficients on BMI in OLS regressions (Standard Error in brackets)
Dependent
Variable
Household income
Recorded BMI
OLS
-.844
(.088)
Self-Reported BMI
N
OLS
7021
-.9456
(.094)
7021
Table 4
Estimated Coefficients on BMI obtained using Conditional Expectation Approach
(Standard Error in brackets)
Dependent
Variable
OLS
Conditional
Household income
Expectation
approach
Recorded BMI
-.844
(.088)
Corrected BMI
-.9467
(.094)
22
N
7021
7021
Table 5
Estimated Coefficients on BMI obtained using IV Approach
(Standard Error in brackets)
Dependent
Variable
OLS
IV approach
Household income
(using child’s
weight as
instrument)
Recorded BMI
-.839
(.089)
IV BMI
-1.397
(.391)
N
6963
6963
Table 6
Comparison of IV with classical and non-classical measurement error using simulated
data (Standard Error in brackets)
Dependent
Variable
OLS
OLS with
IV with
IV with nonHousehold
classical
classical
classical
income
Measurement
Measurement Measurement
Error
Error (using
Error (using
BMI2 and
BMI2 and
BMI3 as
BMI3 as
instruments) instruments)
Recorded BMI
-.844
(.088)
BMI1
-.779
(.083)
BMI1 (IV using
-.835
BM2 and BMI3)
(.082)
self-reported
-.96
BMI (IV using
(.094)
BMI2 and
BMI3)
Hansen Overp-value=.204 p-value=.176
identification
test
N
7021
7021
7021
7021
23
Table 7
Impact of Measurement Error when examining relationship between BMI and
Education using Probit Model (Reported results are Point Estimates and Standard
Error in brackets)
Dependent
Variable
Probit
Probit
Probit with
IV Probit
Conditional
(using BMI2
Mother’s
Education
Expectation
and BMI3 as
Approach
instruments)
Recorded BMI
-.0299
(.003)
Self-Reported
-.034
BMI
(.0036)
Corrected BMI
-.034
using CE
(.0036)
Self-reported
-.0337
BMI (IV using
(.0039)
child’s weight)
N
7462
7462
7462
7462
0
.05
Density
.1
.15
24
60
80
20
40
0
Body Mass Index (BMI) of the primary caregiver - derived from self-reported data
Figure 1: BMI derived from self-reported data (Firstline corresponds to BMI=25, second line
0
Density
.05
.1
to BMI=30) .
10
50
20
30
40
Primary Caregiver's BMI - derived from measured data
60
Figure 2: BMI derived from recorded data (Firstline corresponds to BMI=25, second line to
BMI=30)
0
.1
.2
Density
.3
.4
.5
25
-15
-10
0
-5
10
5
BMI_error
-15
-10
BMI_error
-5
0
5
10
Figure 3: Distribution of Measurement error in reported BMI
10
20
30
40
50
Primary Caregiver's BMI - derived from measured data
60
Figure 4: Scatterplot of Measurement Error in BMI against the true measure of BMI