Getting More from Survey Income Measures: Empirically-based
Recommendations for Improving Accuracy and Efficiency
John L. Czajka* and Gabrielle Denmead**
*Mathematica Policy Research
1100 First Street, NE, 12th Floor
Washington, DC 20002-4221
[email protected]
**Consultant
Stafford, VA 22556
[email protected]
Recent comparative analyses of income measurement in major household surveys found substantial variation in the
sources and distribution of income, and noted areas such as retirement income where survey data lag reality. The
comparisons also found similar aggregates of income, earnings and transfers, despite very large differences in the
number and detail of survey income questions. These findings suggested a range of opportunities for improvements,
and this paper reports on follow-on research related to the Current Population Survey Annual Social and Economic
Supplement (CPS ASEC), with recommendations for improving its estimates of income while reducing the overall
number of questions.
Background
Comparisons of survey income data with benchmarks constructed from administrative records show that surveys
tend to understate total income for most sources (see, for example, Vaughan 1993, Coder and Scoon-Rogers 1996).
This suggests that when one survey is compared to another, the survey that captures more income is probably doing
a better job of measurement. Of the five surveys examined by Czajka and Denmead (2008) that cover the general
population and are conducted by the federal government, the CPS ASEC captured the most total income. Yet the
American Community Survey (ACS), working with a much more limited set of income questions, captured 98
percent as much as the CPS ASEC, and if income was expressed per capita, to adjust for the fact that the ACS
represented, on average, a population nine months earlier than the CPS ASEC, the difference between the two
surveys was reduced to 0.2 percent.
However, the performance of the two surveys was not identical for all categories of income sources. The CPS ASEC
captured 2.8 percent more earned income (that is, wages and salaries plus self-employment income) than the ACS,
and over nine percent more Social Security, but with only five broad questions, the ACS captured a combined total
of eight percent more than the CPS ASEC from all other income sources. Furthermore, while the CPS ASEC and the
ACS captured the same total income from the bottom quintile, the differences by source of income were even more
sizeable than for all income levels. Specifically, the ACS captured 17 percent more earned income while the CPS
ASEC captured 22 percent more Social Security and eight percent more of all other income sources. Lending
support to the suggestion that the CPS ASEC may be picking up too little earnings from the bottom quintile, the
Survey of Income and Program Participation (SIPP), which was explicitly designed to improve income measurement
for lower income families, captured 14 percent more earned income than the CPS ASEC over the same calendar
year, 12 percent less Social Security, and 15 percent more for all non-Social Security unearned income.
Findings from Czajka and Denmead (2008) strongly suggest that survey performance in capturing earned income is
crucial to measuring income and poverty. Earnings comprised at least 82 percent of total income in the five Federal
surveys, and over 90 percent of total income for those under 65. Yet the CPS ASEC obtains less information about
work activity and sources of earnings than it does about Social Security, or property income or about an extensive
list of program and transfer payments other than retirement (some of them now extremely rare) that comprise only
three percent of total income and less than ten percent of the income of the poor. Other income surveys collect data
separately on multiple jobs during the year, as well as multiple self-employment activities. The CPS ASEC collects
detailed data only on the longest-duration work activity during the calendar year, be it a job or farm or non-farm
self-employment, which is supplemented simply by a summary question to capture any other earnings. If interview
topics were selected in order of significance in measuring income and poverty, the time allocated to earnings and to
newer forms of retirement payments would be increased while questions on infrequent income sources were
combined for an overall reduction in interview time.
The success of the ACS in measuring income with a small battery of questions strongly suggests that reducing the
overall number of questions in the CPS ASEC and the SIPP while refocusing their content might have little adverse
effect on the quality of data collected. It is quite possible that an improvement in data would be achieved if item
nonresponse and attrition are lowered. The CPS ASEC will always have higher nonresponse rates than other surveys
with identical questions, because it is a supplement to the basic monthly labor force survey. Some of the monthly
survey respondents decline to participate in the ASEC, and their responses are wholly imputed. However, when
Czajka and Denmead compared item nonresponse and the proportion of income allocated in the ACS to the CPS
ASEC excluding the whole person imputes, they found substantially lower allocation rates for each ACS income
source and for each income quintile. With minor exceptions, this pattern held across all individual sources within
each income quintile. This finding raises the possibility that question simplification may also reduce item
nonresponse. Additionally, fewer questions may also reduce respondent fatigue and elicit more accurate responses
throughout the interview, improving the data for topics such as health insurance that are covered toward the end of
the interview.
The findings from Czajka and Denmead (2008) indicate that the measurement of earnings among low-income
families in the CPS ASEC falls significantly short of the ACS and SIPP. Some reworking and expansion of the
questions on income from employment appears to be in order along with an expansion and restructuring of the
questions on retirement income.
CPS ASEC Income by Source
The CPS ASEC collects data on 20 major sources of income, and the public use file provides both person-level and
family-level measures of recipiency and dollar amounts for each of these sources. Where family-level measures
were not provided, we constructed them. Four of the sources—retirement, survivor, and disability income plus a
residual “other” income—are subdivided into additional, component sources based on a sequence of screeners
followed by check lists. Others, such as Social Security, SSI, veterans benefits and unemployment benefits are
addressed by multiple questions on individual sources (or bases of eligibility). We focus on the 20 major sources.
The first section below examines receipt of these 20 sources among families and unrelated individuals. The second
section looks at the amount of income received by source. The third section examines the additional detail collected
for retirement, survivor, and disability income.
Receipt of Major Sources
Table 1 shows the percent of families and unrelated individuals receiving each of the 20 sources of income by
poverty class. On a recipiency basis, the dominant source of income is earnings, comprised of wages and salaries
and farm and non-farm self-employment income, followed by interest and Social Security income. Overall and
within each poverty class, wage and salary earnings were the most common source, with the percent who received
such income rising with the level of family income. In all, 74.8 percent of families (we use this term to include
unrelated individuals as well) received wage and salary income in 2008, but this varied from a low of 41.5 percent
among families below poverty to a high of 90.5 percent among families above 400 percent of poverty. Another 8.7
percent of families had non-farm self-employment earnings while just 1.4 percent had farm earnings. Both types of
self-employment earnings grew in frequency as family incomes rose.
Interest was claimed by 47.4 percent of all families, but recipiency declined steeply with falling income. Only 14.1
percent of families in poverty reported interest income compared to 72.1 percent of families above 400 percent of
poverty. Interest was less common than Social Security income in the bottom three poverty classes even though
Social Security income—reported by 25.1 percent of families—was only half as common as interest income among
all families. Unlike earnings and interest, the receipt of Social Security income did not rise with relative income.
Rather, the receipt of Social Security income peaked for those with income between 100 and 150 percent of poverty,
and then declined with increasing income.
Two other sources of income were received by at least ten percent of families: dividends and retirement income .
Dividends were not nearly as common as interest, being reported by only 15.4 percent of families, but they were the
third most common source of income among families above 400 percent of poverty. Retirement income increased
with rising income, then leveled off, but might show different levels and relation to income if defined contribution
retirement and IRA payments were included.
Of the other less-common income sources, 12 in all, some increased with rising income, some decreased, and some
were essentially flat. The only source that rose progressively with relative income was the combination of net rent,
royalties, and estates or trusts. Unemployment or strike benefits rose to a peak among families between 200 and 250
percent of poverty but then declined. Veterans’ payments also increased to a point but leveled off. Means-tested
benefits would be expected to decline with rising income, and we see that pattern for Supplemental Security Income
(SSI) and public assistance. Child support and financial assistance from others also declined with rising income but
not as steeply as SSI and public assistance.
Among the rarest income sources, worker’s compensation was claimed by only 0.4 percent of families among the
poor, and on average by only one percent of families above 150 percent of poverty. Other income barely deviated
from a frequency of one percent across all income levels. Disability income (which excludes Social Security
Disability Insurance, SSI, and worker’s compensation) was almost as rare as worker’s compensation and other
income, and alimony was the least common of all income sources, with little variation by income level.
Income by Source
Another way to measure the relative importance of the 20 income sources is to compare their shares of total income.
Table 2 shows the percentage distribution of total income by source for all families (and unrelated individuals) by
poverty class. What is striking here is how little most sources contributed, on average, to the overall total income
received by families at each level of relative income. Wage and salary earnings dominated all sources, of course, and
while such earnings accounted for a greater share of total income at the upper end of the income distribution than at
the lower end, the difference was not as great as the difference between wages and salaries and any other source of
income except Social Security, which partially replaces earnings for the elderly. Wage and salary earnings
accounted for 48.7 percent of the total income of the poor and 80 percent of the total income of families above 400
percent of poverty. Social Security was next in importance, but it accounted for only 6.6 percent of total income
among all families. Among the poor, however, Social Security income represented 22.6 percent of total income, and
among the near poor (100 to 150 percent of poverty), where it peaked, Social Security provided 27.2 percent of total
income. Among families above 400 percent of poverty, however, Social Security contributed only 3.2 percent of
total income, less than contributed by retirement income.
While SSI provided only half a percent of all income, it accounted for 9.8 percent of the total income of poor
families. Among the near poor, however, SSI was only 2.9 percent of total income, and its share declined rapidly as
income rose. Of the remaining 17 major sources collected in CPS ASEC, only one accounted for as much as five
percent of the income within any poverty class. Non-farm self-employment contributed 5.1 percent of the total
income of families above 400 percent of poverty and between 3.5 and 4.4 percent of the income of all other poverty
classes.
Among the poor, five other sources each provided roughly two percent of total income, but it is striking that none
was higher. Each of these sources—unemployment or strike benefits, public assistance or welfare, educational
assistance, child support, and financial assistance from others—declined in importance fairly steeply as income rose.
At the upper end of the income distribution, only interest income (after wage and salary earnings, Social Security,
non-farm self-employment, and retirement income) accounted for as much as two percent of total income.
Alimony contributed no more than 0.13 percent of total income to any poverty class, and other income was nearly as
unimportant, providing 0.3 percent of the income of the poor but between 0.1 and 0.2 percent of the income in every
other poverty class. Other sources that provided less than one percent in every income class were farm selfemployment earnings, worker’s compensation, veterans’ payments, survivor’s income, and disability income.
Retirement, Survivor, and Disability Income: Additional Detail
For Social Security income, the CPS ASEC first establishes the receipt of such income and then establishes the
reason for such receipt—that is, as a retirement, survivor, or disability benefit. For all other post-employment
income, the CPS ASEC first establishes the reason rather than source. Three separate screeners ask if anyone in the
household received income as the result of being retired, disabled, or a survivor, and only those who responded yes
were asked who received such income, what was the source, and how much was received from each source. This
approach does not appear to be as effective or efficient as first establishing the source—such as a former employer
or retirement account. The hard distinction between survivor and retirement income and their separation in the
interview sequence is also somewhat puzzling, since both are associated with retirement, and both (except within
Social Security) may be received by the same person, from the same retirement system. As more and more
employed married women age and become widowed, they are likely to receive survivor’s benefits in conjunction
with their own retirement income. Even more importantly, the wording of the current questions on retirement and
survivors payments is most suitable to identifying income from older defined benefit (pension) retirement systems,
and the CPS ASEC appears not to capture what may be hundreds of billions of dollars in defined contribution and
IRA payments.
Previously we examined receipt by source at the family level (more specifically, for families and unrelated
individuals). Here we are more interested in the patterns of receipt at the person level so that we can determine how
many individuals are receiving income from more than one source.
For individuals identified as receiving retirement income other than Social Security, Table 3 gives the percentage
with each of seven specific sources plus an “other” category. Respondents could list up to two sources for each adult
in the household, so the percentages sum to more than 100 percent. The percentage of persons reporting only one
versus two sources is shown at the bottom of the table. Table 4 shows the percentage distribution of retirement
income by source—again, for persons by family poverty class. Because total income is the denominator, these
percentages sum to exactly 100 percent. The percentage of total dollars reported by persons giving only one versus
two sources is given at the bottom of the table.
Company or union pensions were the dominant source of retirement income at every poverty level, accounting for
58.0 percent of recipients and 43.7 percent of total income. State or local government pensions were next in
frequency, representing 23.0 percent of recipients and 29.4 percent of total income. What may be more notable is
that 97.3 percent of recipients reported only a single source of retirement income, and the income reported by these
recipients was 95.0 percent of the total retirement income (excluding Social Security).
We observed similar patterns for survivor and disability income, with company or union pensions dominant, state or
local government pensions next in frequency and share of total income, and the vast majority of recipients reporting
only a single source. We also found that two of the sources of survivor income—worker’s compensation and estates
or trusts—were picked up elsewhere. Estates and trusts accounted for nearly 14 percent of the total survivor income.
Worker’s compensation was negligible as a source of survivor’s income but is repeated as a source of disability
income, where it accounted for 7.4 percent of the reported income. Unless these two sources are being doublecounted, it is surprising how often respondents who previously answered no to these sources later responded yes.
Comparative Estimates: CPS ASEC, SIPP, and ACS
A comparison of CPS ASEC, SIPP, and ACS income estimates is informative about the income measures in all
three surveys. We established in previous work that the CPS ASEC falls short of the SIPP for all sources of
retirement income but particularly for the newer forms of such income -- payments from defined contribution
(401(k), 403(b), etc.) retirement plans and from any type of IRA (traditional, Roth, SEP or SIMPLE) retirement
account . We have since found that the ACS produces considerably less pension and retirement account income than
SIPP as well—but somewhat more than the CPS ASEC (Czajka and Denmead 2011). Earlier, Czajka and Denmead
(2008) found evidence that aggregate CPS ASEC earnings were smaller than both SIPP and the ACS among lowincome families in 2002, but an update of those findings is needed. Clearly, none of the three surveys is superior to
the others across the board. A comparison of the three surveys on as many sources as possible will identify the
relative strengths and weaknesses of all three surveys.
Our analysis uses the 2009 ACS, which sampled households in all 12 months of the calendar year. Respondents
were asked to report their income for the previous 12 months (that is, January 2008 through December 2008 for
households sampled in January 2009 and progressing to December 2008 through November 2009 for households
sampled in December 2009. Thus the reported incomes span 23 months (January 2008 through November 2009)
centered on December 2008. The Census Bureau applied an inflation adjustment to convert reported incomes from
this rolling reference period to calendar year 2009. For the public use file the Census Bureau provides unadjusted
data and an average adjustment factor. Users can approximate but not exactly replicate the Census Bureau’s
adjustment. Poverty ratios reported on the public use file incorporate the monthly adjustment factors. Because the
Consumer Price Index (CPI) trended downward over part of this period, the average adjustment factor was slightly
less than 1.0, which means that reported incomes were deflated. 1
The ACS collects income for eight sources. To compare estimates of income across the three surveys, we converted
the more detailed CPS ASEC and SIPP income sources to the ACS sources by combining sources to the extent
possible. In a few cases, a CPS ASEC or SIPP income category combined sources that were in different ACS
categories, so it was not possible to align the survey sources perfectly. Sources that were captured in the CPS ASEC
and SIPP but not the ACS were assigned to a separate category. Sources exclusive to the CPS ASEC or SIPP were
excluded from the comparative tables.
Income by ACS Sources
Table 5 compares the ACS, SIPP, and the CPS ASEC with respect to the percent of families and unrelated
individuals receiving each of the eight ACS income sources plus a ninth source that applies only to the latter two
surveys. The fraction of families reporting wage and salary earnings was highest in the ACS—2.8 percent higher
than the CPS ASEC, which was marginally higher than the SIPP. Self-employment was much higher in the ACS
than the CPS ASEC, but was highest in SIPP, some 61.8 percent higher than in the CPS and nearly 40 percent
higher than in the ACS. For every other source but the aforementioned wages and salaries, the SIPP had a
substantially higher percentage of families reporting receipt than either of the other surveys, and for the sources
captured only in the SIPP and CPS ASEC, SIPP’s recipiency rate was three times that of the CPS ASEC. Looking in
turn at the ACS, ACS found more families than CPS ASEC with wages and salaries, with self-employment, with
pensions and with public assistance or welfare. ACS and SIPP had nearly identical fractions reporting public
assistance or welfare, over a million families more than in CPS ASEC. However, ACS found fewer recipients than
the CPS ASEC for Social Security, SSI, property income (interest, dividends, rent, royalties, and income from
estates or trusts) and for all other income sources. The biggest difference was in property income, where fewer than
half as many families reported receipt in the ACS as in the CPS ASEC and fewer than a third as many as in the
SIPP.
Aggregate income tells a different story than recipiency, however. Even without the small amounts captured
for sources not present in the ACS, the survey captured marginally more total income than the CPS ASEC, while
SIPP captured 8.2 percent less (Table 6). Wage and salary earnings in the ACS were slightly lower than in the CPS
ASEC, but SIPP was 15 percent lower. Self-employment earnings were 18 percent higher in the ACS than the CPS
ASEC but 79 percent higher in the SIPP. The results for interest and related sources are a complete reverse of what
we saw for recipiency. Here the ACS was 16 percent higher than the CPS ASEC despite fewer than half as many
recipients while SIPP was much lower (only 37 percent of the CPS ASEC amount) despite finding 44 percent more
recipients. For the remaining sources, the ACS was sometimes higher and sometimes lower than the CPS ASEC,
while SIPP was generally higher. Surprisingly, the ACS obtained 85 percent more welfare income than the CPS
1
CPS ASEC aggregate income was lower in calendar year 2009 than 2008, but this was due to the recession.
The CPS ASEC has no comparable adjustment up or down for inflation, and there were no negative COLA
adjustments to e.g., Social Security or other retirement payments.
ASEC while the SIPP obtained 56 percent more than the CPS ASEC, suggesting there must be an explanation other
than frequent interviews for the two surveys’ comparative success in capturing welfare income.
Looking across poverty classes, we see that the ACS found more income among the poor than the CPS ASEC,
attributable to higher amounts of earnings (both wage and salary and self-employment), welfare, and pensions
(Table 7). The ACS also found slightly more income above 400 percent of poverty, attributable to larger amounts of
each income source except wage and salary earnings and Social Security. However, we also see evidence of a
problem with the ACS data in that the additional welfare income relative to the CPS ASEC grew rapidly as income
rose. Above 400 percent of poverty, the ACS found 11 times as much welfare income as the CPS ASEC. We would
have guessed that this growing difference was due to imputation—specifically, that the Census Bureau imputed
welfare benefits without conditioning adequately on other reported income. We see the same phenomenon for SIPP
in comparison to the CPS ASEC except that the high-income welfare recipients are not nearly as prominent as they
are in the ACS. Additional evidence does not support this interpretation, however. The allocation rate for welfare
income among families above 400 percent of poverty was only 14 percent, which is lower than in any other income
class. In the CPS ASEC, 59 percent of the welfare income among families above 400 percent of poverty was
allocated, but the total welfare income among these families was still very small. The high welfare income among
higher income families in the ACS seems to be a reporting problem rather than an allocation problem, but there is
nothing in the description of this item in the ACS questionnaire that would seem to invite misreports by higher
income respondents. 2 Perhaps there is another explanation either in editing or processing, which only the Census
Bureau can determine.
Looking across poverty classes at SIPP, we see that SIPP found less total income than the CPS ASEC in every
poverty class, with the lowest ratio (89.1 percent) among families above 400 percent of poverty but the highest (99.5
percent) among families between 250 and 400 percent of poverty (Table 8). For the poor, SIPP found less (92.4
percent) total income than the CPS ASEC, and although wage and salary earnings were about the same, there were
virtually no self-employment earnings despite the 79 percent higher self-employment income overall. We have no
explanation for this finding, which merits further investigation. SIPP pension income among the poor was 27
percent lower than the CPS ASEC, again despite the 39 percent higher overall pension income. Similarly, Social
Security income was 23 percent lower among the poor in the SIPP than in the CPS ASEC despite SIPP’s marginally
higher total Social Security income. Besides wage and salary earnings, the only sources for which SIPP was higher
than the CPS ASEC among the poor were interest, dividends and related income and what the ACS classifies as
other sources. At the upper end of the income distribution, SIPP captured only 82 percent as much wage and salary
earnings as the CPS ASEC. For this poverty class, SIPP also had lower amounts than the CPS ASEC for interest,
dividends and related income and for the sources defined as other in the ACS. Topcoding could play a role, as the
CPS top codes for earnings are designed to reproduce the non top-coded total earnings whereas SIPP’s top codes
may not be designed this way. Yet there are four sources for which SIPP had higher amounts than CPS ASEC in the
high income category: self-employment earnings, welfare, pensions, and the sources not mentioned in the ACS but
reported in the CPS ASEC and SIPP.
One other pattern of note concerns Social Security and SSI. Overall, SIPP captured 35 percent more SSI than the
CPS ASEC and about the same amount of Social Security. Between 100 and 250 percent of poverty, SIPP captured
less Social Security benefits than the CPS ASEC but 54 to 95 percent more SSI benefits than the CPS ASEC. A
possible explanation for these reverse results is that some respondents misreported their Social Security income as
SSI. We note, however, that Social Security is listed ahead of SSI on the questionnaire, which would seem to make
this type of confusion unlikely. Furthermore, earlier research with SIPP records linked to Social Security records
shows a tendency only in the reverse direction—that is, SSI recipients reporting their benefits as Social Security
(Koenig 2003).
2
The item reads “Any public assistance or welfare payments from the state or local welfare office.”
Recommendations
The CPS ASEC instrument devotes nearly equal attention to each of 20-some sources of income, seeking an annual
total for each source. The sources vary widely in how much they contribute to total income, however. The current
approach gives disproportionate attention to sources that contribute little to total income. As we demonstrated, a
handful of sources account for around 95 percent of the aggregate income reported in each of the six relative income
classes that we used in this study. The ASEC instrument could be refocused, expanding and strengthening data
collection for the few important income sources and combining and simplifying questions on the remaining income
sources for an overall improvement in data with a shorter interview.
We believe that the overall importance of earnings (wages and salaries and self-employment) as the primary source
of income at all income levels demands that the collection of earnings be as strong as the Census Bureau can make
it. By asking detailed information for only one work activity (the longest duration) during the year, whether it is a
job, self-employment or a business, and lumping together without detail all other sources of earned income, the
instrument seems likely to fall short of full effectiveness. It is not uncommon for people to change from one fulltime job to another during the year, to change from full-time to part-time status or the reverse, to work more than
one job at the same time, to start a small business (self-employment) while still working for someone else, to have
multiple businesses sequentially or simultaneously, and for lower-income workers or certain sectors such as
construction, to have erratic employment. The instrument could be substantially strengthened by collecting data
comparable to that currently collected for the longest main job for at least three or four work activities, before going
to the summary "all other sources of earnings", and ordering these from current to earliest. If additional information
on the longest job (rather than the most current, if it extended back into the calendar year) is desired, the longest job
held during the year could be established after first identifying all or at least the major jobs. In addition, in asking
about earnings, the questions should make clear that the income being requested is prior to all deductions, including
not only taxes but contributions to retirement accounts, health insurance premiums and flexible spending accounts,
dependent care, and transportation benefits.
In the area of self-employment, it is important to distinguish between the salary that a business owner draws and the
profit received due to ownership. A business owner may have paid himself a good salary from a business that lost
money during the year. If the salary is not captured, the owner’s actual income may be substantially understated. It
is also important to recognize the existence of self-employment income that is earned without owning a business,
and to expand the definition of self-employment in the CPS ASEC beyond business ownership, to include
compensation from someone other than one’s employer. The term “consulting” is not mentioned anywhere in the
income questions of the three surveys, but perhaps it should be, as consulting provides a source of income to
professionals who are retired or between jobs, as well as to supplement full-time employment. Periodic consulting
does not require a business, and the term is one that respondents would recognize.
All three surveys would benefit from revisions to their income questions to reflect recent shifts in sources of
retirement income, especially the increasing importance of IRAs as a source of regular distributions, even if such
distributions occur as infrequently as once a year. One suggestion applicable to all three surveys is to use the terms
distribution, withdrawal, and payment in referring to the income taken from retirement accounts generally, to restrict
the term pension income to monthly payments from defined benefit retirement systems, and to drop the regular
versus lump sum distinction and restrict excluded lump sums to unique or one-time payments. The CPS ASEC and
SIPP could consider the approach used in the Survey of Consumer Finances, in which respondents are asked to
identify up to six different retirement plans or accounts and then answer questions on each one. As retirement
accounts multiply, such an approach may become increasingly necessary as a way of ensuring fuller reporting of
income from these sources.
For the CPS ASEC we recommend combining and simplifying the separate questions on retirement, survivor, and
disability benefits from defined benefit pension systems, adding separate questions on distributions for current
income from defined contribution and IRA retirement accounts, and combining all other non-pension disability
income including worker’s compensation under a separate question.
Currently, the CPS ASEC asks separately about retirement, survivor, and disability benefits, and for each one asks
for many of the same sources—including six types of employers that provide defined benefit pensions. This
contrasts with the approach used for Social Security and veterans benefits, where the respondent is first asked if
anyone in the household received any Social Security or veterans’ payments and later asks for the reason, which for
Social Security includes retirement, disability, or any of several specific types of dependent or survivor benefits. We
recommend that the approach used for Social Security be used for other types of employment-related retirement,
survivor, and disability pensions. That is, the questionnaire should first establish whether anyone in the household
received one or more defined benefit pensions from a previous employer of the respondent, spouse or other relative,
then for each determine which of the six types of employer (company or union, federal, state, local, U.S. military, or
Railroad Retirement) is the source, the amount, and the reason (retirement, disability, survivor or dependents
benefit). Separate questions should be asked on payments (other than rollovers) from defined contribution and IRA
retirement accounts, including type of account, and the amount withdrawn during the previous year. Specific types
should at least include 401(k)s, 403(b)s, Roths (now possible within private sector retirement plans), and SEP,
SIMPLE and traditional IRAs. Keoghs should not be listed, since they are very infrequent. This question should
include annuities purchased or provided from defined contribution retirement plans. The other items in the current
retirement/survivor/disability sequence are annuity pay-outs from life insurance, payments from estates or trusts,
worker’s compensation, Black Lung benefits, accident or disability insurance, and state temporary sickness benefits.
Worker’s compensation is currently collected elsewhere in the questionnaire but the very small other disabilityrelated income sources, including the minuscule amounts of their survivors benefits, could be folded into that
question as a single "other" source. Of the remaining sources, the only ones of any size are annuities and paid-up life
insurance, and income from estates or trusts. Estimates from the 2007 SCF indicate that two-thirds of income from
annuities and paid-up life insurance will be accounted for by purchases from retirement plans. The balance of this
item, and payments from estates and trusts could be grouped with other property income, which already includes
estate and trust income not captured in the survivor question.
In conjunction with our recommendations regarding the collection of income from retirement accounts we note that
the CPS ASEC asks respondents to include IRAs when reporting interest and does not explicitly ask respondents to
exclude stocks and mutual funds held in retirement accounts when reporting dividends. This implies a potential
double-count of interest and dividends held in retirement accounts since they are also included in current or
subsequent distributions. Additionally, we recommend updating the possible sources of interest to include widely
held instruments such as bond funds, tax-exempt municipal bond funds and treasuries, in place of savings bonds or
checking accounts.
The dividend question on stocks or mutual funds does not mention capital gain distributions that are paid
automatically by mutual funds for tax reasons. These annual (usually year-end) distributions may be as large or
larger than the annual dividend payments. Absent more detailed instructions, respondents may interpret dividends to
include or exclude capital gain distributions, or may not think of them at all. We recommend that the Census Bureau
revise the wording of the question on dividends to clarify whether capital gain distributions are to be reported. 3
Questions on rent ask the gross amount and the net amount after expenses, but the instrument itself does not clarify
what should be counted as expenses, nor whether net rent should be assessed on a cash flow basis, or according to
the rules, including use of depreciation, that apply on the tax return. Absent more detailed instructions from the
interviewer, respondents may interpret expenses differently or simply not think of all the components that should be
included. We recommend that the Census Bureau provide clearer instructions on what to count as expenses so that
net rent is measured more consistently across households. In a related area, we recommend that possible sources of
property income be expanded to specifically mention not only residential and business rental properties, but other
business investments, to improve the capture of income other than dividends from investment in or ownership of
businesses.
TANF and public assistance generally are among the most poorly reported sources of income in the CPS ASEC, but
our analysis shows that they are one of three sources with more than half of the recipient families reporting at most
one other source of income. The question sequences for both Social Security and for SSI have separate sections on
3
Census Bureau descriptions of CPS money income state explicitly that capital gains are not included (see, for
example, Ruser et al., 2004), but this is not communicated to respondents. Furthermore, pension payments and
withdrawals from IRAs and 401(k)s implicitly include capital gains as well as earnings prior to distribution.
benefits received on behalf of children, asked after the questions for each household member. The questions in these
sections go back to identify any children not already mentioned, on whose behalf benefits are received. Only six
percent of Social Security recipients are under 18, and only 16 percent of SSI recipients. In contrast, about threequarters of the recipients of TANF are children, and the fraction of families receiving TANF in which no adults are
recipients (child-only cases) has ranged from 42 to 46 percent over the last five years. We recommend that the
Census Bureau add a section, modeled after the Social Security and SSI child sections, specifically on TANF for
children, to be asked after the public assistance question. Additionally, recent match studies between the CPS ASEC
and administrative records from Medicaid and the Supplemental Nutrition Assistance Program (formerly the Food
Stamp Program) may provide helpful information on the characteristics of non-reporters, and this information may
be useful in redesigning the CPS ASEC questions.
The CPS ASEC asks separate questions to collect unemployment compensation, supplemental benefits, and strike
benefits. These questions could be combined into a single question, followed by a list of possible benefit types if a
more precise attribution is desired.
Child support, alimony, and financial assistance from others form a set of related sources that could be combined or
at least approached through a common screener. SIPP estimates presented in this paper indicate that the aggregate
income from child support is three times the aggregate income from the other two sources combined. Little
information would be lost by creating a single source, in other words.
Lastly, we are puzzled by the inclusion of educational assistance in CPS money income. Families will not have such
income unless they have educational expenses, and the latter will generally exceed the amount of assistance, so the
net impact of the assistance is at best a wash. More importantly, the poverty threshold does not include the
educational expenses that the assistance is intended to help offset, so the effect of the assistance on measured
poverty is to raise relative income when in fact the receipt of educational assistance implies that the family has fewer
resources to devote to necessities. We recommend that educational assistance be removed from CPS money income
although there may be other, good reasons to continue to collect this variable.
References
Coder, John, and Lydia Scoon-Rogers. Evaluating the Quality of Income Data Collected in the Annual Supplement
to the March Current Population Survey and the Survey of Income and Program Participation. Housing and
Household Economic Statistics Division, U.S. Census Bureau, July 1996, www.sipp.census.gov/sipp/
workpapr/wp215.pdf.
Czajka, John L., and Gabrielle Denmead. “What We’re Missing: How to Obtain Better Estimates of Retirement
Income in Our Major Household Surveys.” Paper presented at the Association for Public Policy Analysis and
Management Fall Research Conference, November 3-5, 2011, Washington, DC.
Czajka, John L., and Gabrielle Denmead. “Income Data for Policy Analysis: A Comparative Assessment of Eight
Surveys.” Final report. Washington, DC: Mathematica Policy Research, December 2008.
Fisher, T. Lynn. “Measuring the Relative Importance of Social Security Benefits to the Elderly.” Social Security
Bulletin, vol. 67, no. 2, 2007.
Koenig, Melissa L. “An Assessment of the Current Population Survey and the Survey of Income and Program
Participation Data Using Social Security Administrative Data.” Federal Committee on Statistical Methodology,
2003 Research Conference papers, pp. 129-137.
Ruser, John, Adrienne Pilot, and Charles Nelson. Alternative Measures of Household Income: BEA Personal
Income, CPS Money Income, and Beyond. Presented to the Federal Economic Statistics Advisory Committee,
December 14, 2004.
Vaughan, Denton R. “Reflections on the Income Estimates from the Initial Panel of the Survey of Income and
Program Participation (SIPP).” Studies in Income Distribution No. 17. Washington, DC: Social Security
Administration, May 1993.
Acknowledgment
The work presented herein draws on research being performed under contract to the Office of the Assistant
Secretary for Planning and Evaluation, Department of Health and Human Services, and the U.S. Census Bureau. The
authors are grateful for their support.
Table 1. Percent of Families and Unrelated Individuals Receiving Each Source of Income in 2008,
by Family Income as a Percent of Poverty: 2009 CPS ASEC
Family Income as Percent of Poverty
Income Source
Wage and salary earnings
Non-farm self-employment earnings
Farm self-employment earnings
Unemployment or strike benefits
Worker's compensation
Social Security
Supplemental Security Income
Public assistance or welfare
Veterans' payments
Survivor's incomea
Disability incomea
Retirement incomea
Interest
Dividends from stocks or mutual funds
Net rent, royalties, estates, or trusts
Educational assistance
Child support
Alimony income
Financial assistance from others
Other incomeb
a
b
< 100%
100% to
< 150%
150% to
< 200%
200% to
< 250%
250% to
< 400%
400% +
Total
41.53
6.05
0.59
3.97
0.38
21.01
11.42
5.60
0.65
0.69
0.67
1.70
14.07
2.60
1.50
4.62
4.96
0.26
3.82
0.99
57.35
6.58
0.78
5.58
0.69
40.84
6.83
2.19
1.50
1.88
1.07
5.57
24.29
4.12
2.40
5.21
4.72
0.24
2.89
0.95
66.75
6.72
0.79
6.21
1.03
35.65
4.25
1.23
1.48
2.60
1.53
10.43
32.78
6.39
3.04
5.56
4.72
0.23
2.00
0.89
75.66
7.43
1.12
7.35
0.89
29.25
2.70
0.56
2.14
2.45
1.30
13.74
37.86
8.18
3.40
5.59
4.46
0.41
1.85
0.86
82.69
8.19
1.50
7.23
1.19
25.30
1.92
0.33
2.61
2.67
1.18
15.56
50.65
13.06
5.25
5.41
3.83
0.44
1.17
0.99
90.52
11.43
2.04
5.10
1.03
18.39
0.90
0.05
2.53
2.27
1.09
15.13
72.09
29.67
11.15
5.00
2.22
0.33
0.65
1.08
74.76
8.66
1.38
5.74
0.92
25.13
3.73
1.29
2.03
2.14
1.11
11.73
47.40
15.45
6.13
5.16
3.66
0.33
1.68
1.00
Separate amounts from as many as two sources are reported in the public use file.
Additional income, which may be from one of the preceding sources, identified in a recipiency flag.
Table 2. Percentage Distribution of Total Income by Source in 2008,
by Family Income as a Percent of Poverty: 2009 CPS ASEC
Family Income as Percent of Poverty
Income Source
< 100%
100% to
< 150%
150% to
< 200%
200% to
< 250%
250% to
< 400%
400% +
Total
Total Income
100.00
100.00
100.00
100.00
100.00
100.00
100.00
48.70
4.14
0.09
1.90
0.30
22.55
9.83
2.38
0.39
0.41
0.51
0.99
0.98
0.34
0.08
2.16
2.25
0.13
1.56
0.31
55.42
4.43
0.25
1.33
0.25
27.19
2.89
0.38
0.64
0.46
0.49
1.49
0.75
0.22
0.17
1.44
1.13
0.09
0.80
0.17
63.12
3.69
0.20
1.05
0.32
21.55
1.41
0.18
0.44
0.48
0.54
2.78
1.04
0.30
0.23
1.16
0.80
0.09
0.47
0.13
69.91
3.65
0.18
1.19
0.23
14.54
0.79
0.03
0.65
0.51
0.37
3.87
1.12
0.34
0.25
0.91
0.69
0.10
0.55
0.10
75.68
3.48
0.32
0.68
0.21
8.98
0.33
0.03
0.65
0.58
0.29
4.76
1.37
0.56
0.40
0.78
0.42
0.09
0.28
0.10
79.96
5.12
0.53
0.27
0.10
3.16
0.07
0.00
0.32
0.41
0.18
3.86
2.70
1.43
1.12
0.38
0.14
0.06
0.09
0.10
76.72
4.66
0.45
0.48
0.14
6.58
0.46
0.07
0.41
0.45
0.24
3.85
2.23
1.12
0.87
0.57
0.31
0.07
0.21
0.11
Wage and salary earnings
Non-farm self-employment earnings
Farm self-employment earnings
Unemployment or strike benefits
Worker's compensation
Social Security
Supplemental Security Income
Public assistance or welfare
Veterans' payments
Survivor's incomea
Disability incomea
Retirement incomea
Interest
Dividends from stocks or mutual funds
Net rent, royalties, estates, or trusts
Educational assistance
Child support
Alimony income
Financial assistance from others
Other incomeb
a
b
Separate amounts from as many as two sources are reported in the public use file.
Additional income, which may be from one of the preceding sources, identified in a recipiency flag.
Table 3. Percentage Distribution of Persons 15 and Older with Retirement Income Other Than
Social Security in 2008 by Source, by Family Income as a Percent of Poverty: 2009 CPS ASEC
Family Income as Percent of Poverty
< 100%
100% to
< 150%
150% to
< 200%
200% to
< 250%
250% to
< 400%
400% +
Total
Company or union pension
Federal government retirement
US military retirement
State or local government retirement
US Railroad Retirement
Regular payments from annuities or
paid-up insurance policies
Regular payments from IRA, Keogh,
or 401(k) accounts
Other or don't know
62.37
4.00
0.64
18.56
0.73
64.51
6.98
1.40
16.13
2.05
69.18
5.56
1.81
17.78
2.44
70.22
4.85
2.50
16.43
1.41
63.00
7.74
4.17
19.45
1.32
49.82
9.51
9.66
28.08
0.57
57.98
8.02
6.25
22.97
1.08
2.30
1.50
1.13
1.52
0.89
0.86
1.01
5.12
7.52
3.17
4.34
1.55
1.84
2.86
1.99
3.58
1.53
3.42
2.10
3.28
2.11
One source only
Two sources
98.79
1.24
99.93
0.07
98.72
1.28
98.21
1.79
98.31
1.69
96.00
4.00
97.31
2.69
Source
Table 4. Percentage Distribution of Retirement Income Other Than Social Security in 2008 by Source,
by Family Income as a Percent of Poverty: 2009 CPS ASEC
Family Income as Percent of Poverty
< 100%
100% to
< 150%
150% to
< 200%
200% to
< 250%
250% to
< 400%
400% +
Total
Company or union pension
Federal government retirement
US military retirement
State or local government retirement
US Railroad Retirement
Regular payments from annuities or
paid-up insurance policies
Regular payments from IRA, Keogh,
or 401(k) accounts
Other or don't know
55.55
3.92
1.48
20.07
0.85
52.00
9.83
4.19
20.30
3.71
56.02
9.41
3.11
24.54
4.38
57.32
9.74
3.79
22.04
2.48
53.02
12.16
5.03
23.70
1.90
38.98
13.29
10.03
32.19
0.48
43.69
12.67
8.32
29.42
1.05
2.71
1.81
0.44
1.24
0.52
0.56
0.61
8.79
6.63
2.92
5.24
0.87
1.24
1.78
1.61
2.72
0.95
2.46
2.00
2.47
1.79
One source only
Two sources
97.85
2.15
99.90
0.10
97.37
2.63
98.10
1.90
97.79
2.21
93.69
6.31
95.00
5.00
Source
Table 5. Comparison of ACS, SIPP, and CPS ASEC with Respect to the Percentage of Families
and Unrelated Individuals Receiving Each Source of Income in 2009
Income Source
Wage and salary earnings
Self-employment earnings
Interest, dividends, net rental income, royalty
income, or income from estates and trustsa
Social Security or Railroad Retirement
Supplemental Security Income
Public assistance or welfare
Retirement, survivor, or disability pensionsb
Any other sources of income received regularlyc
Other sources of income not explicitly mentioned
in the ACS and received regularly, as reported in
CPS and SIPPd
a
Percent of Families with
Each Source by Survey
CPS
ACS
SIPP
ASEC
Ratio of
ACS to
CPS
ASEC
Ratio of
SIPP to
CPS
ASEC
75.22
10.24
72.72
14.19
73.19
8.77
1.028
1.168
0.994
1.618
21.06
24.84
3.27
2.40
15.42
14.08
67.83
29.41
6.07
2.41
19.93
19.02
47.23
25.39
3.67
1.36
13.60
15.75
0.446
0.978
0.893
1.770
1.134
0.894
1.436
1.158
1.657
1.779
1.466
1.208
0.00
6.36
2.04
0.000
3.116
SIPP income from estates and trusts is included with pension income below.
The ACS instructions mention benefits from companies and unions, federal, state, and local governments,
and the U.S. military. They also ask the respondent to include regular income from annuities and IRA or
Keogh retirement plans but do not mention 401(k) plans. Both the CPS ASEC and SIPP include 401(k) plans.
c
Other sources named in the ACS question are Veterans' payments, unemployment compensation, child
support, and alimony. The instructions add all other regular payments such as Armed Forces transfer payments,
assistance from private charities, and regular contributions from persons not living in the household.
d
Other sources collected in both the CPS and SIPP but not named specifically in the ACS include income
from paid-up life insurance (but combined with annuities so we include this source with pensions), workers'
compensation, Black Lung benefits, and sources identified as other income. It is possible that ACS
respondents might include Black Lung benefits among retirement, survivor, or disability pensions.
b
Table 6. Comparison of ACS, SIPP, and CPS ASEC with Respect to Aggregate Dollars
of Income in 2009 for Comparable Sources
Millions of Dollars of
Income by Survey
ACS
SIPP
CPS
ASEC
Ratio of
ACS to
CPS
ASEC
Total Income (sum of sources)
8,058,115
7,365,466
8,021,524
1.005
0.918
Wage and salary earnings
Self-employment earnings
Interest, dividends, net rental income, royalty
income, or income from estates and trustsa
Social Security or Railroad Retirement
Supplemental Security Income
Public assistance or welfare
Retirement, survivor, or disability pensionsb
Any other sources of income received regularlyc
Other sources of income not explicitly mentioned
in the ACS and received regularly, as reported in
CPS and SIPPd
6,081,541
429,367
5,219,581
652,802
6,130,675
364,255
0.992
1.179
0.851
1.792
397,816
497,359
35,444
11,276
428,229
177,085
126,015
585,394
52,717
9,506
491,540
167,951
341,983
581,105
39,168
6,085
353,175
183,734
1.163
0.856
0.905
1.853
1.213
0.964
0.368
1.007
1.346
1.562
1.392
0.914
0.00
59,961
21,344
0.000
2.809
Income Source
a
SIPP income from estates and trusts is included with pension income below.
The ACS instructions mention benefits from companies and unions, federal, state, and local governments,
and the U.S. military. They also ask the respondent to include regular income from annuities and IRA or
Keogh retirement plans but do not mention 401(k) plans. Both the CPS ASEC and SIPP include 401(k) plans.
c
Other sources named in the ACS question are Veterans' payments, unemployment compensation, child
support, and alimony. The instructions add all other regular payments such as Armed Forces transfer payments,
assistance from private charities, and regular contributions from persons not living in the household.
d
Other sources collected in both the CPS and SIPP but not named specifically in the ACS include income
from paid-up life insurance (but combined with annuities so we include this source with pensions), workers'
compensation, Black Lung benefits, and sources identified as other income. It is possible that ACS
respondents might include Black Lung benefits among retirement, survivor, or disability pensions.
b
Ratio of
SIPP to
CPS
ASEC
Table 7. Ratios of ACS to CPS ASEC Aggregate Dollars by Source of Income in 2009, Mapped to ACS Sources,
by Family Income as a Percent of Poverty
Family Income as Percent of Poverty
< 100%
100% to
< 150%
150% to
< 200%
200% to
< 250%
250% to
< 400%
Total Income
1.037
0.981
0.955
0.936
Wage and salary earnings
Self-employment earnings
Interest, dividends, net rental income, royalty
income, or income from estates and trustsa
Social Security or Railroad Retirement
Supplemental Security Income
Public assistance or welfare
Retirement, survivor, or disability pensionsb
Other sources of income explicitly mentioned in
the ACS and received regularlyc
Other sources of income not explicitly mentioned
in the ACS and received regularly, as reported in
CPS and SIPPd
1.117
1.201
1.076
1.052
1.055
1.000
0.775
0.914
0.813
1.266
1.632
0.897
0.832
0.761
1.675
1.489
0.984
0.000
Income Source
a
400% +
Total
1.005
1.013
1.005
0.973
1.203
1.019
1.157
0.979
1.196
0.992
1.179
0.804
0.734
0.860
1.775
1.218
0.972
0.757
0.867
2.145
1.130
0.958
0.874
1.022
3.605
1.094
1.201
0.921
1.471
11.085
1.247
1.163
0.856
0.905
1.853
1.213
0.741
0.718
0.735
0.878
1.196
0.964
0.000
0.000
0.000
0.000
0.000
0.000
SIPP income from estates and trusts is included with pension income below.
The ACS instructions mention benefits from companies and unions, federal, state, and local governments, and the U.S. military.
They also ask the respondent to include regular income from annuities and IRA or Keogh retirement plans but do not mention
401(k) plans. Both the CPS ASEC and SIPP include 401(k) plans.
c
Other sources named in the ACS question are Veterans' payments, unemployment compensation, child support, and alimony.
The instructions add all other regular payments such as Armed Forces transfer payments, assistance from private charities, and
regular contributions from persons not living in the household.
d
Other sources collected in both the CPS and SIPP but not named specifically in the ACS include income from paid-up life
insurance (but combined with annuities so we include this source with pensions), worker's compensation, Black Lung benefits,
and sources identified as other income. It is possible that some ACS respondents might include Black Lung benefits among
retirement, survivor, or disability pensions.
b
Table 8. Ratios of SIPP to CPS ASEC Aggregate Dollars by Source of Income in 2009, Mapped to ACS Sources,
by Family Income as a Percent of Poverty
Family Income as Percent of Poverty
< 100%
100% to
< 150%
150% to
< 200%
200% to
< 250%
250% to
< 400%
400% +
Total
Total Income
0.924
0.978
0.988
0.925
0.995
0.891
0.918
Wage and salary earnings
Self-employment earnings
Interest, dividends, net rental income, royalty
income, or income from estates and trustsa
Social Security or Railroad Retirement
Supplemental Security Income
Public assistance or welfare
Retirement, survivor, or disability pensionsb
Other sources of income explicitly mentioned in
the ACS and received regularlyc
Other sources of income not explicitly mentioned
in the ACS and received regularly, as reported in
CPS and SIPPd
1.007
0.008
0.954
0.974
0.986
1.274
0.887
1.324
0.928
1.623
0.819
1.956
0.851
1.792
0.701
0.772
0.993
1.386
0.733
0.644
0.905
1.676
1.446
1.396
0.450
0.857
1.952
2.033
1.231
0.540
0.946
1.537
1.656
1.164
0.577
1.118
1.326
2.077
1.339
0.336
1.075
1.070
2.781
1.441
0.368
1.007
1.346
1.562
1.392
1.094
0.962
1.028
0.835
0.986
0.812
0.914
1.387
1.501
1.393
1.539
1.927
4.070
2.809
Income Source
a
SIPP income from estates and trusts is included with pension income below.
The ACS instructions mention benefits from companies and unions, federal, state, and local governments, and the U.S. military.
They also ask the respondent to include regular income from annuities and IRA or Keogh retirement plans but do not mention
401(k) plans. Both the CPS ASEC and SIPP include 401(k) plans.
c
Other sources named in the ACS question are Veterans' payments, unemployment compensation, child support, and alimony.
The instructions add all other regular payments such as Armed Forces transfer payments, assistance from private charities, and
regular contributions from persons not living in the household.
d
Other sources collected in both the CPS and SIPP but not named specifically in the ACS include income from paid-up life
insurance (but combined with annuities so we include this source with pensions), worker's compensation, Black Lung benefits,
and sources identified as other income. It is possible that some ACS respondents might include Black Lung benefits among
retirement, survivor, or disability pensions.
b