What Distributional Impacts Mean: Welfare Reform
Experiments and Competing Margins of Adjustment
Patrick Kline
Melissa Tartari
UC Berkeley/NBER
Yale
February, 2013
Abstract
We study the impact of the Connecticut Jobs First (JF) experiment on the labor
supply and program participation decisions of a sample of welfare applicants and recipients. A rich optimizing model is developed incorporating under-reporting decisions,
labor supply constraints and heterogeneity in welfare stigma, hassle e¤ects, and …xed
costs of work. Qualitatively, our model rationalizes the large empirical impacts of the
JF experiment on the distribution of administrative earnings despite the absence of
bunching at a program induced notch in agents’budget sets. We show that the model
places nonparametric restrictions on experimental impacts that can be used to develop
bounds on the magnitude of a variety of allowable intensive and extensive margin responses to reform. In addition to incentivizing work at the extensive margin, we …nd
that the JF experiment induced a substantial welfare “opt-in”response among women
with relatively high earnings potential.
We thanks Andres Santos and seminar participants at Bates-White, Yale, Queens University, and UC
Berkeley for useful comments. This research was supported by NSF Grant #0962352.
1
A large empirical literature studies behavioral responses to transfer policies (Mo¢ tt,
2002 provides a review). Traditional neoclassical models of labor supply predict that the
price distortions induced by such policies will yield complex intensive margin labor supply
responses, with some households working more and some less, often with discrete bunching
of earnings at particular levels (Hausman, 1985; Mo¢ tt, 1990). However, many studies
…nd that key qualitative predictions of these frictionless models fail empirically. In both
survey and administrative data, earnings tend not to exhibit much bunching at kinks in
agents’budget sets (Heckman, 1983; Saez, 2010). Moreover, several studies exploiting policy
variation fail to …nd evidence of important intensive margin responses (Ashenfelter, 1983;
Eissa and Liebman, 1996; Eissa and Hoynes, 2006; Meyer and Rosenbaum, 2001; Meyer,
2002). These facts have led many to conclude that behavioral responses on the intensive
margin are often sharply constrained (e.g., as in Chetty et al., 2011) and that adjustment to
policy reforms is likely to occur primarily on the extensive margin.1
In an important contribution, Bitler, Gelbach, and Hoynes (BGH, 2006) provide nonparametric evidence that intensive margin responses may have played a substantial role in
mediating the e¤ects of welfare reform. They show, using experimental data from Connecticut’s Jobs First (JF) program, that welfare reform induced a nuanced pattern of quantile
treatment e¤ects (QTEs) on earnings consistent with the unconstrained neoclassical labor
supply model. In particular, they …nd that the Jobs First program boosted the middle
quantiles of earnings while lowering the top quantiles, yielding a mean earnings e¤ect near
zero. BGH argue that the negative impacts on upper quantiles provide suggestive evidence
of an “opt-in” response to welfare (Ashenfelter, 1983), whereby some workers who would
have worked without welfare in the absence of reform are induced to lower their earnings
in order to qualify for bene…ts. The …nding of a strong opt-in e¤ect suggests important
intensive margin responsiveness among the population of poor women with children, setting
this study apart from most prior evidence in the literature.
In this paper, we extend BGH’s analysis of the Jobs First experiment to formally identify
the magnitude of intensive and extensive margin adjustments to the program’s various work
incentives. To conclude that adjustment occured along the intensive margin, one must infer
features of the joint distribution of women’s potential earnings under the two policy regimes.
As noted by Heckman, Smith, and Clements (1997), joint distributions are only identi…ed by
QTEs under the “rank invariance”assumption that potential earnings under the two regimes
1
Heckman (1993), for instance, concludes that “elasticities are closer to 0 than 1 for hours-of-work equations (or weeks-of-work equations) estimated for those who are working. A major lesson of the past 20 years
is that the strongest empirical e¤ects of wages and nonlabor income on labor supply are to be found at the
extensive margin.” (emphasis in original)
2
exhibit perfect positive dependence, a condition which BGH acknowledge is likely to be
violated in their sample.2 Our analysis replaces the rank invariance assumption with a set of
joint restrictions on earnings and program participation behavior derived from an optimizing
model where agents face constraints on their labor supply decisions and may choose to underreport their earnings to case workers. We show that these restrictions are refutable with
experimental data and allow the development of informative bounds on the probability of
opting into welfare along with the probability of responding along a number of other margins.
We develop estimators and inferential methods for these response probabilities and …nd that
the Jobs First program induced a large opt-in response along with a substantial extensive
margin response. To our knowledge, these estimates are the …rst to separately identify the
extensive and intensive margin responses to a policy reform without parametric assumptions.
We begin our analysis by developing the standard frictionless model of labor supply and
deriving its key qualitative predictions for the impact of the JF program on earnings and
program participation choices. Empirically, most of these predictions appear to be satis…ed
in the data. We show that the program induced many women to work, raised welfare
participation, and increased the incidence of work while on welfare. We also show that
the program “warped” the distribution of earnings in the theoretically expected manner,
with work becoming more common at low earnings levels and less common at high earnings
levels. Finally, we show that the pattern of distributional impacts varies across subgroups
in a manner consistent with theory.
However, one notable prediction of the model does not hold. The opt-in e¤ects of welfare
reform should be accomplished, in part, via a point mass of women locating at the JF
eligibility notch. We show that there is no evidence of bunching at this notch, despite its
obvious salience. We suggest this is likely due to a combination of constraints on workers’
ability to adjust their earnings levels and under-reporting behavior. Indeed, many women
with ineligible earning levels nonetheless receive bene…ts under JF, suggesting that underreporting of earnings is likely to be common among program participants, a …nding in line
with much of the previous literature (Greenberg, Mo¢ tt, and Friedman, 1981; Greenberg
and Halsey, 1983; Hotz, Mullin, and Scholz, 2003).
We then turn to developing the quantitative implications of an augmented model where
workers may lack …ne control over their earnings but are able to make crude intensive margin
adjustments, to switch between program participation states, and to under-report their earnings. The addition of under-reporting behavior introduces additional margins of adjustment
2
In a related analysis, BGH (2005) …nd strong evidence of rank reversals in the Canadian Self-Su¢ ciency
Project experiment.
3
that have not typically been incorporated into prior models but are potentially quite important for a number of transfer programs. Notably, the addition of under-reporting behavior
yields violations of the rank invariance condition as reform may induce some women to leave
welfare and earn relatively large amounts which are then under-reported to case workers.
Unlike traditional parametric models of nonlinear budget sets (e.g. Burtless and Hausman,
1978; Hoynes, 1996; Keane and Mo¢ tt, 1998), our constrained model has no refutable predictions for the cross-sectional distribution of earnings and program participation under a
given policy regime.3 We show however that the model places strong restrictions on the set of
potential transitions between coarse earnings and welfare participation categories (“response
margins”) that may be induced by the experiment. We use these theoretical restrictions
to derive analytical bounds on the probabilities of making transitions along each allowable
margin.
Applying our identi…cation results, we …nd evidence of substantial intensive and extensive
margin responses to treatment. Jobs First appears to have incentivized many women who
would not have worked to do so and many who would have worked o¤ welfare at low earnings
to participate in the JF program. Corroborating the analysis of BGH, we …nd a strong opt-in
response among women who would have worked at relatively high earnings levels. We also
…nd evidence that the hassle associated with the “work-…rst”component of JF induced some
women to work but under-report their earnings in order to maintain eligibility for bene…ts.
Our results demonstrate that a theoretically motivated coarsening of earnings data can
allow researchers to bound the size of competing response margins under weaker assumptions
than have been previously considered in the literature. This insight appears to be new
and may be applicable to other settings with ostensibly continuous choices (e.g. capital
taxation or electricity demand) where incentives are highly nonlinear. More generally, using
theoretical restrictions on the set of potential transitions induced by reform provides a feasible
approach to going beyond mean e¤ects and estimating impact distributions, a goal which has
been argued by many to be important for appropriately interpreting experimental evidence
(Heckman, Smith, and Clements, 1997; Heckman, 2001; Deaton, 2009).
The bounding approach developed here is similar in spirit to the recent work of Manski
(2012) who uses revealed preference arguments to set-identify features of policy counterfactuals. However, our analysis explores the identifying power of experimental variation in
isolating response margins, a subject he does not consider. We also work with a richer model
3
See Macurdy, Green, and Paarsch (1990) for an early critique of parametrically structured econometric
models of labor supply with nonlinear budget sets.
4
of labor supply behavior incorporating …xed costs of work, under-reporting behavior, the effects of welfare hassle, and labor supply constraints. Blundell, Bozio, and Laroque (2011a,b)
also have a recent bounds based analysis of labor supply behavior but are concerned with
a statistical decomposition of ‡uctuations in aggregate hours worked rather than formal
identi…cation of policy counterfactuals.
Our …ndings strongly suggest that households respond to policy variation of the sort
involved in welfare reform along several margins, some of which are best described as intensive
margin “opt-in”e¤ects. As noted by Saez (2002) and Laroque (2005), responsiveness on the
intensive margin has important implications for the optimal design of transfer policy. An
important question for future work is whether the responses uncovered in our analysis are an
artifact of the sharp (and particularly salient) eligibility notch created by the JF program.
1
The Jobs First Program
With the passage of the Personal Responsibility and Work Opportunity Reconciliation Act
(PRWORA) in 1996, all …fty states were required to replace their Aid to Families with Dependent Children (AFDC) programs with Temporary Assistance to Needy Families (TANF)
programs. This change involved a series of major reforms including the imposition of time
limits, work requirements, and improved …nancial incentives to work. The state of Connecticut responded to PRWORA by implementing the Jobs First (JF) program which involved
each of the major features of the TANF program including a particularly strong (and salient)
change in the …nancial incentives to work.
To study the e¤ectiveness of the reform, the state contracted with the Manpower Development Research Corporation (MDRC) to conduct a randomized evaluation, comparing
the Jobs First TANF program to the earlier state AFDC program. Between January 1996
and February 1997, MDRC collected a baseline sample of about 4,800 single parent AFDC
recipients and applicants and randomly assigned them to either the new JF program or the
old AFDC program with equal probability. To conduct the evaluation, administrative data
on earnings and welfare participation were collected for several years prior to and following
the date of random assignment and merged with a baseline survey conducted by MDRC.
This experiment has been heavily studied, with several analyses …nding that the program
encouraged work and had important e¤ects on the distribution of both earned and unearned
income for program participants (Adams-Ciardullo et al., 2002; BGH, 2006, 2010).
Table 1 provides a summary of the key JF and AFDC program features. A distinguishing
feature of the JF program is the 100% earnings disregard which provides a dramatic reduction
5
in the implicit tax on earnings faced by women on welfare relative to AFDC. The JF earnings
disregard incentivizes work at low earnings levels but also creates an eligibility “notch” in
the agent’s budget set with a windfall loss of the entire grant amount occurring if she earns a
dollar more than the federal poverty line. This creates strong incentives not to earn amounts
just above the poverty line and, when possible, to under-report income in order to maintain
program eligibility.4
Another important feature of JF, like all TANF programs, is the imposition of time limits.
As documented in MDRC’s …nal report (Adams-Ciardullo et al., 2002) and BGH (2006),
this induces time varying e¤ects of the program as JF cases eventually become ineligible
for program bene…ts. For this reason, we restrict our analysis to the …rst 7 quarters of the
experiment, a horizon over which time no case was in danger of reaching the limit.5 Finally,
the JF program involved a series of formal work requirements and imposed sanctions on
cases that failed to seek work. Evidence from the …nal report suggests the sanctions were
not heavily enforced. Nevertheless, in our empirical work, we allow for the possibility that
these work requirements made life less pleasant for women who chose to stay on welfare
without working.
2
Budget Set and Optimizing Behavior
Figure 1 provides a stylized depiction of the budget faced by a woman with 2 children who
is new to welfare under the AFDC and JF policy rules respectively. The vertical axis of the
graph gives total income (earned income + transfers) while the horizontal axis gives earned
income E. The JF budget set contains a large notch at the federal poverty line beyond which
welfare eligibility expires. Under JF, total income received when monthly earnings equal the
federal poverty line (F P L) exceeds that for earnings in the range (F P L; F P L + G) where G
is the base grant amount which is common to JF and AFDC. This is to be contrasted with the
4
The program also induced a second notch for the sample of JF applicants who faced a strict earnings
test in order to establish eligibility. AFDC did not have an earnings test, but bene…ts for that program
phased out at an amount above the JF earnings test. Hence, it became harder under JF for high earning
applicants to establish eligibility. Because our analysis is static, we follow BGH in ignoring this feature of
the policy. See Card and Hyslop (2005) and Ferrall (2012) for empirical analyses of dynamic responses to
eligibility incentives. We have experimented with restricting our analysis to recipients only and found similar
results however this group contains substantially fewer high earners which limits our ability to detect opt-in
behavior.
5
If agents are forward looking, this restriction may not fully remove the in‡uence of the time limits on
behavior (Grogger and Michalopoulos, 2003). BGH (2005) argue that this is probably not a major concern
in this sample but we cannot fully rule out the possibility that our results would be di¤erent if time limits
were not present.
6
AFDC budget set which phases out smoothly, yielding an implicit tax rate of approximately
49% on earnings above a $120 disregard.
We summarize these policy rules with the transfer function Gti (E) which gives the
monthly grant amount associated with welfare participation at earnings level E under policy
regime t 2 fa; jg (AFDC or JF respectively). The i subscript acknowledges that the grant
amount may vary across women with the same earnings due to variation in the size of the
assistance unit (AU). Thus,
Gai (E) = max Gi
Gji
1 [E >
i ] (E
i)
i; 0
(1)
(E) = 1 [E < F P Li ] Gi
where i and 1
i are the …xed and proportional AFDC earnings disregards, both of
which may vary across households depending upon how long the woman has been receiving
bene…ts.6 Note that the base grant amount Gi and the federal poverty line F P Li also vary
across households due to di¤erences in AU size.
To study the e¤ects of the program, we begin by considering a conventional static utility
maximization framework where agents have complete control over their earnings. We assume
women in our sample have heterogenous convex non-satiated utility functions
Ui (E; C)
over earnings and a consumption equivalent C. As in Saez (2010), utility is increasing
in consumption C but decreasing in earnings E which require e¤ort to generate. In this
framework, one can think of heterogeneity in the disutility of earnings as worker skill. The
consumption equivalent C incorporates a variety of psychic and monetary costs and takes
the form:
t
(2)
C = E + Gti (E)
i 1 [E > 0] :
i
i 1 [E = 0] D
The variable D 2 f0; 1g is an indicator for the woman participating in welfare. The parameter i
0 gives the dollar value of welfare stigma which may vary arbitrarily across
women and explain why some women with no earnings don’t participate in welfare. We also
allow for a …xed cost of work i which may vary arbitrarily across women. Fixed costs will
discourage work at low earnings levels.7 Finally, ti 0 gives the dollar value of the hassle
a woman faces from authorities when not working and receiving bene…ts. This hassle may
6
7
Under AFDC, women are allowed four months use of the proportional disregard before it expires.
See Meyer and Heim (2004) for a careful analysis of labor supply behavior with …xed costs of work.
7
depend on the policy regime. Because JF includes stronger work requirements and sanctions,
we assume that
j
a
i 8i.
i
The woman’s decision problem is to:
max
E 0; D2f0;1g
Ui (E; C) subject to (1) and (2)
We now summarize our predictions from this model. Consider …rst a woman at point
A in Figure 1 who, under AFDC rules, would participate in welfare without working. She
may be incentivized by JF rules to work while on welfare both by the reduction in implicit
tax rates on earnings and the hassle associated with JF. If …xed costs are large enough,
the additional hassle associated with not working under JF may induce her to leave welfare
entirely and earn more than the federal poverty line. Alternatively, she may simply respond
to the hassle by opting out of welfare and continuing to not work. Finally, she may not
respond at all and simply continue to stay on welfare and not work.
A woman working on welfare at point B under AFDC will face a reduction in her implicit
tax rate under JF. Like any uncompensated increase in the wage, this change could lead to
increases or decreases in the amount of work undertaken, but in any case will lead her to
continue working on welfare. If, as much of the literature on female labor supply suggests
(Blundell and Macurdy, 1999), the substitution e¤ect dominates the income e¤ect, a woman
will be expected to work more but not more than the federal poverty line as this level of
earnings was in her choice set under AFDC and not chosen.
A woman who chooses to locate at point A’under AFDC rules must face high welfare
stigma as she is forgoing the full grant amount G. Not working under JF will be at least as
unattractive since the base grant amount is una¤ected by the reform. She will also not want
to work on JF because the bene…ts of work are no greater under JF than o¤ welfare and she
chose not to work while o¤ welfare. Therefore, her behavior will be una¤ected by reform.
A woman who works o¤ welfare under AFDC rules at point B’must also dislike welfare
participation. Yet she may be induced by the reforms to work under JF. This is likely if
stigma is strong but so is the substitution e¤ect. In such a case, the woman will value the
reduction in tax rate under JF enough to justify participating in welfare.
A woman working on welfare under AFDC rules at point C in Figure 1 will choose
to participate in JF which would o¤er an increase in income for less work with the same
8
amount of stigma. Indeed, the windfall associated with JF participation may well induce an
income e¤ect leading her to reduce her earnings below the federal poverty line. Much the
same response is likely for a woman who would work o¤ welfare under AFDC rules at point
D. If however the woman faces severe stigma with regard to welfare participation she may
nonetheless choose to remain o¤ welfare and continue to earn at point D.
A high earning woman at point E in Figure 1 may face incentives to opt-in to welfare
under JF which would involve lowering her earnings to the federal poverty line. At this point
she will unambiguously want to work more as participation involves a reduction in income
for her. She will not do so because of the large eligibility notch she faces. Thus, we have the
sharp nonparametric prediction that opt-in should lead to a mass point of workers locating
exactly at the poverty line.
The key predictions of the model then, are that:
The fraction of women who work should increase under JF
The fraction of women on welfare and not working should decrease
The fraction of women with earnings less than or equal to the FPL should increase
A mass of workers should locate at the FPL exactly
We turn now to an examination of whether these predictions are actually satis…ed in the
data.
3
Data
Our data come from the MDRC Jobs First Public Use Files. The data contain a baseline survey of demographic and family composition variables merged with administrative
information on welfare participation, rounded welfare payments, family composition, and
rounded earnings covered by the state unemployment insurance (UI) system. There are
a number of important limitations to the Public Use Files that place constraints on our
analysis. While welfare payments are measured monthly, UI earnings data are only available
quarterly. Because we are studying the relationship between program participation and labor
supply decisions, it is important that we be able to compare these variables on a consistent
time scale. The Public Use Files do not directly provide the month of randomization, making it di¢ cult to aggregate the monthly welfare information to the quarterly level. We infer
the month of randomization by contrasting monthly assistance payments with an MDRC
9
constructed variable providing quarterly assistance payments. For each case, we have found
that a unique month of randomization leads the aggregation of the monthly payments to
match the quarterly measure to within rounding error.
Another di¢ culty is that our administrative measure of the size of the assistance unit is
missing for most cases. For the Jobs First sample we are able to infer an AU size in most
months from the grant amount while the women are on welfare. If however AU size changes
while o¤ welfare we are not able to detect this change. Moreover, in some cases the grant
amount is reduced, presumably because the household reported some unearned income. In
both cases we use the grant amount in other months to impute AU size in a month for which
we cannot observe it. For the AFDC sample, the grant amount depends on many unobserved
factors, preventing us from inferring the AU size even while on welfare.
In cases where we need to compute treatment e¤ects by AU size we rely on another
variable (kidcount) collected in the baseline survey which asks for the number of children
in the household at the time of random assignment. This variable is top-coded at three
children. Appendix Table 1 gives a cross-tabulation in the JF sample of kidcount with our
more reliable AU size measure inferred from grant amounts. The tabulation suggests the
kidcount variable is a reasonably accurate measure of AU size over the …rst 7 quarters postrandom assignment conditional on the number of children at baseline being less than three.
As might be expected, the kidcount variable tends to underestimate the true AU size as
women may have additional children over the 7 quarters following the baseline survey. To
deal with this problem we try in‡ating the kidcount based AU size by one in order to avoid
understating the location of the poverty line for most households.
Table 2 shows descriptive statistics for our analysis sample. We have 4,802 cases with
complete earnings and pre-random assignment characteristics, yielding essentially the same
baseline sample as in BGH (2006).8 As the Table makes clear, there are some mildly signi…cant di¤erences between the AFDC and JF groups in their baseline characteristics, however
these di¤erences are not jointly signi…cant. We follow BGH (2006) in using propensity score
methods to adjust for these baseline di¤erences in order to increase the e¢ ciency of our
estimation procedures. These techniques are described in the Appendix. After adjustment,
the means of the AFDC and JF groups are very similar.
We also examine three subgroups of the data characterized by their earnings seven quarters prior to random assignment. These groups were studied previously by BGH (2010), with
8
We drop one AFDC case from our analysis with unrealistically high quarterly earnings that sometimes
led to erratic results.
10
the zero earnings group having no earnings in pre-assignment quarter 7, and the low and
high earning groups having quarterly earnings below or above the median of the non-zero
observations respectively. Descriptive statistics for these groups are also provided in Table
2. Because pre-assignment earnings proxy for tastes and earnings ability, the switch to JF
likely presented these groups with di¤erent incentives. We document substantial treatment
e¤ect heterogeneity between them later on.
The Eligibility Notch and Under-reporting Behavior
Before examining the e¤ects of JF it is useful to verify that transfers do in fact obey the JF
policy rules. To examine this, Figure 2 plots the empirical relationship between quarterly
UI earnings and welfare payments for women in the JF sample. Because the poverty line
varies by AU size we rescale earnings of each case relative to the relevant poverty line which
in this …gure was inferred from the grant amount. Our measure of AU size is based upon the
assumption that size has not changed since the last time the woman was on welfare and had
an unreduced grant amount.9 The close correspondence at low earnings levels between the
median monthly grant (which includes cases not on welfare) and the statutory unreduced
grant amount indicates that grants are rarely reduced.
Figure 2 strongly suggests that the JF eligibility notch was enforced, with the fraction of
women on assistance throughout the quarter falling quickly as earnings exceed the eligibility
notch at three times the monthly poverty line. This drop is equally pronounced in several di¤erent measures of the duration of welfare participation which suggests lags between
earnings and eligibility decisions are not particularly important.
We interpret the fact that participation does not drop to zero immediately above the notch
as evidence that women substantially under-report their earnings. This is consistent with
the analysis of Hotz, Mullin, and Scholz (2003) who analyzed data from a welfare reform
experiment in California. Comparing administrative earnings records from the California
unemployment insurance system to earnings reported to welfare over a similar time period,
they …nd a highly nonlinear reporting relationship with cases having quarterly UI earnings above $2500 being disproportionately more likely to under-report and under-reporting
greater amounts on average. This threshold is somewhat below the federal poverty line for a
typical AU and close to the eligibility threshold of the California program. In section 6, we
more carefully consider how to distinguish under-reporting responses from true labor supply
responses.
9
We have also tried throwing out cases that experience a change in AU size and found similar results.
11
4
Impacts on Earnings and Participation
We turn now to examining the qualitative predictions of the standard neoclassical model in
section 2. Figure 3 provides reweighted CDFs of earnings in the AFDC and JF groups over
the seven quarters following random assignment. We rescale earnings relative to three times
the monthly FPLs faced by these women based upon our survey measure of AU size (3F P L
is the maximum amount that can be earned in a quarter while maintaining welfare eligibility
under JF). To avoid understating the location of the poverty line these households face, we
in‡ate the implied AU size by one.10
Several of the qualitative predictions of the theory can be assessed from this …gure alone.
A reweighted Kolmogorov-Smirnov test strongly rejects the null hypothesis that the two
CDFs are identical.11 Clearly more women had quarters with positive earnings in the JF
sample than under AFDC, indicating that JF successfully incentivized many women to work.
The earnings CDF rises more quickly in the JF sample than under AFDC, signaling excess
mass at low earnings levels. Also, the CDFs cross below the notch suggesting a small optin response with the fraction earning below 3F P L slightly greater for the JF sample than
among the AFDC controls. There is no evidence of a spike in the distribution of earnings at
3F P L, an issue we will explore more carefully in the next section using a di¤erent measure
of AU size.
These distributional e¤ects conceal substantial heterogeneity across subgroups. Figures
4a-4c provide corresponding CDFs in three subsamples considered by BGH (2010) de…ned
by their earnings in the seventh quarter prior to random assignment.12 These groups are
of interest because pre-random assignment earnings are a strong predictor of post-random
assignment earnings, hence they proxy for the relevant range of the budget set an agent
would face under AFDC. Cases with high pre-random assignment earnings are most likely to
exhibit an opt-in e¤ect while cases with zero earnings are likely to be pushed into the labor
force by JF. The …gures con…rm that the expected pattern of heterogeneity is in fact present,
with the high earnings group experiencing no impact on the fraction of cases working but
a large (though only marginally signi…cant) impact on the fraction of cases with earnings
10
That is, we use the mapping from kidcount to AU size: 0–>3, 1–>3, 2–>4, 3–>5. This mapping is
conservative in ensuring that earnings levels below the FPL are indeed below it. We have found that our
qualitative results are relatively robust to alternate codings including in‡ating the AU size by two and not
in‡ating it at all.
11
See the appendix for details on the computation of this test.
12
The seventh quarter prior to random assignment is a useful stratifying variable because welfare applicants
and recipients generally experience a severe dip in earnings in the quarters immediately prior to random
assignment.
12
less than or equal to the poverty line. The zero earnings group, by contrast, exhibits a large
impact on the fraction of cases working, but essentially no impact on the fraction of cases
with earnings less than or equal to the poverty line.
Table 3 quanti…es the distributional impacts of JF on the CDF of earnings and provides
standard errors generally con…rming the visual impression of the prior …gures. JF yielded
a large decrease in the fraction of quarters with earnings below the monthly poverty line,
suggesting many workers were incentivized to work greater amounts than before by the
reduction in implicit tax rates. We also see a tendency for the fraction earning less than
three times the poverty line (the maximum quarterly earnings one can obtain under JF while
maintaining eligibility) to increase, especially in the higher earning groups, suggesting the
presence of an opt-in e¤ect.
Consistent with the model’s other predictions, welfare participation grew sharply under
JF in each group and the fraction of quarters spent on welfare with no earnings fell. We
also see that average earnings while on welfare grew, which is likely driven by a combination
of non-random selection into welfare and substitution e¤ects among working woman who
would participate in welfare under either set of policy rules.
5
Bunching Evidence
Our evidence so far strongly suggests the presence of an opt-in response to the JF intervention, with the magnitude of the response being greatest in subsamples of the population
where one would most expect it. The basic neoclassical model in section 2 predicts that
such responses should be accomplished in part via bunching at the eligibility notch. Here
we examine this prediction more carefully by looking at bunching in the cross-sectional distribution of earnings in the JF sample where we have a means of accurately measuring AU
size.
Figure 5 provides a histogram of earned income rescaled relative to the federal poverty
line. Not only do we not see a spike in the mass of cases located at the notch, the earnings density actually appears to be declining through this point. Moreover, this decline is
relatively smooth through the notch which should be a dominated earnings region for most
households. However, compared to individuals not on welfare at all in the quarter there is
arguably an excess “mound”in the density of earnings below the notch.
One explanation for these …ndings is that agents cannot fully control their earnings
(Heckman, 1983; Saez, 2010; Chetty et al, 2011). For example, women may engage in
13
directed search for jobs with earnings near the notch level. If however a job pays a few
hundred dollars less than the notch, a woman may take it because it would be too costly to
turn an o¤er down and keep searching for one directly at the notch level. We also suspect
that under-reporting behavior is important for these patterns, as households may report
earnings at the notch while earning higher amounts, placing them in what appears to be the
dominated region of earnings. The next section develops an augmented model that nests
these concerns.
6
An augmented model
Thus far we have found that most of the qualitative predictions of the frictionless static model
regarding earnings and program participation appear to hold in the data but that predictions
regarding a mass point at the eligibility notch fail. We also found that many seemingly
ineligible households receive welfare bene…ts. We consider now a generalized version of
the standard labor supply model of section 2 that can accomodate these observations by
allowing constraints on worker choices and under-reporting behavior. Working with this
model, we derive in the next section a robust approach to identi…cation of the magnitude of
the intensive margin opt-in e¤ect, the extensive margin decision to work, and various sorts
of under-reporting responses.
We begin by assuming that, in addition to the welfare participation decision (D) and the
choice of earned income (E), workers may also choose a level of earnings (E r ) to report to
the welfare agency. We assume that women can under-report but not over-report earnings
so that E r E. Grant amounts are determined based upon reported earnings, so that the
transfer function becomes:
Gai (E r ) = max Gi
Gji
1 [E r >
i ] (E
r
i)
(3)
i; 0
(E r ) = 1 [E r < F P Li ] Gi :
The costs of under-reporting earnings are given by i
0 which may vary arbitrarily
across women. One can think of i as the psychic and pecuniary costs of concealing a job
from the welfare agency.13 Like stigma and hassle, under-reporting costs are assumed to
enter as dollar equivalents in consumption. Thus, we augment our earlier speci…cation of the
consumption equivalent to be:
C = E + Gti (E r )
13
i
t
r
i 1 [E
= 0] D
i 1 [E
See Saez (2010) for another analysis of …xed reporting costs.
14
> 0]
i 1 [E
r
< E] D:
(4)
Note that women face hassle when they report zero earnings regardless of whether or not
their actual earnings are zero, hence women concealing an income source may choose to
report earnings equal to the positive …xed disregard level i instead of zero.
We make one additional restriction on psychic costs relative to section 2 which will
greatly simplify our analysis in the next section: we assume that monthly welfare stigma i
is bounded from below for women eligible for the proportional disregard. Speci…cally, we
assume i Gai (F P Li ). This assumption guarantees that women will not choose to report
earnings above the federal poverty line while on AFDC (e.g. point C in Figure 1). Because
under AFDC the proportional disregard phases out just above the poverty line, this bound
is not very stringent –at most it requires a monthly stigma of $75.14 Moreover, most of the
women in our sample are long term participants ineligible for the proportional disregard in
which case no restriction applies.
To model constraints, we suppose that each woman draws a pair of earnings o¤ers (Oi1 ; Oi2 )
from the bivariate distribution Fi (:). She can choose between these o¤ers or reject them both,
in which case she earns nothing. The o¤ers drawn are invariant to the policy regime t to
which the woman is assigned. Thus, the woman’s objective is to:
max
E2f0;Oi1 ;Oi2 g; D2f0;1g; E r 2[0;E]
Ui (E; C) subject to (3) and (4)
Note that the presence of two o¤ers provides the possibility of an intensive margin earnings response to the policy change. This response may or may not be constrained as the
o¤ers (Oi1 ; Oi2 ) could coincide with the woman’s unconstrained choices of Ei under the two
policy regimes. The presence of constraints provides a simple explanation for the absence of
a spike in the earnings distribution at the JF eligibility notch. Likewise, our allowance for
under-reporting behavior provides an explanation for welfare participation among households
with administrative earnings above the eligibility notch.
The introduction of under-reporting behavior introduces new margins of adjustment not
present in the model of section 2. Figure 6 illustrates the decision problem in earnings and
consumption equivalent space for a woman with substantial …xed costs of work and low
costs of under-reporting. The e¤ective budget sets are discontinuous at zero earnings due
to the …xed costs of work . In the Figure, the hassle costs j of not working under JF are
14
As a check on the plausibility of this assumption we examined the fraction of women in the AFDC
sample who participated in welfare while receiving a rounded bene…t of $50. We found this fraction to be
approximately 1% which corroborates the notion that most women in our sample do not …nd it worthwhile
to participate in welfare in exchange for very low bene…t levels.
15
substantially larger than those under AFDC represented by a , but both are smaller than .
In comparison with the …xed costs of work and hassle, the costs of under-reporting ( ) are
depicted as being relatively small. The under-reporting line is equivalent under AFDC and
JF because in either case the woman can report earnings arbitrarily close to zero, obviating
any implicit taxes on her true earnings.
A women with this con…guration of psychic costs who under AFDC would receive bene…ts
without working (point A) may now under JF be induced to earn above the poverty line
while under-reporting earnings (e.g. point B) in order to maintain eligibility. This occurs
because the work …rst component e¤ectively removes point A from the budget set due to
increased hassle. With preferences of the sort depicted in the Figure, options like point B will
be an attractive response if the woman draws an earnings o¤er in that range. Conversely,
a woman who works while on AFDC but under-reports earnings (e.g. point C) may choose
under JF to work and truthfully report her earnings (e.g. point D) as the earnings disregard
reduces the return to under-reporting. Thus, reform has mixed e¤ects on reporting behavior
which may lead to an increase or a decrease in the total rate of under-reporting. In the next
section we seek to formally identify the magnitude of these and other responses.
7
Identi…cation of response margins
Our augmented model is su¢ ciently general that it places no restrictions on the crosssectional distribution of earnings and program participation choices under a given policy
regime. The right mix of preferences and earnings o¤ers could support all women choosing
any earnings level and program participation state. Hence, the right mix of preferences and
o¤ers across women could support any cross-sectional distribution of choices.
However, as we have already discussed, the model does yield predictions about how di¤erent groups of women may respond to policy variation. In particular, it rules out transitions
between certain states of the world de…ned by broad earnings and program participation
categories. For example, a woman who would have worked on welfare under AFDC with
earnings below the eligibility notch will not choose to stop working under JF. This conclusion
follows from a revealed preference argument –she could have not worked under AFDC and
received the same grant amount (along with less hassle). Since she did not, she must prefer
working to this choice.
In this section we enumerate a complete set of restrictions on transitions between coarsened earnings categories and welfare participation states. Our focus on coarsened earnings
16
categories simpli…es our analysis and highlights the fact that the experiment only di¤erentially manipulates the relative attractiveness of labor supply choices across broad earnings
ranges.15 We use these restrictions to test the model and form bounds on the fraction of
women in each state who make each of the allowable transitions to other states.
Response margins
ei is de…ned by
We begin by introducing some notation. Our coarsened measure of earnings E
the relation:
8
>
if Ei = 0
< 0
ei =
E
1 if Ei F P Li
>
:
2 if Ei > F P Li
That is, it indicates whether a woman works, and if so, whether she earns enough to be
ineligible for bene…ts under JF. This choice of earnings categories is crucial as the model
rules out many transitions between (but not within) these categories in response to the JF
reform. Moreover, our assumptions so far imply that, under either policy regime, a woman
ei = 2 who participates in welfare must be under-reporting her earnings to the welfare
with E
agency.
Intersecting these earnings categories with the decision to participate in welfare and the
under-reporting decision there are seven possible earnings / participation / reporting states
allowed by the model:
S f0n; 1n; 2n; 0r; 1r; 1u; 2ug :
The number of each state refers to the woman’s earnings category, while the letter n denotes welfare non-participation, r denotes participating in welfare while truthfully reporting
earnings, and u denotes participating in welfare while under-reporting earnings.
Let Sia denote woman i’s potential state under AFDC and Sij her potential state under
JF. De…ne the conditional probability of occupying state sj 2 S under JF given the choice
15
In some respects, our approach is similar to the common practice in the structural modeling literature of
coarsening the choice set (e.g. Hoynes, 1996; Keane and Mo¢ tt, 1998). Relative to that literature, we do not
presume that women lack …ne control over earnings within the relevant earnings category. Also in contrast
to the structural literature, which typically works with data driven categories such as part-time and full-time
work, we coarsen the outcomes based upon features of the policy rules, breaking earnings into regions de…ned
by their relation to the JF eligibility notch. This choice allows us to make strong nonparametric predictions
about behavior.
17
of state sa 2 S under AFDC as:
sa ;sj
P Sij = sj jSia = sa :
These transition probabilities will be our objects of interest as they summarize the strength
of the various margins along which agents can respond to the JF program. Our model
constrains the 7 7 matrix of state transition probabilities to take the following form:
State under
AFDC
0n
1n
2n
0r
Earnings / Reporting State under Jobs First
1n
2n
0r
1r
0n
1
0
0n;1r
0
0
0r;0n
1
0
0
1n;1r
0
0
1
0
0
0
2n;1r
1
0r;0n
1r
1u
2u
0
0
0
0
0
0
0
0
0
2u
2n;1r
0
0
0
0
0
0
0r;1r
0
0n;1r
1n;1r
0r;2n
0r;2n
0r;1r
1u
0r;2u
0r;2u
0
0
0
1
1
2u;1r
0
0
0
0
0
1
2u;1r
A proof that the matrix takes this form is given in the Appendix. Note that each of the
allowable transitions corresponds to a response discussed in sections 2 or 6. Key to identi…cation of the underlying transition probabilities are the many transitions that the model
says cannot happen. As discussed in the Appendix, these transitions can be ruled out by
revealed preference arguments.
The model allows eight response margins along which individuals can switch states:
A fraction 1n;1r of cases who would work but earn less than the poverty line and not
participate in welfare under AFDC will work while on welfare under JF.
A fraction 2n;1r of cases who would earn more than the poverty line under AFDC will
“opt-in”to welfare and reduce their earnings below the poverty line under JF.
A fraction 0r;1r of cases who participate in welfare without working under AFDC will
work and remain on welfare under JF .
A fraction 0r;0n of cases who participate in welfare without working under AFDC will
leave welfare and continue not to work.
18
A fraction 0r;2n of cases who participate in welfare without working under AFDC will
leave welfare and earn above the poverty line.
A fraction 0r;2u of cases who participate in welfare without working under AFDC will
earn above the poverty line but remain on welfare by under-reporting earnings.
A fraction 2u;1r of cases who earn above the poverty line but under-report earnings in
order to qualify for bene…ts under AFDC will reduce their earnings below the poverty
line under JF and truthfully report earnings.
Note that some of these transitions ( 0r;2n ; 0r;2u ) involve moving from earnings category
0 to category 2, while others ( 2n;1r ; 2u;1r ) involve moving from earnings category 2 to
category 1. Thus, rank invariance may be violated in our model, even across coarsened
earnings categories.
A variety of other adjustments may occur within each of our three earnings ranges. Thus
the restriction that 1p;1p = 1 should not be taken to imply that no response is present among
those who work on welfare, as many such workers may earn more within that range. Without
further restrictions, however, we cannot infer the magnitude of any such adjustments.
Identi…cation
Identi…cation of the transition probabilities is hindered by the fact that the states f1u; 1rg
in S are not empirically distinguishable. The function g (s) : S !Se de…ned as
8
>
0n
if s = 0n
>
>
>
>
>
1n
if s = 1n
>
>
>
< 2n
if s = 2n
g (s) =
>
0p
if s = 0r
>
>
>
>
>
1p if s 2 f1r; 1ug
>
>
>
: 2p
if s = 2u
maps the set of latent states S to the set of observed states Se
f0n; 1n; 2n; 0p; 1p; 2pg.
As before, the number of each state refers to the woman’s earnings category and the letter
n refers to welfare non-participation. The letter p denotes welfare participation, which is
directly observable. Note that state 2p can only be occupied via under-reporting.
Let Sei denote the value of woman i’s observed state, Seia = g (Sia ) her potential observed
state under AFDC, and Seij = g Sij her potential observed state under JF. The experimental
19
status variable Ti 2 fa; jg denotes the policy regime into which woman i was actually
randomized. Random assignment of cases to the two policy regimes guarantees that the
following condition holds:
(5)
Ti ? Sia ; Sij
where the ? symbol denotes independence.
P (Sit = s) denote the probability of occupying state s 2 S under treatment
Let qst
regime t. Likewise, de…ne
X
ptse P Seit = se =
qst
s: g(s)=e
s
as the probability of occupying observable state se 2 Se under treatment regime t. The
marginal state probabilities ptse are identi…ed by the relation:
ptse = P Sei = sejTi = t ;
which follows from (5). Thus, experimental variation allows identi…cation of the marginal
distributions of potential outcomes. We can write the probability of transitioning between
observed states in terms of the probabilitities of unobserved states as follows:
P Seij = sej jSeia = sea =
X
a
sa ;sj qsa
(sa ;sj ): g(sa )=e
sa ; g(sj )=e
sj
X
qsaa
:
(6)
sa : g(sa )=e
sa
De…ne the vectors of observed state probabilities:
0
pj
pj0n ; pj1n ; pj2n ; pj0p ; pj1p ; pj2p
pa
pa0n ; pa1n ; pa2n ; pa0p ; pa1p ; pa2p :
0
Summing (6) across observed AFDC states yields,
pj =
where, from (6), the 6 6 transition matrix
pa
(7)
consists of the elements of P Seij = sej jSeia = sea
20
and takes the following form:
State under
AFDC
0n
1n
2n
0p
Earnings / Participation State under Jobs First
1n
2n
0p
1p
0n
1
0
0n;1r
0
0
0r;0n
1
0
0
1n;1r
0
1
0
0
0
2n;1r
1
0
0
0
0
0
0
0
0
0n;1r
1n;1r
2n;1r
0r;0n
0r;2n
0r;2n
0r;1r
0r;1r
1p
2p
2p
0
0
0r;2u
0r;2u
0
0
0
1
2u;1r
1
2u;1r
Somewhat surprisingly, these transition probabilities do not depend on the unobserved fraca
tion (q1u
) of the population who would earn below the poverty line yet under-report their
earnings under AFDC policy rules. This occurs because women who would work and earn
less than the poverty line under AFDC rules are predicted to do the same under Jobs First,
regardless of whether they would have under-reported under AFDC or not.
The transition matrix
implies a set of six equations, one of which is redundant given
the constraint that the state probabilities sum to one in each regime. Since the matrix
involves eight parameters, the system is clearly under-determined. One of the transition
probabilities may be solved out explicitly as:
1n;1r
pa1n pj1n
=
pa1n
(8)
This leaves us with the following system of four non-redundant linear restrictions on seven
parameters:
pj0n
pa0n =
pa0n
0n;1r
+ pa0p
0r;0n
pj2n
pa2n =
pa2n
2n;1r
+ pa0p
0r;2n
pj0p
pa0p =
pa0p (
pj2p
pa2p = pa0p
0r;1r
0r;2u
+
0r;2u
pa2p
2u;1r
+
(9)
0r;2n
+
0r;0n )
It is straightforward to show that the system implies the following testable non-parametric
21
restrictions:
pj0p
pa0p
pj1p
(10)
0
pj1n
pa1n
pa1p
0
In addition to testing for these restrictions, we can solve for bounds on the transition
probabilities entering (9) because we have the additional constraint that each transition
probability lies in the unit interval. The upper and lower bounds on each of the transition probabilities can be represented as the solution to the constrained linear programming
program of the form:
max
0
(11)
2 [0; 1]7 and (9)
s.t.
where
[ 0n;1r ; 0r;0n ; 2n;1r ; 0r;2n ; 0r;1r ; 0r;2u ; 2u;1r ]0 . Solving the above problem for
= [1; 0; 0; 0; 0; 0; 0]0 yields the upper bound on 2u;1r , while choosing = [ 1; 0; 0; 0; 0; 0; 0]0
yields the lower bound.
One can also use this representation to derive bounds on linear combinations of the parameters. We consider parameters summarizing the prevalence of three “composite” margins
of adjustment:
pa0p
( 0r;0n + 0r;2n )
0r;n
pa0p
n;p
pa2n
pa0p (
0;1+
+ pa1n 1n;1r + pa0n
pa2n + pa1n + pa0n
2n;1r
0r;1r
+
0r;2n
pa0p
0n;1r
+ 0r;2u ) + pa0n
+ pa0n
0n;1r
:
The parameter 0r;n gives the fraction of women who would claim bene…ts without working
under AFDC that are induced to get o¤ welfare
JF. Upperiand lower
bounds for this pa-i
h under
h
pa
pa
pa
pa
0p
0p
0p
rameter can be had by solving (11) with = 0; pa ; 0; pa ; 0; 0; 0 and 0; p0p
; 0; 0; 0
a ; 0;
pa
0p
0p
0p
0p
respectively. We also examine the fraction n;p of women who are induced to take up welfare
under JF and the fraction 0;1+ who are induced by JF to work. Because no women who
would work under AFDC will choose not to work under JF, this latter fraction is point identi…ed by the proportional reduction in the fraction of women not working under JF relative
to AFDC.
The bounds de…ned by the solution to (11) are sharp and summarize the restrictions of the
22
model. Routines for the solution of such problems are standard in most numerical packages.
However, …nding analytical solutions to these problems is useful for conducting inference. In
the appendix we list analytical upper and lower bounds on each parameter found by solving
the relevant optimization problems by hand. This is a tedious but manageable process which
involves systematically checking a large number of special cases. A representative example
are the bounds on the opt-in probability 2n;1r which obey:
(
pa2n pj2n
max 0;
pa2n
)
2n;1r
min
8
>
>
>
>
>
>
>
>
>
<
>
>
>
>
>
>
>
>
>
:
1;
pa
2n
j
j
a
pa
2n p2n +p0p p0p
;
pa
2n
j
j
a
a
a
p2n p2n +p0p p0p +p0n pj0n
;
pa
2n
j
j
j
a
a
+p
p
+p
p
pa
p
2p
0p
2n
2p
0p
2n
;
pa
2n
j
j
j
a
a
a
p2n +p0p p0p +p0n p0n +p2p pj2p
pa
2n
9
>
>
>
>
>
>
>
>
>
=
>
>
>
>
>
>
>
>
>
;
:
Note that there are two lower bounds, one of which is zero. This turns out to be a generic
feature of the lower bounds for each of the seven transition parameters. Which lower bound
binds is unknown a priori. Likewise, which of the …ve upper bounds will be smallest is a
question which can only be deduced by looking at the data.
Consistent estimators of the least upper and greatest lower bounds can be had by using
sample analogs of the marginal probabilities and computing the relevant min f:g and max f:g
expressions. Inference is complicated by the fact that the limit distribution of the least upper
and greatest lower bounds depends upon sampling uncertainty in the relative ranks of the
bounds –i.e. in which of the upper and lower bounds are binding. Naive bootstrap inference
on the min f:g and max f:g of the relevant sample bound estimates will fail to provide
coverage of the true parameter with …xed probability (Andrews and Han, 2009).
We report con…dence intervals for the transition probabilities based upon two inference
procedures. The …rst simply ignores the sampling uncertainty in the ranks of the bounds and
assumes the bound that appears relevant given the sample analogues does in fact bind. In
such a case, results from Imbens and Manski (2002) imply a 95% con…dence interval for the
parameter in question can be constructed by extending the upper and lower bounds by 1:65b
where b is a standard bootstrap estimate of the standard error of the sample moment used
to de…ne the relevant bound.16 The second approach is a conservative bootstrap procedure
described in the appendix which covers the true parameter with …xed probability regardless
16
per
For example,
bound
is
if the relevant lower bound for
j
j
a
pa
2n p2n +p0p p0p
,
pa
2n
then
the
95%
23
2n;1r
bootstrap
is
j
pa
2n p2n
pa
2n
con…dence
and the relevant upinterval
for
2n;1r
is:
of which of the bounds bind. The lower limit of this con…dence interval coincides with that
of the naive procedure because there is only one sample moment in the max f:g operator
for each parameter. But the upper limit of the con…dence interval from our conservative
procedure exceeds that from the naive procedure, substantially so in some cases.
8
Results
We estimate the relevant set of joint probabilities for each of our samples in Table 4. Notably
the basic sign restrictions in (10) are satis…ed by the point estimates. We also see a small but
statistically signi…cant increase in the fraction of quarters on welfare with earnings above
the quarterly poverty line indicating that some fraction of the population was induced by
the treatment to under-report their earnings.
Table 5 provides estimates of the underlying transition probabilities that rationalize the
impacts in Table 4. The sole point identi…ed transition probability 1n;1r is computed by
plugging in sample analogues of the probabilities in (8). We …nd a strong e¤ect of JF on
entry into the program by the working poor. Our bootstrap con…dence intervals suggest
between 31% and 46% of the women who would have worked o¤ welfare under the AFDC
system at earnings levels below the poverty line were induced to participate in JF at eligible
earning levels.
We also …nd a substantial opt-in response among women who would have worked o¤
welfare at earning levels above the poverty line. Our estimated bounds imply that 2n;1r
:28. That is, at least 28% of those women with ineligible earnings under AFDC decided
to work at eligible levels under JF and participate in welfare. Accounting for sampling
uncertainty in the bounds extends this lower limit to 19%, which is still quite substantial.
The upper bounds for this parameter were not informative leading us to conclude that the
opt-in probability is somewhere in the interval [:19; 1] with 95% probability.
We also …nd suggestive evidence of a second opt-in e¤ect from non-participation. Our
sample bounds imply 0n;1r 2 [:06; :62]. However, uncertainty in the bounds prevents us from
rejecting the null that this response probability is actually zero. We also …nd a small but
signi…cant under-reporting response attributable to the hassle e¤ects of JF. A conservative
n a j
p
b
p
b
max 0; 2npba 2n
2n
p
ba
p
bj
o
p
ba
1:65bl ; min 1; 2n
p
bj2n +b
bj0p
pa
0p p
p
ba
2n
+ 1:65bu
ror of 2npba 2n and bu the bootstrap standard error of
2n
denote reweighted sample analogues.
24
p
ba
bj2n
2n p
p
ba
2n
+
with bl the bootstrap standard erp
ba
bj0p
0p p
p
ba
2n
and where hats on probabilities
95% con…dence interval for 0r;2u is [:02; :13]. Thus, JF induced at least one subpopulation
to under-report earnings, and in the process violate the standard rank invariance condition
implicit in BGH’s analysis.
The remaining response probabilities: 0r;0n ; 0r;2n ; 0r;1r ; 2u;1r have zero lower bounds,
meaning we cannot reject the null that each of them are zero. However, we can reject the
null that they are jointly zero. From (9) such a joint restriction would imply that:
pj0p
pj2p
pa0p =
pa2p
which is easily rejected by our data. Thus, at least some of these margins of adjustment are
present. Among the probabilities in question, the candidate that seems most likely to be
positive is 0r;1r which is the extensive margin response through which welfare reform has
traditionally been assumed to operate. However, we cannot be sure that the abundance of
women working at low earning levels under JF are in fact coming from state 0r rather than
state 2u.
We also computed bounds on the three composite adjustment margins described in the
previous section. First is the probability 0;1+ that a woman transitions along the extensive
margin from nonwork to work. A conservative 95% con…dence interval for this probability
is [0:13; 0:21]. Thus JF seems to have induced a substantial fraction of women to work who
would not have done so under AFDC.
We also …nd a relatively tight con…dence interval on the fraction n;p of women induced
to take up welfare by JF. Although JF unambiguously increased the fraction of women on
welfare, our model suggests some women may also have been induced to leave welfare, breaking point identi…cation of this margin. According to our conservative inference procedure,
at least 19% (and at most 51%) of women o¤ welfare under AFDC were induced to claim
bene…ts under JF.
Finally, we cannot reject the null hypothesis that JF failed to induce any of the women
who would have not worked while claiming AFDC bene…ts to leave welfare under JF, as the
lower bound for the parameter 0r;n is zero. We are however able to conclude that at most
24% of such women left welfare, which may limit concerns that the JF reforms pushed a
large fraction of women potentially unable to work o¤ assistance.
25
9
Conclusion
Our analysis of the Jobs First experiment suggests that women responded to the policy
incentives of welfare reform along several margins, some of which would traditionally be
considered intensive and some of which are clearly extensive. This …nding is in accord with
BGH’s original interpretation of the JF experiment and with recent evidence from Blundell,
Bozio, and Laroque (2011a,b) who …nd that secular trends in aggregate hours worked appear
to be driven by both intensive and extensive margin adjustments.
An important question is the extent to which our …nding of intensive margin responsiveness might generalize to other transfer programs that lack sharp budget notches but still
involve phase-out regions that should discourage work. It seems plausible that the JF notch
would be more salient than, say, the budget kink induced by the EITC phase-out region. On
the other hand, enforcement of the EITC rules involves less ambiguity than those governing
the JF notch. Progress on the this question may be had via an application of the methods
developed here to quasi-experimental estimates of the counterfactuals associated with other
heavily studied policies (e.g., the di¤erence in di¤erences design exploited by Meyer and
Rosenbaum 2001).
Finally, the methods developed here depart from the usual focus in the literature on
identi…cation of labor supply elasticities. In the frictionless model, elasticities provide useful
summaries of the e¤ects of marginal reforms. However in our model, which involves general
labor supply constraints, structural elasticities are not su¢ cient for predicting the e¤ects of
reform, even locally. Whether meaningful restrictions can be developed on the constraints
that restore the usefulness of elasticities as in Chetty et al. (2011) without invoking strong
parametric assumptions is an important question for future research.
References
1. Adams-Ciardullo, Diana; Bloom, Dan; Hendra, Richard; Michalopoulos, Charles; Morris, Pamela; Scrivener, Susan and Walter, Johanna. 2002. Jobs First: Final Report
on Connecticut’s Welfare Reform Initiative. New York: Manpower Demonstration
Research Corporation.
2. Andrews, Donald and S. Han. 2009. “Invalidity of the Bootstrap and m Out of
n Bootstrap for Interval Endpoints De…ned by Moment Inequalities.” Econometrics
Journal 12(s1): S172-S199.
26
3. Andrews, Donald and Panle Jia Barwick. 2012. “Inference for Parameters De…ned by
Moment Inequalities: A Recommended Moment Selection Procedure.” Econometrica
80(6): 2805–2826.
4. Ashenfelter, Orley. 1983. “Determining Participation in Income-Tested Social Programs.”Journal of the American Statistical Association 78(383):517-525.
5. Bitler, Marianne, Jonah Gelbach, and Hilary Hoynes. 2005. “Distributional Impacts
of the Self-Su¢ ciency Project.”NBER Working Paper #11626.
6. Bitler, Marianne, Jonah Gelbach, and Hilary Hoynes. 2006. “What Mean Impacts
Miss: Distributional E¤ects of Welfare Reform Experiments.” American Economic
Review 96(4): 988–1012.
7. Bitler, Marianne, Jonah Gelbach, and Hilary Hoynes. 2010. “Can Variation in Subgroups’Average Treatment E¤ects Explain Treatment E¤ect Heterogeneity? Evidence
from a Social Experiment.”working paper.
8. Blundell, Richard and Thomas Macurdy. 1999. “Labor Supply: A Review of Alternative Approaches.”In Handbook of Labor Economics, Vol. 3A, ed. Orley Ashenfelter
and David Card, 1559–1695. Amsterdam: Elsevier.
9. Blundell, Richard, Antoine Bozio, and Guy Laroque. 2011a. “Extensive and Intensive
Margins of Labour Supply: Working Hours in the US, UK and France.”IZA Discussion
Paper #6051.
10. Blundell, Richard, Antoine Bozio, and Guy Laroque. 2011b. “Labour Supply and
the Extensive Margin.” American Economic Review, Papers and Proceedings 101(3):
482-486.
11. Burtless, Gary and Jerry Hausman. 1978. “The E¤ect of Taxation on Labor Supply: Evaluating the Gary Negative Income Tax Experiment.”The Journal of Political
Economy 86(6):1103-1130.
12. Card, David and Dean Hyslop. 2005. “Estimating the E¤ects of a Time-Limited
Earnings Subsidy for Welfare-Leavers.”Econometrica, 73(6): 1723-1770.
13. Chetty, Raj, John Friedman, Tore Olsen, and Luigi Pistaferri. 2011. “Adjustment
Costs, Firm Responses, and Micro vs. Macro Labor Supply Elasticities: Evidence
From Danish Tax Records.”Quarterly Journal of Economics 126(2): 749–804.
27
14. Chetty, Raj, John Friedman, and Emmanuel Saez. 2012. “Using Di¤erences in Knowledge Across Neighborhoods to Uncover the Impacts of the EITC on Earnings.”NBER
Working Paper #18232.
15. Chernozhukov, Victor, Sokbae Lee, and Adam Rosen. 2009. “Intersection bounds:
estimation and inference.”working paper.
16. Deaton, Angus. 2009. “Instruments of Development: Randomization in the tropics,
and the search for the elusive keys to economic development.” NBER Working Paper
#14690.
17. Eissa, Nada, and Je¤rey B. Liebman. 1996. “Labor Supply Response to the Earned
Income Tax Credit.”Quarterly Journal of Economics, 111(2): 605–37.
18. Eissa, Nada, and Hilary Hoynes. 2006. “The Hours of Work Response of Married
Couples: Taxes and the Earned Income Tax Credit.”In Tax Policy and Labor Market
Performance, ed. Jonas Agell and Peter Birch Sorensen, 187–227. Cambridge: MIT
Press.
19. Ferrall, Christopher. 2012. “Explaining and Forecasting Results of The Self-Su¢ ciency
Project.”Review of Economic Studies, forthcoming.
20. Giné, Evarist and Joel Zinn. 1990. “Bootstrapping General Empirical Measures.”
Annals of Probability 18(2): 851-869.
21. Greenberg, David, Robert Mo¢ tt, and John Friedmann. 1981. “Underreporting and
Experimental E¤ects on Work E¤ort: Evidence from the Gary Income Maintenance
Experiment.”Review of Economics and Statistics 63(4): 581-589.
22. Greenberg, David and Harlan Halsey. 1983. “Systematic Misreporting and E¤ects of
Income Maintenance Experiments on Work E¤ort: Evidence from the Seattle-Denver
Experiment.”1(4): 380-407.
23. Grogger, Je¤rey and Charles Michalopoulos. 2003. “Welfare Dynamics under Time
Limits.”Journal of Political Economy 111(3): 530-554.
24. Hausman, Jerry. 1985. “The Econometrics of Nonlinear Budget Sets.” Econometrica
53(6): 1255-1282.
25. Heckman, James, Je¤rey Smith, and Nancy Clements. 1997. “Making the Most Out
of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in
Programme Impacts.”Review of Economic Studies 64(4): 487-535.
28
26. Heckman, James J. 1983. “Comment.”In Behavioral Simulation Methods in Tax Policy
Analysis, ed. Martin Feldstein, 70–82. Chicago: University of Chicago Press.
27. Heckman, James. 1993. “What Has Been Learned About Labor Supply in the Last 20
Years?”American Economic Review Papers and Proceedings 83(2): 116-121.
28. Heckman, James. 2001. “Micro Data, Heterogeneity, and the Evaluation of Public
Policy: Nobel Lecture.”Journal of Political Economy 109(4): 673-748.
29. Heim, Bradley and Bruce Meyer. 2004. “Work Costs and Non-Convex Preferences in
the Estimation of Labor Supply Models.” Journal of Public Economics 88(11): 23232338.
30. Hotz, Joseph , Charles Mullin, and Karl Scholz. 2003. “The E¤ects of Welfare Reform
on Employment and Income: Evidence from California.”working paper.
31. Hoynes, Hilary. 1996. “Welfare Transfers in Two-Parent Families: Labor Supply and
Welfare Participation Under AFDC-UP.”Econometrica 64(2): 295-332.
32. Keane, Michael, and Robert Mo¢ tt. 1998. “A Structural Model of Multiple Welfare Program Participation and Labor Supply.”International Economic Review, 39(3):
553–89.
33. Kosorok, Michael. 2008. Introduction to Empirical Processes and Semiparametric
Inference Springer: New York.
34. Laroque, Guy. 2005. “Income Maintenance and Labor Force Participation.” Econometrica 73(2): 341-376.
35. Macurdy, Thomas, David Green, and Harry Paarsch. 1990. “Assessing Empirical
Approaches for Analyzing Taxes and Labor Supply.” Journal of Human Resources
25(3): 415-490.
36. Meyer, Bruce and Dan Rosenbaum. 2001. “Welfare, the Earned Income Tax Credit,
and the Labor Supply of Single Mothers.” Quarterly Journal of Economics 116(3):
1063-1114.
37. Meyer, Bruce. 2002. “Labor Supply at the Intensive and Extensive Margins: the
EITC, Welfare, and Hours Worked.”American Economic Review 92(2): 373-379.
38. Mo¢ tt, Robert. 1990. “The Econometrics of Kinked Budget Constraints.”Journal of
Economic Perspectives 4(2): 119-139.
29
39. Mo¢ tt, Robert. 2002. “Welfare Programs and Labor Supply.” In Handbook of Public
Economics. Vol. 4, ed. Alan J. Auerbach and Martin S. Feldstein, 2393–2430. New
York: North-Holland.
40. Saez, Emmanuel. 2002. “Optimal Income Transfer Programs: Intensive Versus Extensive Labor Supply Responses.”Quarterly Journal of Economics 117(3): 1039-1073.
41. Saez, Emmanuel. 2010. “Do Taxpayers Bunch at Kink Points?” American Economic
Journal: Economic Policy 2(3): 180-212.
Appendix
Propensity Score Reweighting
To deal with the chance imbalances between the AFDC and JF groups, we estimate a
parametric propensity score model. Following BGH (2006) we estimate a logit of the JF
assignment dummy on: quarterly earnings in each of the 8 pre-assignment quarters, separate
variables representing quarterly AFDC and quarterly food stamps payments in each of the 7
pre-assignment quarters, dummies indicating whether each of these 22 variables is nonzero,
and dummies indicating whether the woman was employed at all or on welfare at all in the
year preceding random assignment or in the applicant sample. We also include dummies
indicating each of the following baseline demographic characteristics: being white, black,
or Hispanic; being never married or separated; having a high-school diploma/GED or more
than a high-school education; having more than two children; being younger than 25 or
age 25–34; and dummies indicating whether baseline information is missing for education,
number of children, or marital status.
The predicted values from this model by pbi . The propensity score weights used to adjust
the moments of interest are given by:
Ti
pbi
!i = X T +
j
j
pbj
1 Ti
1 pb
X 1iT
j
1 pbj
j
That is, they are inverse probability weights, re-normalized to sum to one within policy
group. When examining subgroups we always recompute a new set of propensity score
weights and subsequently re-normalize them.
30
Kolmogorov-Smirnov Tests for Equality of Distribution
We use a bootstrap procedure to compute the p-values for our reweighted KolmogorovSmirnov (K-S) tests for equality of distribution functions across treatment groups. Let Fnt (e)
be the reweighted empirical distribution function (EDF) of earnings in treatment group t.
That is,
X
! i I [Ei e; Ti = t]
Fnt (e)
i
De…ne the corresponding bootstrap EDF as:
Fnt (e)
X
! i I [Ei
e; Ti = t]
i
The K-S test statistic is given by:
d
KS
sup jFnj (e)
e
Fna (e) j
To obtain a critical value for this statistic, we compute the bootstrap distribution of the
recentered K-S statistic:
KS
sup jFnj (e)
Fna (e)
e
Fnj (e)
Fna (e) j:
Recentering is necessary to impose the correct null hypothesis on the bootstrap DGP (Giné
and Zinn, 1990). We compute an estimated p-value b for the null hypothesis that the two
distributions are equal as:
1 X h
I KS
1000 b=1
1000
b
where b indexes the bootstrap replication.
(b)
d
> KS
i
List of Bounds
The analytical expressions for the bounds on the transition probabilities are as follows:
2n;1r
2n;1r
)
pa2n pj2n
max 0;
;
pa2n
8
pa pj +pa pj
pa pj +pa pj +pa pj
<
1; 2n 2npa 0p 0p ; 2n 2n 0ppa 0p 0n 0n ;
2n
2n
min
j
j
j
j
j
j
a
a
0n p0n +p2p
: pa2n p2n +pa0p a p0p +pa2p p2p ; pa2n p2n +pa0p p0p +p
a
(
p2n
p2n
31
(12)
pj2p
9
=
;
;
(13)
)
pa0n pj0n
;
max 0;
pa0n
8
pa pj +pa pj
pa pj +pa pj +pa pj
<
1; 0n 0npa 0p 0p ; 0n 0n 0ppa 0p 2n 2n ;
0n
0n
min
j
j
j
j
j
j
a
a
2p p2p +p2n
: pa0n p0n +pa0p a p0p +pa2p p2p ; pa0n p0n +pa0p p0p +p
a
(
0n;1r
0n;1r
p0n
(
max 0;
2u;1r
min
2u;1r
0r;1r
0r;1r
(
max 0;
min
8
>
>
>
<
>
>
>
:
0r;2n
0r;2n
0r;2u
0r;2u
8
<
:
pa0p
pa
0p
pj2p
pa2p
pa2p
)
pj2n
p0n
pj0n
pa0p
;
;
;
pa pj
pa pj
pa pj +pa pj +pa pj
1; 2ppa 2p + 0ppa 0p ; 2p 2p 0ppa 0p 2n 2n ;
2p
2p
2p
j
j
j
j
j
j
j
a
a
a
a
a
pa
pa
0p p0p +p0n p0n p2p p2p +p0p p0p +p2n p2n +p0n p0n
2p p2p
+
;
a
a
a
p2p
p2p
p2p
pj0p
9
=
pj2n
pj2p
)
9
=
;
;
;
j
j
j
j
j
j
j
a
a
a
a
a
a
pa
0p p0p p0p p0p +p0n p0n p0p p0p +p2n p2n p0p p0p +p2p p2p
;
;
;
;
a
a
a
a
p0p
p0p
p0p
p0p
j
j
j
j
j
j
j
j
j
a
a
a
a
a
a
a
p0p +p2n p2n +p2p p2p p0p p0p +p0n p0n +p2n p2n p0p p0p +p2p p2p +pa
0n p0n
;
;
;
a
a
a
p0p
p0p
p0p
j
j
j
j
a
a
a
a
p0p p0p +p0n p0n +p2n p2n +p2p p2p
pa
0p
(
max 0;
min
8
<
:
(
min
:
)
;
j
j
j
j
j
a
a
a
a
pj2n pa
0p p0p p0p p0p +p2p p2p p0p p0p +p0n p0n
;
;
;
;
a
a
a
a
p0p
p0p
p0p
p0p
j
j
j
a
a
a
p0p p0p +p2p p2p +p0n p0n
pa
0p
max 0;
8
<
pa2n pj2n
pa0p
pa2p pj2p
pa0p
)
;
9
=
;
;
;
j
j
j
j
j
a
a
a
a
pj2p pa
0p p0p p0p p0p +p2n p2n p0p p0p +p0n p0n
;
;
;
;
pa
pa
pa
pa
0p
0p
0p
0p
j
j
j
a
a
a
p0p p0p +p2n p2n +p0n p0n
pa
0p
32
9
=
;
9
>
>
>
=
>
>
>
;
;
(
max 0;
0r;0n
min
0r;0n
8
<
:
)
pa0n pj0n
pa0p
;
j
j
j
j
j
a
a
a
a
pj0n pa
0p p0p p0p p0p +p2p p2p p0p p0p +p2n p2n
;
;
;
;
pa
pa
pa
pa
0p
0p
0p
0p
j
j
j
a
a
pa
0p p0p +p2p p2p +p2n p2n
pa
0p
Derivation of Bounds
9
=
;
:
The transition matrix in conjunction with (7) induces a linear system of 6 equations in
eight transition probabilities. The equations of the system, numbered for convenience, are:
1 : pj0n
pa0n =
pa0n
0n;1r
2 : pj1n
pa1n =
pa1n
1n;1r
pa2n
3 :
pj2n
=
+ pa0p
0r;0n
pa2n 2n;1r
+ pa0p
0r;2n
pa0p (
+
4 : pj0p
pa0p =
5 : pj1p
pa1p = pa0n
0n;1r
+ pa1n
6 : pj2p
pa2p = pa0p
0r;2u
pa2p
0r;1r
0r;2u
(14)
+
1n;1r
+
0r;0n )
2n;1r
+ pa0p
0r;2n
+ pa2n
0r;1r
+ pa2p
2u;1r
2u;1r
We drop Equation 5 as the state probabilities sum to one under each treatment status.
The bounds on each transition probability are derived by enumerating the inequality restrictions placed on the parameter of interest by the system in (14). The process proceeds
by sequential substitution of unknowns in equations related to parameters of interest and
consideration of extreme values (0 or 1) of nuisance transition probabilities. Dominated
inequality restrictions are then eliminated.
We illustrate the procedure here for the opt-in probability 2n;1r . For purposes of this
exercise, denote 2n;1r = where is some constant in [0; 1]. The equations of the full rank
system are:
1 : pj0n
pa1n
pa0n = pa0p
pj1n
=
pa1n
3 : pj2n pa2n (1
2 :
pa0n
0r;0n
0n;1r
1n;1r
) = pa0p
4 : pa0p
pj0p = pa0p (
6 : pj2p
pa2p = pa0p
0r;1r
0r;2n
+
0r;2u
33
0r;2u
pa2p
+
2u;1r
0r;2n
+
0r;0n )
We have two restrictions from 3:
pj2n
pa2n (1
)
0
pj2n
pa2n (1
)
pa0p
We can solve for transition rate
pj0p
40 : pa0p
0r;2n
pj2n
and substitute into equation 4 to yield:
pa2n (1
We can use equation 1 to substitute pa0p
400 : pa0p
pj0p
pj2n
pa2n (1
)
) = pa0p (
0r;0n
4000 : pa0p
pj0p
pj2n
pa2n (1
pa0n = pa0p (
0r;2u
pj2p
)
+
+
0r;2u
0r;1r
+
0r;2u )
+ pa0n
0n;1r
0r;1r
+
0r;0n )
+ pa2p
2u;1r
0r;0n
and pa0p
in 40 :
pa2p = pa0p (
We can also use both equation 1 and equation 6 to substitute pa0p
equation 4’:
40000 : pa0p pj0p
pj2n
pa2n (1
pj0n
)
0r;0n )
in 40 :
pj0n
We can also use equation 6 to substitute pa0p
0r;1r
pj2p
pa0n
pa2p = pa0p
0r;2u
in
a
a
0r;1r +p0n 0n;1r +p2p 2u;1r
Thus, for any , we get the following inequalities which arise from considering extreme
values of the nuisance transition probabilities17 :
from 3 : pj2n
pa2n (1
)
0
from 3 : pj2n
pa2n (1
)
pa0p
from 40 : pa0p
pj0p
pj2n
pa2n (1
)
0
from 40 : pa0p
pj0p
pj2n
pa2n (1
)
pa0p
from 400 : pa0p
pj0p
pj2n
pa2n (1
)
pj0n
pa0n
0
from 400 : pa0p
pj0p
pj2n
pa2n (1
)
pj0n
pa0n
pa0p + pa0n
from 4000 : pa0p
pj0p
pj2n
pa2n (1
)
pj2p
pa2p
0
from 4000 : pa0p
pj0p
pj2n
pa2n (1
)
pj2p
pa2p
pa0p + pa2p
from 40000 : pa0p
pj0p
pj2n
pa2n (1
)
pj0n
pa0n
pj2p
pa2p
0
from 40000 : pa0p
pj0p
pj2n
pa2n (1
)
pj0n
pa0n
pj2p
pa2p
pa0p + pa0n + pa2p
pj0p
0.
17
Recall from (10) that pa0p
34
Rewriting these inequalities to isolate
from 3 :
from 3 :
from 40 :
from 40 :
from 400 :
from 400 :
from 4000 :
from 4000 :
from 40000 :
from 40000 :
we have:
pa2n pj2n
pa2n
pa2n pj2n + pa0p
(redundant)
pa2n
pa2n pj2n + pa0p pj0p
pa2n
pa2n pj2n pj0p
(redundant)
pa2n
pj0n pa0n
pa2n pj2n + pa0p pj0p
pa2n
pa2n pj2n pj0p pj0n
(redundant)
pa2n
pa2n pj2n + pa0p pj0p
pj2p pa2p
pa2n
pa2n pj2n pj0p pj2p
(redundant)
pa2n
pa2n pj2n + pa0p pj0p
pj0n pa0n
pj2p
pa2n
pa2n pj2n pj0p pj0n pj2p
(redundant)
pa2n
pa2p
Inequalities labeled redundant can be shown to be dominated by the other inequalities in
the list. Note that the list of nonredundant expressions coincide with the upper and lower
bounds in (12; 13).
Inference on Bounds
We begin with a description of the upper limit of our con…dence interval. For each parameter
we have a set of upper bounds fub1 ; ub2 ; :::; ubK g. We know that:
min fub; 1g
ub
min fub1 ; ub2 ; :::; ubK g
A consistent estimate of the least upper bound
n ub can be hadoby plugging in consistent
p
b k ! ubk and using ub
b
b 1 ; ub
b 2 ; :::; ub
b K as an estimate of ub. This
sample moments ub
min ub
estimator is consistent by continuity of probability limits. We can then form a corresponding
35
consistent estimator b
n
o
b 1 of .
min ub;
To conduct inference on , we seek a critical value r such that:
b + r = 0:95;
ub
P ub
as such an r implies
P
n
o
b + r; 1
min ub
(15)
n
o
b + r; 1
min ub
n
o
b + r; 1
P ub min ub
h
i
h
b
b
b +r
= P ub ub + r 1 ub + r < 1 + 1 ub
h
i
h
i
b + r < 1 + 1 ub
b +r 1
= 0:95 1 ub
P
i
1
0:95
with the …rst inequality binding with equality when
and the second when ub < .
=
We can rewrite (15) as:
P
or equivalently
n
b1
min ub
n
P max ub
b2
ub; ub
b 1 ; ub
ub
bK
ub; :::; ub
b 2 ; :::; ub
ub
o
ub
bK
ub
o
r = 0:95
r = 0:95
n
o
b
b
b
It is well known that the limiting distribution of max ub ub1 ; ub ub2 ; :::; ub ubK depends on which and how many of the upper bounds bind. Several approaches to this problem
have been proposed which involve conducting pre-tests for which constraints are binding (e.g.
Andrews and Barwick, 2012).
We will take an alternative approach to inference that is simple to implement and consistent regardless of the bounds that bind. Our approach is predicated on the observation
that:
n
P max ub1
b 1 ; :::; ubK
ub
bK
ub
o
r
n
P max ub
b 1 ; :::; ub
ub
bK
ub
o
r
(16)
with equality holding in the case where all of the upper bounds are identical. We seek an r0
such that:
n
o
b 1 ; :::; ubK ub
bK
P max ub1 ub
r0 = :95
(17)
36
From (16),
n
P max ub
b 1 ; :::; ub
ub
bK
ub
with equality holding when all bounds are identical.
o
r0
:95
p
A bootstrap estimate r ! r0 of the necessary critical value can be had by considering
the bootstrap analog of condition (17) (see Proposition 10.7 of Kosorok, 2008). That is, by
computing the 95th percentile of:
n
b1
max ub
b ; :::; ub
bK
ub
1
b
ub
K
o
across bootstrap replications, where stars refer to bootstrap quantities. An upper limit U of
the con…dence region for can then be formed as:
o
n
b
U = min ub + r ; 1
Note that this procedure is essentially an unstudentized version of the inference method of
Chernozhukov et al. (2009) where the set of relevant upper bounds (V0 in their notation) is
taken here to be the set of all upper bounds, thus yielding conservative inference.
We turn now to the lower limit of our con…dence interval. Our greatest lower bounds are
all of the form:
max flb; 0g :
p
b !
We have the plugin lower bound estimator lb
lb. By the same arguments as above we
want to search for an r00 such that
P lb
b
lb
r00 = 0:95:
b is just a scalar sample mean, we can choose r00 = 1:65 lb where lb is the asymptotic
Since lb
b in order to guarantee the above condition holds asymptotically. To
standard error of lb
account for the propensity score reweighting, we use a bootstrap standard error estimator
blb of lb which is consistent via the usual arguments. Thus, our conservative 95% con…dence
interval for a given parameter is:
h
n
b
max 0; lb
oi
o
n
b + r ;1
1:65blb ; min ub
This con…dence interval covers the parameter
95%.
37
with asymptotic probability of at least
Derivation of Transition Matrix
We prove here that the zero entries of the matrix
correspond to state transitions that,
given our model assumptions, cannot occur in response to the JF experiment.18 To facilitate
our exposition, we start with some de…nitions:
De…nition 1 An allocation is an earnings and consumption equivalent pair (E; C).
e D; E r
De…nition 2 For treatment regime t 2 fa; jg, the state st = h E;
e D; E r from its domain f0; 1; 2g
tion h maps the triple E;
in the text.
f0; 1g
where the func-
R+ to Se as explained
De…nition 3 An allocation (E 0 ; C 0 ) is compatible with state st if E 0 evaluates to the same
0
e and: i) if D = 0, C 0 = E 0
earnings category as E
i 1 fE > 0g, ii) if D = 1 there exists an
r
0
r
r
t
0
t
E r 2 [0; E] such that C 0 = E 0
i 1 fE 6= E g.
i
i 1 fE > 0g + Gi (E )
i 1 fE = 0g
De…nition 4 An allocation (E 0 ; C 0 ) is available if it is compatible with state st and E 0
corresponds to one of the earning draws fOi1 ; Oi2 g or E 0 = 0.
Assumption 1 Treatment regime a allows a small …xed earning disregard denoted by
belonging to range 1.
and
Proposition 1 Under either treatment regime, the optimal reporting rule entails either
truthful reporting or reporting an earnings level that, without loss of generality, equals .
Proof. Omitted.
Lemma 1 Consider transition rate
sa ;sj
for any (sa ; sj ) such that:
L.1 sj 6= sa ;
L.2 the utility value of any allocation compatible with sa is at least as high under regime j
as under regime a;
L.3 the utility value of any allocation compatible with sj is at most as high under regime j
as under regime a.
Then, the labor supply model implies that
sa ;sj
= 0.
Proof. Omitted.
18
In future updates we will prove that the nonzero entires of
38
correspond to transitions that can occur.
Assumption 2 A woman indi¤erent between two available allocations will not switch between them in response to a change in treatment regime.
Lemma 2 The transition probability
8
>
>
>
>
>
>
<
>
>
>
>
>
>
:
sa ;sj
equals zero for all (sa ; sj ) in the collections:
(0n; 1n) ; (0n; 2n) ; (0n; 0r) ; (0n; 2u) ;
(1n; 0n) ; (1n; 2n) ; (1n; 0r) ; (1n; 2u) ;
(2n; 0n) ; (2n; 1n) ; (2n; 0r) ; (2n; 2u) ;
(1u; 0n) ; (1u; 1n) ; (1u; 2n) ; (1u; 0r) ; (1u; 2u) ;
(2u; 0n) ; (2u; 1n) ; (2u; 2n) ; (2u; 0r)
9
>
>
>
>
>
>
=
>
>
>
>
>
>
;
f(1r; 0n) ; (1r; 1n) ; (1r; 2n) ; (1r; 0r) ; (1r; 2u)g
f(0r; 1n) ; (0r; 1u)g
(18)
(19)
(20)
Proof. We start with the collection in (18). It is su¢ cient to prove that these transitions
obey conditions L.1-L.3. L.1 holds trivially. L.2 and L.3 are met because all allocations
compatible with states (0n; 1n; 2n; 1u; 2u) have the same value under either regime.
Turning to the collection in (19), allocations compatible with state 1r must yield utility at
F P Li . Therefore
least as high under JF as under AFDC because Gai (E) < Gi for all E
condition L.2 is met. L.3 follows because (0n; 1n; 2n; 1u; 2u) have the same value under either
regime.
Next, we turn to the collection in (20). Here we argue by contradiction that 0r;1n and
0r;1u must be zero. Suppose …rst that there is a woman i who optimally chooses an allocation
compatible with state 0r under regime a and switches to an available allocation compatible
with state 1n under regime j, say at Oik . By optimality of the allocation compatible with
a
state 0r under regime a we have Ui 0; Gi
Ui (0; 0) which implies Gi
i
i
i
a
a
0. By optimality of the allocation compatible with state 1n under regime
i where i
k
k
j, Ui Oi ; Oi
Ui Oik ; Oik
0. If ai > 0
i which implies Gi
i
i
i + Gi
or ai = 0; Gi 6= i this entails an immediate contradiction. If ai = 0; Gi = i then the
woman must have been indi¤erent between the allocation compatibile with state 0p and that
l
compatible with 0n under regime a which means that Ui (0; 0) Ui Oil ; Oil
i for any Oi
in range 1 including Oik . By optimality of the allocation compatible with state 1n under j,
Ui Oik ; Oik
Ui (0; 0). Thus, if this last inequality is strict we have a contradiction.
i
If an equality obtains, the woman must have been indi¤erent under regime a between the
allocation compatible with state 0n and the allocation entailing earnings Oik o¤ assistance.
From assumption 2, if she did not choose Oik over 0n under regime a, given indi¤erence, she
39
will not make a di¤erent choice under j.
Next, suppose that there is a woman i who optimally chooses an allocation compatible
with state 0r under a and switches to an available allocation compatible with state 1u under
j. By Proposition 1, no allocation compatible with 1u may be optimal under regime j which
produces a contradiction.
40
Figure 1: Hypothetical Budget Sets under AFDC and JF
Notes: Figure provides hypothetical monthly budget constraints faced by assistance unit of size 3 under AFDC and JF policy rules. Figure assumes household
rd
would have access to the $30 + 1/3 disregard under AFDC. FPL refers to federal poverty line, G is base grant amount, and MBS is AFDC needs standard.
Figure 2: Benefits as a Function of Earnings in JF Sample
Notes: The sample includes the JF-TANF cases, all months in Q1−Q7 post-RA such that imputed AU size is between 2 and 5. The horizontal line is drawn at
(unrounded) maximum grant of a size 3 AU in 1997. Grant amounts and earnings are rescaled using, respectively, the maximum grant and FPL of size 3 AU in
1997. Median monthly grant computed across all cases including those not on welfare in the quarter.
.8
.9
1
Figure 3: CDFs of Quarterly Earnings Relative to 3 x Monthly Federal Poverty Line
.5
.6
.7
p-value for equality of distributions = 0.000
0
.5
1
1.5
2
2.5
Quarterly Earnings as a fraction of 3*FPL implied by kidcount (next size up).
Control
JF
Notes: Figure gives reweighted CDFs of quarterly UI earnings (in quarters 1-7 post-RA) in JF and AFDC samples relative to 3 x the monthly federal poverty line
associated with year and AU size. AU size determined by baseline survey question kidcount. To deal with increases in family size since random assignment, we
use next AU size up relative to size directly implied by kidcount (see text for details). The bootstrapped p-value for a Kolmogorov-Smirnov test of equality of the
two distributions is 0.000 (based on 1,000 replications, see Appendix for details).
Figure 4: CDFs of Quarterly Earnings Relative to 3 x Federal Poverty Line (Subgroups)
.8
.9
1
a) Zero Earnings pre-RA
.5
.6
.7
p-value = 0.000
0
.5
1
1.5
2
2.5
Quarterly Earnings as a fraction of 3*FPL implied by kidcount (next size up).
Control
JF
c) High Earnings pre-RA
.8
.8
1
1
b) Low Earnings pre-RA
p-value = 0.159
.2
.2
.4
.4
.6
.6
p-value = 0.006
0
.5
1
1.5
2
Quarterly Earnings as a fraction of 3*FPL implied by kidcount (next size up).
Control
JF
2.5
0
.5
1
1.5
2
2.5
Quarterly Earnings as a fraction of 3*FPL implied by kidcount (next size up).
Control
JF
Notes: Figures give reweighted CDFs of quarterly UI earnings (in quarters 1-7 post-RA) in JF and AFDC samples relative to 3 x the monthly federal poverty line associated
with year and AU size. AU size determined by baseline survey question kidcount. To deal with increases in family size since random assignment, we use next AU size up
relative to size directly implied by kidcount (see text for details). The bootstrapped p-value for a Kolmogorov-Smirnov test of equality of the two distributions is is 0.000
for the Zero Earning subgroup, 0.006 for the Low Earning subgroup and 0.159 for the High Earning subgroup (based on 1,000 replications, see Appendix for details).
Figure 5: Distribution of Quarterly Earnings Centered at 3 x Monthly Federal Poverty Line
0
100
200
Frequency
300
400
500
a) Unconditional
-1000
-500
0
500
c) Off Assistance all 3 Months of the Quarter
100
Frequency
300
0
0
100
50
200
Frequency
400
150
500
b) On Assistance all 3 Months of the Quarter
1000
-1000
-500
0
500
1000
-1000
-500
0
500
1000
Notes: Restricted to Jobs First sample quarters 1-7 post-RA. Assistance unit size has been inferred from monthly aid payment. AU sizes above 8 have been excluded. The bins in
the histograms are $100 wide with bin 0 containing three times the monthly federal poverty line corresponding to the size and the calendar year of the quarterly observation.
Figure 6: Earnings and Participation Choices with Under-reporting
Table 1 - Summary of Policy Differences Between AFDC and Jobs First
Eligibility
Earnings disregard
Time Limit
Work requirements
Other
JF
AFDC
Earnings Below Poverty Line
Earnings level at which benefits
are exhausted (see disregard
parameters below)
All earned income disregarded up to
poverty line
Months 1-4: $120 + 1/3
Months 5-12: $120
Month >12: $90
21 months
(6 month extensions possible)
None
Mandatory work first
(exempt if child <1)
Education / training
(exempt if child < 2)
●Sanctions (moderate enforcement)
●Asset limit $3,000
●Partial family cap (50 percent)
●Two years transitional Medicaid
●Child care assistance
●Child support: $100 disregard, full
pass-through
Sources: Adams-Ciardullo et al. (2002) and Bitler, Gelbach, and Hoynes (2005).
●Sanctions (rarely enforced)
●Asset limit $1,000
●100 hour rule and work history
requirement for two-parent families
●One year transitional Medicaid
●Child support: $50 disregard, $50
maximum pass-through
Table 2: Mean Sample Characteristics
Overall Sample
Demographic Characteristics
White
Black
Hispanic
Never married
Div/wid/sep/living apart
HS dropout
HS diploma/GED
More than HS diploma
More than 2 Children
Mother younger than 25
Mother age 25-34
Mother older than 34
Average quarterly pretreatment values
Earnings
Cash welfare
Food stamps
Fraction of pretreatment quarters with
Any earnings
Any cash welfare
Any food stamps
# of Cases
Zero Earnings Q7 pre-RA
Difference
Jobs First AFDC Difference
(adjusted)
Low Earnings Q7 pre-RA
Difference
Jobs First AFDC Difference
(adjusted)
High Earnings Q7 pre-RA
Difference
Jobs First AFDC Difference
(adjusted)
Jobs First
AFDC
Difference
Difference
(adjusted)
0.362
0.368
0.207
0.624
0.317
0.331
0.550
0.066
0.227
0.289
0.410
0.301
0.348
0.371
0.216
0.631
0.312
0.313
0.566
0.062
0.206
0.297
0.418
0.286
0.014
-0.003
-0.009
-0.007
0.005
0.018
-0.016
0.004
0.021
-0.007
-0.007
0.015
0.001
0.000
-0.001
0.000
0.000
0.000
0.001
0.000
0.000
0.000
0.000
0.000
0.331
0.360
0.250
0.631
0.314
0.369
0.521
0.060
0.252
0.288
0.411
0.301
0.321
0.349
0.267
0.627
0.320
0.370
0.521
0.051
0.242
0.269
0.418
0.313
0.010
0.011
-0.016
0.005
-0.006
-0.001
0.000
0.009
0.010
0.019
-0.007
-0.012
-0.002
0.001
0.000
0.000
0.000
0.000
0.000
0.009
0.000
-0.001
0.000
0.001
0.443
0.367
0.126
0.664
0.272
0.303
0.583
0.054
0.179
0.381
0.361
0.258
0.421
0.428
0.098
0.708
0.237
0.224
0.668
0.056
0.123
0.448
0.358
0.194
0.022
-0.061
0.028
-0.044
0.035
0.078
-0.085
-0.002
0.056
-0.067
0.004
0.064
0.000
-0.002
-0.001
0.004
-0.008
-0.001
0.000
-0.001
-0.007
0.007
-0.001
-0.007
0.425
0.406
0.086
0.550
0.376
0.180
0.655
0.109
0.157
0.204
0.456
0.340
0.385
0.403
0.127
0.568
0.357
0.165
0.651
0.115
0.142
0.256
0.478
0.266
0.040
0.003
-0.041
-0.019
0.019
0.014
0.004
-0.006
0.015
-0.051
-0.022
0.074
0.003
-0.004
-0.004
-0.002
0.003
0.007
-0.014
0.017
-0.004
-0.002
-0.001
0.003
679
(1304)
891
(806)
352
(320)
786
(1545)
835
(785)
339
(304)
-107***
(41)
56**
(23)
13
(9)
-1
(51)
-1
(2)
0
(1)
175
(465)
1041
(812)
397
(326)
189
(490)
1013
(802)
394
(310)
-14
(17)
28
(28)
2
(11)
2
(4)
0
(3)
1
(1)
770
(683)
785
(722)
329
(300)
865
(710)
696
(660)
297
(267)
-94
(51)
90
(51)
32
(21)
-7
(28)
1
(22)
0
(20)
2924
(1890)
298
(517)
168
(232)
3209
(2441)
232
(417)
151
(219)
-285
(159)
66
(34)
17
(16)
-23
(443)
-24
(80)
-14
(45)
0.322
(0.363)
0.573
(0.452)
0.607
(0.438)
2,396
0.351
(0.372)
0.544
(0.450)
0.598
(0.433)
2,407
-0.029***
(0.011)
0.029**
(0.013)
0.009
(0.013)
0.000
(0.001)
-0.001
(0.001)
0.000
(0.001)
0.137
(0.211)
0.644
(0.440)
0.664
(0.428)
1,677
0.144
(0.216)
0.630
(0.442)
0.669
(0.422)
1,623
-0.008
(0.007)
0.014
(0.015)
-0.004
(0.015)
0.000
(0.001)
0.000
(0.001)
0.001
(0.001)
0.667
(0.273)
0.564
(0.448)
0.602
(0.433)
357
0.694
(0.259)
0.532
(0.441)
0.588
(0.423)
397
-0.027
(0.019)
0.032
(0.032)
0.014
(0.031)
0.000
(0.008)
0.001
(0.01)
0.002
(0.01)
0.839
(0.216)
0.254
(0.365)
0.345
(0.391)
362
0.864
(0.181)
0.198
(0.305)
0.312
(0.362)
387
-0.025
(0.015)
0.056
(0.025)
0.033
(0.028)
0.003
(0.02)
-0.010
(0.04)
-0.009
(0.04)
Notes: Adjusted differences are computed via propensity score reweighting. Numbers in parentheses are standard deviations for columns 1, 2, 3, 5, 6, 7; those for columns 4 and 8 are standard errors calculated via 1,000 bootstrap
replications.
Table 3: Mean Outcomes Post-Random Assignment
Average Earnings
Overall
Zero Earnings Q7 pre-RA
Low Earnings Q7 pre-RA
High Earnings Q7 pre-RA
Adjusted
Jobs First AFDC Difference
1,191
1,086
105
(29)
(30)
(36)
Adjusted
Jobs First AFDC Difference
930
751
179
(32)
(30)
(42)
Adjusted
Jobs First AFDC Difference
1,362 1,291
70
(66)
(101)
(118)
Adjusted
Jobs First AFDC Difference
2,124 2,362
-238
(114)
(151)
(179)
Fraction of quarters
with positive earnings
0.440
0.520
(0.008) (0.007)
0.080
(0.010)
0.445
0.349
(0.009) (0.009)
0.096
(0.012)
0.691 0.590
(0.018) (0.022)
0.100
(0.028)
0.680 0.690
(0.022) (0.028)
-0.011
(0.035)
Fraction of quarters with earnings below
monthly FPL (AU size implied by kidcount+1)
0.665
0.710
(0.004) (0.005)
-0.046
(0.006)
0.731
0.789
(0.004) (0.004)
-0.058
(0.006)
0.570 0.636
(0.012) (0.017)
-0.066
(0.020)
0.477 0.438
(0.019) (0.025)
0.039
(0.030)
Fraction of quarters with earnings below
3FPL (AU size implied by kidcount+1)
0.906
0.897
(0.007) (0.007)
0.009
(0.009)
0.938
0.940
(0.008) (0.008)
-0.002
(0.010)
0.906 0.881
(0.019) (0.021)
0.025
(0.027)
0.777 0.722
(0.023) (0.030)
0.055
(0.036)
Fraction of quarters on welfare
0.674
0.748
(0.007) (0.007)
0.074
(0.010)
0.771
0.718
(0.008) (0.008)
0.053
(0.011)
0.764 0.674
(0.019) (0.022)
0.091
(0.029)
0.637 0.475
(0.024) (0.033)
0.162
(0.040)
Average earnings in quarters
with any month on welfare
Fraction of quarters with no earnings and
at least one month on welfare
929
(24)
526
(19)
0.363
0.437
(0.007) (0.007)
403
(28)
-0.074
(0.010)
762
(25)
404
(18)
0.426
0.508
(0.009) (0.009)
359
(30)
-0.082
(0.012)
1,123
(53)
694
(48)
0.231 0.334
(0.015) (0.022)
428
(69)
-0.103
(0.027)
1,524
(96)
1,075
(119)
0.221 0.220
(0.019) (0.028)
449
(147)
0.001
(0.033)
# of cases
2,318
2,324
1,630
1,574
343
384
345
366
Notes: Sample covers quarters 1-7 post-random assignment. Adjusted differences are computed via propensity score reweighting. Numbers in parentheses are standard errors calculated via
1,000 bootstrap replications. Sample units with kidcount missing are excluded.
Table 4: Probability of Earnings / Participation States
Pr(State=0n)
Jobs First
0.127
Overall
AFDC
0.136
Difference
-0.009
Pr(State=1n)
0.076
0.130
-0.055
Pr(State=2n)
0.068
0.099
-0.031
Pr(State=0p)
0.366
0.440
-0.074
Pr(State=1p)
0.342
0.185
0.157
Pr(State=2p)
0.022
0.009
0.013
Overall - Adjusted
Difference
Jobs First
AFDC
0.128
0.135
-0.007
(0.006)
(0.006)
(0.008)
0.078
0.126
-0.048
(0.004)
(0.005)
(0.006)
0.069
0.096
-0.027
(0.004)
(0.005)
(0.006)
0.359
0.449
-0.090
(0.008)
(0.008)
(0.012)
0.343
0.184
0.159
(0.008)
(0.006)
(0.009)
0.023
0.009
0.014
(0.002)
(0.001)
(0.002)
# of quarterly observations
16,226
16,268
16,226
16,268
Notes: Sample covers quarters 1-7 post-random assignment during which individual is either always on or
always off welfare. Number of state refers to earnings level, with 0 indicating no earnings, 1 indicating
earnings below 3 times the monthly FPL, and 2 indicating earnings above 3FPL. The letter n indicates welfare
nonparticipation throughout the quarter while the letter p indicates welfare participation throughout the
quarter. Poverty line computed under assumption AU size is one greater than amount implied by baseline
kidcount variable. Adjusted probabilities are adjusted via the propensity score reweighting algorithm
described in the Appendix. Standard errors computed using 1,000 bootstrap replications.
Table 5: Point and Set-identified Transition Probabilities
Estimate
Standard Error
0.381
0.038
95% CI
(deterministic bound)
95% CI
(conservative)
[0.306, 0.455]
[0.306, 0.455]
Point-identified
Off welfare, low earnings → On welfare, low earnings (π 1n,1r)
Set-identified
On welfare, not working → Off welfare, not working (π 0r,0n)
{0.000, 0.169}
[0.000, 0.210]
[0.000, 0.245]
On welfare, not working → On welfare, low earnings (π 0r,1r)
{0.000, 0.169}
[0.000, 0.210]
[0.000, 0.250]
On welfare, not working → Off welfare, high earnings (π 0r,2n)
{0.000, 0.154}
[0.000, 0.170]
[0.000, 0.302]
On welfare, not working → On welfare, high earnings (π 0r,2u)
{0.031, 0.051}
[0.022, 0.059]
[0.022, 0.131]
Off welfare, not working → On welfare, low earnings (π 0n,1r)
{0.059, 0.618}
[0.000, 0.755]
[0.000, 0.875]
Off welfare, high earnings → On welfare, low earnings (π 2n,1r)
{0.281, 1.000}
[0.193, 1.000]
[0.193, 1.000]
On welfare, high earnings → Off welfare, low earnings (π 2u,1r)
Composite Margins
Not working → Working (π0,1+)
{0.000, 1.000}
[0.000, 1.000]
[0.000, 1.000]
[0.129, 0.206]
[0.129, 0.206]
Off welfare → On welfare (π n,p)
{0.232, 0.444}
[0.188, 0.486]
[0.188, 0.509]
On welfare, not working → Off welfare (π 0r,n)
{0.000, 0.169}
[0.000, 0.210]
[0.000, 0.244]
0.168
0.020
Notes: Estimates inferred from probabilities in Table 4, see text for formulas. Low earnings refers to quarterly earnings less than or equal to three times the monthly federal poverty line (E=1), high
earnings refer to quarterly earnings above three times the monthly federal poverty line (E=2). Numbers in braces are estimated upper and lower bounds, numbers in brackets are 95% confidence
intervals. Column labeled "deterministic bound" ignores uncertainty in which moment inequalities bind. Column labeled "conservative" uses inference procedure described in appendix which covers
true parameter with at least 95% probability regardless of which constraints bind. See text for details.
Table A1: Cross Tabulation of grant-inferred AU size and kidcount
kidcount
Inferred Size
1
2
3
4
5
6
7
8
# of monthly observations
0
1
2
3
0.17
0.53
0.17
0.11
0.00
0.00
0.03
0.00
840
0.08
0.84
0.06
0.01
0.00
0.00
0.00
0.00
11,361
0.04
0.19
0.72
0.05
0.00
0.00
0.00
0.00
8,463
0.01
0.06
0.17
0.53
0.14
0.01
0.07
0.00
8,043
Total
0.05
0.42
0.29
0.17
0.04
0.00
0.02
0.00
28,707
Notes: Analysis conducted on Jobs First sample over quarters 1-7 post-random assignment. Kidcount variable, which is obtained from baseline
survey, is tabulated conditional on non-missing. The AU size is inferred from (rounded) monthly grant amounts and MAP values up to a size of 8.
Starting with size 5, the unique correspondence between AU size and rounded grant amount obtains only for units which do not receive housing
subsidies. Thus, inferred sizes from 5 to 8 rest on this assumption. The size inferred during months on assistance is imputed forward to months off
assistance and to months that otherwise lack an inferred size.