The Impact of Non-Cognitive Skills Training on Academic and Non-academic
Trajectories: From Childhood to Early Adulthood1
Yann Algan (Sciences Po), Elizabeth Beasley (CEPREMAP),
Frank Vitaro (University of Montréal), Richard E. Tremblay (UCD, University of Montréal)
Non-cognitive skills are closely associated with adult socio-economic success. However, it is unclear
whether interventions targeting those skills exclusively, rather than cognitive skills, can improve
adult outcomes, and whether the window for the effective ages of intervention is wide or narrow. We
show that an intervention focused on self-control and social skills at school entry change the lifetime
trajectories for children with disruptive behavior, increasing self-control and trust in adolescence,
improving education achievement, and outcomes in early adulthood such as criminality, education,
employment and social capital. We show that improvements in trust and self-control explain much of
the impact on education and young adult outcomes, and argue that social skills are an important but
neglected aspect of non-cognitive skill development. Using conservative assumptions in a simple
framework, we estimate that, as a lower bound, $1 invested in this program yields about $14 in
benefits over the lifetime of the participants.
1
We thank Roland Benabou, Sam Bowles, Raj Chetty, Bruno Crepon, Angus Deaton, Angela Duckworth, Esther Duflo,
Ernst Fehr, Nicole Fortin, James Heckman, Karla Hoff, Larry Katz, Jens Ludwig, Mark Stabile, Philip Oreopoulos, Bob
Putnam, Jean Marc Robin and Emmanuel Saez for very helpful comments. We also thank participants at the HCEO
workshop on The Effects of socioeconomics status on Identity and Personality 2014, the NBER Education and Crime
Summer Institute 2014, the NBER Political Economy workshop 2014, and seminars at Berkeley, Chicago, Harvard,
Princeton, Paris School of Economics, Stockholm. All the data from the MLES experiment are located at GRIP: Groupe
de recherche sur l’inadaptation psychosociale chez l’enfant, Hopital Sainte-Justine, Montréal Canada. The research
leading to these results has received funding from the European Research Council for Yann Algan under the European
Community‘s Seventh Framework Program (FP7/2007-2013) / ERC grant agreement n° 240923. Elizabeth Beasley
gratefully acknowledges the support of the Abdul Latif Jameel Poverty Action Lab (J-PAL). The experimental
intervention and the follow-up assessments were funded by grants from the Canadian Institutes of Health Research, the
Social Sciences and Humanities Research Council of Canada, the Québec Research Funds for Health and the Québec
Research Funds for Society and Culture.
Introduction
If schools seek to make people more prosperous and productive by improving cognitive
skills, should non-cognitive skills be a formal part of the curriculum? Substantial evidence shows
that non-cognitive skills like self-control, motivation, and sociability are strongly associated with
favorable school, economic and social outcomes (Borghans et al., 2008, Almlund et al., 2011,
Duckworth et al., 2012). Understanding whether interventions in elementary school can promote
non-cognitive skills, and subsequently increase positive adult outcomes, is critical, especially for
children who arrive at school with low levels of social skills and self-control. These children are
more likely to struggle in school, have behavior problems, and be locked into poverty in adulthood.
Providing more equal opportunities for all children may not depend only on teaching in math and
reading, but also self-control and socialization.
Much of the large benefit of early childhood interventions may be due to improvements in
non-cognitive, rather than cognitive, skills. Experiments measuring the impact of investments in
early childhood cognitive development, such as the Abecedarian project, the Perry Preschool
program, Head Start or Project STAR, suggest that a substantial part of the powerful long-term
impact of these programs is due to increases in skills that are not measured by grades or IQ tests –
suggesting a very important role for non-cognitive skills (Heckman et al. 2013, Chetty et al. 2011). It
is important to know what these non-cognitive skills are, whether it makes sense to target them
directly, and whether the window for intervention is wide or narrow – in particular, is elementary
school “too late”? It is well established that early childhood is a critical period for formation of noncognitive skills (Almond et al., 2010, Heckman, 2006). The last question is particularly important for
communities where pre-school education is not universal, and so behavioral issues may not even be
identified until children begin primary school.
This paper provides evidence on these questions by estimating the impact of a randomized
non-cognitive skills training program at school entry for disruptive kindergarten boys from low
socioeconomic environments, in particular on lifetime trajectories. The intervention consisted of a 2year program aimed at enhancing self-control and social abilities beginning at age 7. In the spring of
1984, the Montreal Longitudinal Experimental Study (MLES) evaluated 1037 boys at the end of
kindergarten in low socio-economic areas schools. From this original sample, 250 boys were targeted
for the experiment based on teacher ratings of disruptive behavior. These 250 boys were randomly
2
assigned to either participate in the training program or to be part of the control group. Over the twoyear period, 19 sessions were carried out by a team of professional childcare workers, a social worker
and a psychologist. The sessions included mixed disruptive boys with a majority of boys with good
social skills. Sessions included social abilities training (such as how to ask why, how to invite a
bystander to play, or how to help) and self-control strategy lessons (such as how to react to teasing or
how to react when angry, how to correctly identify the intentions of others). The parents received a
home-based program to reinforce the skills taught at school. The control group did not have access to
this program but had access to all of the standard programs and resources available to the Montreal
public school children in this period. Over the last three decades the MLES collected detailed
longitudinal data from these cohorts on the boys’ bio-psycho-social development and later adult
outcomes.
We examine the impact of the training on the development of a large set of non-cognitive
skills and school performance throughout adolescence, and connect these improvements with
existing evidence on adult crime and education outcomes and new evidence on adult economic and
social capital outcomes. We also provide a cost-benefit analysis of the program. The rich dataset
allows us to crack open the “black box” of non-cognitive skills and sheds new light on our
understanding of skill formation (Heckman, 2006, Cunha and Heckman 2008, Cunha et al. 2010), the
malleability of non-cognitive skills at school entry, and the triggering effect of non-cognitive skills
on latter academic achievement.
We find that, in the early adolescent period, the intervention increased aggression control,
attention-impulse control, and trust. During the late adolescent period, the intervention improved
school performance, aggression control, and trust. For impacts in early adulthood, we present new
data showing that the treated group is more likely to be active fulltime in either work or school from
ages 17-27, and to belong to a civic or social group. This result complements earlier findings
(Boisjoli et al., 2007) showing higher rate of secondary-school graduation for the treatment group
and a large reduction in criminal behavior. We then present evidence that the changes in late
adolescent school performance and young adult outcomes are explained by changes in behavior in
the early adolescent period. We provide suggestive evidence on the amount of the impact on each
outcome that can be explained by changes in Self-Control and Trust. The results of this exercise
provide evidence that trust is a likely to be a separate but understudied mechanism that merits more
attention from the education community.
3
This suggested mechanism is directly connected to the debate in the literature about the
interaction between non-cognitive and cognitive skills, the contribution of non-cognitive skills to
cognitive outcomes, school performance, and their related outcomes in adulthood (Heckman and
Kautz, 2013). In particular, our results are related to non-experimental longitudinal studies showing a
strong association between childhood self-control and a variety of adult outcomes such as school
achievement, health and criminality. The crucial contribution of this study is that there is an
exogenous variation in self-control levels since the intervention was randomized, whereas most
existing evidence on the effect of self-control is based on non-experimental longitudinal studies,
which cannot address the issue of causality (see Duckworth 2011 and Moffitt et al. 2011 for surveys
on the role of self-control; and Duckworth and Seligman 2005, Diamond et al. 2007 and Pingault et
al. 2014 for the impact of self-control on academic achievement in non-experimental studies).
This study makes a new contribution to the rich and growing literature on childhood
development programs. It draws novel insights because of differences in the characteristics of the
target population, length of follow up, the strength of causal identification and the content of the
program.
First, this study targets a different population than most previous studies: children with high
deficits in social skills at school entry. The selection criterion of previous studies was mostly based
on low IQ or low-income status of parents or neighborhoods. Second, it provides evidence of
effectiveness of an intervention after early childhood, and there is little evidence on the effectiveness
of non-cognitive skills training during the early elementary school years (see the meta-analysis by
Heckman and Kautz 2013 and Durlak et al. 2011). As discussed above, understanding whether the
window of intervention is narrow or wide has significant policy relevance. Programs have begun
either at birth, like the Nurse-Family Partnership (Howard and Brooks-Gunn, 2009) or the
Abecedarian Program (Campbell et al. 2002, Campbell et al. 2014), or before the age of 3, like the
Jamaican Supplementation Study (Gertler et al., 2013), the Perry Preschool Program (Heckman et al.
2010), or the Head Start program (Currie and Duncan 1995, Ludwig and Miller 2007). Third, this
study has a longer follow-up than most previous studies. The few interventions that have targeted
elementary school students generally do not have long-term follow-up on both the development of
skills during adolescence and later adult outcomes (Heckman and Kautz 2013). In a meta-analysis of
213 school-based emotional learning programs, only 15% of those programs have a follow-up that
lasts beyond 6 months, and the other ones have very short follow-up programs compared to the
4
MLES (Durlak at al. 2011). A randomized experiment of non-cognitive skill training in Chicago is
close to ours (Heller et al. 2015) but not enough time has elapsed since implementation to measure
long-term impacts. The Seattle Social Development Project had unusually long-term follow up that
showed positive impacts (Hawkins et al. 1991, Hawkins et al. 2008). The MLES program is different
because it was randomized at the individual level, providing an opportunity for strong causal
inference, and because it has a broader set of observations and more frequent assessments of a large
variety of non-cognitive skills from childhood to early adulthood.
Finally, the content of the MLES curriculum targeted only non-cognitive skills while most of
the other early childhood interventions taught cognitive skill development (like Abecedarian and
Head Start) or combined non-cognitive training with cognitive training (Perry Preschool). In this
way, the MLES is also different from the Fast Track program (Bierman et al. 2013), which included
both academic tutoring and social skills training. The Cambridge-Somerville Youth Study evaluated
training in non-cognitive skills with a long-term follow-up, which actually had a negative impact on
participants at age 30 (McCord and McCord, 1959), but the program intervention was itself
fundamentally different from that of the MLES: children with behavioral problems were grouped
together for treatment, and this may have given rise to the stigma or negative influence of deviant
peers that led to the unintended negative consequences.
Several dimensions of the MLES program have been discussed in the psychology literature,
but it is generally centered on testing measures of and theories on the development of aggressiveness
during adolescence and primarily relies on scales designed to test one psychological construct at a
time. Other studies have examined the long-term effect of the MLES on secondary completion and
crime (Boisjoli et al. 2007, Vitaro et al. 2012). In this study we investigate the impact of the
intervention on a wide set of non-cognitive and cognitive skills at adolescence, using new data that
has not been previously presented or analyzed. We take a much broader approach to identify the
development of all potential cognitive and non-cognitive skills during adolescence rather than
restricting the analysis to aggressiveness. This approach opens the way to identifying new,
potentially important, skills, which are then systematically examined for their relationship to
academic achievement during adolescence and adult outcomes using the same framework. While we
do find some skills related to disruptiveness as addressed in previous papers, there are notable
differences, and we add several new skills: new data includes information on school performance and
grades, trust, altruism, self-esteem, friends, and parent behaviors. We also extend the analysis of
5
these diverse cognitive and non-cognitive skills as potential pathways for impacts on education,
crime, economic and social outcomes and provide suggestive evidence on the relative contribution of
several different skills.
2. The Montreal Longitudinal Experimental Study (MLES)
2.1 Experimental Design and Timeline
Figure 1 shows the timeline of the experiment and of data collection. Kindergarten teachers of 53
schools in low SES areas of Montreal, Canada, were asked to rate the behavior of their male students
at the end of the 1984 school year with the Social Behavior Questionnaire (Tremblay et al., 1987).
Almost all (87%) of the teachers provided ratings for a total of 1,161 boys. To create a homogenous
sample, only participants whose parents were Canadian-born with French as a first language and 14
years or less of schooling were included in the longitudinal study, which reduced the number to 1037
boys.
The disruptiveness scale of the Social Behavior Questionnaire was used to select the at risk
boys for the intervention. The scale measures the frequency of physical aggressions, oppositional
behavior and hyperactive behavior (Cronbach α = 0.93). Boys with a score above the 70th percentile
(N = 250) on this disruptiveness scale were considered to be at high risk of later antisocial behavior
(Vitaro et al., 2001).
These 250 participants were randomly assigned to a treatment (69 boys) and a control group
(181 boys) by drawing names from a box. The control group was initially divided into two groups, a
control group with contact only for yearly assessments and a control group with intensive
assessments (at home, at school and in the laboratory). This second control group was used for
detailed studies of family interactions, peer interactions and bio-psycho-social development, and in
this way served to control for the intensive measurement attention the intervention group received (it
is possible that studies with intensive measurement show impact due not to the intervention but to the
measurement itself). The two control groups showed no difference in outcomes and so the two
groups were collapsed into one control group.
2.2 Validity of the experiment design and baseline controls
The validity of the causal inference from the MLES rests on the quality of the randomization.
6
To test for successful randomization, we carry out a balance check. Table 1 shows the baseline
values of the two groups for several critical variables measured prior to randomization. Among the
32 baseline variables, there are significant differences on 4 variables: initial anxiety measures, age of
father at birth of subject, prestige of the mother’s employment (all at the 10% level), and number of
sisters (at the 1% level). The fact that there are some differences does not indicate that the selection
process was non-random. It is not surprising to find imbalances for a handful of variables, given that
the sample size is small and 32 variables were tested for differences and so we do not fear that the
randomization protocol was violated. However, since these variables might impact the mechanisms
and outcomes we wish to examine, we control for them in the analysis that follows. We have
imputed certain baseline measures that are necessary to eliminate an imbalance between the groups
but omit observations when the intermediate or outcome variable of interest is missing. Father’s age,
prestige of the mother’s work, and number of sisters were imputed using all other available baseline
information. We include a dummy equal to one when the value is imputed. Note that this imputation
is limited to the baseline variables, and not to outcome variables.
Compliance was not complete. Some families (78 out of 250) from both the treatment and the
control groups refused to participate in some elements of the study, particularly in the elements
involving parent participation, but were included in the longitudinal data collection. The rate of nonparticipation was the same across groups. These participants are included in the analysis as belonging
to their initially assigned treatment groups (intention-to-treat analysis). Of those assigned to the
treatment group, 67% agreed to participate. Table 2 shows the difference between compliers and
non-compliers on a number of baseline variables. Differences are significant at the 10% level or
higher for 4 out of 21 variables tested: Prestige of mother’s job (compliers mothers have lower
prestige jobs, about 0.5 SD), age of mother (compliers have older mothers, about 0.5 SD), initial
aggression (compliers have lower aggression, about 0.15 SD) and initial fighting (compliers have
lower fighting, about 0.6 SD). We should not expect compliers and non-compliers to be similar –
since the treatment required parent cooperation and some investment of parent time, it is not
surprising that some parent characteristics are different between the two groups. In part because there
are differences on observables, our preferred specification uses Intention to Treat (ITT) estimation,
so that all subjects assigned to the treatment group are considered treated for the analysis, and we
provide Treatment on Treated (TOT) estimates in for completeness.
7
Table 3 presents rates of attrition for the different outcome variables, and Table 4 presents the
difference in baseline characteristics for attriters and non-attriters.
Attrition is significantly
negatively related to parent employment and parent education, and significantly positively related to
family adversity and mother’s age. This suggests that there may be selective attrition, where subjects
from more disadvantaged families are more likely to attrit. We also examine the relationship of
attrition to interactions of treatment and baseline variables, and find significant coefficients for 2 out
of 25 interactions tested (father’s initial employment status and initial hyperactivity.
Table 5
presents the results of the tests of the relationship of attrition to baseline characteristics with respect
to treatment status. Note that the size and direction of the significant coefficients imply that treatment
subjects whose father worked in 1984 were more likely to have attritted than control subjects whose
father worked, and treatment subjects who had higher levels of initial hyperactivity were less likely
to have attritted than control subjects with higher levels of initial hyperactivity. We include these
variables as controls in all regressions using the adult data.
2.3 Intervention Program
The intervention program was implemented over a 2-year period, from ages 7 to 9 years
(Grades 2 and 3). The main aspect of the intervention consisted of direct training on social abilities
and self-control to children (Supplementary Materials section A provides additional detailed
information on the program and contents of the training sessions). The experiment drew on
randomized and non-randomized studies of children on emotional regulation and impulse control,
social-information processing and how to interpret other’s intent (Cartledge and Milburn, 1980;
Kettlewell and Kausch, 1983; Michelson et al., 1983; Schneider and Byrne, 1987; Weiss et al., 1992;
Dodge 2003 and 2007). The training sessions were conducted at school (outside the classroom), in
groups of four to seven children, of which one or two would be the treatment participants, and the
rest would be boys identified by their teachers as highly pro-social.2 This arrangement was intended
to provide positive role models for the treatment participants and also avoid stigmatizing the
treatment participants. The sessions were held once a week for 45 minutes, during lunch or after
school.
During the first year, nine sessions of social behavior training were offered. Sessions included
2
An interesting research experiment would have been to examine the impact of participating in these groups on the prosocial boys. Unfortunately, we have no follow up data on them and so this is not possible.
8
how to invite a bystander to play, how to ask “why”, how to give a compliment, and how to help.
The second year included 10 sessions of problem solving and self-control strategies (Camp et al.,
1977; Goldstein et al., 1980). Some stimulus situations for these sessions were how to react to
teasing, how to react when angry, and what to do if other children refuse to play with you. For each
situation, the children reviewed ways to define the problem, identified the intentions of the other
person, analyzed their feelings if they were in the role of the victim, suggested different action plans
to solve the problem, anticipated their consequences, selected one action plan and, finally, reinforced
themselves for their work. Verbal instructions, coaching, modeling, behavior rehearsal, and positive
(verbal and material) reinforcement were all used. Children were encouraged to use their newly
learned skills before the next training session. At the following meeting, the children were
congratulated for having performed their new skills in the interim. Teachers and parents were
informed through one-page letters of the new skills learned by the children during each session. They
were encouraged to praise the child for using these new skills as often as possible.
For example, one session covered Self Control. The facilitator introduced the topic, and
talked about situations where children are upset and might want to make an angry outburst, like a
spilled glass of milk or a disappointment. The facilitator then modeled a situation: he has been
playing tag, and he just got tagged and is now out. He’s upset because he is the first person to be
tagged out, and he’s angry and disappointed. The facilitator demonstrates how children can respond
in this situation: noticing clues in his body that he is going to lose control (clenching fists, feeling
hot), he thinks about what happened to make him feel this way (he got tagged first, is worried other
kids will laugh at him), he chooses a way to avoid making an angry outburst (count to ten, move
away, say to himself “calm down”, breathe), and then he acts and praises himself. The facilitator then
invites children to perform additional role-plays based at school (one child bumps another’s desk and
their pen falls down), at home (someone suddenly turns off the TV because it’s time for dinner) or
while playing (a friend takes a ball that was dropped). Together, the group makes observations about
what the actors are doing, how they are following the steps, and gives feedback. At the end of the
session, the facilitator fills out a workbook with the children to explain how they can practice selfcontrol until the next session ("homework”).
Two university-trained child-care workers, one a psychologist and one a social worker, all
working full time, carried out the training and support activities. The team was coordinated by a fifth
professional who worked on the project half time.
9
2.4 Data
Following the intervention, the MLES collected detailed longitudinal data on development
during adolescence at ages 10-17, including self-reported behavior, grades, relationship with parents,
and teacher-reported behavior, and later outcomes at ages 17-27, including educational achievement,
crime, employment and social capital from both administrative data and self-reported surveys. The
surveys took place around age 20-21 and 26-27. The MLES also collected the same longitudinal data
for the boys who were not identified as disruptive during the kindergarten year (N=787). We use data
from both groups as the comparison between the experimental group and the non-disruptive group
helps understand the size of the impact by estimating to what extent the intervention helped the
disruptive boys “catch up” to the non-disruptive boys.
2.4.1 Data on Cognitive and Non-Cognitive Skills during Adolescence
Measures relative to different skills in adolescence were collected annually from ages 10 and
17 years. While previous studies on this sample have examined measures of disruptiveness and have
found that participants are on different trajectories of disruptive behavior (Tremblay et al., 1991,
Vitaro et al. 2012), we pursue a more agnostic and comprehensive approach in this study. We
investigate the initial and latter impact of the intervention on all the observed non-cognitive and
cognitive skills at adolescence by exploiting all the measures for which we have balanced
observation every year from ages 10 to 17. We break this period into two and analyze data that are
available from both periods: early adolescence (ages 10 to 13, or 1988-1991), and late adolescence
(ages 14 to 17, or 1992-1995). The year 1992 was chosen as the break year because it is in this year
that the treatment and control groups begin to diverge in whether or not they have repeated a year in
school, and repeating a grade (being “held back”) has a large correlation with later life outcomes, as
discussed below.
Our identification of skills is based on exploratory factor analysis. Combining all data
available and averaging over the years available, we use factor analysis to examine how the factors
combine into groups potentially measuring the same latent variable. We include the original
questions that were used to identify the disruptive sub-sample, and questions from several wellknown psychological inventories (Jesness and Wedge, 1983; Kovacs, 1983; March, 1990;
Rosenberg, 1965; Lacourse et al., 2002; Tremblay et al, 1992). We use individual questions from the
10
different measuring instruments rather than the original scales themselves. This approach allows for
the possibility that individual questions might cluster together effectively and also allows subjectreported and teacher-reported data to be used together when possible. While we acknowledge that
there are likely to be discrepancies between teacher and student reports, we still expect them to be
related when they are measuring the same trait and, moreover, to provide different points of data
collection that cannot be obtained from the same informant (that is, the subjects can tell us how they
feel and about hidden behaviors, while the teachers can tell us how they behave in the class). If this
is not the case – and there is evidence of discrepancies between the teacher and student reported data
- this should prevent us from finding sensible groupings of variables from these two sources. In fact,
all of the skills that we identify, except for altruism, include both teacher and student reported
variables that have sufficiently high alphas when grouped together.
We identify two skills that deal with self-control, based largely on the behavioral dimensions
used to identify the disruptive sub-sample in kindergarten: Aggression Control, that is, control over
aggressive behavior towards persons or towards property (such as fighting, bullying, and destroying
objects), and Attention-Impulse Control, that is, control over impulsive behavior in tasks that require
self-control (sitting still, remaining on task, focusing). We also identify four additional skills: Trust,
Friends, Altruism and Self-Esteem. Trust includes trust in others and also the ability of perspective
taking, that is, to understand the intentions of another person. Our measure of trust is thus more
subtle than a definition where someone may be “trusting” if they simply tend to believe what other
people tell them, it is more closely related to someone’s attitude toward the intentions of institutions
and other people (in particular, people who are outside of their immediate social circle). Friends
measures the closeness of relationships friends, Altruism measures voluntary altruistic and
compassionate behavior, and Self-Esteem measures feelings of value and self-worth. There is also a
measure of social capital, Group Membership, which is the only non-cognitive skill that is measured
by a single variable (whether or not the individual belong to a social group at ages 16 or 17).
For cognitive skills and school performance, verbal IQ was tested when the subjects were
around 13 years old using the Sentence Completion Test, and we have data on yearly grades in Math
and French, as well as whether the subject had been held back or was in special education class each
year.
Figure 2 shows the distributions for the treatment and control groups, as well as the nondisruptive sub-sample of the non-cognitive skills, IQ and school performance. Full details for each
11
skill, including all distributions, summary statistics, the number of components, the Cronbach alpha
index for internal consistency, and the mean and number of observations for each sub-sample are
given in Table 6. Details about the component variables and factor loadings are given in the
Supplementary Materials Section B.
We also measure, but find no impact on, various aspects of parent behavior, reported in
Supplementary Materials Section D.
2.4.2 Adult data
The adult outcome variables are measured with administrative data and questionnaires. We
use self-reported data on activities from ages 17 to 27 to construct a variable of the percent of
reported years from 17-27 where the subject was active full time by either school or work or both. To
supplement the data on fulltime activity, we also provide estimates on whether the subject was
employed at age 27, employed fulltime at age 27, their wages3, and the percent of years from ages
17-27 during which the subject reported receiving government transfers, and the percent of years
from ages 17-27 that the subject was inactive. We also measure the hours worked at age 27 which we
use as one measure of cost-effectiveness in section 5. We use self-reported data on whether or not the
subject reported belonging to a civic or social group at age 21 or age 27. We use administrative
information on whether each subject had received a secondary-school degree, the number of criminal
offenses and having a record for a violent or non-violent offense, collected when the participants
were around 23-24 years old. Summary statistics on adult data are given in Table 6.
3. Impact of the Program
3.1 Specifications
Figures 2 and 3 show the raw differences in selected outcomes, and Tables 7-10 show the
results from different specifications for selected outcomes. Supplementary Materials Section C
3
For the unemployed, we estimate replacement wages by using data from the entire sample on wages, baseline family
and behavioral characteristics, previous employment and secondary completion to impute wage levels, then multiplying
by the historical replacement rate. If we do not impute hourly wages, we have two choices: count the unemployed as
receiving zero, which would lead to a massive treatment effect, or drop the unemployed, which would reduce our sample
substantially. However, because of this imputation, we do not focus on hourly wages as a principal outcome.
12
provides details on the specifications and detailed results for all outcomes. We use Intention to Treat
(ITT) estimation, so that all subjects assigned to the treatment group are considered treated for the
analysis whether or not they actually participated, which we believe to be the correct specification in
this case, particularly as, first, the compliers are different from the non-compliers on observables,
and, second, ITT estimates are potentially more relevant for public policy evaluation, though we
provide Treatment on Treated (TOT) estimates for completeness in the Supplementary Materials
Section C. 4
We use several specifications to estimate the total impact of treatment on cognitive and noncognitive skills. We provide the p-value of the raw difference in means between the treatment and
control group for each skill (Column (2) of Tables 7-10). We also conduct a series of OLS
regressions. In Tables 7 and 8, column (3) reports the results from a simple OLS regression of
treatment on the skill with no controls on the disruptive sub-sample, and column (4) reports the same
regression as in column (3) but including, as controls, any variable that is imbalanced between
treatment and control groups. In Table 9 and 10, column (3) reports logit estimates with controls as
appropriate, column (4) reports the unconditional treatment effect from OLS, and column (5) reports
the conditional treatment effect. The remaining columns provide results from the robustness checks
(described in Section 4, below).
3.2 Impact of the Treatment in Early Adolescence
Table 7 shows the treatment impact in early adolescence and Figures 2A-2J show the
differences in the distributions. We find that the treatment has a significant impact on non-cognitive
skills: Externalized Self Control is higher in the treatment group (0.15 standard deviations,
significant at the 10% level, about 36% of the difference between the disruptive and non-disruptive
sub-samples), Internalized Self-Control (0.19 standard deviations, significant at the 5% level, about
4
While the TOT estimates are large, we do not think that they are worrisome. First, we are confident that the differences
between the treatment and the control groups are due to the program, as we have no reason to suspect that the
randomization was compromised. In addition, we find no impact on self-esteem, altruism, friends, or parent behavior,
but only on those skills likely to be enhanced by the program based on the program design. The program may have had
such a large impact because it was targeted to those children who were at the tail of the distribution in terms of
disruptiveness, and who thus had the most to gain from changes in behavior. It may also be that the program was
particularly well monitored and executed (as is often the case with trials and pilot projects), which means that, as with the
results of all trials, scaling up should happen incrementally and may not have exactly the same impact as the trial. Note
that of treatment subjects with data at age 27 (25 total), 17 were compliers and 8 were non-compliers. Of treatment
subjects with data on the combined outcomes at age 21 and 27 (39 total), 27 were compliers and 12 non-compliers.
13
57% of the difference between the disruptive and non-disruptive sub-samples) is higher, and Trust is
higher (0.17 standard deviations, significant at the 5% level, about 62% of the difference between
disruptive and non-disruptive).
There is no impact on the other behavioral factors in early adolescence: Friendship, SelfEsteem, and Altruism, and no impact on IQ scores, grades, being held back, or being placed in a
Special Education Class. This is in line with what might be expected, given the nature of the
program: a focus on social abilities and self-control. In terms of narrowing the gap with the nondisruptive group, the treatment was more effective for Attention-Impulse Control and Trust.
3.3 Impact of the Treatment in Late Adolescence
Table 8 reports the treatment impact in late adolescence and Figures 2A-2J show the
differences in the distributions. Treatment has a significant impact on Externalized Self Control (0.18
standard deviations, significant at the 5% level, or 61% of the gap between the disruptive and nondisruptive sub-samples) and Trust (0.18 standard deviations, significant at the 5% level, or 68% of
the gap between disruptive and non-disruptive sub-samples). We find no impact on Internalized Self
Control in late adolescence.
In contrast to the early adolescent period, we observe a large impact on school performance in
the late adolescent period: Grades (about 0.30 standard deviations, or 74% of the gap between
disruptive and non-disruptive sub-samples), percent of years Held Back (14 percentage points, 65%
of the gap between disruptive and non-disruptive sub-samples) and percent of years in Special
Education (14 percentage points, or 65% of the gap between disruptive and non-disruptive samples)
(all significant at the 5% level). In addition, there is an increase in social group membership (10
percentage points, significant at the 5% level). The impact on social group membership is very large
relative to the gap between the disruptive and the non-disruptive sub-samples, over five times as
large, but this is because the gap itself is quite small. Indeed, overall group membership is low.
3.4 Impact of the Treatment on Adult Outcomes
Table 9 shows the treatment impact on crime and secondary completion and Figure 3 shows
the difference between the means of each group. The treatment group committed 1.1 fewer crimes
14
per person, significant at the 10% level, bringing the treatment group about 80% of the way “back”
to the non-disruptive sub-sample. The impact is largely driven by reductions in nonviolent crimes
(for example, drug offenses) rather than violent crimes. Participants in the treatment group are 18
percentage points more likely to have received a secondary school diploma than participants in the
control group, significant at the 1% level (note that only 32% percent of control group participants
completed secondary school). Our specification shows that the treatment reduced the gap between
the non-disruptive and disruptive sub-samples by 80%.
We also find a significant impact on economic performance and social capital, as shown in
Table 10, and again in Figure 3. Treatment participants were active fulltime in work or school
between 8 and 12 percentage points more than the control group (the control group average is 80%),
significant at the 5% level. We also observe significant effects on percent of years 17-26 working,
and employment, full-time employment, number of hours worked and hourly wages at age 26. There
is no significant impact on the percent of government transfers, though the point estimates are in the
expected direction. Treatment group members were 22 percentage points more likely to belong to a
social group, from 32% in the control group, significant at the 5% level. Both the impacts on percent
of years when subjects were active fulltime and on social capital are large and greater than the size of
the gap between the disruptive and non-disruptive groups. (Recall that the impact on social group
membership in adolescence also exceeded the gap between the disruptive and non-disruptive subsamples).
4. Robustness checks on the treatment impact
4.1 Clustering and permutation
There were 53 schools in 1984, with an average of 20 total participants, 2 treatment
participants and 4 control participants, per school. Since randomization was carried out at the
individual level (within schools), and control and treatment participants are present in each school,
neither fixed effects nor clustering are required, but we present these results as robustness checks.
For these specifications, we use the full sample of subjects (disruptive and non-disruptive, where the
disruptive subjects are those who were randomized into control and treatment groups) because there
are very few treatment or control subjects in each school. We do this by including a dummy variable
representing disruptive or non-disruptive, and a dummy variable for treatment (the coefficient of
which would be identical to the coefficient on treatment in a specification including only the
15
experimental group, but in practice varies slightly due to different coefficients estimated on the
covariates). We present a specification that includes clustered standard errors at the level of the
school in 1984, and a specification that includes fixed effects at the level of the school in 1984. The
results are reported in Columns (5) and (6) of Tables 7 and 8 for adolescent outcomes and Columns
(6) and (7) – of Tables 9 and 10 for adult outcomes. We do not find any substantial differences in the
specification with clustering and school fixed effects.
Since the sample size is small, we also include the p-value from a permutation
(randomization) test of the difference in means (where treatment group assignment was randomly
permuted within the sample) with 2000 draws. That is, treatment group assignment was randomly reassigned 2000 times, and the simple difference in means (or proportions) was calculated for each
draw. The p-value is the proportion of draws that have a difference in means as large (in absolute
value) as the difference observed in the true sample. The results are reported in Column (8) of tables
7 and 8, and Column (10) of Tables 9 and 10 for adolescent and adult outcomes respectively.
4.2 Attrition
Attrition is marginal for observations on cognitive and non-cognitive skills during
adolescence, and for secondary-school completion and criminal records that are based on
administrative data. But is a potentially important issue for the economic and social adult outcomes,
with a rate of attrition of 38 percent for our main variables of interest percentage of time full-time
active and group membership. As discussed above, Table 4 provides a summary statistics for attrition
from each group.
We test in whether attrition for adult outcomes is related to 21 variables collected prior to the
program, and whether it is related to treatment conditional on those baseline variables. There are two
variables of potential concern: father’s initial work status and hyperactivity (presented in Table 5).
We control for these variables in all regressions using adult data subject to attrition (that is, the
economic and social adult outcomes).
In order to get an idea of the possible direction and magnitude of any bias, we estimate the
bias introduced by the same level of attrition into the estimate of the program impact on secondary
school completion. Administrative data on secondary school completion is available for nearly the
entire sample (242 out of 250 participants, or 97%). We use the following procedure. We create a
false secondary school variable that takes the value of missing if the subject is missing data for our
16
principal economic outcome (percent of years active fulltime) and the value of the true secondary
school variable if they do have economic outcome data. In this way we mimic the level of attrition in
the adult questionnaire data in the administrative data (we call this the “attrited” dataset). We then
estimate the impact of the program on secondary school completion using our preferred specification.
We compare this to an estimate of the impact of the program using data from the entire sample (the
“true” dataset). The validity of this comparison rests on the assumption that the direction of the bias
induced in the secondary school completion is likely to be the same as the direction of the potential
bias in the fulltime occupation data. The results presented in Table 11 show that if we had observed
the same pattern of attrition for secondary completion as we do for percent active fulltime, we would
underestimate the impact. This test provides some reassurance that our results are not driven by
attrition bias.
We also use inverse probability weighting (IPW) as an additional check. IPW gives higher
weight to observations that are similar to those missing outcome data (see Campbell et al., 2014).
This procedure cannot recover the true distribution of outcomes except under strong assumptions, but
provides an additional robustness check. We use a logit specification with a dummy variable for
attrition as the dependent variable and all available baseline variables as the independent variables.
Using the coefficients from this estimate and the values of the baseline variables, we calculate the
likelihood that each subject will attrit regardless of actual attrition status. Participants are weighted
using the inverse of the likelihood of having data on the outcome, so that participants who are similar
to attriters have higher weight in the estimate of program impact. As with the specifications that
control for school level effects, we include the entire sample (both disruptive and non-disruptive).
We include the estimate using IPW for variables with attrition, and we do this both for the adolescent
period (where there is relatively low attrition) and the adult period. The results are reported in
Column (7) of Tables 7 and 8 for adolescent outcomes and Column (8) of Tables 9 and 10 for adult
outcomes. In general, the IPW estimates of impact are slightly larger than the unweighted estimates,
which is in line with the results of the falsification test, which suggest that if anything, attrition bias
is reducing the estimated treatment effect.
5. Which skills might be acting as mechanisms for which outcomes?
A natural question, given the results, is whether the skills on which we observe impacts are
acting as mechanisms or channels for the impacts on adult outcomes. One hint comes from the fact
17
that the noncognitive skills impacted by treatment (trust, externalized self-control and internalized
self-control) explain at least as much of the variance in Special Education status and being Held Back
as IQ, and substantially more of the variation in Grades, as shown in Figure 4.
There are several different strategies for examining the likely channels of impact. Heckman
et al (2013) carry out a formal decomposition of impact of the Perry Preschool Program, finding that
changes in non-cognitive skills explain the bulk of the impact of the program. In their strategy, the
latent skills were orthogonal to one another. In this paper, we are examining sub-sets of noncognitive skills, which are by their nature correlated (attention control and aggression control, for
example).
We instead follow Heller et al (2015) to provide evidence on likely channels. In that paper,
the authors examine several different potential explanations for program impact by multiplying the
treatment coefficient on the potential channel by the coefficient of the channel on the outcome in the
control group, and divide by the total impact, given an estimate of the percent of impact explained by
changes in each channel (in isolation).
They also provide upper and lower bounds for the
explanation of impact.
This exercise then provides as estimate of the percent of the impact that can be explained by
the changes in each skill if they were changed in isolation. In fact, we know that they were not
changed in isolation, as self-control and trust both increased for the treatment group. The usefulness
of this exercise is to show that the skills that explain the most of the impact for each different
outcome are those that we would expect, a priori, to act as mechanisms for each particular outcome.
In addition, they give support to the idea that changes in behavior at early ages impacts behavior and
outcomes in adolescence, which in turn have large impacts on adult outcomes. The results are
presented in Tables 12 and 13.
First, with respect to adolescent outcomes, we observe that changes in internalized selfcontrol (related to concentration) explain 37-40% of the impact on school outcome variables in late
adolescence (grades, held back, and special education). These variables, in turn play an important
role in explaining the impact on secondary completion (40-65%), criminality (24-46%), and, percent
of years working fulltime (around 30% for Special Education and Held Back, though the impact on
Grades explains very little of the impact on labor market outcomes).
Examining the extent to which the direct effect on behavior explains adult outcomes, we
observe that the impact on internalized self-control (related to concentration) in early adolescence
18
explains a substantial proportion of the impact on Secondary Completion (38%), the impact on
externalized self-control (related to delinquency) in late adolescence explains a large proportion of
the impact on criminality (62%), and the impact on trust in early and late adolescence explains a
moderate proportion of the impact on labor market outcomes (23%-24%). The impact on group
membership is not explained very well by any of the impacts on behavior, though the largest is the
impact on trust in late adolescence (14%).
This pattern fits well with our theoretical understanding of which types of behavior are
related to which adult outcome: attention and concentration in the classroom is related to school
outcomes, trust is related to labor market and social outcomes, criminal behavior is related to
delinquency and aggression, and both labor market and criminal behavior are related to adolescent
school outcomes which are themselves related to earlier attention and concentration.
6. Cost-effectiveness and Rate of Return
In order to provide policymakers with an idea of the value of programs like this one, we
provide simple estimates of the likely rate of return and cost-effectiveness of the intervention
program in this sample. We do not compare our results to any particular threshold, as we lack high
quality data on the costing of most of the benefits and are thus obliged to make assumptions, which
make conservatively, and thus our estimates are lower-end estimates.
6.1 Cost of the program
We estimate the cost of the program based on known staff costs, as the principle cost of the
program was the salary of the trainers and implementers, and no other particular inputs were used.
The implementation team was composed of one full time social worker, two full time childcare
specialists (BA level), one psychologist, and one half-time program administrator, full time over the
course of the program (two years). We do not include the cost of evaluation or questionnaires. We
use median reference hourly wages for these professions in Quebec in 2011 (from the Labor Force
survey), adjusted for inflation to 1985 (half a year), 1986 (full year), and 1987 (half a year). We
assume 40 hours per week paid for 52 weeks. We assume that other costs amount to 30% of salaries
(photocopies, transport, training, and so on). Under these assumptions, we calculate that the total
program cost per person (for two years) was around 9,500 in 2013 USD.
19
6.2 Cost Effectiveness
In order to understand how the benefits relate to the costs of the intervention, we first
calculate how much each increment of benefit “cost” under the intervention. For example, how much
did it cost, using this program, to avert one crime? This type of estimate of cost-effectiveness
measures the effectiveness of a program in terms of the cost of attaining a desired outcome. It does
not rely on monetizing the outcomes. It can be useful for making general comparisons between
programs that have similar policy goals (Dhaliwal et al. 2013). Table 14 shows the main results of
the cost-effectiveness for criminality, education and labor market outcomes.
6.2.1 Criminality
To calculate the cost-effectiveness of the program by crime averted, we use the coefficient
from column 5 of Table 9 which estimates the reduction in the number of crimes from age 18-25 (1.1
fewer crimes in the treatment group over that period). We make some conservative assumptions
about the persistence of this effect over time, taking into account that the rate of crime commission
overall decreases as people age. First, for simplicity, we assume that this reduction is spread evenly
over the seven years, so a reduction of about 0.16 crimes per person per year. We assume that this
difference fades out over time (see Sampson and Laub, 2003) at a rate of 10% per year after age 25
(so that at age 26, the treatment group commits 0.14 fewer crimes, at age 27, 0.13 fewer crimes, and
so on) and that by age 35 there is no difference between the treatment and the control group in terms
of crime commission. We then compare the cumulative number of crimes avoided to the cost (using
a 3% discount rate in both cases) at age 35 when there is assumed to be no remaining treatment
effect. Under these assumptions, we calculate that the cost of averting on crime is around 3,100 in
2013 USD.
6.2.2 Education
Using the number of additional graduations and the cost per person (both evaluated in 1996 when the
participants were 18 years old) from Column 5 of Table 9, we find that the cost of each additional
secondary graduation is around 29,300 in 2013 USD. We also calculate, for each year, the number of
years of special education and repeated class (held back) averted by the treatment each year (the
coefficient on the average for ages 14-17 is given in Table 9, regressions for individual years not
shown but available on request). The coefficient of treatment on Held Back at age 14 is 0.06, age 15
20
is 0.15, and age 16 and 17 is 0.20. The coefficient of treatment on Special Education at age 14 is 0.1,
at age 15 is 0.11, at age 16 is 0.17, and age 17 is 0.30. We again add the cumulative number of years
of repetition or Special Education avoided, and compare it to the cost of the program. Using this, we
estimate that the cost of each repeated year avoided is 8,600 in 2013 USD and the cost of each year
of special education avoided is 7,700 in 2013 USD.
6.2.3 Labor Market
We use the estimate of the treatment impact on hours of work per week at age 27 in Table 10. On
average, the treatment group reported 7 more hours of work per week, which, if people work for 50
weeks per year, is 350 additional hours of work per year. Assuming that people work until they are
60 years old (the minimum age to receive retirement pension in Quebec), people in the treatment
group will have worked an average of about 6 more years. Our base case assumes that the treatment
effect remains constant. Calculating the cumulative additional hours of work and comparing it to the
cost of the program, this implies a cost of each additional year of fulltime work of about 1,800 in
2013 USD.
We provide a sensitivity analysis for the assumptions underlying the duration of labor market
benefits in Table 15 using alternative estimates for the cost per additional year for fulltime work if
the treatment impact fades by 5%, 10%, and 20% per year (under the last set of assumptions, there is
very little treatment effect after age 40, about 18 minutes per week). The implied costs per additional
year of fulltime work under these scenarios are 3,400, 5,200 and 9,100 in 2013 USD.
6.3 Rate of return
Next, we calculate the overall rate of return based on a set of reasonable and conservative
assumptions, and show the proportion of return accounted for by each type of benefit. This estimate
gives a general idea of how good an investment this type of intervention is likely to be, compared to
other interventions with different policy goals (see Heckman et al., 2010).
Our calculation of the total monetary benefits of the program is a lower bound: it excludes
monetary benefits from increased hourly wage, because calculating the benefits in increased hourly
wage requires making assumptions about the replacement wage of the unemployed, increased high
school graduation, and reduced social cost of crime. To provide a lower bound on the monetary
benefits due to reduced crime we use the administrative costs of crime (cost of arrest, holding, court
21
time, and administration). This approach is justified, given that the treatment impact seems to be
driven by reductions in non-violent crime, so benefit from reduced costs to victims would be small.5
This estimate does not include the costs of policing, which would substantially increase the estimate
of the monetary benefit. Based on information from the Canada Department of Justice in 2008, these
costs were 2,164 per crime in 2013 USD (Department of Justice, 2008).
We monetize the benefits of reduced grade repetition and special education, based on the cost
avoided (additional years of schooling and additional costs of the services required for special
education). Based on figures from the Ministry of Education in Quebec, the cost of education per
student per year (representing the cost of being held back and additional year) is 8,305 in 2013 USD
(Ministère de l’Éducation, du Loisir et du Sport, 2014) and special education classes cost an
additional 30%, or 2,492 in 2013 USD.
We use the increase in working hours to give a lower bound estimate of the effect on income. It
is a lower bound, because we assume that during those additional hours, workers make only the
minimum wage (this is almost certainly not the case, but we have no way of reliably measuring the
hourly wage for this particular population from the dataset). Our main specification assumes that the
difference in hours between the treatment and the control group remains constant, but we provide
alternative estimates based on the difference decreasing by 5% and 10% per year. We assume people
work until age 60 (which is the earliest age at which one can begin receiving a retirement pension in
Quebec). For ages 27-36 (2004-2014), the minimum wage for each year is obtained from the Quebec
Commission on Labor Standards, and from age 36 onwards we assume that the minimum wage
increases at the same average rate as it did from 2004-2014 (3.3% per year).
Under these assumptions, we calculate that each dollar invested yields about 14 dollars in net
benefits over 53 years. An equivalent compound interest rate, taking into account the 53 years
between investment at age 7 (the intervention) and the full benefits accrued at age 60, is around 5%.
A sensitivity analysis to assumptions about the treatment effects is given in Table 15. When the
treatment effect on hours worked declines by 5%, 10% and 15% per year, the respective rates of
return are 7, 4, and 2.5, and the respective interest rates are 4%, 3%, and 2%. In all cases, the return
is positive and respectably high for a social program, suggesting that interventions based on
5
Monetary benefits due to reduced criminal behavior are difficult to estimate, as they include not only the cost of
enforcement and, if applicable, incarceration, but also the cost to society of the crime committed. Different strategies to
monetize the cost of crime are using jury awards as an estimate of the monetary damage due to different types of crimes
(Heller et al., 2015), or the value of a statistical life (for murders) and victim assault costs (Heckman et al., 2010). Our
data do not include the type of crime committed, so we cannot do this type of analysis.
22
increasing non-cognitive skills, even after early childhood, may be very prudent investments for
society. If, in the most extreme case, labor market returns are zero, the intervention still breaks even.
In all cases, except when there are no returns on the labor market, the bulk of the returns are due to
increases in the number of hours worked.
Conclusion
We find that a non-cognitive skills training program conducted in inner-city Montreal in the
1980s with disruptive boys led to substantial improvements in non-cognitive skills in early and late
adolescence, improvements in school performance in late adolescence, and long-term positive effects
early adult outcomes. We use a rich longitudinal dataset to examine cognitive and non-cognitive
skills and find that the program changed levels of trust and self-control (both in terms of controlling
aggression and controlling attention and impulses) in adolescence, and we provide suggestive
evidence that these non-cognitive skills are the mechanisms by which adult outcomes are improved.
An important result from this paper is the evidence that self-control is critically important for
academic achievement as an adolescent, and directly linked to young adult outcomes. Our findings
also highlight the role of trust as an important factor for explaining academic achievement and adult
outcomes. A growing literature shows an important association between social capital and trust and
community or country-level outcome like income per capita (Banfield, 1958, Putnam, 2000, Algan
and Cahuc, 2014). A recent study shows that inter-generational mobility is highly correlated with
social capital at the local level, measured by voter turn-out or civic associations in US districts
(Chetty et al, 2014). At the same time, relatively little research attention has been paid to the causal
impact of trust on individual success (as compared to self-control) and the mechanisms behind the
relationship between trust and aggregate outcomes are poorly understood. We present evidence at the
individual level on this mechanism: higher levels of trust help people function better in secondary
school and on the job market, controlling for other cognitive and non-cognitive skills. Our evidence
on the important role of trust suggests that new investment should be made in refining the measure of
trust in others and perspective taking, and how to build those social skills in early childhood.
The results reported in this paper demonstrate that increased investment in early childhood
development programs at school entry for disruptive boys from low socioeconomic environments is
likely to be an efficient and profitable public policy, especially where such programs explicitly
23
incorporate simple strategies to foster the development of social skills. Outside of the public interest
in the welfare of these children, such investments are likely to be particularly prudent in terms of cost
to the education system, the welfare system, the juvenile justice system and the criminal justice
system.
24
References
Y. Algan, P. Cahuc, Trust, Growth and Well-Being: New evidence and policy implication.
Handbook of Economic Growth, vol.1, eds S. Durlauf and P. Aghion. Amsterdam: Elsevier (2014).
M. Almlund, A. Duckworth, J. J. Heckman, T. Kautz, Personality psychology and economics. In E.
A. Hanushek, S. Machin, L. Woessmann (Eds.), Handbook of the Economics of Education, Volume
4, 1-181. Amsterdam: Elsevier. (2011).
D. Almond, J. Currie, Human Capital Development Before Age Five. Handbook of Labor
Economics, Volume 4, Elsvier (2010).
E. Banfield, The Moral Basis of a Backward Society. New York: Free Press (1958).
K. L. Bierman, J. Coie, K. Dodge, M. Greenberg, J. Lochman, R. McMohan, E. Pinderhughes,
School Outcomes of Aggressive!Disruptive Children: Prediction From Kindergarten Risk Factors
and Impact of the Fast Track Prevention Program. Aggressive behavior, 39(2), 114-130 (2013).
R. Boisjoli, F. Vitaro, E. Lacourse, E. D. Barker, R. Tremblay, Impact and clinical significance of a
preventive intervention for disruptive boys: A 15-year follow-up. British Journal of Psychiatry, 191,
415-419 (2007).
L. Borghans, A. L. Duckworth, J. J. Heckman, B. ter Weel. The economics and psychology of
personality traits. Journal of Human Resources 43(4), 972-1059 (2008).
S. Bowles, A (Science-Based) Poor Kids’ manifesto, (Review of Heckman James, Giving Kids a
Fair Chance: A strategy that works, Cambridge MIT), Science, 340, 1044-1045 (2013).
F. Campbell, G. Conti, J. Heckman, S. Moon, R. Pinto, E. Pungello, Y. Pan, Early childhood
investments substantially boost adult health. Science, 343, 1478 (2014).
F. Campbell, C. T. Ramey, E. Pungello, J. Sparling S. Miller-Johnson, Early childhood education:
Young adult outcomes for the Abecedarian project. Applied Development Science, 6, 42-57 (2002).
Canada Department of Justice (2008), Costs of Crime in Canada, 2008, available at
http://www.justice.gc.ca/eng/rp-pr/csj-sjc/crime/rr10_5/a.html.
N. Castellanos-Ryan, J. Séguin, F. Vitaro, S. Parent, R. E. Tremblay, A multimodal intervention for
disruptive kindergarten children reduces substance-use across adolescence: a randomized control
trial. British Journal of Psychiatry, forthcoming (2013).
R. Chetty, J. Friedman, N. Hilger, E. Saez, D. W. Schanzenbach, D. Yagan, How does your
kindergarten classroom affect your earnings? Evidence from project Star. Quarterly Journal of
Economics 126(4) (2011).
R. Chetty, N. Hendren, P. Kline, E. Saez, Where is the land of opportunity? The geography of
intergenerational mobility in the United States. NBER Working Paper, 19843 (2014).
F. Cunha, J. J. Heckman, Formulating, identifying and estimating the technology of cognitive and
non-cognitive skill formation. Journal of Human Resources 43(4), 738-782 (2008).
25
F. Cunha, J. J. Heckman, S. M. Schennach, Estimating the technology of cognitive and noncognitive
skill formation. Econometrica 78(3), 883-931 (2010).
J. Currie, T. Duncan, Does Head Start make a difference? American Economic Review, 85, 341-364
(1995).
I. Dhaliwal, E. Duflo, R. Glennerster, C. Tulloch, Comparative cost-effectiveness analysis to inform
policy in developing countries: a general framework with applications for education. Cambridge,
MA: MIT Press (2013).
R. Diamond, W. S. Barnett, J. Thomas, S. Munro, Preschool program improves cognitive control.
Science 318, 1387–1388 (2007).
A. L. Duckworth, The significance of self-control. Proceedings of the National Academy of
Sciences, 108(7), 2639-2640, 2011.
A. L. Duckworth, M. E. P. Seligman, Self-discipline outdoes IQ in predicting academic performance
of adolescents. Psychol Science 16, 939–944 (2005).
A. L. Duckworth, D. Weir, E. Tsukayama, D. Kwok, Who does well in life? Conscientious adults
excel in both objective and subjective success. Frontiers in psychology, 3 (2012).
J. A. Durlak, R. P. Weissberg, A. B. Dymnicki, R. D. Taylor, K. B. Schellinger, The impact of
enhancing students' social and emotional learning: A meta-analysis of school-based universal
interventions. Child Development 82 (1), 405-432. (2011).
P. Gertler, J. Heckman, R. Pinto, A. Zanolini, C. Vermeersch, S. Walker, S. Chang, S., GranthamMcGregor, Labor market returns to early childhood stimulation: A 20-year follow up to an
experimental intervention in Jamaica. NBER Working Paper 19185 (2013).
J. D. Hawkins, E. Von Cleve, R. F. Catalano Jr., Reducing early childhood aggression: Results of a
primary prevention program. Journal of the American Academy of Child & Adolescent Psychiatry
30(2), 208-217 (1991).
J. D. Hawkins, R. F. Catalano, R. Kosterman, R. D. Abbott, K. G. Hill, Preventing adolescent
health-risk behaviors by strengthening protection during childhood. Archives of Pediatrics and
Adolescent Medicine 153(3), 226-234 (1999).
J. D. Hawkins, R. Kosterman, R. F. Catalano, K. G. Hill, R. D. Abbott, Promoting positive adult
functioning through social development intervention in childhood. Archives of Pediatrics and
Adolescent Medicine 159(1), 25-31 (2005).
J. D. Hawkins, R. Kosterman, R. F. Catalano, K. G. Hill, R. D. Abbott, Effect of social development
intervention in childhood 15 years later. Archives of Pediatrics and Adolescent Medicine 162(12),
1133-1141 (2008).
J. J. Heckman, Skill formation and the economics of investing in disadvantaged children. Science
312, 1900-1902 (2006).
J. J. Heckman, T. Kautz, Fostering and Measuring Skills: Interventions that improve Character and
26
Cognition, IZA working paper (2013).
J. J. Heckman, L. Malofeeva, R. Pinto, P. Savelyev, Understanding the mechanisms through which
an influential early childhood program boosted adult outcomes. American Economic Review, 103(6),
2052-86 (2013).
J. J. Heckman, S. H. Moon, R. Pinto, P. A. Savelyev, A. Q. Yavitz, Analyzing social experiments as
implemented: A reexamination of the evidence from the High Scope Perry Preschool. Quantitative
Economics 1(1), 1-46 (2010).
J. J. Heckman, S. H. Moon, R. Pinto, P. A. Savelyev, A. Yavitz, The rate of return to the HighScope
Perry Preschool Program. Journal of Public Economics 94(1), 114-128 (2010).
S.B. Heller, A.K. Shah, J. Guryan, J. Ludwig, S. Mullainathan & H. A. Pollack, Thinking, Fast and
Slow? Some Field Experiments to Reduce Crime and Dropout in Chicago. NBER Working Paper
21178 (2015).
K. S. Howard, J. Brooks-Gunn, The role of home-visiting programs in preventing child abuse and
neglect. The Future of Children, 19(2), 119-146 (2009).
E. Lacourse, S. Cote, D. S. Nagin, F. Vitaro, M. Brendgen, R. E. Tremblay, A longitudinalexperimental approach to testing theories of antisocial behavior development. Development and
Psychopathology 14, 909-924 (2002).
I. Lorge, R. A. Thorndike. The Lorge-Thorndike Intelligence Tests. New York, NY: Houghton
Mifflin (1950).
J. Ludwig, D.L. Miller, Does Head Start improve children’s life chances? Evidence from a
regression-discontinuity design. Quarterly Journal of Economics 122(1), 159-208 (2007).
J. McCord, W. McCord. A follow-up report on the Cambridge-Somerville Youth Study. Annals of
the American Academy of Political and Social Science, 322(1), 89-96 (1959).
Ministère de l’Éducation, du Loisir et du Sport (2014), Indicateurs de Gestion 2012-2013 :
Commissions Scolaires, Montreal : Government of Quebec.
T. E. Moffitt, L. Arseneault, D. Belsky, N. Dickson, R. J. Hancox, H. Harrington, R. Houts, R.
Poulton, B. Roberts, S. Ross, M. Sears, W. Thomspon, A. Caspi, A gradient of childhood selfcontrol predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences,
108(7), 2693-2698. (2011).
J. B. Pingault, S. M. Côté, E. Lacourse, C. Galéra, F. Vitaro, R. E. Tremblay, Childhood
hyperactivity, physical aggression and criminality: a 19-year prospective population-based study.
PloS one, 8(5), e62594 (2013).
J. B. Pingault, S. M. Côté, F. Vitaro, B. Falissard, C. Genolini, R. E. Tremblay, The developmental
course of childhood inattention symptoms uniquely predicts educational attainment: A 16-year
longitudinal study. Psychiatry Research 219(3), (2014).
J. B. Pingault, R. E. Tremblay, F. Vitaro, R. Carbonneau, C. Genolini, B. Falissard, S. M. Côté,
27
Childhood trajectories of inattention and hyperactivity and prediction of educational attainment in
early adulthood: a 16-year longitudinal population-based study. American Journal of Psychiatry,
168(11), 1164-1170 (2011).
R. Putnam Bowling Alone: The Collapse and Revival of American Community. New York: Simon &
Schuster (2000).
R. J. Sampson, J. H. Laub, Life!Course Desisters? Trajectories Of Crime Among Delinquent Boys
Followed To Age 70. Criminology, 41(3), 555-592 (2003).
Statistics Canada, Labour Force Survey (2011).
R. E. Tremblay, The development of aggressive behavior during childhood: what have we learned
during the past century. International Journal of Behavioral Development, 24, 129-141 (2000).
R. E. Tremblay, E. McCord, J. Boileau, P. Charlebois, C. Gagnon, M. LeBlanc, Can disruptive boys
be helped to become competent? Psychiatry, 54(2), 148-161 (1991).
R. E. Tremblay, L. Pagani-Kurtz, L. C. Mâsse, F. Vitaro, R. O. Pihl, A bimodal preventive
intervention for disruptive kindergarten boys: its impact through mid-adolescence. Journal of
consulting and clinical psychology, 63(4), 560 (1995).
R. E. Tremblay, F. Vitaro, L. Bertrand, M. Le Blanc, H. Beauchesne, H. Boileau, Parent and child
training to prevent early onset of delinquency: The Montreal longitudinal experimental study. In J.
McCord, R. E. Tremblay (Eds.), Preventing antisocial behavior: Intervention from birth through
adolescence (Guilford Press, New York, 1992), 117-138.
F. Vitaro, E. Barker, M. Brendgen, R. E. Tremblay, Pathways explaining the reduction of adult
criminal behaviour by a randomized preventative intervention for disruptive kindergarten children.
The Journal of Child Psychology and Psychiatry 53(7), 748-756 (2012).
F. Vitaro, M. Brendgen, S. Larose, R. E. Tremblay, Kindergarten Disruptive Behaviors, Protective
Factors, and Educational Achievement by Early Adulthood. Journal of educational psychology,
97(4), 617 (2005).
F. Vitaro, M. Brendgen, R. E. Tremblay. Early predictors of high school completion: The
developmental interplay between behavior, motivation, and academic performance. In M. Boivin, K.
Bierman (Eds.), Promoting school readiness and early learning: implications of developmental
research for practice (15-45, ch. 2). New York, NY: Guilford Press (2014).
F. Vitaro, D. Larocque, M. Janosz, R. E. Tremblay, Negative social experiences and dropping out of
school. Educational Psychology 21, 401-415 (2001).
28
Tables and Figures
Figure 1. Timeline of the experimental study
29
Figures 2A- 2J: Skills in Adolescence
Figure 2A. Aggression Control
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.04 for ages 10-13 and 0.01 for ages 14-17.
Figure 2B. Attention-Impulse Control
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.02 for ages 10-13 and 0.99 for ages 14-17.
Figure 2C. Trust
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.00 for ages 10-13 and 0.22 for ages 14-17.
30
Figure 2D. Friends
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.55 for ages 10-13 and 0.96 for ages 14-17.
Figure 2F. Self-Esteem
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.82 for ages 10-13 and 0.39 for ages 14-17.
Figure 2G. Altruism
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.27 for ages 10-13 and 0.89 for ages 14-17.
31
Figure 2H. Grades
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions gives a p-value of 0.02 for ages 10-13 and 0.06 for ages 14-17. Note that
the treatment impact on grades from age 10-13 is not significant under any other specification.
Figure 2I. Verbal IQ.
Kolmorgorov-Smirnov test for equality of Treatment and Control distributions (age 13) gives a p-value of 1.00.
1
0
Percent of subjects held back
.2
.4
.6
Percent of subjects in Special Ed
1.2
1.4
1.6
.8
Figure 2J. Held Back and Special Education
10
12
14
Age
Control
Non-disruptive
16
Treatment
18
88
90
92
Year
Control
Non-disruptive
94
96
Treatment
P-value from unconditional test for equality of Treatment and Control group averages for ages 10-13: 0.96 (Held Back) and 0.95 (Special Education).
P-value from test for equality of Treatment and Control group averages for ages 14-19: 0.12 (Held Back) and 0.11 (Special Education)
32
0
2
Figure 3. Young Adult Outcomes.
Number of crimes
% Sec. diploma
Control
Non-Disruptive
% Active
% Social group
Treatment
As young adults, treatment subjects commit fewer crimes, are more likely to graduate from secondary school, are more likely to be active fulltime in
school or work, and are more likely to belong to a social or civic group. The intervention closed part or all of the gap between boys ranked as
disruptive in kindergarten but not treated (the control group) and the non-disruptive boys (who represent the normative population). Raw differences
are significant for secondary diploma (p-value=0.04) and group membership (p-value=0.05), conditional differences (controlling for group
imbalances) are significant for number of crimes (p-value=0.09) and percent active fulltime (p-value=0.03).
0
.1
R-squared
.2
.3
.4
.5
Figure 4. School achievement explained by IQ and non-cognitive skills.
Grades
Special Education
Held Back
Secondary
IQ and Noncognitive Skills
IQ
Noncognitive Skills
The non-cognitive skills measured in this paper explain a higher proportion of school performance than IQ. The bars plot the adjusted R-squared from
uncontrolled OLS regressions of IQ or non-cognitive skills (Trust, Aggression Control, and Attention-Impulse Control), or both, on different measures
of school achievement.
33
Table 1. Baseline Characteristics and Randomization Check
NonDisruptive
disruptive
population
Control
Treatment
mean
mean
N
sd
mean
N
sd
Difference
C-T
p-value
Age
Attended Pre-school
Birth order
Number brothers
Number sisters
Number half brothers
Number half sisters
Lives with both parents
6.00
0.16
1.66
0.56
0.53
0.04
0.04
0.59
6.03
0.21
1.56
0.5
0.33
0.03
0.04
0.53
181
181
181
180
180
181
181
181
0.30
0.41
0.82
0.69
0.52
0.21
0.32
0.50
5.97
0.19
1.50
0.47
0.57
0.00
0.02
0.46
69
69
68
68
68
68
68
69
0.29
0.40
0.72
0.68
0.68
0
0.12
0.50
0.05
0.02
0.06
0.03
-0.24
0.03
0.02
0.06
0.20
0.71
0.61
0.76
0.00
0.19
0.55
0.39
Age of mother
Age of father
Mother education
Father education
# of children in HH
Adversity index
Mother works
Mother fulltime
Father Works
Father fulltime
Mother prestige
Father prestige
25.69
28.67
10.67
10.81
1.14
0.30
1.61
1.70
1.12
1.97
39.35
40.74
23.99
26.90
9.97
9.70
0.97
0.43
1.73
1.69
1.21
1.94
36.03
35.19
180
161
180
160
181
181
177
48
148
115
161
156
4.18
5.34
2.23
2.45
0.90
0.24
0.45
0.47
0.41
0.24
11.02
9.58
24.01
28.28
9.90
9.93
1.07
0.43
1.78
1.57
1.20
1.92
33.16
35.22
68
56
68
60
68
68
68
14
49
38
60
53
4.71
5.33
2.28
2.42
0.80
0.27
0.42
0.51
0.41
0.27
10.13
9.83
-0.02
-1.38
0.07
-0.24
-0.10
-0.00
-0.05
0.12
0.01
0.02
2.87
-0.03
0.97
0.10
0.83
0.52
0.42
0.96
0.42
0.43
0.86
0.70
0.08
0.99
Initial Aggression
Initial Anxiety
Initial Opposition
Initial Prosociality
Initial Combativeness
Initial Inattention
Initial Hyperactivity
Initial Antisociality
4.00
2.65
1.63
8.21
0.82
2.23
0.98
0.84
14.51
3.55
5.62
6.52
3.53
4.19
2.79
0.99
181
181
181
181
181
181
180
181
4.78
2.73
2.19
4.79
1.59
2.35
1.21
1.11
14.62
4.26
5.81
6.99
3.48
4.19
2.96
1.21
69
69
69
69
69
69
68
68
4.58
2.82
1.93
4.51
1.54
2.18
1.19
1.23
-0.11
-0.71
-0.19
-0.47
0.05
0.01
-0.16
-0.21
0.86
0.07
0.53
0.49
0.83
0.99
0.35
0.20
Data from baseline data collection, 1984.
34
Table 2. Characteristics of Compliers and Non-Compliers
N
NonNonComplier
Complier
Mean
SD
Father Works
69
3.30
3.48
Mother Works
69
2.00
1.60
Prestige of Mother
60
37.03
11.25
Prestige of Father
53
35.26
8.13
Family Adversity
68
0.45
0.29
Number of Kids in Fam
68
1.09
0.81
Father Education
60
9.84
1.77
Mother Education
68
10.14
2.42
Age of Mother
68
22.32
4.55
Age of Father
56
28.48
5.36
Lives with both Parents
69
0.48
0.51
Age
69
6.00
0.30
Attended Preschool
69
0.13
0.34
Initial Aggression
69
13.30
4.68
Initial Anxiety
69
4.65
2.95
Initial Opposition
69
5.65
1.77
Initial Prosociality
69
6.00
4.80
Initial Fighting
69
2.91
1.65
Initial Antisocial
68
1.52
1.31
Initial Hyperactivity
68
2.70
1.18
Initial Inattention
69
4.78
1.88
Complier Complier
Mean
SD
3.41
1.83
31.36
35.20
0.42
1.07
9.98
9.78
24.82
28.18
0.46
5.96
0.22
15.28
4.07
5.89
7.48
3.76
1.04
3.09
3.89
3.47
0.38
9.17
10.58
0.26
0.80
2.70
2.23
4.61
5.39
0.50
0.28
0.42
4.43
2.76
2.01
4.33
1.42
1.17
1.18
2.27
Difference
0.11
-0.17
-5.66
-0.06
-0.03
-0.03
0.13
-0.35
2.50
-0.30
-0.02
-0.05
0.09
1.98
-0.59
0.24
1.48
0.85
-0.48
0.39
-0.89
Equality of
Means pvalue
0.90
0.48
0.04
0.98
0.71
0.90
0.84
0.55
0.04
0.85
0.87
0.54
0.39
0.09
0.42
0.63
0.20
0.03
0.13
0.20
0.11
Data from baseline data collection, 1984.
35
Table 3. Attrition
(1)
NonDisruptive
(2)
% missing
(3)
(4)
(5)
Control
Treatment
Difference
(T-C)
p-value of
difference
Trust
Friends
Aggression Control
Attention-Impulse Control
Self Esteem
Altruism
Grades
Held Back
Special Education
Early Adolescent Outcomes: Age 10-13
2%
2%
4%
1%
1%
1%
1%
1%
1%
1%
1%
1%
4%
7%
9%
1%
1%
1%
7%
13%
10%
1%
0%
0%
1%
0%
0%
2%
1%
1%
1%
2%
1%
-3%
0%
0%
0.36
0.48
0.48
0.48
0.57
0.48
0.58
Trust
Friends
Aggression Control
Attention-Impulse Control
Self Esteem
Altruism
Grades
Held Back
Special Education
Group Member
Late Adolescent Outcomes: Age 14-17
10%
16%
13%
10%
16%
13%
10%
16%
13%
11%
17%
15%
13%
20%
16%
13%
21%
19%
10%
14%
13%
1%
1%
0%
1%
1%
0%
17%
26%
20%
-2%
-2%
-2%
-2%
-4%
-2%
-1%
-1%
-1%
-6%
0.63
0.63
0.63
0.69
0.42
0.71
0.79
0.54
0.38
0.35
Group Member
% Fulltime
% Working
% Receiving Transfer
Employed Age 27
Employed Fulltime Age 27
Hours per week Age 27
Hourly Wage Age 27
25%
25%
25%
25%
45%
45%
45%
45%
Adult Outcomes
36%
38%
38%
38%
58%
58%
58%
58%
2%
6%
6%
6%
6%
6%
6%
6%
0.80
0.39
0.39
0.39
0.37
0.37
0.37
0.37
38%
44%
44%
44%
64%
64%
64%
64%
Column (1) shows the percent of participants in the non-disruptive sub-sample that have missing data. Column (2) shows the percent of
participants in the control group with missing data. Column (3) shows the percent of participants in the treatment group with missing
data. Column (4) shows the difference between the percent missing in the treatment group and the control group. Column (5) provides the
p-value for the difference, from testing the difference in means of a dummy variable equal to 1 between the treatment and control group.
36
Table 4. Difference in baseline characteristics, Adult Outcome attriters vs. non-attriters
Non-Attriters
Attriters
N
Mean
SD
Mean
SD
Diff
Father Works
197
0.87
0.34
0.65
0.48
-0.22
Mother Works
245
0.32
0.47
0.16
0.37
-0.16
Prestige of Mother's Job
221
36.04
10.98
33.89
10.53
-2.16
Prestige of Father's Job
209
35.57
10.10
34.51
8.67
-1.06
Family Adversity
249
0.41
0.24
0.46
0.25
0.06
Number of Children
249
1.01
0.93
0.98
0.78
-0.03
Father Education
220
9.96
2.46
9.42
2.39
-0.54
Mother Education
248
10.15
2.10
9.64
2.42
-0.51
Mother Age
248
24.16
4.33
23.73
4.32
-0.43
Father Age
217
26.80
4.82
28.05
6.14
1.25
Lives with both parents
250
0.49
0.50
0.53
0.50
0.04
Subject Age
250
6.02
0.30
6.00
0.29
-0.01
Preschool
250
0.18
0.38
0.24
0.43
0.07
Initial Aggression
250
14.56
4.82
14.51
4.57
-0.05
Initial Anxiety
250
3.80
2.87
3.66
2.61
-0.14
Initial Opposition
250
5.67
2.21
5.68
1.98
0.01
Initial Prosociality
250
6.33
4.78
7.14
4.59
0.81
Initial Fighting
250
3.51
1.62
3.51
1.51
0.00
Initial Antisociality
249
1.06
1.23
1.04
1.01
-0.02
Initial Hyperactivity
248
2.90
1.14
2.74
1.29
-0.16
Initial Inattention
250
4.18
2.33
4.20
2.26
0.02
Equality of
means pvalue
0.00
0.00
0.15
0.45
0.07
0.77
0.12
0.08
0.45
0.10
0.57
0.70
0.20
0.94
0.70
0.96
0.18
0.99
0.90
0.32
0.95
Data from baseline questionnaire, 1984. Attriters are those who are missing data for the Percent Active Fulltime variable, indicating
that they are missing self-reported adult outcome data
Table 5. Attrition, baseline characteristics, and Treatment
(1)
(2)
(3)
Dependent Variable: Missing Percent Active Fulltime (17-27)
Treatment
0.246
(0.288)
Treatment * Father Works
Father Works
-0.647
(0.736)
1.797**
(0.831)
-1.745***
(0.431)
Treatment * Initial Hyperactivity
-0.508***
(0.154)
0.647*
(0.373)
-0.557**
(0.247)
0.0323
(0.132)
-0.589
(0.401)
250
197
248
Initial Hyperactivity
Constant
Observations
1.893**
(0.764)
Robust standard errors in parentheses, ***p<0.01, **p<0.05, *p<0.10. Controls for treatment group imbalances (age of father,
prestige of mother’s work, and initial anxiety) included. Non-significant results from other baseline variable interactions not shown
but available on request.
37
Table 6. Adolescent and Young Adult Summary Statistics
Non-Disruptive
Control
mean
N
mean
N
Ages 10-13
0.4
780
-0.01
180
Num.
Var
Cronbach
Alpha
Aggression Control
13
0.83
Attention-Impulse Control
Trust
Friends
Self-Esteem
Altruism
Grades
Held Back
Special Ed
Verbal IQ (Age 13)
6
12
13
18
10
2
0.73
0.61
0.64
0.82
0.92
0.85
0.35
0.29
0.12
0.2
0.11
0.38
0.11
0.08
9.17
Aggression Control
Attention-Impulse Control
Trust
Friends
Self-Esteem
Altruism
Grades
Held Back
Special Ed
Group member (16 or 17)
13
6
12
13
18
10
2
0.86
0.67
0.68
0.65
0.75
0.90
0.77
0.25
0.25
0.22
0.11
0.12
-0.01
0.43
0.34
0.22
0.04
Number of crimes 18-23
Any non-violent crime 18-23
Any violent crime 18-23
Secondary School Diploma
% of years 17-27 Active
% of years 17-27 Working
% of years 17-27 Transfers
Employed at age 27
Employed fulltime at age 27
Hours per week at age 27
Hourly wage at age 27
Group member (21 or 27)
0.69
0.04
0.13
0.58
0.82
0.72
0.07
0.91
0.85
35.97
16.74
0.46
780
-0.01
775
0
780
0
754
0
778
0
734
0
783
0.26
783
0.21
681
8.57
Ages 14-17
706
-0.01
704
0
706
-0.04
706
0
686
-0.01
687
0
707
0
781
0.60
779
0.46
651
0.02
Young Adult
787
787
787
785
591
591
591
433
433
433
433
591
2.16
0.11
0.28
0.31
0.76
0.71
0.14
0.86
0.82
34.17
14.96
0.32
Treatment
mean
N
0.14
68
180
177
180
169
180
158
181
181
148
0.11
0.15
-0.07
0.03
-0.11
0.13
0.26
0.20
8.54
68
66
68
63
68
63
69
69
56
153
151
153
153
144
143
155
180
179
134
0.18
0.03
0.14
0
-0.02
-0.04
0.23
0.50
0.36
0.13
60
59
60
60
58
56
60
69
69
55
181
181
181
180
113
113
113
77
77
77
77
116
1.13
0.09
0.19
0.45
0.83
0.80
0.12
0.96
0.96
40.60
16.53
0.49
69
69
69
69
39
39
39
25
25
25
25
43
Aggression Control, Attention-Impulse Control, Trust, Friends, Self-Esteem, Altruism, and Grades are the z-score averages of the
component variables presented in Tables S 7 – S 13, except for Held Back and Special Ed which are the percent of years where the
subject was held back or in special education.
38
Table 7. Treatment Impact in Early Adolescence (Age 10-13)
(1)
(2)
(3)
(4)
Control
Group
Mean
p-value of raw
difference in
means (t-test,
Disruptive
Sample)
OLS Treatment
Effect:
Disruptive
Sample
0
0.0221
Self Control
-0.01
0.0307
Self Control (Ext)
-0.01
0.0543
Self Control (Int)
-0.01
0.0575
Friends
0
0.260
Self Esteem
0
0.683
Altruism
0
0.329
8.57
0.931
0
0.722
Special Ed
0.21
0.951
Held Back
0.26
0.956
0.156**
(0.0764)
0.149**
(0.0662)
0.154*
(0.0809)
0.160**
(0.0788)
-0.0786
(0.0722)
0.0304
(0.0689)
-0.109
(0.100)
-0.0319
(0.376)
0.0515
(0.146)
-0.00290
(0.0513)
-0.00294
(0.0526)
Trust
IQ
Grades
Conditional
Treatment
Effect:
Disruptive
Sample
(5)
Conditional
Treatment
effect with full
sample,
including
clusters
(6)
Conditional
Treatment
effect with full
sample,
including Fixed
Effects
(7)
Conditional
Treatment
effect with full
sample,
including IPW
0.181**
(0.0717)
0.150**
(0.0719)
0.149*
(0.0780)
0.177**
(0.0716)
-0.0296
(0.0713)
0.0379
(0.0781)
-0.0859
(0.108)
0.188
(0.399)
0.103
(0.173)
-0.0316
(0.0501)
-0.0446
(0.0587)
0.173**
(0.0811)
0.155***
(0.0594)
0.147*
(0.0778)
0.192***
(0.0602)
-0.0352
(0.0777)
0.0527
(0.0735)
-0.101
(0.0895)
0.156
(0.344)
0.125
(0.149)
-0.0227
(0.0420)
-0.0318
(0.0500)
0.177**
(0.0718)
0.150**
(0.0644)
0.146*
(0.0774)
0.179**
(0.0846)
-0.0480
(0.0742)
0.0664
(0.0863)
-0.0946
(0.0997)
0.261
(0.386)
0.112
(0.159)
-0.0174
(0.0495)
-0.0342
(0.0568)
0.168**
(0.0698)
0.152**
(0.0651)
0.143*
(0.0783)
0.188**
(0.0826)
-0.0334
(0.0727)
0.0415
(0.0761)
-0.0985
(0.105)
0.201
(0.382)
0.128
(0.158)
0.00117
(0.0744)
0.00518
(0.0790)
(8)
P-value of raw
difference in
means
(permutation
test, Disruptive
Sample)
0.0240
0.0240
0.0420
0.0870
0.248
0.686
0.316
0.938
0.702
0.937
0.944
Each cell of column (1) provides the control group average. Each cell of column (2) provides the p-value for the raw difference between the treatment and the control group for each outcomes (in rows).
Each cell of columns (3)-(7) give the regression coefficient of the treatment dummy variable on each of the potential mechanisms (in rows). Columns (3)-(7) include robust standard errors in parentheses.
*** p<0.01, ** p<0.05, *p<0.10. Columns (1)-(4) and (8) use data from the disruptive (experimental) sample only. Columns (5)-(7) use data from the entire sample. Columns (1) -(3), and (8) include no
controls. Columns (4)-(7) include controls for imbalances between the treatment and control groups. Column (7) uses inverse probability weighting to adjust for attrition under certain assumptions. The
p-values in column (8) are obtained from a permutation test of the difference of the means (where the permuted value is treatment group) with 2000 repetitions. Clustered standard errors and fixed effects
are at the school level in 1984.
Table 8. Treatment Impact in Late Adolescence (Age 14-17)
(1)
(2)
(3)
(4)
Control
Group
Mean
p-value of raw
difference in
means (t-test,
Disruptive
Sample)
OLS Treatment
Effect:
Disruptive
Sample
-0.04
0.0422
0
0.322
Self Control (Ext)
-0.01
0.0377
Self Control (Int)
0
0.654
Friends
0
0.842
-0.01
0.982
Altruism
0
0.743
Grades
0
0.0301
Special Ed
0.46
0.108
Held Back
0.6
0.117
0.176**
(0.0772)
0.0767
(0.0797)
0.186**
(0.0870)
0.0412
(0.0934)
0.0135
(0.0693)
-0.00149
(0.0615)
-0.0393
(0.108)
0.338**
(0.164)
-0.0989*
(0.0577)
-0.0963
(0.0605)
Trust
Self Control
Self Esteem
Conditional
Treatment
Effect:
Disruptive
Sample
(5)
Conditional
Treatment
effect with full
sample,
including
clusters
(6)
Conditional
Treatment
effect with full
sample,
including Fixed
Effects
(7)
Conditional
Treatment
effect with full
sample,
including IPW
0.199**
(0.0823)
0.0513
(0.0769)
0.170**
(0.0838)
0.0113
(0.0915)
0.0493
(0.0674)
0.0161
(0.0653)
-0.0772
(0.138)
0.397**
(0.161)
-0.143**
(0.0577)
-0.148***
(0.0565)
0.176*
(0.0925)
0.0704
(0.0928)
0.176*
(0.0985)
0.0425
(0.0749)
0.0564
(0.0648)
0.0122
(0.0746)
-0.0770
(0.134)
0.401**
(0.167)
-0.139**
(0.0541)
-0.144**
(0.0696)
0.200**
(0.0871)
0.0896
(0.0783)
0.200**
(0.0841)
0.0623
(0.0914)
0.0667
(0.0751)
0.0430
(0.0703)
-0.104
(0.117)
0.452***
(0.162)
-0.152***
(0.0575)
-0.150**
(0.0658)
0.174**
(0.0816)
0.0709
(0.0752)
0.175**
(0.0814)
0.0480
(0.0887)
0.0449
(0.0668)
0.0115
(0.0682)
-0.0691
(0.124)
0.423**
(0.177)
-0.137**
(0.0608)
-0.0218
(0.0860)
(8)
P-value of raw
difference in
means
(permutation
test, Disruptive
Sample)
0.0170
0.229
0.00900
0.669
0.853
0.989
0.736
0.0140
0.0610
0.119
Each cell of column (1) provides the control group average. Each cell of column (2) provides the p-value for the raw difference between the treatment and the control group for each outcomes (in rows).
Each cell of columns (3)-(7) give the regression coefficient of the treatment dummy variable on each of the potential mechanisms (in rows). Columns (3)-(7) include robust standard errors in parentheses.
*** p<0.01, ** p<0.05, *p<0.10. Columns (1)-(4) and (8) use data from the disruptive (experimental) sample only. Columns (5)-(7) use data from the entire sample. Columns (1) -(3), and (8) include no
controls. Columns (4)-(7) include controls for imbalances between the treatment and control groups. Column (7) uses inverse probability weighting to adjust for attrition under certain assumptions. The
p-values in column (8) are obtained from a permutation test of the difference of the means (where the permuted value is treatment group) with 2000 repetitions. Clustered standard errors and fixed effects
are at the school level in 1984.
40
Table 9. Impact on Criminality and Secondary Completion (Administrative Data)
(1)
(2)
(3)
Control
Group
Mean
p-value of
raw
difference in
means (ttest,
Disruptive
Sample)
Number of crimes 18-23
2.15
0.162
Any non-violent crime 18-23
0.28
0.132
Any violent crime 18-23
0.11
0.587
Secondary School Diploma
0.31
0.0409
(4)
(5)
(6)
(7)
(8)
P-value of
raw
difference in
means
(permutation
test,
Disruptive
Sample)
Logit
Treatment
Effect:
Disruptive
Sample
OLS
Treatment
Effect:
Disruptive
Sample
Conditional
Treatment
Effect:
Disruptive
Sample
Conditional
Treatment
effect with
full sample,
including
clusters
Conditional
Treatment
effect with
full sample,
including
Fixed
Effects
Criminality
-1.272*
(0.668)
-0.0939
(0.0579)
-0.0305
(0.0447)
-1.091*
(0.629)
-0.0979
(0.0635)
-0.0273
(0.0445)
-1.147*
(0.599)
-0.0862
(0.0586)
-0.0329
(0.0425)
0.0460
-0.0934
(0.0706)
-0.0235
(0.0429)
-1.024*
(0.557)
-0.0934*
(0.0566)
-0.0235
(0.0407)
0.138*
(0.0711)
0.138*
(0.0721)
Education
0.176**
(0.0692)
0.185**
(0.0775)
0.189***
(0.0728)
0.0510
0.0660
0.461
Each cell of column (1) provides the control group average. Each cell of column (2) provides the p-value for the raw difference between the treatment and the control group for each of the outcomes
(in rows). Each cell of columns (3)-(7) give the regression coefficient of the treatment dummy variable on each of the potential mechanisms (in rows). Columns (3)-(7) include robust standard errors
in parentheses. *** p<0.01, ** p<0.05, *p<0.10. Columns (1)-(4) and (8) use data from the disruptive (experimental) sample only. Columns (5)-(7) use data from the entire sample. Columns (1),
(2), (4), and (8) include no controls. Columns (3) and (5)-(7) include controls for imbalances between the treatment and control groups. The p-values in column (8) are obtained from a permutation
test of the difference of the means (where the permuted value is treatment group) with 2000 repetitions. Clustered standard errors and fixed effects are at the school level in 1984. Our estimate of the
treatment effect on the dummy variable of whether or not the subject ever committed a crime is very slightly smaller than that in Boisjoli et al. (2007) and nonsignificant, due to corrections in the
dataset, but the substantive result remains the same.
41
Table 10. Impact on Economic Activity and Social Capital (Self-Reported Data)
(1)
(2)
Control
Group
Mean
p-value of
raw
difference in
means
(Disruptive
Sample)
% years fulltime age 17-26
0.78
0.112
% years working 17-26
0.69
0.0577
% years transfers age 17-26
0.14
0.555
Employed fulltime age 26
0.82
0.0834
Hourly wage age 26
14.7
0.332
34
0.0484
Weekly hours worked age 26
(3)
Logit
Treatment
Effect:
Disruptive
Sample
0.142***
(0.0479)
(4)
OLS
Treatment
Effect:
Disruptive
Sample
0.0758*
(0.0392)
0.0920**
(0.0444)
-0.0254
(0.0359)
0.142***
(0.0535)
1.567
(1.566)
6.431**
(2.681)
(5)
(6)
(7)
(8)
(9)
Conditional
Treatment
Effect:
Disruptive
Sample
Conditional
Treatment
effect with
full sample,
including
clusters
Conditional
Treatment
effect with
full sample,
including
Fixed
Effects
Conditional
Treatment
effect with
full sample,
including
IPW
P-value of
permutation
test (no
controls)
0.119***
(0.0456)
0.114***
(0.0431)
-0.0552
(0.0404)
0.165***
(0.0633)
2.415
(2.071)
6.142*
(3.442)
0.126**
(0.0494)
0.109**
(0.0449)
-0.0530
(0.0425)
0.193***
(0.0699)
2.867*
(1.542)
8.097***
(2.905)
0.202**
(0.0951)
0.0958**
(0.0444)
0.208**
(0.0866)
0.0989**
(0.0474)
Economic Activity
0.122**
0.116***
(0.0530)
(0.0414)
0.117**
0.116***
(0.0457)
(0.0363)
-0.0294
-0.0514
(0.0438)
(0.0370)
0.189***
0.159**
(0.0693)
(0.0663)
1.790
2.398*
(1.720)
(1.269)
7.990***
6.995**
(2.884)
(2.732)
0.113
0.0680
0.443
0.0850
0.724
0.0450
Social Capital
Member of group (21 or 26)
0.32
0.0491
Member of a group (16 or 17)
0.02
0.00328
0.169*
(0.0998)
0.105**
(0.0499)
0.169*
(0.0883)
0.105**
(0.0478)
0.169*
(0.0982)
0.0868*
(0.0513)
0.229***
(0.0733)
0.100**
(0.0487)
0.0630
0.00200
Each cell of column (1) provides the control group average. Each cell of column (2) provides the p-value for the raw difference between the treatment and the control group for each outcomes (in rows).
Each cell of columns (3)-(7) give the regression coefficient of the treatment dummy variable on each of the potential mechanisms (in rows). Columns (3)-(7) include robust standard errors in parentheses.
*** p<0.01, ** p<0.05, *p<0.10. Columns (1)-(4) and (9) use data from the disruptive (experimental) sample only. Columns (5)-(7) use data from the entire sample. Columns (1), (2), (4), and (9) include
no controls. Columns (3) and (5)-(8) include controls for imbalances between the treatment and control groups. Column (8) uses inverse probability weighting to adjust for attrition under certain
assumptions. The p-values in column (9) are obtained from a permutation test of the difference of the means (where the permuted value is treatment group) with 2000 repetitions. Clustered standard errors
and fixed effects are at the school level in 1984.
42
Table 11. Attrition falsification test for Social and Economic Adult Outcomes.
Treatment
Constant
Observations
R-squared
(1)
Secondary
Completion
(All)
(2)
Secondary
Completion
(non-Attriters)
(3)
Secondary
Completion
(Attriters)
0.176***
(0.0676)
-0.141
(0.182)
0.148
(0.0962)
0.104
(0.262)
0.249***
(0.0901)
-0.618***
(0.224)
249
0.099
151
0.051
98
0.219
Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Column (1)
shows the treatment impact on secondary completion for the entire sample. Column (2)
shows the treatment impact on secondary completion for those that have data for percent
fulltime (the non-attriters). Column (3) shows the treatment impact on secondary
completion for those missing data for percent fulltime (the attriters).
Table 12. Proportion of Treatment Effect on Late Adolescent Outcomes Explained by Treatment Effect on Early
Adolescent Outcomes.
Grades 14-17
Skill Age 10-13
Trust
Impact of
Treatment on
Skill
(1)
0.180
, 0.298
0.147
[ 0.016 , 0.278
0.177
[ 0.041 , 0.314
[ 0.061
Self Control (Ext)
Self Control (Int)
Association of
Skill to Grades
14-17
(2)
]
]
]
0.605
, 0.858 ]
0.543
[ 0.373 , 0.713 ]
0.661
[ 0.479 , 0.844 ]
[ 0.351
Proportion of
impact on
Grades 14-17
Explained by
skill
(3)
0.387
, 0.912 ]
0.284
[ 0.021 , 0.707 ]
0.418
[ 0.070 , 0.944 ]
[ 0.076
Held Back 14-17
Skill Age 10-13
Trust
Self Control (Ext)
Self Control (Int)
Impact of
Treatment on
Skill
(1)
0.180
[ 0.061 , 0.298 ]
0.147
[ 0.016 , 0.278 ]
0.177
[ 0.041 , 0.314 ]
Association of
Skill to Held
Back 14-17
(2)
-0.218
[ -0.325 , -0.112 ]
-0.165
[ -0.244 , -0.087 ]
-0.313
[ -0.386 , -0.241 ]
Proportion of
impact on Held
Back 14-17
Explained by
skill
(3)
0.268
[ 0.135 , 0.229 ]
0.166
[ 0.026 , 0.165 ]
0.379
[ 0.107 , 0.516 ]
Special Ed 14-17
Proportion of
impact on
Impact of
Association of
Special Ed 14Treatment on
Skill to Special Ed
17 Explained
Skill
14-17
by skill
Skill Age 10-13
(1)
(2)
(3)
Trust
0.180
-0.231
0.293
[ 0.061 , 0.298 ]
[ -0.339 , -0.123 ]
[ 0.146 , 0.260 ]
Self Control (Ext)
0.147
-0.167
0.173
[ 0.016 , 0.278 ]
[ -0.246 , -0.088 ]
[ 0.027 , 0.172 ]
Self Control (Int)
0.177
-0.297
0.372
[ 0.041 , 0.314 ]
[ -0.368 , -0.226 ]
[ 0.106 , 0.501 ]
Column (1) presents the impact of treatment on each skill, with the 10% confidence interval below.
Column (2) presents the coefficient from a non-experimental regression of the skill against the late
adolescent outcome, using data from the control group only with no controls, again with the 10%
confidence interval below. Column (3) multiplies point estimate from Column (1) by the point estimate
in Column (2) and then divides by overall treatment impact on the late adolescent outcome, with
confidence intervals given below. Confidence intervals in Column (3) are calculated by multiplying the
90% confidence interval in Column (1) by the 90% confidence interval in Column (2)
44
Table 12. Proportion of Treatment Effect on Late Adolescent Outcomes Explained by Treatment Effect on Early
Adolescent Outcomes (continued).
Trust 14-17
Skill Age 10-13
Impact of
Treatment on
Skill
(1)
Association of
Skill to Trust
14-17
(2)
Proportion of
impact on Trust
14-17 Explained
by skill
(3)
Trust
0.180
0.522
0.475
, 0.298 ]
[ 0.370 , 0.674 ]
[ 0.114 , 1.017 ]
Self Control (Ext)
0.147
0.469
0.348
[ 0.016 , 0.278 ]
[ 0.327 , 0.610 ]
[ 0.026 , 0.857 ]
Self Control (Int)
0.177
0.385
0.345
[ 0.041 , 0.314 ]
[ 0.267 , 0.502 ]
[ 0.055 , 0.797 ]
Column (1) presents the impact of treatment on each skill, with the 10% confidence interval below.
Column (2) presents the coefficient from a non-experimental regression of the skill against the late
adolescent outcome, using data from the control group only with no controls, again with the 10%
confidence interval below. Column (3) multiplies point estimate from Column (1) by the point
estimate in Column (2) and then divides by overall treatment impact on the late adolescent outcome,
with confidence intervals given below. Confidence intervals in Column (3) are calculated by
multiplying the 90% confidence interval in Column (1) by the 90% confidence interval in Column (2)
[ 0.061
Delinquency 14-17
Proportion of
impact on
Association of
Delinquency
Impact of
Skill to
14-17
Treatment on
Delinquency
Explained by
Skill
14-17
skill
Skill Age 10-13
(1)
(2)
(3)
Trust
0.180
0.583
0.617
[ 0.061 , 0.298 ]
[ 0.407 , 0.758 ]
[ 0.147 , 1.333 ]
Self Control (Ext)
0.147
0.663
0.575
[ 0.016 , 0.278 ]
[ 0.534 , 0.792 ]
[ 0.049 , 1.300 ]
Self Control (Int)
0.177
0.384
0.402
[ 0.041 , 0.314 ]
[ 0.244 , 0.525 ]
[ 0.059 , 0.971 ]
Column (1) presents the impact of treatment on each skill, with the 10% confidence interval below.
Column (2) presents the coefficient from a non-experimental regression of the skill against the late
adolescent outcome, using data from the control group only with no controls, again with the 10%
confidence interval below. Column (3) multiplies point estimate from Column (1) by the point
estimate in Column (2) and then divides by overall treatment impact on the late adolescent outcome,
with confidence intervals given below. Confidence intervals in Column (3) are calculated by
multiplying the 90% confidence interval in Column (1) by the 90% confidence interval in Column (2)
45
Table 13. Proportion of Treatment Effect on Adult Outcomes Explained by Adolescent Outcomes
Secondary Completion
Trust
[
Ages 10-13
Self Control (Ext)
[
Self Control (Int)
[
Trust
[
Self Control (Ext)
[
Ages 14-17
Grades
[
Special Ed
[
Held Back
[
Impact of
Treatment on Skill
(1)
0.180
0.061 , 0.298
0.147
0.016 , 0.278
0.177
0.041 , 0.314
0.198
0.056 , 0.339
0.169
0.025 , 0.314
0.280
0.070 , 0.491
-0.142
-0.245 , -0.038
-0.146
-0.245 , -0.047
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
Association of
Skill to Secondary
Completion
(2)
0.236
0.112 , 0.359
0.221
0.134 , 0.308
0.380
0.302 , 0.457
0.258
0.168 , 0.348
0.247
0.152 , 0.342
0.254
0.203 , 0.305
-0.573
-0.680 , -0.467
-0.788
-0.879 , -0.696
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
Proportion of
impact on
Secondary
Completion
Explained by
skill
(3)
0.240
0.039 , 0.608
0.184
0.012 , 0.485
0.382
0.070 , 0.813
0.289
0.053 , 0.669
0.238
0.022 , 0.609
0.405
0.081 , 0.851
0.460
0.944 , 0.102
0.654
1.224 , 0.187
]
]
]
]
]
]
]
]
Number of Crimes
Proportion of
impact on
Number of
Impact of
Association of
Crimes
Treatment on
Skill to Number of
Explained by
Skill
Crimes
skill
(1)
(2)
(3)
Trust
0.180
-3,242
0.458
[ 0.061 ,
0.298 ]
[ -5,804
,
-0.681 ]
[ 0.278
,
0.160]
Self Control (Ext)
0.147
-2,857
0.330
Ages 10-13
[ 0.016 ,
0.278 ]
[ -4,769
,
-0.945 ]
[ 0.059
,
0.207]
Self Control (Int)
0.177
-1.148
0.160
[ 0.041 ,
0.314 ]
[ -2,093
,
-0.203 ]
[ 0.067
,
0.050]
Trust
0.198
-3,209
0.499
[ 0.056 ,
0.339 ]
[ -5,600
,
-0.818 ]
[ 0.247
,
0.218]
Self Control (Ext)
0.169
-4,687
0.624
[ 0.025 ,
0.314 ]
[ -7,255
,
-2,118 ]
[ 0.144
,
0.522]
Grades
0.280
-1.084
0.239
Ages 14-17
[ 0.070 ,
0.491 ]
[ -2,076
,
-0.091 ]
[ 0.115
,
0.035]
Special Ed
-0.142
4,147
0.461
[ -0.245 ,
-0.038 ]
[ 2,297
,
5,997 ]
[ 0.442
,
0.181]
Held Back
-0.146
3,417
0.393
[ -0.245 ,
-0.047 ]
[ 1.920
,
4,913 ]
[ 0.370
,
0.183]
Column (1) presents the impact of treatment on each skill, with the 10% confidence interval below. Column (2) presents
the coefficient from a non-experimental regression of the skill against the adult outcome, using data from the control
group only with no controls, again with the 10% confidence interval below. Column (3) multiplies point estimate from
Column (1) by the point estimate in Column (2) and then divides by overall treatment impact on the adult outcome, with
confidence intervals given below. Confidence intervals in Column (3) are calculated by multiplying the 90% confidence
interval in Column (1) by the 90% confidence interval in Column (2)
46
Table 13. Proportion of Treatment Effect on Adult Outcomes Explained by Adolescent Outcomes (continued)
Percent Fulltime
Trust
[
Ages 10-13
Self Control (Ext)
[
Self Control (Int)
[
Trust
[
Self Control (Ext)
[
Ages 14-17
Grades
[
Special Ed
[
Held Back
[
Impact of
Treatment on Skill
(1)
0.180
0.061 , 0.298
0.147
0.016 , 0.278
0.177
0.041 , 0.314
0.198
0.056 , 0.339
0.169
0.025 , 0.314
0.280
0.070 , 0.491
-0.142
-0.245 , -0.038
-0.146
-0.245 , -0.047
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
Association of
Skill to Percent
Employed
(2)
0.155
0.048 , 0.263
0.073
-0.030 , 0.176
0.086
0.020 , 0.151
0.137
0.029 , 0.245
0.088
-0.010 , 0.186
0.036
-0.019 , 0.092
-0.291
-0.400 , -0.182
-0.240
-0.337 , -0.143
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
Proportion of
impact on
Percent
Employed
Explained by
skill
(3)
0.240
0.025 , 0.674
0.093
-0.004 , 0.421
0.131
0.007 , 0.408
0.233
0.014 , 0.713
0.128
-0.002 , 0.501
0.087
-0.012 , 0.386
0.354
0.841 , 0.060
0.302
0.711 , 0.058
]
]
]
]
]
]
]
]
Group Membership
Proportion of
Association of
impact on Group
Impact of
Skill to Group
Membership
Treatment on Skill
Membership
Explained by skill
(1)
(2)
(3)
Trust
0.180
0.021
0.022
[ 0.061 , 0.298 ]
[ -0.152 , 0.193 ]
[ -0.056 , 0.348 ]
Self Control (Ext)
0.147
-0.045
-0.040
Ages 10-13
[ 0.016 , 0.278 ]
[ -0.193 , 0.103 ]
[ -0.018 , 0.173 ]
Self Control (Int)
0.177
-0.020
-0.022
[ 0.041 , 0.314 ]
[ -0.150 , 0.109 ]
[ -0.037 , 0.208 ]
Trust
0.198
0.115
0.138
[ 0.056 , 0.339 ]
[ 0.003 , 0.228 ]
[ 0.001 , 0.468 ]
Self Control (Ext)
0.169
0.028
0.029
[ 0.025 , 0.314 ]
[ -0.080 , 0.136 ]
[ -0.012 , 0.258 ]
Grades
0.280
-0.019
-0.032
Ages 14-17
[ 0.070 , 0.491 ]
[ -0.120 , 0.082 ]
[ -0.051 , 0.243 ]
Special Ed
-0.142
0.052
-0.045
[ -0.245 , -0.038 ]
[ -0.118 , 0.223 ]
[ 0.175 , -0.052 ]
Held Back
-0.146
0.005
-0.004
[ -0.245 , -0.047 ]
[ -0.165 , 0.175 ]
[ 0.245 , -0.050 ]
Column (1) presents the impact of treatment on each skill, with the 10% confidence interval below. Column (2) presents
the coefficient from a non-experimental regression of the skill against the adult outcome, using data from the control
group only with no controls, again with the 10% confidence interval below. Column (3) multiplies point estimate from
Column (1) by the point estimate in Column (2) and then divides by overall treatment impact on the adult outcome, with
confidence intervals given below. Confidence intervals in Column (3) are calculated by multiplying the 90% confidence
interval in Column (1) by the 90% confidence interval in Column (2)
47
Table 14. Cost-Effectiveness Estimates
Cost of…
2013 USD
Averting one crime
3,087
One more secondary diploma
29,341
Avoiding one year of repeating a grade
8,606
Avoiding one year of special education
7,743
One year of fulltime work
1,846
Cost of program per person: 9,500 in 2013 USD. All accrued costs and benefits estimated
at age 60, using a discount rate of 3%. Assumptions: treatment effect on crime fades by
10% per year and disappears at age 35, and treatment effect on hours worked does not
fade.
Table 15. Cost-Benefit Estimate and Sensitivity Test
(1)
(2)
(3)
(4)
Yearly Decline of Labor Market Returns
(5)
No labor
market
returns
0
5%
10%
15%
1,846
3,373
5,204
9,126
Dollars of benefits per dollars spent
13.8
6.8
4.2
2.5
0.7
Comparable compound interest rate
5%
4%
3%
2%
1%
Cost per additional year working fulltime
(2013 USD)
Cost of program per person: 9,500 in 2013 USD. All accrued costs and benefits estimated at age 60, using a discount rate of
3%. Assumptions: treatment effect on crime fades by 10% per year and disappears at age 35. Workers are assumed to make
only the minimum wage. Column 1 provides the preferred estimate. Sources for cost and minimum wage: Quebec Ministry of
Justice, Ministry of Education, and Commission on Labor Standards.
48