Meeting 6, February 14, 2008
In this class, we discuss issues related to test score gaps.
A lot of research shows that early test scores are predictive of future labor market success:

Neal and Johnson (1996) showed, using NLSY data, that AFQT scores taken at
adolescents are predictive of their future educational attainment. In particular, if
one conditions on adolescent AFQT, there is little or no racial gap in adult wage
earning regressions. (Neal and Johnson 1996: The Role of Permanent Factors in
Black-White Wage Differences, Journal of Political Economy, 104 (5): 869-895.

Keane and Wolpin (1997) showed that two thirds of the adult wage variations
could be explained by unobserved types in adolescence, where the unobserved
type presumably captures whatever cognitive or non-cognitive capacities that an
individual has.
These research findings are crucially important in shaping the current policy debate about
how to close racial gap, how to achieve equality? We will see more of this in the book by
Heckman and Krueger.
First, we will step back and ask the following question: if cognitive achievement by
adolescence is so important, what do we know about how to improve children's cognitive
achievements? To address this question, we need to have some idea about the nature of
the production function for cognitive skills, that is, how do various inputs, inputs at home
and school, translate into test scores.
The paper we discuss is Todd and Wolpin (2007).
Q1: If we see a racial gap in test scores at age 14, say, should we attribute this only to
factors that is prior to age 14?
This is the key question, and the different answers to this question is the main reason for
the different opinions as expressed in the debate between Heckman and Krueger.
pre-market factors vs. post-market factors.
Pre-market factors are broadly interpreted to represent family and school influences on
skills and abilities valued in the labor market.
Post-market factors refer to events that would happen after one finishes school and enters
labor market.
Why would post-market factors matter for test scores at age 14, before one even finishes
schooling?
The reason is that investment decisions are made with expectations about future labor
market returns. (You will have no incentives to invest in human capital, other than the
pure consumption of value of knowledge, if you do not expect to have any job prospects!)
Thus, theoretically it is very possible that racial gaps emerge in adolescence due to postmarket factors because, for examples, blacks and hispanics expect to face discrimination
in the labor market which will dampen their returns to skill investment.
Heckman and his coauthors argue that post-market factors are unlikely to explain the test
score gaps we see in children as young as 7. But the evidence is not conclusive.
New surveys, such as the NLSY 1997 (a new cohort of National Longitudinal Survey of
Youth) started asking a lot of questions about expectations.
For example: in the baseline survey, parents are asked the following set of questions
regarding events that may happen when their children's outcomes when he/she is 20 years
old:

The next questions ask about [name of youth]'s school and work situation at the time
of [his/her] 20th birthday. What is the percent chance that [he/she youth] will have
received a high school diploma by the time [he/she youth] turns 20?

What is the percent chance that [name of youth] will be serve time in jail or prison
between now and when [he/she] turns 20?

What is the percent chance that [name of youth] will become the [mother/father] of a
baby sometime between now and when [he/she] turns 20?

Now think ahead to when [name of youth] turns 30 years old. What is the percent
chance that [name of youth] will have a four-year college degree by the time [he/she]
turns 30?

What is the percent chance that [name of youth] will be working for pay more than 20
hours per week when [he/she] turns 30?
Moreover, in the same survey, the youths are also asked a bunch of questions about their
expectations about the future:

The next questions ask about your school and work situation at the time of your 20th
birthday. What is the percent chance that you will have received a high school
diploma by the time you turn 20?

What is the percent chance you will serve time in jail or prison between now and
when you turn 20?

What is the percent chance that you will become the [mother/father] of a baby
sometime between now and when you turn 20?

What is the percent chance that you will die (from any cause -- crime, illness,
accident, and so on) between now and when you turn 20?

Now think ahead to when you turn 30 years old. What is the percent chance that you
will have a four-year college degree by the time you turn 30?

What is the percent chance that you will be working for pay more than 20 hours per
week when you turn 30?
Analysis from these expectations show some very interesting patterns:
See the graph in the next page. (taken from "TEEN EXPECTATIONS FOR
SIGNIFICANT LIFE EVENTS" by BARUCH FISCHHOFF et al. 2000 in Public
Opinion)




Significant pessimism among youth about the probability of death by age 20;
Significant pessimism among youth about the probability of serving time in jail or
prison before one turns 20.
The pessimism were true for all races and both genders; but more so among
blacks than among whites.
If one thinks that he/she may be dead by 20, they certainly would have little or no
incentive to invest in schools!
We don't know yet why different expectations about the future emerge among the youth.
But it could be related to what they see happening to the adults in their social network.
Such evidence suggests the importance of post-market factors in shaping the racial gap in
test scores at early ages.
What are the racial gaps in test scores that we are talking about?
See Table 1 in Todd and Wolpin (2007), but one can also get similar pictures by looking
at the NCES (National Center for Education Statistics) reports.


here we again run into the issue of how to compare the distributions of test scores
in two populations.
explain here the percentile scores
Q2: Suppose that we want to put aside the debate on the importance of pre-market factors
vs. post-market factors in affecting the children's test scores. Suppose for the moment that
we could not do too much to change the post-market factors. What can we do to narrow
the test score gaps? This requires us to understand the importance of various pre-market
factors in the production of cognitive skills. This is the task undertaken in Todd and
Wolpin (2007). In particular, they want to distinguish the effects of home inputs, school
inputs and family background in accounting for racial/ethnic differences in achievement.
Ideally, in analyzing cognitive achievement of children, it would be useful to have data
on all past and present home and school inputs, as well as information on children's
heritable endowments. But no data set is so comprehensive. Thus researchers have to
confront problems of missing or imprecisely measured variables.
Just to give you a hint of how difficult the empirical problem is, let us imagine that you
want to ask whether a small class size will improve test scores. So suppose that you hire
more teachers in your school, and the class size is lowered. Even better, you may
randomly choose which classes get more teachers. Then you can compare the end-of-year
test scores of classes with high or low Pupil-teacher Ratio. Can you attribute the
difference to class size (which is a school input here)? The main challenge is that home
inputs may respond. Parents may react to the increased attention their children are getting
from teachers in school by reducing home tutoring, for example (this is a case of
substitution between school inputs and home inputs); or the opposite can happen as well:
parents may find reading to their children more productive after their children get more
prepared for reading in school (this is a case of complementarity between school inputs
and home inputs). The problem is that, without measuring home inputs, we are unlikely
to find out the true effect of school inputs on achievement.
Some attempts in the literature:
1. School fixed effects to address the issue of unmeasured school inputs: under the
assumption that children within the same school receive the same school inputs.
(Murnane, Maynard and Ohls (1981).
2. Sibling fixed effect to address the issue of missing home inputs under the assumption
that children at the same household receive the same home inputs. Rosenzweig and
Wolpin (1994) and Altonji and Dunn (1996)


Big problem: Results differ substantially depending on the adopted empirical
approach.
Todd and Wolpin (2007) paper also proposes a cumulative production function
for children's cognitive achievement, i.e. children's achievement at a given age
depends on lifetime history of family and school inputs as well as on mother's
abilities and unmeasured heritable endowments.

Describe Todd and Wolpin's Approaches to Modeling and Estimating the
Production Function for Achievement
See extracted pages 5-9 from Todd and Wolpin paper

Describe their estimation results.
See extracted tables 3a and 3b from Todd and Wolpin paper

An idea for how to select various models: random holdout sample and crossvalidation
There are many models that one can think about estimating. How do we know which
model is better than others? This has been one of the most important research areas in
empirical economics. The hypothesis here is that, if the model is good, we will be able to
extrapolate one's estimates from an estimation sample to other samples that we did not
use in our estimation -- the so called "hold out sample".
How can one measure whether one model's ability to extrapolate to the hold out sample is
better than other models? Root of Mean Square Errors (RMSE)
See Table 5a extracted from Todd and Wolpin.
Finally, what do the estimation results tell us about the sources of racial gap in test scores?
Table 6a and 6b from Todd and Wolpin.
Conclusion:

Across almost all the empirical specifications considered in the paper, the authors
found that mother's accumulated ability, as measured by her AFQT, and home
inputs (both contemporaneous and lagged) are substantive determinants of
children's test scores in math and reading.

The estimated magnitude of lagged home input effects is often similar to that of
the current inputs and the effect of current inputs tends to be overstated in
specifications that ignore lagged input effects.

The coefficients associated with the school inputs were for the most part not
precisely estimated.

Cross-validation using hold out samples generally provides the most support for
the value-added plus lagged inputs model.

About half of the racial test score gaps (both for black/white gap and
hispanic/white gap) are accounted for by the differences in mothers' AFQT test
scores; differences in home inputs account for about 10-20 percent of the test
score gaps. Differences in school inputs and in mothers' schooling account for a
very small portion of the test score gaps.
Discussion: What are the policy implications?


Do not have direct answer to the question of what is the most efficient way to
reduce the test score gap.
But does point toward early intervention because of the finding that lagged inputs
have persistent effects. ("More bangs for the buck" if spent early!)