Efficiency or equality: why should we choose? The
differentiated effect of high schools on students’ achievements
Pauline Givord (INSEE-CREST)∗
Milena Suarez (INSEE-CREST) †
PRELIMINARY VERSION – DO NOT CITE NOR CIRCULATE
Abstract
This article proposes a new value-added statistical model, which encompasses the ability of schools
to mitigate or intensify the expected differences in grades between students of similar background.
Measuring the performance of a school requires to control for the student characteristics (notably
prior performance). However, while usual value-added indicators measure inter school performance,
they do not account for differences in intra school achievements. Some schools may improve mostly
the achievements of already high-performing students, while others help the students the most likely
to drop out. We thus propose to estimate value-added indicators at different levels of the distribution
of final achievements. We apply this method to exhaustive data of the 2015 French high-school
diploma. We find evidence that almost one third of the French high schools behave in a "egalitarian"
or on the contrary unequal way, meaning that they significantly reduce, or on the contrary increase,
the dispersion in final grades which were expected given the school population. We observe that the
highest performing high schools on average are over-represented among the egalitarian high schools.
Key words: Value added; quantile regression; Student Growth Percentiles
JEL Classification: I24; Value added; quantile regression; Student Growth Percentiles
Introduction
Assessing the performance of schools has become standard in most developed countries. A first aim
is to provide tools for public action, for example by evaluating the effectiveness of resources allocation
between schools. These evaluations may also aim at providing useful information to families, as long
as they can choose the school of their children. Several performance metrics have been proposed in
the literature. They are generally focused on the average school impact, which may mask significant
disparities in educational investment among students. In this paper, we propose to complement the
classical school value-added indicator by a measure of equality of this effect among students. We aim
at measuring the capacity of schools to teach equally to all or conversely give priority to some specific
students, by reducing or increasing the expected dispersion given student characteristics.
Evaluating school performance raises a number of issues on the definition, the estimation but also the
interpretation of school performance indicators. First, to measure school efficiency one must dissociate
∗ 18
† 15
Boulevard Adolphe Pinard, 75014 Paris; [email protected]
Boulevard Gabriel Péri, 92240 Malakoff; [email protected]; 01 41 17 60 18
1
school’s own effect on students success from the many factors influencing academic achievement (initial
academic level but also family environment). Students are not randomly assigned to schools, and the
differences in the mean school performances also reflect this unequal assignment of pre-existing potentials.
For example, a school that selects students based on their academic achievement is expected to have very
high success rates, with little effort from its teaching staff. As pointed out by Raudenbush and Willms
[1995], and given the difficulty of obtaining experimental data, it is generally impossible to isolate the
effect attributable to pedagogical action from effects related to its composition (in particular, peer effects).
However, one may more modestly measure the difference between a child’s actual performance and the
performance that would have been expected if he had attended an average school.
Several types of indicators have been proposed to isolate the school effect from these selection effects.
The most classic ones are based on so-called value-added models (Koedel et al. [2015], Raudenbush and
Willms [1995]). Their principle is to model the success of students given their individual characteristics
(as measured over the whole population), plus a random or fixed effect characterizing the specific action
of the school. This effect characterizes what exceeds or conversely reduces the expected mean level, given
the school enrollment. In their simplest form, the value-aded estimates are based on a linear hypothesis:
the expected progression of the students in the school is assumed to be identical, once their individual
characteristics are taken into account.
On the contrary, the individual dimension is put forward by the indicators known as Student Growth
Percentiles (SGP) (Guarino et al. [2014], Tzavidis and Brown [2010], Walsh and Isenberg [2013]), originally developed in some American states. These indicators are based on an estimate of the individual
progression for each student, using quantile regression of actual score on past scores. In practice, the
SGP is the student’s grade percentile in the distribution of grades conditional on his past scores. School
performance is defined by agregating student progressions, as estimated by this model. These indicators
are based on a fairly intuitive principle therefore readable by all actors on the ground.
However, as pointed out by Guarino et al. [2014], Walsh and Isenberg [2013], controlling only for
students’ initial level can lead to inequitable estimates of the schools efficiency. This will be the case if
the family context also affects school progression, as evidenced by many previous works. Overlooking
the social composition can lead to underestimate the pedagogical action of schools that welcome a
more disadvantaged population. This question is all the more sensitive in a context where, as in some
American states, the means depend on these performance measures. Some schools may be tempted to
exclude students from disadvantaged backgrounds, amplifying social segregation. A relevant evaluation
must correct as much as possible for the observed characteristics correlated to students’ success and
student’s sorting (Rothstein [2009]). This requires detailed data, in particular with information on both
initial performance and social background.
A second question related to the measurement of the performance of the schools is their ability to
make all their students succeed, beyond an average performance. In fact, as SGP estimates show, there is
a great heterogeneity in individual performances within the same school. Apparently equally-performing
2
schools may reveal varying investments in the achievements of their students, depending on their profile:
for example, some may enable only their initial high-performing (or on the contrary low-performing)
students to progress the most. Average results can be pushed upward by some of the students at the
cost of the failure of others, or may be a signal of modest but homogeneous results for all.
Highlighting this within school heterogeneity is essential to draw an accurate picture of school performance. The school system equity, and its ability to benefit to all students is undoubtedly a current
concern. Following the diffusion of international comparisons in particular, the capacity of school systems
to combine efficiency and equity is questioned. In particular, it is necessary to evaluate the capacity to
produce many «high-performing» students but also to prevent less-favored students’ drop out.
This article therefore proposes indicators of school value-added that encompass overall performance,
both in level and spread. These indicators distinguish schools whose performance is average, but where
the majority succeed from those where the benefits only go to the most brilliant students. In practice,
these models are inspired from both value-added and SGP models. We estimate the school specific effect
adjusting for individual controls (initial performance, social origin, gender) using quantile regressions.
From these models we construct a typology of schools, depending on whether they reduce or increase
the dispersion of performance (compared to those predicted by the model). The more or less egalitarian
behaviours of schools may also be confronted with their effectiveness, as measured conventionally.
Estimates are based on an exhaustive database on French high schools, focusing on success at the 2015
high-school diploma (French baccalauréat), the final national exam of secondary school (cf Duclos and
Murat [2014]). These data provide detailed grades but also information on the individual characteristics
of the students. In particular we measure their social position index (cf Rocher [2016]) and their initial
performance as measured by a pre high school national exam (brevet des collèges). A quantile regression
is applied to several quantiles (second decile, median and eighth decile), to estimate school effect on the
success of its students at different levels of the distribution.
Recent contributions to the econometric literature on quantile regressions establish convergence results in panels data (Koenker [2004], Canay [2011], Kato et al. [2012]), which can be extended to the
framework of grouped data (high school × students). Two important differences must be emphasized.
On the one hand, whereas in panel data, fixed effects are rarely of interest per se, there are here the
main parameters of interest. Moreover, the number of observations per high school, on which the estimation of the fixed effects is based, is much higher here than in the standard case of panel data (where
there are generally only a few observations per individual). This limits the risk of biasing our results
due to the incidental parameter problem. In our non-linear framework, the measurement error due to
fixed effects could be reflected in all the coefficients. We verify that our estimates do not appear to be
affected by this bias. In practical terms, the numerical estimation of a large number of coefficients can
be computationally complex. We use the adaptation of the Frish Newton algorithm for sparse matrices
proposed by Koenker and Ng [2005]. We use a boostrap procedure to compare fixed effects estimated at
different levels of distribution for a single high school.
3
By comparing school effects in the top and the bottom of the distribution of final grade, we distinguish
high schools whose effect is more "egalitarian" in the sense that they tend to reduce performance gaps
between students compared to what would be expected given their recruitment, "inegalitarian" high
schools, which on the other hand tend to increase achievement differences. In the general stream, the
usual stepping stone to university, we observe that these two categories account for almost a third of the
high schools, in an almost equal proportion. The other high schools have a "homogeneous" effect on all
their students, in the sense that their impact, positive or negative, does not have a significantly different
effect at the top and bottom of the distribution. This classification can be crossed with the fixed effect as
measured at the median level, which is close to the "classic" value-added indicators (which correspond to
the average effect). The two dimensions, mean performance or egalitarianism, bring different elements,
and are not perfectly correlated. Nevertheless, there is an over-representation of positive value-added high
schools (in the sense that the fixed effect measured at the median level is significantly positive) among
the egalitarian high schools, and an under-representation of these high schools among the unequal high
schools. We verify that this typology is robust to student mobilities, which may partly reflect strategic
management of certain schools.
The following section presents the econometric model, detailing in particular the issues related to
fixed-effect quantile estimation and the interpretation that can be made of these estimates. The third
part describes the data, and the fourth details the results obtained. The fifth part discusses the robustness
of the results, particularly related to the consideration of possible selection of students during the high
school years.
Methods
The quantile regressions describes the effect of an explanatory variable at different levels of the distribution of the variable of interest.
qτ (Yi |Xi ) = m(τ ) + Xi β(τ )
(1)
The Students Growth Percentiles (SGP) rely on this model estimated on all students, where the
covariates X are reduced to past test scores, to explain the present score Y . Each student is associated
with a conditional distribution of final results Y , that of his fellows in terms of X. Its final score
determines at which quantile it is located in this distribution. Its positioning, between 0 and 1, defines
its SGP. Thus, each student is placed in the distribution of the achievements of other students who have
similar past test scores to deduce his rank. More formally, the rank of student i is computed from
SGPi = max{τ : qτ (Y |Xi ) ≤ Yi }
A school j is assigned an aggregation measure of its students’ SGP, for example τ̂j =< τ̂i >i∈j . A limit
of this model, as emphasized by Walsh and Isenberg [2013], is that it omits dimensions that can explain
4
variying rate of progression, at identical initial academic level. This model, therefore, could wrongly
attribute credit to school action whereas good results could be due to a more favored composition.
Controlling for observable characteristics that can affect academic success is therefore important from
an equity perspective. Indeed, students are not randomly assigned to high schools, and it is important
to control these effects. However, it is not sufficient to introduce into a SGP model the same variables as
those introduced in an value-added model to address the issues raised by this unequal repartition. If the
characteristics of the students are also correlated with the expected effectiveness of the high schools (for
example, if students from a privileged background have easier access to the "best" high schools where the
most effective teachers are), a SGP model overlooks these selective matches, hence will produce biased
estimates. Guarino et al. [2014] compare by simulation the multilevel model with fixed effects and the
SGP model, with for only control past score, in a case where the students are not randomly assigned
to a teacher but where a selection process operates. When students are assigned to school by "dynamic
grouping", the two types of model have similar performances, the SGPs models allowing to control the
effects of prior level. On the other hand, if the professors are moreover allocated non-randomly according
to their effectiveness, or value-added (dynamic grouping and positive or negative assignment), the SGP
model provides a classification of teachers less correlated with the real classification than the multilevel
model with fixed effects. The hypothesis of absence of correlation between the characteristics of students
and the value added of the high school is not necessary for multilevel models, the estimation of the VA
model is therefore convergent, whereas the theoretical framework of the global quantile regression used for
SGPs is not suitable. In the same way as for the mean effect, introducing a fixed effect into the quantile
regression responds to this criticism, while providing an estimator with an interesting interpretation.
Moreover, in our sample, the selection effect related to the social position index appears to be massive:
the introduction of fixed effects reduces the role of the social background in determining the achievement
of students, indicating that the well-to-do students probably benefit from the best-performing schools.
qτ (Yij |Xij , 1{i ∈ j}) = m(τ ) + Xij β(τ ) +
|
{z
}
Expectation
αj∗ (τ )
| {z }
(2)
School effect
Here, the distribution of baccalauréat grades is taken as a function of the individual characteristics of
the student, initial level, social position index, gender and a repeating dummy, which can be seen as an
expected distribution. An effect specific to the high school is added and varies by quantile. By nature,
this statistical approach is collective and adapts poorly to an individual interpretation except if one is
willing to make the restrictive hypothesis of ranks invariance. This is equivalent to assuming that after
a marginal change of a covariate, which corresponds to a passage from one conditional distribution to
another, the ranks of the students in the two distributions are the same.
The school effect modifies the inverse distribution by adding or subtracting an effect to different
quantiles. For example, a positive value of α∗ (τ = 0.2) means that the grade reached by 80% of students
is higher than what was expected given the high school students’ characteristics. The sign of these
coefficients reveals whether or not the distribution of scores falls short of expectations. We can then
5
α(τ = 0.5)* > 0
0.5
0.5
Whole population
0.4
0.4
0.3
0.3
In the high school
0.2
0.2
0.1
0.1
α(τ = 0.2)* < 0
α(τ = 0.5)* = 0
α(τ = 0.8)* > 0
0
2
0.0
0.0
τ=0.2
−5
0
5
−4
10
0.3
0.4
<
α2(τ = 0.8)*
0.1
0.2
α2(τ = 0.2)*
0.0
τ=0.2
−5
0
5
10
(c) Heterogeneous and positive effect : τ → α∗ (τ ) is positive
increasing
Figure 1: Interpreting α∗ (τ ) coefficients: type of school effects on the expected grade distribution
define a τ → αi∗ (τ ) function, which describes how these high school-specific effects evolve along the
distribution. A growing function (vs decreasing) suggests that the baccalauréat grades within this high
school are more (or less) scattered than expected. A series of examples of what can be expected is
presented in figure 1.
The high school effect will be said heterogeneous when this function is strictly monotonic. Rather
than a shift in the overall distribution of student outcomes, high school tends either to widen inequalities
in outcomes from what would be expected or to reduce these inequalities by making outcomes converge.
A translation of the results gives a constant effect at any quantile. Note that if a high school that
intensifies inequalities in outcomes has a more scattered distribution than expected, its impact can be
positive at any quantile. And conversely, a high school that reduces the inequalities of results can do so
by a negative effect on its highest quantile. We must therefore consider the two dimensions, sign and
monotonicity. Since inequalities in outcomes are greater at the exit of a high school characterized by an
increase in α∗ , we will refer to the school as "unequal". On the contrary, for a high school where results
6
4
(b) Heterogeneous effect: τ → α∗ (τ ) is increasing
0.5
(a) Homogeneous and positive effect : τ → α∗ (τ ) = α0 > 0
−2
are tighter than expected, the term "egalitarian" is used.
The estimation of the coefficients of interest is done by solving a convex optimization program for J
high schools:
({α̂j (τ )}1≤j≤J , β̂(τ )) = argmin
X
ρτ (Yij − µ(τ ) − αj (τ ) − Xij β(τ ))
(3)
i,j
where ρτ (u) = |u| (τ 1{u ≥ 0} + (1 − τ )1{u < 0}) et α1 (τ ) = 0. Indeed, one can show that qτ (Y ) =
argminq E[ρτ (Y − q)], equation 3 being the empirical counterpart (Koenker [2005]). To simplify the
P
P
1
1
interpretation, we normalize the coefficients with αj∗ (τ ) = αj (τ ) − |J|
l αl (τ ) et m(τ ) = |J|
l αl (τ ) +
µ(τ ).
If guarantees of convergence exist (Kato et al. [2012]), to our knowledge there are no results of inferences on fixed effects by quantile. Moreover, in general, the results of inference on the coefficients of
the quantile regression are restricted to a very specific framework when it comes to estimate the asymptotic variance. It seems more reasonable to prefer empirical sampling methods such as the bootstrap to
restrictive assumptions allowing a complete estimation of the asymptotic variance, and it is the choice
we make here. A bootstrap sample is stratified by high school, with intra-high school sampling with
replacement, the sampled unit being the student, i.e. the score he obtained at the baccalauréat and his
individual characteristics.
Significance criteria are based on the B = 500 bootstrap b draws. The 95% confidence intervals
∗,(b)
computed with [q0.025 ({ατ
∗,(b)
}b∈B ), q0.975 ({ατ
}b∈B )] allow to conclude whether the effect is significant
∗,(b)
∗,(b)
∗
∗
or not. If the empirical probability of order conservation Pb (order(α0.2 , α0.8 ) = order(α̂0.2
, α̂0.8
)) is
greater then 90%, the high school is assigned to the category of corresponding heterogeneous effect
"unequal" or "egalitarian", otherwise, the effect is assumed homogeneous.
Data
The analysis is based on several sources: the exhaustive database of the results at the baccalauréat
national exam, matched with the FAERE database (Fichier anonymisé d’élèves pour la recherche et les
études, i.e. anonymous file of students for research and studies), data obtained from the school monitoring
by the ministry of Education. The FAERE database contains the results at the DNB (diplôme national
du brevet), the national pre high school diploma. This exam is normally taken three years before the
baccalauréat. The DNB grade used is a raw score, ie before harmonization and resit: it represents a
homogeneous initial level for all students, with the limit that candidates for the 2015 baccalauréat did
not all take the exam the same year; an indicator of repeating a class before entering high school, and
student characteristics such as gender, socioprofesional characteristics of parents, and age. The social
position index is that of Rocher [2016]. The school of each student in the year of the baccalauréat is
recorded in both bases, while the FAERE files allow to recover the past schools history of students. The
7
APAE database contains information on the high school: sector, staff, teaching options...
Table 1: Characteristics of students applying for the baccalauréat 2015
Stream
General
Mean
sd
Technology
Mean
sd
DNB grade (past score)
Social position index
Repeaters
Girls
12.3
122.4
5.7%
56.5%
9.7
104.6
18.3%
50.0%
2.3
35.0
2.0
33.6
At the end of the first session of the baccalauréat exam, the candidate gets a grade, before resit but
after harmonization jury. The harmonization Jury may increase the score obtained at the first session
in order to allow a candidate to obtain 10, the threshold for passing the exam, or 12, 14 or 16, the
thresholds for particular distinctions (the raw grade, before harmonization, is not available). The final
mark (after the second session) obtained at the baccalauréat shows a very strong threshold effect at 10,
due to a large extent to the students who barely pass. We choose to work on the note at the first session,
i.e. the average of the scores on the tests with their respective weightings, which represents a relatively
20000
15000
Frequency
10000
10000
Frequency
10000
0
5
10
15
(a) DNB grades (raw prior score)
20
5000
0
0
0
5000
5000
Frequency
15000
25000
15000
20000
homogeneous test for the whole sample. This grade is reduced centered.
0
5
10
15
20
0
5
10
15
20
(b) Baccalauréat grades (first session) (c) Baccalauréat grades (after second
session, i.e. resit)
Figure 2: For candidates of the 2015 baccalauréat, empirical prior and baccalauréat grades distributions
The models are estimated separately for general and technology streams with very distinct distributions of grades. The general streams usually leads to university while the technology streams leads
to shorter higher education degrees. To estimate fixed effects that are not biased, we limit ourselves to
high schools with a "sufficient" number of students. In practice, only high schools with more than 65
students in general streams or 25 students in the technological streams are kept. The choice of these
thresholds is the result of a trade-off between the representativeness of the sample (95% of students are
kept both curriculum) and the ambition to estimate the parameters of a distribution by high school (it
seems reasonable to keep more than twenty students).
Finally, to account for the peculiarities in the grading in each série (stream) leading to the baccalauréat
exam, a fixed "série" effect is introduced. Each série has a different weighting for subjects and students
choose two years before taking the baccalauréat which exam path they will prepare. Thus, empirically,
8
Table 2: Students, high schools and sample restriction
High schools
Students
Students per school
Mean
Sd
Median
Candidates of the baccalauréat 2015
General streams
Technology streams
All
2248
1895
2521
340469
130887
471356
151.5
69.1
187.0
95.9
49.3
125.7
140
58
169
Sample restricted to headcounts ≥ 65 (general), ≥ 25 (technology)
General streams
Technology streams
All
1759
1549
2113
318222
122286
440508
180.9
78.9
208.5
82.3
46.3
118.2
166
67
195
at given initial level and equal characteristics, a student taking the scientific série(S) will on average
have a lower score than a student of taking the humanities série (L). This observation is probably due
to the fact that the students are graded and therefore compared between students within the same path.
Students taking the scientific série have more often than humanities students favorable characteristics
for academic success. In practice, when the grading is partly standardized, scientific students compete
with students who are brigther and with equal characteristics, their results are on average worse.
Results
The estimation of the quantile regression model with and without fixed effects is presented in the table
3, separating general and technology streams. Overhall, the distribution of grades is mainly determined
by the student’s past score. The social position index has a positive and homogeneous effect on grades
distribution. On the other hand, the results of the girls are both better and less dispersed, and the
opposite holds for those who repeated a class. Additionning the fixed effects is not neutral on the
estimates: their absence accentuates the measured impact of student’s characteristics, in particular that
of the social position index. This suggests a selection effect of the most favored students in the best
schools. Given the results obtained by Guarino et al. [2014], the addition of fixed effects appears as an
improvement of the SGP model, biased in the case of this type of selection.
The effects are normalized to zero by quantile, and their distribution is presented in the 4 table. At
the median, in the general streams, 50% of high schools have a significantly different from zero effect, in
the technology streams, 46%. Heterogeneous effects (equal or unequal) are detected for 28% and 26% of
high schools. The characteristics of these high schools are presented in table 5 and 6: in the egalitarian
high schools, the performances are much more often positive. In the general streams, 43.7% of egalitarian
high schools have a positive effect at the median (α̂∗ (τ = 0.5) > 0) and 19.9 % negative effects, i.e. +22.6
points et -9 points compared to "homogeneous" high schools. The egalitarian high schools are thus overrepresented among high-performing high schools. Compared to unequal high schools, their population
is more favored and more homogeneous, they are of smaller size and have fewer repeaters. Teachers
9
Table 3: Estimation results on control variables - β̂
With fixed effects
Quantile
20%
50%
Without fixed effects
80%
20%
50%
80%
General stream
Intercept
Prior score
Prior score squared
Social position
Repeaters
Girls
L (réf ES)
S
-0.705
0.593
0.107
0.079
-0.271
0.080
0.074
-0.194
(0.059)
(0.002)
(0.001)
(0.002)
(0.008)
(0.004)
(0.005)
(0.004)
-0.097
0.632
0.105
0.079
-0.245
0.052
0.086
-0.172
(0.082)
(0.002)
(0.001)
(0.002)
(0.007)
(0.003)
(0.005)
(0.004)
0.683
0.646
0.082
0.079
-0.193
0.032
0.088
-0.147
(0.143)
0.329
0.408
0.034
0.027
-0.228
0.189
0.291
0.511
0.624
0.717
0.456
(0.138)
(0.002)
(0.001)
(0.002)
(0.008)
(0.004)
(0.006)
(0.004)
-0.586
0.622
0.108
0.108
-0.284
0.080
0.065
-0.214
(0.004)
-1.004
0.388
0.009
0.060
-0.299
0.235
0.218
0.339
0.329
0.450
0.354
(0.022)
(0.002)
(0.001)
(0.002)
(0.008)
(0.004)
(0.005)
(0.004)
-0.004
0.655
0.105
0.106
-0.253
0.051
0.077
-0.184
(0.004)
-0.423
0.404
0.021
0.049
-0.268
0.206
0.235
0.368
0.428
0.579
0.407
(0.019)
(0.002)
(0.001)
(0.002)
(0.008)
(0.003)
(0.005)
(0.004)
0.607
0.661
0.079
0.102
-0.201
0.035
0.070
-0.162
(0.004)
0.176
0.409
0.029
0.043
-0.236
0.189
0.234
0.451
0.547
0.637
0.430
(0.024)
(0.002)
(0.001)
(0.002)
(0.008)
(0.004)
(0.005)
(0.004)
Technology stream
Intercept
Prior score
Prior score squared
Social position
Repeaters
Girls
ST2S (réf.HOT)
STD2A
STI2D
STL
STMG
-0.518
0.358
0.018
0.034
-0.285
0.242
0.205
0.362
0.371
0.500
0.360
(0.171)
(0.004)
(0.003)
(0.004)
(0.009)
(0.007)
(0.057)
(0.066)
(0.059)
(0.061)
(0.057)
-0.139
0.392
0.025
0.027
-0.258
0.211
0.229
0.385
0.464
0.604
0.397
(0.089)
(0.003)
(0.002)
(0.003)
(0.007)
(0.006)
(0.039)
(0.049)
(0.039)
(0.039)
(0.038)
(0.004)
(0.002)
(0.003)
(0.009)
(0.007)
(0.056)
(0.060)
(0.054)
(0.055)
(0.055)
(0.003)
(0.002)
(0.003)
(0.009)
(0.008)
(0.023)
(0.030)
(0.023)
(0.027)
(0.022)
(0.003)
(0.002)
(0.003)
(0.007)
(0.006)
(0.020)
(0.026)
(0.020)
(0.022)
(0.019)
(0.003)
(0.002)
(0.003)
(0.009)
(0.008)
(0.025)
(0.030)
(0.025)
(0.028)
(0.024)
The continuous explanatory variables are centered-scaled.Séries names: L: Humanities, ES: Economics and Scial Sciences, S: Sciences.
ST2S: Sciences and Technologies in Health and Social, HOT: Hotel and restaurants management
STD2A: Sciences and Technologies in Design and Applied Arts, STI2D:Industrial Science and Technologies and sustainable development,
STL:Laboratory Science and Technologies, STMG: Management Sciences and Technologies
are on average older and also more feminized. Half of the egalitarian high schools are private schools.
Access rates are particularly low, and this is mostly due to the private high schools of this category
(-2 on average against -0.5 in public schools of the same category for the access rate from first year of
high school to second year, -1 against -0.6 for the access rate from second to third year of high school).
Access rates are computed by the share of students going to the next level in the school among the
students present who are not repeaters in the school. It assesses a school’s ability or willingness to keep
its students throughout their schooling, allowing repeaters if necessary. These descriptive statistics may
suggest that homogeneous success may be the result of an eviction process during schooling.
These estimates provide a more accurate picture of the effectiveness of a high school. The figures 3
give an illustration of these estimates on two high schools: we compare the empirical distribution of the
results Yij |yij and the corresponding high school effect by quantile, as well as the distribution of SGPi
in the high school.
∗
The high schools where αi,τ
, τ ∈ {0.2, 0.5, 0.8} is increasing have indeed higher empirical variances
var(Yij |yij ) than high schools where the effect tends to bring achievements together.
Robustness
The estimation here is based on the use of a large number of fixed effects, estimated on a finite number
of points. One can worry about the problems associated with the imprecision that can result from the
estimation of these coefficients, and in return on the set of covariates coefficients of the models that
are correlated to them. This phenomenon, often called incidental parameter problem, is well known for
the panels (where the number of observations for the same individual T is often small compared to the
10
15
3
0.5
2
α*
Nbre d'élèves
10
1
0.0
0
Note bac 1ère session
5
−1
−0.5
−2
0
−3
Ensemble
Ce lycée
−3
−2
−1
0
1
2
3
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0
τ
Note Dnb
20
40
60
80
100
Rang de l'élève dans sa distribution conditionnelle
20
Nbre d'élèves
15
α*
0.0
0
0
−3
Ensemble
Ce lycée
5
−0.5
−2
10
−1
Note bac 1ère session
1
25
0.5
2
30
3
35
(a) Egalitarian high school
−3
−2
−1
0
Note Dnb
1
2
3
0.2
0.3
0.4
0.5
0.6
0.7
0.8
τ
0
20
40
60
80
100
Rang de l'élève dans sa distribution conditionnelle
(b) Unequal high school
Figure 3: From left to right: Distribution of baccalauréat grades conditional on prior score estimated in
the high school (red curves) and in the whole population (black curves); High school effect by quantile;
Distribution of SGPs in the school
11
Table 4: Estimation results: quantile fixed effects α̂(τ )
General stream
Quantile
Min.
q25
q50
Mean
Sd
q75
Max.
20%
50%
80%
-0.558
-0.144
-0.020
0
0.206
0.117
0.792
-0.532
-0.138
-0.011
0
0.193
0.114
0.741
-0.595
-0.127
-0.005
0
0.186
0.120
0.583
-1.300
-0.172
0.003
0
0.272
0.167
1.030
-1.118
-0.154
-0.006
0
0.245
0.143
0.868
-0.973
-0.165
-0.010
0
0.251
0.149
0.902
Technology stream
Min.
q25
q50
Mean
Sd
q75
Max.
number of individuals N ). It leads to prefer methods of estimation which get rid (by differentiation for
example) of the fixed effects (which are not of direct interest). In order to assess whether our estimates
are actually affected by this phenomenon, the alternative estimation of the impact of observed covariates
proposed by Canay [2011] is used. In the framework of quantile regression models, it adapts the within
differentiations by eliminating the αi (τ ) parameter. At the cost of hypotheses that are certainly stronger
on the model generating the data, we obtain a convergent estimate of the coefficients β(τ ) corresponding
to the observable variables, as well as an estimate of their precision. These estimates appear very close
to those obtained by explicitly estimating the fixed effects (table 7). We also compare the estimates of
the precision obtained by the bootstrap procedure and those obtained by the Canay procedure, which
appear to be practically identical (see figures 6,7 in the appendix).
In addition to the sensitivity of the coefficients to overparametrization, the robustness of the "label"
attributed to a high school can also be understood as the impossibility for the evaluated actor to manipulate the result. In particular, high schools may try to exclude students who are less likely to succeed in
the baccalauréat, or students whose results are far from the group on which the school is efficient. It is
possible, for example, to encourage a reorientation towards an academic path not available in the school.
Private high schools, which control their recruitment, may be induced to refuse the re-registration of
certain students on the basis of their level in order to obtain better rates of success. In this case, the
coefficients αi∗ (τ ) would be partially manipulable.
One way to quantify this phenomenon is to evaluate the high schools on students applying for the
baccalauréat 2015 present in 2014 in the school, regardless of the school they attended the year of the
exam. Thus, high schools which benefited from favorable arrivals or departures between the two years
12
Table 5: High school characteristics: homogeneous, egalitarian and unequal. General stream
Homogeneous
mH
Negative median value-added
Positive median value-added
Median prior grade (DNB)
Academic heterogeneity (DNB interquartile)
Median social position index (SPI)
Social heterogeneity (SPI interquartile)
% girls
% repeaters
Headcount
% teachers laureate of the agrégation
% teachers laureate of the certificat
Mean teacher age
%
%
%
%
%
%
teachers less than 30 years old
teachers between 30 and 50
teachers over 50 years old
teachers with years in the school < 2
- between 2 and 5
- between 5 and 8
% - >8
% part-time teachers
% women among teachers
Access rate 2nd to 3rd year
Access rate 1rst to 2nd year
Students per class
% students living in deprived areas (ZUS)
Discipline staff per student
% private high schools
Number
Unequal
mI − mH p-value
28.9%
21.1%
12.2
3.5
122.1
60.6
55.8
6.0
181.7
28.5
60.1
44.7
17.2
52.5
30.4
30.8
14.1
14.3
40.8
7.3
57.5
-0.3
-0.4
29.0
4.7
0.04
23.9
+0.7
-6.9
-0.4
+0.2
-6.6
+3.3
1.4
1.7
+36.1
+1.2
-1.1
-0.1
+0.3
-0.2
-0.1
-0.5
+0.4
-0.1
+0.2
-0.2
-2.3
+0.4
+1.0
+0.3
+2.5
+0.004
-16.6
1261
233
0.04
−
0.000
0
0.000
0.000
0.01
0.000
0
0.3
0.3
0.6
0.7
0.8
0.9
0.4
0.4
0.8
0.8
0.4
0.001
0.02
0.01
0.2
0.001
0.02
0.000
Egalitarian
mE − mH p-value
∗∗
∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗
∗∗∗
∗∗∗
∗∗
∗∗∗
-8.9
+22.6
+0.5
-0.3
+7.8
-7.5
0.03
-1.3
-19.1
+1.3
-2.7
+0.3
+0.1
-1.9
+1.8
-0.1
+0.7
+0.9
-1.5
-0.04
+3.7
-0.5
-0.9
+0.1
-1.3
-0.01
+27.9
261
The p-values are those of the equality tests of Welch’s (unequal variance) and chi2 tests
13
0
−
0
0
0
0
1.0
0.000
0.000
0.4
0.1
0.1
0.9
0.001
0.003
0.9
0.3
0.2
0.2
0.9
0.000
0.3
0.1
0.7
0.001
0.02
0
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗
∗∗∗
∗∗
∗∗∗
Table 6: High school characteristics: homogeneous, egalitarian and unequal. Technology stream
Negative median value-added
Positive median value-added
Median prior grade (DNB)
Academic heterogeneity (DNB interquatile)
Median social position index (SPI)
Social heterogeneity (SPI interquartile)
% girls
% repeaters
Headcount
% teachers laureate of the agrégation
% teachers laureate of the certificat
Mean teacher age
%
%
%
%
%
%
teachers less than 30 years old
teachers between 30 and 50
teachers over 50 years old
teachers with years in the school < 2
- between 2 and 5
- between 5 and 8
% - >8
% part-time teachers
% women among teachers
Access rate 2nd to 3rd year
Access rate 1rst to 2nd year
Students per class
% students living in deprived areas (ZUS)
Discipline staff per student
% private high schools
Homogeneous
mH
19.0%
16.8%
9.7
2.8
103.1
55.8
49.5
17.9
78.0
25.1
59.2
44.8
17.0
52.0
31.0
31.4
13.7
13.7
41.2
7.1
56.3
-0.4
-0.4
27.9
5.7
0.05
18.3
1149
Number
Unequal
mI − mH p-value
+2.8
0.2
-5.1
−
-0.5
0
+0.1
0.004
-5.9
0.000
+2.5
0.005
-0.2
0.9
3.2
0.000
+25.9
0.000
+1.2
0.2
+0.4
0.7
-0.5
0.02
+2.1
0.01
-1.1
0.1
-1.0
0.1
+0.9
0.2
+0.2
0.7
-0.7
0.2
-0.4
0.7
-0.8
0.004
-1.1
0.2
+0.5
0.1
+0.4
0.5
+0.3
0.3
+4.8
0.000
+0.002
0.6
-13.5
0.000
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗∗
∗∗
∗∗∗
∗
∗∗∗
∗
∗∗∗
∗∗∗
Egalitarian
mE − mH p-value
-4.2
0.04
+6.5
−
+0.4
0
-0.1
0.003
+2.7
0.02
-0.4
0.6
-0.1
1.0
-2.0
0.003
-6.6
0.04
-2.6
0.02
-0.6
0.7
+0.7
0.000
-2.3
0.000
+0.4
0.5
+1.9
0.01
-1.7
0.04
-0.7
0.1
+0.7
0.3
+1.7
0.1
-0.5
0.1
+1.3
0.2
-0.3
0.3
+0.1
0.8
-1.1
0.000
-2.3
0.000
-0.01
0.07
+22.1
0
188
∗∗
∗∗
∗∗∗
∗∗∗
∗∗
∗∗∗
∗∗
∗∗
∗∗∗
∗∗∗
∗∗
∗∗
∗
∗∗∗
∗∗∗
∗
∗∗∗
233
The p-values are those of the equality tests of Welch’s (unequal variance) and chi2 tests
Table 7: Comparison of the common parameters estimated by the Canay procedure and with fixed effects
to evaluate incidental parameter problem
With fixed effects
Quantile
20%
50%
Canay
80%
20%
50%
80%
Série Générale
Intercept
Prior grade
Prior grade squared
Social position
Repeaters
Girl
L (ref ES)
S (ref ES)
-0.705
0.593
0.107
0.079
-0.271
0.080
0.074
-0.194
(0.059)
(0.002)
(0.001)
(0.002)
(0.008)
(0.004)
(0.005)
(0.004)
-0.097
0.632
0.105
0.079
-0.245
0.052
0.086
-0.172
(0.082)
-0.139
0.392
0.025
0.027
-0.258
0.211
0.229
0.385
0.464
0.604
0.397
(0.089)
(0.002)
(0.001)
(0.002)
(0.007)
(0.003)
(0.005)
(0.004)
0.683
0.646
0.082
0.079
-0.193
0.032
0.088
-0.147
(0.143)
0.329
0.408
0.034
0.027
-0.228
0.189
0.291
0.511
0.624
0.717
0.456
(0.138)
(0.002)
(0.001)
(0.002)
(0.008)
(0.004)
(0.006)
(0.004)
-0.569
0.601
0.106
0.084
-0.273
0.079
0.072
-0.200
(0.004)
-0.562
0.372
0.015
0.040
-0.286
0.241
0.193
0.342
0.356
0.481
0.335
(0.023)
(0.002)
(0.001)
(0.002)
(0.007)
(0.003)
(0.005)
(0.004)
-0.011
0.633
0.105
0.080
-0.245
0.051
0.086
-0.173
(0.003)
-0.008
0.391
0.026
0.027
-0.254
0.212
0.226
0.378
0.459
0.603
0.389
(0.021)
(0.002)
(0.001)
(0.002)
(0.007)
(0.003)
(0.004)
(0.004)
0.573
0.643
0.084
0.076
-0.192
0.033
0.094
-0.148
(0.004)
0.562
0.398
0.035
0.022
-0.230
0.190
0.244
0.479
0.580
0.678
0.416
(0.020)
(0.002)
(0.001)
(0.002)
(0.009)
(0.004)
(0.005)
(0.004)
Technology stream
Intercept
Prior grade
Prior grade squared
Social position
Repeaters
Girl
ST2S (ref.HOT)
STD2A
STI2D
STL
STMG
-0.518
0.358
0.018
0.034
-0.285
0.242
0.205
0.362
0.371
0.500
0.360
(0.171)
(0.004)
(0.003)
(0.004)
(0.009)
(0.007)
(0.057)
(0.066)
(0.059)
(0.061)
(0.057)
(0.003)
(0.002)
(0.003)
(0.007)
(0.006)
(0.039)
(0.049)
(0.039)
(0.039)
(0.038)
The continuous explanatory variables are centered-scaled.
14
(0.004)
(0.002)
(0.003)
(0.009)
(0.007)
(0.056)
(0.060)
(0.054)
(0.055)
(0.055)
(0.003)
(0.002)
(0.003)
(0.008)
(0.007)
(0.023)
(0.029)
(0.023)
(0.027)
(0.023)
(0.003)
(0.002)
(0.003)
(0.007)
(0.006)
(0.022)
(0.027)
(0.022)
(0.024)
(0.021)
(0.003)
(0.002)
(0.003)
(0.008)
(0.007)
(0.021)
(0.030)
(0.021)
(0.024)
(0.020)
could see their evaluation change. In the sample, 8.7% of arrivals and 5.6% of departures were identified.
Departures are underestimated because the school of the previous year is not known for 2% of students
(potentially all departures, as opposed to students staying in the same school). In addition, some
changes may occur outside the sample to smaller high schools. By evaluating the schools on their 2015
candidates population in 2014, we observe 9 % and 10.5 % school transitions (see table 8), but without
transition between extreme categories. More mobility and / or specific SGP compositions suggest that
students’ selection (active or passive) may influence categorization: coefficients change where there is
movement. For example, high schools classified in the "homogeneous" category on the 2014 population,
while classified as "egalitarian" in 2015, had an average of +1.4% of departures, +1.8% of and arrivals’
SGPs were 5.7% higher than arrivals in high schools who did not change labels. The access rates to the
last high school year confirm these observations: they are much lower in those who pass from the label
"homogeneous" to the label "egalitarian"(figure 4). However, these changes appear possible only at the
margin since there is no transition between high schools classified as egalitarian and classified as unequal.
Table 8: Transition between categories when taking into account the mobility of students
General stream
Students 2015/Students 2014
U
H
E
U
H
E
205
44
0
28
1161
40
0
52
220
Students 2015/Students 2014
U
H
E
U
H
E
148
34
0
37
1067
51
0
43
171
Technology stream
15
0
−2
−6
−4
Acess rates
2
4
6
Figure 4: Access rate from 2nd to last year of high school according to the change of label between 2014
and 2015
Same Cat
Hom−>Uneq
Hom−>Ega
Uneq−>Hom
Ega−>Hom
References
Ivan A Canay. A simple approach to quantile regression for panel data. The Econometrics Journal, 14
(3):368–386, 2011.
Marc Duclos and Fabrice Murat. Comment évaluer la performance des lycées. Éducation & formations,
(85):74, 2014.
Cassandra M Guarino, Mark D Reckase, Brian W Stacy, and Jeffrey M Wooldridge. A comparison of
growth percentile and value-added models of teacher performance. working paper# 39. Education
Policy Center, Michigan State University, 2014.
Kengo Kato, Antonio F Galvao, and Gabriel V Montes-Rojas. Asymptotics for panel quantile regression
models with individual effects. Journal of Econometrics, 170(1):76–91, 2012.
Cory Koedel, Kata Mihaly, and Jonah E Rockoff. Value-added modeling: A review. Economics of
Education Review, 47:180–195, 2015.
Roger Koenker. Quantile regression for longitudinal data. Journal of Multivariate Analysis, 91(1):74–89,
2004.
Roger Koenker. Quantile regression. Number 38. Cambridge university press, 2005.
Roger Koenker and Pin Ng. A frisch-newton algorithm for sparse quantile regression. Acta Mathematicae
Applicatae Sinica, 21(2):225–236, 2005.
Stephen W Raudenbush and JDouglas Willms. The estimation of school effects. Journal of educational
and behavioral statistics, 20(4):307–335, 1995.
16
Thierry Rocher. Construction d’un indice de position sociale des élèves. Éducation & formations, 90,
2016.
Jesse Rothstein. Student sorting and bias in value-added estimation: Selection on observables and
unobservables. Education, 4(4):537–571, 2009.
Nikos Tzavidis and James Brown. Using m-quantile models as an alternative to random effects to model
the contextual value-added of schools in london. 2010.
Elias Walsh and Eric Isenberg. How does a value-added model compare to the colorado growth model?
Technical report, Mathematica Policy Research, 2013.
17
Figure 5: Access rate in public and private schools (Left: general stream, right: technology stream)
F. Technologique
−10
−8
−6
−5
−4
−2
0
0
2
5
4
6
F. Générale
PR
PU
PR
18
PU
note_dnb
I(note_dnb^2)
0.05
−0.6
−0.5
−0.10 −0.05
−0.2
0.0
0.00
0.5
0.2 0.4 0.6
0.10
(Intercept)
1.0
1.5
2.0
2.5
3.0
1.0
1.5
2.5
3.0
2.0
2.5
3.0
2.0
0.05
1.5
2.0
2.5
3.0
2.5
3.0
1.0
1.5
2.0
−0.1
−0.05
0.0
0.00
0.05
0.1
0.10
0.2
serS
−0.2
−0.10
1.5
2.0
2.5
3.0
3.0
−0.05
1.0
serL
1.0
2.5
0.00
0.0
−0.2 −0.1
1.5
1.5
sexe_bac2
0.1
0.05
0.00
−0.05
1.0
1.0
retard
0.2
indice_pcs
2.0
1.0
1.5
2.0
Figure 6: General stream - Bootstrap and Canay confidence intervals
19
2.5
3.0
I(note_dnb^2)
1.0
1.5
2.0
2.5
3.0
0.00
1.0
1.5
2.5
3.0
2.0
2.5
3.0
2.5
3.0
0.2
−0.2
1.0
serST2S
1.5
2.0
2.5
3.0
1.0
serSTD2A
1.5
2.0
2.5
3.0
serSTI2D
1.0
1.5
2.0
2.5
3.0
−0.6
−0.4
−0.6 −0.2
0.0
0.2
0.4
0.6
0.0 0.2 0.4
2.0
0.0
0.2
−0.2
1.5
1.5
sexe_bac2
0.0
0.00
−0.04
1.0
1.0
retard
0.04
indice_pcs
2.0
−0.04
−0.6
−0.4
−0.2
0.2
0.0 0.2 0.4
0.04
note_dnb
0.6
(Intercept)
1.0
2.0
2.5
3.0
1.0
1.5
2.0
serSTMG
1.0
1.5
2.0
2.5
3.0
−0.6
−0.5
−0.2
0.0
0.2
0.5
0.6
serSTL
1.5
1.0
1.5
2.0
2.5
3.0
Figure 7: Technology stream - Bootstrap and Canay confidence intervals
20
2.5
3.0