Inter-American Development Bank (IDB)
The Quality of Education
in Argentina
An IDB Research Project
FINAL DRAFT
Research team: Sebastián Auguste, María Echart and Francisco Franchetti
January 2008
1
Table of Contents
I. Background and Objectives............................................................................................. 6
II. The Argentine Educational System.............................................................................. 11
II.1. A Brief History of the Argentine Educational System...................................... 11
II.2. Educational System Design............................................................................... 16
II.3. Argentine Educational System in Numbers ...................................................... 23
III. Data and Methodology................................................................................................ 27
III.1. Data .................................................................................................................. 27
III.2. Methodology .................................................................................................... 29
IV. International Benchmarking ....................................................................................... 36
IV.1. Structure and Coverage.................................................................................... 36
IV.2. Performance in International Tests.................................................................. 39
IV.3. Quality of Education from a Mincerian Perspective ....................................... 47
IV.4. Recent Evolution in the Quality of Education................................................. 48
IV.5. Equity in the Distribution of the Quality of Education.................................... 51
IV.6. Comparing the Educational Systems ............................................................... 58
V. Quality of Education with-in Argentina....................................................................... 78
VI. A Hierarchical Lineal Model with PISA .................................................................... 88
VI.1. Variance Decomposition ................................................................................. 88
VI.2. School Factors Related to Quality of Education.............................................. 94
VI.3. Oaxaca-Blinder Decomposition....................................................................... 96
Grouping Variables................................................................................................. 104
VII. A Hierarchical Lineal Model with PIRLS............................................................... 106
VII.1. Variance Decomposition .............................................................................. 112
VII.2. School Factors Related to Quality of Education .......................................... 114
VII.3. Oaxaca-Blinder Decomposition ................................................................... 117
VII. A Hierarchical Lineal Model with ONE ................................................................. 122
IX.1. Variance Decomposition ............................................................................... 122
IX. Peer-Group Effects in the Classroom ....................................................................... 129
X. Teachers and the Quality of Education ...................................................................... 132
X.1. Who are the teachers now? ............................................................................. 132
X.2. International Benchmarking............................................................................ 135
X.3. Recent Evolution............................................................................................. 138
XI. Conclusions and Policy Implications........................................................................ 143
References....................................................................................................................... 146
Annex A. Educational Indicators for Argentina.
Annex B. Educational Indicators. International Benchmarking.
Annex C. Measures of Inequality
Annex D. Measuring the Quality of Education from a Mincerian Perspective
Annex E. Peer Group Effect in the Classroom in Argentina
Annex F. Teacher Characteristics
2
Index of Tables
Table 1. Human Development Index Position/1............................................................................. 15
Table 2. Proportion of right answers, ONE test, Argentina .......................................................... 26
Table 3. Argentina compared to LATAM and Upper-Middle Income countries.......................... 37
Table 4. Argentina and selected comparators, 2004...................................................................... 39
Table 5. Performance of Argentina in International Tests (Language) ......................................... 39
Table 6. Evolution of Argentina compared to other Latin American countries ............................ 40
Table 7. Country characteristics and Expenditure in Education.................................................... 46
Table 8. Ranking by Implicit Quality of Education ...................................................................... 48
Table 9. Ranking by Implicit Quality of Education ...................................................................... 48
Table 10. Relative Evolution of Argentina in terms of Quality of Education ............................... 50
Table 11. Performance in PISA 2000 according the SES quantiles .............................................. 52
Table 12. Country Characteristics ................................................................................................. 60
Table 13. Performance in PISA..................................................................................................... 61
Table 14. Performance in PIRLS................................................................................................... 61
Table 15. Comparing PISA and PIRLS performance.................................................................... 63
Table 16. Other quality related indicators ..................................................................................... 64
Table 17. Repetition Rate by Grade .............................................................................................. 64
Table 18. Number of school hours per year .................................................................................. 65
Table 19. % of Students with Reading Problems. PIRLS ............................................................. 65
Table 20. Use of reading specialists. PIRLS ................................................................................. 66
Table 21. Parents’ Involvement. PIRLS........................................................................................ 67
Table 22. Reading Instructions. PIRLS......................................................................................... 67
Table 23. Teaching Time Allocation. PIRLS ................................................................................ 68
Table 24. Teaching methodology .................................................................................................. 68
Table 25. How teachers work with student groups ....................................................................... 69
Table 26. Use of Reading Instructional Material for students at different reading levels ............. 70
Table 27. Income Level and Educational Resources. PISA 2000 ................................................. 71
Table 28. Index Of Early Home Literacy Activities. PIRLS 2001................................................ 75
Table 29. Index Of Early Home Literacy Activities ..................................................................... 75
Table 30. Importance given by family to reading. PIRLS 2001.................................................... 75
Table 31. Availability of school resources. PIRLS 2001 .............................................................. 76
Table 32. Between-classes variation ............................................................................................. 79
Table 33. Between-schools variation ............................................................................................ 80
Table 34. Ranking of Provinces according to different measures of Quality of Education .......... 85
Table 35. Expenditure and Quality across provinces .................................................................... 86
Table 36. Percentage of variance in student performance in reading, mathematical and scientific
literacy .................................................................................................................................. 89
Table 37. Percentage of variance in student performance in reading, mathematical and scientific
.............................................................................................................................................. 91
Table 38. Variance Decomposition. Reading Literacy Achievement ........................................... 92
Table 39. Decomposition of the between school variation in explained and unexplained factors 93
Table 40. Estimated coefficients for school level in a 3-levels HLM model ................................ 94
Table 41. School Climate Factors ................................................................................................. 95
Table 42. School Resources Factors.............................................................................................. 96
Table 43. Oaxaca-Blinder decomposition for Similar development countries.............................. 97
Table 44. Oaxaca-Blinder decomposition for Similar Culture countries .................................... 101
Table 45. Oaxaca-Blinder decomposition for High performing countries .................................. 104
3
Table 46. Oaxaca-Blinder decomposition ................................................................................... 105
Table 47. Distribution of Reading Achievement......................................................................... 107
Table 48. Achievement in Reading for Literacy and for Informational Purposes....................... 108
Table 49. Percentages of Students Reaching PIRLS International Benchmarks in..................... 109
Table 50. HLM Estimations results............................................................................................. 115
Table 51. Length the students stay with the same teacher........................................................... 116
Table 52. Oaxaca-Blinder Decomposition, similar income level countries ................................ 118
Table 53. Variance Decomposition for between school and classrooms variation ..................... 127
Table 54. Estimated Coefficients for Classroom and School Levels .......................................... 128
Table 55. Estimated coefficients for the peer-group effects variables ........................................ 130
Table 56. Exploring the functional form of the peer group effect............................................... 131
Table 57. Evolution of the Teacher Composition........................................................................ 132
Table 58. Teachers’ parents education ........................................................................................ 135
Table 59. % of teachers with an ISCED 5A qualification in the language of assessment .......... 136
Table 60. Index of teacher participation in school decisions. PISA 2000 ................................... 137
Table 61. Teacher Characteristics ............................................................................................... 138
4
Index of Figures
Figure 1. Evolution of private sector enrollment share ................................................................. 14
Figure 3. Evolution of GDP per capita .......................................................................................... 15
Figure 4. Basic Education System in Argentina............................................................................ 19
Figure 5. GNI per capita and performance in PISA ...................................................................... 41
Figure 6. GNI per capita and performance in international tests................................................... 42
Figure 7. School life expectancy and performance in international tests ...................................... 43
Figure 8. Public Expenditure in Education as a % of GNI............................................................ 43
Figure 9. Public Expenditure in Education as a % of GDP ........................................................... 44
Figure 10. Public Expenditure in Education as a % of total expenditure ...................................... 44
Figure 11. Public Expenditure in Education per pupil as a % of GDP.......................................... 45
Figure 12. Change in relative ranking between 1980 measure and 2000 measure........................ 51
Figure 13. Difference in average score between top and lowest 20% according to wealth .......... 53
Figure 14. Difference in average score between top and lowest 20% according to household
socioeconomic index ............................................................................................................ 53
Figure 15. Difference in average quality between top and lowest 20% according to household
socioeconomic index ............................................................................................................ 54
Figure 16. Gini coefficient for test score....................................................................................... 55
Figure 17. Gini coefficient for quality........................................................................................... 56
Figure 18. Gini coefficient for parents’ SES ................................................................................. 56
Figure 19. Relationship between inequality in quality and inequality in SES at country level..... 57
Figure 20. Distribution of students according to number of books at home. PISA 2000.............. 72
Figure 21. Proportion of students with more than 50 books at home............................................ 72
Figure 22. Index of Home Educational Resources according to SES category. PISA 2000 ......... 73
Figure 23. Income level . PISA 2000 ............................................................................................ 73
Figure 24. Proportion of Students with more than 25 Child Books at home . PIRLS 2001.......... 74
Figure 25. Proportion of Students with minimum educational resources at home . PIRLS 2001 . 74
Figure 26. Index of school violence .............................................................................................. 76
Figure 27. Relationship between Average Test Score and GDP per capita (in ppp) at province
level) ..................................................................................................................................... 81
Figure 28. Relationship between the Coefficient of Variation in Test Score ................................ 81
Figure 29. Relationship between Test Quality and GDP per capita (in ppp) at province level) ... 82
Figure 30. Inequality and GDP per capita (in ppp) at province level) .......................................... 83
Figure 31. Between and within school variance decomposition with PISA.................................. 90
Figure 32. Average Reading Achievement and Early Home Literacy Activities........................ 110
Figure 33. Between and within school variation ......................................................................... 113
Figure 34. Ratio of students per teacher, PISA 2000 .................................................................. 136
Figure 35. Accumulated Distribution of teachers by deciles....................................................... 140
Figure 36. Teacher Distribution according to husband educational level ................................... 140
Figure 37. Accumulated Distribution of teachers by deciles....................................................... 142
5
I. Background and Objectives
Argentina is a middle-income country with an unsuccessful history in terms of economic
growth, which mirrors in education. Not a long time ago Argentina was regionally
considered a country with high level of human capital and good quality of education. It
was one of the first countries in Latin America to achieve almost full coverage in primary
level, it reached fairly high coverage in secondary education earlier than other countries,
and it is among the top countries in the region in terms of years of schooling and
proportion of adult population with tertiary education. The main issue now, rather than
coverage, is the quality of education.
The perception is that the quality of education in Argentina has deteriorated as a result of
low investment and lack of appropriate policies. Education seems not to be providing
equal opportunities any more, deepening the pattern of increasing inequality observed in
the last decades, in a globalized world where human capital is the key to economic
development and quality of life. As a consequence, the country is falling behind the
world and the region.
Nevertheless, the factors affecting the quality of education in Argentina remain relatively
unexplored; it has not been quantitatively proved whether the quality has deteriorated in
the last decades or what factors explain the difference in achievement in international test
between Argentina and the rest of world. We do not know much about what is working or
what is not working in the country, what we are doing differently from successful
countries, or what are the local needs and appropriate public policies. To fill that gap in
just one paper is a very ambitious goal, even risky, given the poor information available.
Our intention is to provide some evidence to help understand better the current situation
and to identify areas for further research. Since we are interested in policy implications,
we focus on those factors associated with the formal educational process that increase the
individual human capital beyond the students and families’ characteristics that affect the
learning process. In particular, our work analyzes the effects of primary and secondary
6
school inputs in the educational process, what lies in the “school effectiveness” branch of
the literature.1 In focusing on this topic, we exclude the very important analysis of the
effect of monetary inputs on achievement, more related to a cost-benefit analysis, what
Lockheed and Hanushek (1988) defines “school efficiency”. Clearly, public policies in a
world of scarce resources should be based on cost-benefit analysis, which gives a ranking
of priorities, but the production function analysis of the literature on school effectiveness
is the prerequisite for this more elaborated analysis.
The process of learning is complex, and the school effectiveness literature has had
problems to associate inputs with results. First, it is not easy to summarize in just one
dimension an educational process that is by nature multiproduct. In a multidimensional
product space quality is not univocally defined. The literature has followed two
alternative approaches: i) linking wages (or labor market outcomes) with school inputs or
characteristics; and ii) linking a measure of performance (typically student achievement
in a standardized test) with school inputs. Both have limitations. For instance, focusing
on labor market outcomes assumes that this is the only dimension that matters (what the
market demands) and assumes that the labor markets work perfectly and prices correctly
reflect scarcity. Using test scores, on the other hand, depends critically on the test design
and whether the test is capable of capturing what is optimal for the students to learn at
school.
Both approaches, thus, do not solve the problem of capturing correctly in one dimension
the process of education, and both are constrained by data availability. This led us to the
second problem in the literature, to measure inputs quantitatively. We can tautologically
define a good teacher, a good school, or a good policy as that one that increases the most
the (difficult to measure) human capital of the student, but it is not easy to measure what
characteristics are behind that input. The problem is that not only the product of
education is multidimensional, but also the main inputs, students and teachers, are also
multidimensional or heterogeneous, and there are many alternative production functions
1
There are several comprehensive reviews of this literature, see for instance Scheerens (1992), Scheerens
(2000), and Pennycuick (1998).
7
relating those inputs (such as school organization and system of incentives). All these
aspects are difficult to measure and much of the heterogeneity is unobservable for the
analyst, what generates technical problems to establish causality.
The ideal scientific experiment would be to study causality in education by carrying out a
succession of randomized experiments where we assign individuals to institutions and
‘treatments’ at random and observe their responses and performances. We would
randomly assign any chosen student, teacher, and school factors, a few at a time, to judge
their effects and thus, slowly, would discover which forms of organization, curriculum,
classroom composition, and so on are associated with desirable outcomes (Goldstein
(1997)). Obviously, in the real world we cannot do this, and natural experiments are not
very common. Most of the school effectiveness literature tries to infer causality in a nonrandom experiment framework where students, teachers, and other inputs are allocated
probably based on unobservable characteristics. A double selection problem is common:
a) students select schools and schools select students, but also b) schools select teachers
and teachers select schools. In this non-randomized world it is extremely difficult to
establish causality, and causality is critical for policy implications as we cannot base our
policy prescriptions in spurious relationships.
Therefore, despite of the measure of performance chosen or the way the
multidimensionality problem is solved, the most important limitation in the school
effectiveness literature is the econometric problem of identification. At best, value added
estimates can be used as crude screening devices to identify ‘outliers’, but they cannot be
used as definitive statements about the effect of a school per se (Goldstein & Thomas,
1996; Goldstein & Spiegelhalter, 1996).
It is not surprising, therefore, that the usefulness of this literature to provide policy
implications has been constantly challenged (e.g. Goldstein and Thomas (1996)). In fact,
according to Rowe, Hill, and Holmes-Smith (1995), current policy initiatives are poorly
supported by the available evidence, and a clear message is yet to emerge from the school
effectiveness literature.
8
The first studies (such as Coleman et al (1966), Jencks et al (1972), and Plowden (1967))
were pessimistic regarding what the school can add, finding that family and
neighborhood characteristics were much more important than school characteristics to
explain students’ performance, what led to the conclusion that, against what most people
think, school does not matter. But these papers, and most of the literature before the mid
1980s, based on simple OLS regressions, were contaminated with selection bias and
endogeneity. New techniques were designed to obtain better results, such as the
multilevel analysis (Mortimore et al (1988) being one of the first) and the natural
experiment approach; nonetheless, the technical problem of identification persists. There
is controversy about the impact of different school and teacher level variables, fed by the
diversity in the results.
As Hill and Rowe (1996) point out, very few studies satisfy the minimum conditions for
satisfactory inference, suggesting that few positive conclusions can be derived from
existing evidence. The minimum conditions can be summarized, Goldstein (1997), as:
1. a study is longitudinal so that pre-existing students’ differences and subsequent
contingent events among institutions can be taken into account
2. a proper multilevel analysis is undertaken so that statistical inferences are valid
and, in particular, so that ‘differential effectiveness’ is explored
3. some replication over time and space is undertaken to support replicability
4. some plausible explanation of the process in which schools become effective is
available.
In the particular case of Argentina, the data limitation problems are even more critical.
There is no way to link workers with the school where they studied to follow a labor
outcome approach, and all the standardized tests available (either national or
international) are cross sectional, what does not allow to capture the evolution of the
student, what is the ultimate goal of the school effectiveness approach.
9
Having these technical limitations in mind and given the data restrictions, this study
analyzes the quality of education for Argentina and the factors that affect it based on test
scores and following a comprehensive approach using different techniques. Basically, our
strategy to analyze the quality of education in Argentina has three components. A
qualitative analysis of the Argentine educational system and its recent evolution, an
international benchmarking analysis based on PISA and PIRLS, and a within-Argentina
analysis based on a local test (ONE) administrated to all the students in 6th grade. We
have combined simple benchmarking with econometric models such as Hierarchical
Lineal Models, Blinder-Oaxaca Decomposition, Juhn-Murphy-Pierce decomposition,
cluster analysis, and cluster regressions. Our goal, rather than trying to establish
causality, is to find stylized facts providing a rich and comprehensive descriptive
analysis, which, with all its limitations, results extremely useful for Argentina given the
lack of studies of this nature and the abovementioned data limitations.
The rest of the work is organized as follows. Section 2 briefly describes the Argentine
educational system and its recent evolution. Section 3 describes the data sets used in this
study and the econometric methodology. Section 4 does a comprehensive international
benchmarking to understand where Argentina is located and how Argentina differs from
other countries. Section 5 explores the variation in quality of education within Argentina.
Sections 6 to 8 make an econometrical analysis of the variation in internationally- and
locally-based test scores on a Hierarchical Lineal Model. Section 9 estimates peer group
effect in the classroom for Argentina. Section 10 analyzes the teachers’ characteristics
and the recent evolution. And finally Section 11 presents the main conclusions and policy
implications.
10
II. The Argentine Educational System
II.1. A Brief History of the Argentine Educational System
Argentina attained its independence in 1816, but it took several years for the country to
organize. A long civil war and armed conflicts delayed the organization of the country,
which finally started with the National Constitution in 1853.
The organization of the educational system started in 1852, but it was only in 1884, when
the 1420 Law of Basic Education was passed, that the National System started to shape.
The 1420 Law established the State’s obligation to provide public education to children
in schooling age. Any city or town with more than one thousand inhabitants had to have a
primary public school. This law had a very significant effect on schooling. In 1883 only
one third of the children between six and thirteen years old went to school, ratio that
increased to 50% in 1914, and 75% in 1931. The number of primary schools doubled in
the first 10 years, and they increased from 1,279 in 1880 to 10,776 in 1930.
In terms of the federal organization of the educational system, at the beginning primary
schools were the Provinces’ responsibility, but since 1890 the Federal government had
started to influence. First, it conditioned the transfers to provinces (since most of the tax
revenue in Argentina had been (almost from the birth of the country) collected at federal
level), fulfilling the directions of the National Council of Education. Second, in 1905, the
Làinez’s Law authorized the Federal Government to build schools and provide education
in those cities without provincial schools. In practice, federal schools were established in
almost every large city of the country, what created a somewhat asymmetric competition
between provincial and national schools. Usually federal schools were in a more
advantageous situation, with more resources, better paid teachers, and, consequently, a
reputation for better quality.
11
Federal schools were directly managed by the National Government, which imposed a
relatively homogenous standard across the country. Some argue that this federal
intermission was harmful for the system, for instance Aguerrondo (2006), but there is no
evidence supporting or against this hypothesis. In fact, it is possible that federal schools
helped to improve the quality, first increasing the school competition, and second, forcing
a more homogenous system.
As a consequence of the national school competition, the market share of provincial
schools fell quickly, and by 1930 only 59% of the total primary schools in the country
were provincial.
In 1978, under the military government, the advance of the federal government on the
provincial jurisdiction was reverted, with the transfer of 6000 national schools to the
provinces, what represented 25% of the total number of public school at that time, or
98% of the total public federal schools (the remaining federal schools were finally
transferred in 1992, so currently all the primary and secondary schools are managed by
the provincial governments). Some argue that the transfers were not done on the basis of
increasing quality or accountability but rather to diminish the burden of these schools on
the federal budget (Aguerrondo (1992)). The 1978 transfers (under the military
government) were a drastic change, mandatory and without an increase of the fiscal
revenue that the federal government transfered to the provincial government, what might
have affected the quality of the schools.
Secondary schools had developed at a slower pace. At the beginning of the XX, they
were restricted to the national elite only. In 1914, for instance, only 3% of the population
between 13 and 18 years old attended secondary school. The situation changed with the
Peronist government (1946-1955), as it generalized access, but by 1960, the net
enrollment rate was only 23% and increased slowly to 40% by 1970 (Wiñar (1974)) or
54% by 1991. By 1994, in a general reform, the first two years of the secondary
education became mandatory, and now around 80% of the students in secondary level age
actually attend school.
12
The secondary school curriculum in Argentina has always been biased to a humanistic
formation. Historically, there have been some attempts to articulate secondary schools
with labor market more by means of technical schools, but technical and agricultural
schools never had an enrollment share larger than 28%. With the reform in the 90s, they
were abolished, and with the 2006 reform they were reinstituted, what shows the typical
Argentine cycle of reforms and counter-reforms.
In international terms, as a consequence of the early promotion of its education,
Argentina was able to reach fairly good coverage and years of schooling early in the
century. By 1960, Argentina had, on average, 5 years of schooling for those over 25 years
old compared to 2.1 in Italy and 3.4 in Spain, the countries where most of the local
population comes, closer to the developed economies than the Latin American average.
After 40 years, Argentina has now the same years of schooling as the Latin American
average, and well below the average for Italy and Spain respectively, when in 1960
Argentina was 3 and 2.5 years above them.
Participation of the Private Sector
In the last century, the secondary and primary schools showed opposite trends in terms of
private sector participation. In the secondary level, the enrollment share grew quickly at
the beginning of the 1900s as a response to the demand for more education in a fast
growing country, demand that the public sector was not covering. By 1965, 32% of the
secondary school students were attending a private school. But since this peak, the
private sector has been steadily losing share, as the government has increased the number
of public schools, and at present the private sector share is around 25%. The recent
national trend has two patterns: in the richest provinces the private sector participation
has been increasing, but this has been overcompensated by the increase of the public
sector share in the poorest provinces, mainly due to the increase in the net enrollment
share (covered mostly by public schools).
13
In primary schools, the private school enrollment share, which was almost 20% at the end
of the XIX century, fell to 7.3% by 1940, as the state increased the public school offer in
its attempt to generalize this level of education. But since then we observe an increasing
share of private schools, which is more marked in the richest provinces such as Buenos
Aires. The development of the private school sector was facilitated by the Domingorena’s
Law in 1955, which made easier for the state to subsidize private schools (particularly
catholic schools). Nonetheless, the recent increase in private school share is more related
with a perception of a deterioration of the quality of public education (the most notorious
fact is the loss of school days per year due to teachers’ union strikes, which were very
common in the 90s).
Figure 1. Evolution of private sector enrollment share
0.35
Private Sector Enrollment Share
0.3
Secondary School
0.25
0.2
0.15
Primary School
0.1
0.05
0
1910 1920 1930 1940 1952 1955 1960 1965 1970 1975 1980 1985 1988 2000 2005
The problems
Since the middle XX century, the economy has started a stop-and-go (no long-run)
growth process; fiscal crises have become a recurrent problem, what has translated into
lack of investment in the educational sector, and less competitive teachers’ wages.
14
Figure 2. Evolution of GDP per capita
GDP per capita (1990 International Geary-Khamis dollars)
25,000
Australia
20,000
15,000
Italy
10,000
Argentina
5,000
02
99
20
96
19
93
19
90
19
87
19
84
19
81
19
78
19
75
19
72
19
69
19
66
19
63
19
60
19
57
19
54
19
51
19
48
19
45
19
42
19
39
19
36
19
33
19
30
19
27
19
24
19
21
19
18
19
15
19
12
19
09
19
06
19
03
19
19
19
00
0
Source: Madisson (2001)
By 1950 the country was 14th in terms of the Human Development Index, a similar
position than in 1900, but after 1950, the country has started to lose positions and at
present is 36th. Australia, on the other hand, a country with similar factor endowments as
Argentina at the beginning of the XX century, was able to hold its position among the top
5 countries according to this index.
Table 1. Human Development Index Position
Year
1900
1930
1950
1974
2004
Australia
4
9
4
7
3
Argentina
13
11
14
18
36
Notes: Index based on: life expectancy, literacy, school enrolment,
access to health care, GDP and other indicators. Position among
the top 50 countries.
Source: Angus Madison (2001). UNDP Human Development
Report 2005.
Regularly the outcome of the problems between the government and the teachers’ union
was strikes, which became more and more frequent in the 80s and 90s. The problems are
still present; a clear piece of evidence is the murder of a teacher in a violent provincial
strike this year.
15
The popular perception is that teaching as an occupation has lost social status, and today
only those who cannot access other, more profitable, professions choose to be a teacher.
In addition, the country has not made significant changes in the curricula or the way
teachers are trained (still it is a three-year, non-university instruction).
In addition, the system shows very high repetition rates and very high drop-out rates,
what means that many students do not finish school at the corresponding age. This has
been a historical problem in Argentina. For instance, for the cohort starting first grade by
1937, only 25% reached third grade on time, and only 19% reached sixth grade on time,
what means that for each of the 30 students starting primary school, only three finished
on time. The problem is less severe now, for each 100 students that start first grade, 54
finish the primary school on time; nevertheless, there is still a long way to go compared
to international standards. In 2004, for instance, just 10% of the cohort starting the school
that year repeated the first grade. The average repetition rate for the entire primary school
is 6.5% per grade, and the average drop-out rate 1.8% per grade; in secondary school it is
8.8% and 16.9% per year respectively. According to the teachers, by fourth grade, 60%
of the students still have serious reading difficulties.
Regional disparity in terms of access to education and school indicators are high, and
although the least developed provinces have been catching up with the national average
in the last decade, the ranking of provinces has not been altered in 100 years. The ranking
of provinces according to the illiteracy rate in the most recent Census is the same as the
obtained in the 1869 or 1895 Censuses. The pattern of inequality of school indicators
holds for other variables such as income level, infant mortality rates, poverty rates, and
other indicators related to the quality of life, what shows a structural problem.
II.2. Educational System Design
Recently there have been some new laws to impulse the sector. The two most important
ones are the Ley Federal de Educación (Federal Law of Education) (1993) and the Ley
16
de Educación Nacional (Naitonal Law of Education) (Law 26.206), sanctioned at the end
of 2006.
The Federal Law of Education, Law 24.195, tried to revise and reformulate the system.
This Law was intended to serve as a flexible framework to define the national and
jurisdictional responsibilities. The Federal Council of Education was created to provide
the main guidelines, and the mandatory years of schooling were extended from 7 to 10.2
Before the reform, Argentina had a 7-year primary school and a 5-year secondary school.
The system was changed to a new cycle called General Basic Education Level (EGB),
which comprised the previous primary school plus the first two years of the secondary
school, organized in three cycles of three years each, and the secondary school was
reduced to three years (called Polimodal), eliminating technical schools. The mandatory
schooling changed from the primary level (7 years) to the EGB (9 years) plus Initial
Education (Kinder 5).
The National Law of Education (2006) introduced some modifications to the previous
law, reverting some reforms, although granting more importance to the Federal Council
of Education. The new law defines the pre-school as a pedagogical unity since the child is
45 days old; consequently, child care institutions (“jardines maternales”) are also
included in the national system. It also reverts the structure to a primary (comprising
EBG 1 and 2) and secondary school (where the EGB 3 is added as a common cycle to all
the orientations). The law revitalizes technical schools, emphasizing a more labor-market
oriented education in the last three years of the secondary school. Some of the objectives
are: to deepen the theoretical knowledge as a whole according to the different
orientations (humanistic, social, and techno-scientific) and to develop important human
abilities so that students may access the production sector, incorporating work as a
pedagogical element. Therefore, Polimodal can be linked with the Technical Professional
Stage (TPS), offering training in more specific fields and granting the diploma of
2
The “Consejo Federal de Educación” (Federal Council of Education) is an organism in charge of
conducting the educational system composed by the Provincial Ministers of Education, the Minister of
Education of the Ciudad Autónoma de Buenos Aires, and the National Minister of Education. The purpose
of the Consejo is to establish the general principles of the educational policy for the whole country.
17
technician in the chosen specialty. The Common Basic Contents (CBC) were designed to
help all students acquire a series of basic competencies, conceptualize different fields of
knowledge and contribute to social and productive activities.
In addition, the 2006 law extends the mandatory years of schooling to the entire
secondary school, i.e., 13 years of mandatory schooling.
The Structure of the Educational System
Basic education comprises:
•
Pre-school level: For children from 3 to 5 years old; only Kinder 5 is compulsory.
The purpose of this level is to stimulate maturity, social integration, and the bond
between family and the educational institution. All the pre-school settlements are
authorized and monitored by the jurisdictional educational authorities.
•
General Basic Education Level (EGB): Three cycles of three years each. Some of its
objectives are “to achieve the acquisition and instrumental command of socially
significant values: oral and written communication, language and math operations,
natural sciences and ecology, exact sciences, technology and informatics, social
sciences and national, Latin-American, and universal culture.”
•
Polimodal Level: three years (from 15 to 17/18 years old). I
For all these levels, the school year begins in early March and ends in late November,
and, according to the Federal and provincial law, the government must guarantee of 180
days of instruction. A day of instruction comprises at least 4 hours, where most of public
schools have 20 hours a week for the whole curriculum, whereas private schools
(particularly in large cities) are increasingly offering full-day schooling (even though
many of these private schools are not bilingual). There are just very few full-day public
schools. In this sense, the increasing labor force participation of women might have also
worked in favor of private schools (in addition to the strikes in public schools).
18
Figure 3. Basic Education System in Argentina
In addition to the Basic Education, the system includes:
•
Tertiary Level: it includes non-university and university studies that offer
professional and academic education at undergraduate level. The academic autonomy
and economic autarchy of universities is stated.
•
Graduate Level: it is under the responsibility of universities and academic, scientific,
and professional institutions of renowned level. The universities are national,
autarchic and autonomous institutions, while National Government is basically
responsible for the financing.
•
Special Regimes: they mainly comprise special education, adult education, artistic
education, and non-formal education (the reform has explicitly incorporated these
regimes into the system and recognized their importance.)
In terms of organization and governance, Argentina has 24 jurisdictions, each responsible
for education services. In 1993, functions were redefined for the National Ministry of
Education, Science, and Technology and the Federal Council of Culture and Education
19
(in which the 24 jurisdictions take part). Institutional autonomy has been increased, but
provincial and national teams have a complementary function in the system. The
decentralization did not reach municipalities (as in Chile), what still leaves a somewhat
centralized system.
In the current system, at the national level, the Ministry of Education is responsible for
assisting the President in all matters connected to it, defining goals and schools’ course
syllabi, assigning budgets for programs and their management, setting procedures for
projects, establishing institutional and methodological structures for schools and their
relationship with provincial governments, monitoring compliance with rules, and visiting
schools. In view of this, the main functions of the Ministry of Education at national level
are the coordination and monitoring of the educational system, the orientation of the
different jurisdictions, and the formulation of objectives and area policies.
Jurisdictions are in charge of designing, financing, and executing said educational policy
and of hiring the Initial Level (EGB and Polimodal) teachers, who may be employed
either part time or full time (multiple positions is not uncommon). The Constitution
demands that the provinces ensure “primary education;” consequently, all of them have
sanctioned constitutional norms according to such demand while each jurisdiction has
elaborated laws on education following the national norms, so they may promote
education at any level within their territories. Similarly, the Constitution does not forbid
cities from organizing and spreading education at any level with the purpose of
encouraging general knowledge. This is the third action level of the official education3.
Regarding public education with private management, the National Constitution supports
this activity in the Preamble and in the Chapter on declaration, rights, and guarantees;
consequently, private and official education are regulated by the same norms.
In terms of class size and classroom organization, the structure varies considerably from
place to place across the country. Class size ranges from 15 to 40 students (an average of
3
The education at city level has not a relevant development.
20
25), with one exclusive teacher per grade. Students frequently are seated facing the
blackboard, and teacher-student communication predominates over student-student
interactions. Schools do not have reading specialists, except for a few private schools. No
second language is taught in most public schools, in contrast to private ones.
Nevertheless, a few children whose mother tongue is not Spanish but an aboriginal
language receive bilingual education.
Teacher Formation
The teaching degree in Argentina is non-university and emphasizes the pedagogical
rather than the content aspects. Before 1969, teachers were trained in special secondary
schools (Normal Schools created in 1870). In addition, the provinces founded Institutions
for Secondary Schools Teachers’ Formation at tertiary level (both public and private).
Since the mid-1980s, these institutions have strived to achieve greater heterogeneity and
complexity with respect to diversity of dependency, diplomas, and curricular planning.
Within this context and the regulatory framework of the Federal Law of Education,
teachers’ development is centered on the following:
• Evaluation of the adequacy of training programs regarding higher quality,
coverage, and relevance to requirements of the 10 years of compulsory education and the
implementation of the new structure of the system.
• Stipulation of a framework articulating the federal policies of continual
development for teachers
• Updating of professional education through the Common Basic Contents
published in 1995
• Creation of the Federal Network of Continual Teachers’ Training.
In addition to this effort to increase the teaching force, about sixty percent of the teaching
force at primary level does not hold a higher education title. This has been a side effect of
21
the extension of compulsory education from seven to ten years, since highly qualified
teachers were allocated to higher levels of education.
Curricula and evaluation
National reading policy states that all students should be able to read (and write)
independently by the end of grade 3, but the actual policy is that children should be able
to “sound out” texts much earlier (by the end of first grade). Although the reading
curriculum is set at national level, there is a large heterogeneity in the “real curriculum”
throughout the country. Reading instruction formally starts in the first grade of EGB1,
but schools are starting earlier in Kinder 5, where basic writing takes place at this early
stage as a collaborative enterprise between teacher and students. The Argentine
curriculum does not dissociate reading from writing. Language as a subject incorporates
both reading and writing instruction. This instruction comprises oral language, written
language (reading and writing), language awareness, literary discourse, procedures
involved in comprehension and production of texts, and attitudes toward uses of
language. Most of the instructional time in the first three years is devoted to reading and
writing, and only a small fraction is used for math. The importance of math and science
increases as students evolve to more abstract reasoning.
In terms of materials, teachers use books (manuales), but most classes (and even schools)
do not have a bookshelf or reading corner to be used by students. The only book available
for each student is the graded reader or his own textbook.
Achievement is assessed by teachers through teacher-made tests and teachers’
observations. These include oral reading and written answers to questions about what has
been read. Since 1993, a reading comprehension test (Operativo Nacional de Evaluación,
ONE) for third, sixth, ninth, and twelfth graders has been administered to nationally
representative samples of students (except for 2000 when it covered the entire population
(Census Type). Commercial standardized tests are not used regularly.
22
II.3. Argentine Educational System in Numbers
Main characteristics
According to the most recent publication of educational statistics (2005), there are in
Argentina 12.2 millions of students, what represents approximately 30% of the total
population. Of this total, 81.3% are in the Basic Level (up to secondary level), what is the
object of study in this paper. 98% of the students in the age range of primary level (EGB
1 and 2) are enrolled in school; whereas in EGB3 and Polimodal, this ratio is 78.4% and
53.6%, respectively.
The literacy rate is 97%, above the average for Latin America (89.9%) and upper-middle
income (UMI) countries (94.3%). The school life expectancy (or years of schooling) is
15.4 years, again over the average for Latin-America and UMI countries (13.1 and 13
years respectively).
Between 1991 and 2007, the number of students in tertiary education level has more than
doubled, in primary level the coverage has been very constant, and there has been an
important increase in secondary education, particularly in the 90s.
From the official information (see Annex A) we conclude that:
a) The gross enrollment ratio in secondary level is substantially higher (104.7%
and 73.7% respectively), what shows that a large proportion of individuals are finishing
the secondary school older than expected (this is in part explained by the high repetition
rate and high school drop-out ratio at secondary level).
b) Argentina has reached a relatively high enrollment ratio earlier than other
countries in the region. For instance, in 1970 Argentina had a gross enrollment ratio in
secondary school of 44%, level that Brazil, for instance, only reached by the middle 90s.
c) The goal of universal access of primary education (EGB 1 and 2) has been
already reached (Argentina shows high net enrollment ratios since the late 70s, being the
first country in the region achieving this goal).
23
d) In secondary education (EGB 3 and Polimodal) the country still has problems
to reach universal access, at least in terms of net enrollment ratios(many finish this level
as adult education).
e) The deep economic crisis suffered in 2001/2002 seems to have had an impact
on enrollment ratios. For instance, the gross enrollment ratio in primary school fell from
117.8% in 2000 to 112.7%, and in secondary, from 96.7 to 86. But the effect on the net
enrollment ratio was much smaller (null in secondary and a fall of 0.5 percentage points
in primary), what suggests that either adult education suffered the most or that the catchup possibilities for adults have been exhausted.
d) There are significant differences across regions, with a strong correlation
between the indexes and economic development.4
e) The common feature at all educational levels and regions is the difference
between the gross and net schooling. In this sense, the Argentine educational system has
the ability to incorporate most of the population that potentially demands education but
shows a significant fault in the ability to encourage students to finish school.
f) The spending per student (as % of per capita GDP) significantly increased in
the 90s for all the levels, but it was affected by the economic crisis, and the current levels
are still the 2000 levels, particularly for tertiary level (which represents now 11.7
compared to the peak reached in 1995 of 21.5).
A characteristic of the Argentine system is that establishments use more teachers
compared to OECD countries, but a high proportion of these teachers are not actually in
front of the class. It is estimated that approximately 20 to 30% of the teachers are
working in administrative tasks or supporting the educational process from outside the
classroom.
The ratio of students per teacher in primary school is as low as 12.4 in the Southern
provinces. Across provinces, this ratio shows a high dispersion, indicating a high degree
4
The regions are: a) Center: Buenos Aires, Ciudad de Buenos Aires, Córdoba, Entre Ríos, La Pampa, and
Santa Fe; b) Cuyo: Mendoza, San Juan and San Luis; c) NEA: Chaco, Corrientes, Formosa, and Misiones;
d) NOA: Catamarca, Jujuy, La Rioja, Salta, Santiago del Estero, and Tucumán; e) South: Chubut, Neuquén,
Río Negro, Santa Cruz, and Tierra del Fuego.
24
of discretionarily across provinces. Sometimes, the ratio of students per teacher is used to
proxy the quality of education (under the assumption that a lower ratio is better), but in
the case of Argentina, the ratio should be interpreted as a sign of inefficiency.
Achievement in Primary and Secondary Education
As a proxy of achievement, we use the following indicators:
•
Effective Promotion Rate: share of students that pass and are enrolled in the next
corresponding grade
•
Effective Repetition Rate: share of students that are enrolled in the same grade again.
•
Drop-out Rate
•
Scores in National-wide Standardized Tests (ONE tests)
The main results are:
•
Indicators are in general correlated with the regional level of development; the
more developed the regions, the better the indicators are. An exception is the
effective promotion and repetition rate in the Center, which is the richest.
•
The drop-out rate is particularly high in EGB3 and Polimodal. In EGB, the
fall in enrollment happens mainly at the end of the EGB2 (eight grade) and the
beginning of EBG3 (9th grade).
•
Polimodal has a much higher drop-out rate, particularly in the first year.
Regionally, the Northern provinces show the highest drop-out rates.
The next table shows the average score of the tests administrated in 2000 and 2003
(Language and Mathematics Tests) for the country and regions. The main results are:
•
On average, students respond less than 70% of the questions correctly, which is
supposed to be the break even point to pass an exam. The only exception is the
Language test in 3rd grade (EGB1) for urban private schools in 2000.
25
•
The performance is worse for higher levels of education in both tests, what shows that
students have increasing problems to reach the increasing standard (what might be
interpreted as problems to develop complex reasoning and logic).
•
On average, the rate of correct answers is lower in 2003 (a year of economic crisis)
•
Private schools obtain better scores systematically (for all the regions and tests).
•
The provinces with the lowest scores are the Northern provinces, which are the
poorest and with poorest overall educational indicators (such as enrollment rates and
average years of schooling)
Table 2. Proportion of right answers, ONE test, Argentina
Primary School
3rd grade (EGB1)
6th grade (EGB2)
Language
Mathematic
Language
Mathematic
2003
2000
2003
2000
2003
2000
2003
2000
Total
Urban Public Schools
Urban Private Schools
Rural Schools
Regions
Center
Cuyo
NEA
NOA
South
59.4
58.5
69.3
57.6
61.9
59.6
71.6
59.0
59.5
58.7
68.5
58.2
59.5
58.0
64.9
58.8
54.1
51.5
64.0
51.4
61.6
59.0
72.1
54.8
56.4
54.1
64.9
54.0
57.9
55.5
67.5
50.8
60.7
61.0
60.1
56.0
62.0
62.6
63.3
58.6
60.0
63.8
61.0
60.5
60.0
56.0
63.4
59.9
59.8
57.0
59.7
60.8
54.6
55.6
49.8
52.9
57.4
62.9
62.2
56.4
58.6
62.4
56.6
60.4
52.7
54.7
58.9
58.8
59.2
52.8
55.5
59.3
Secondary School
9th grade (EGB3)
12th grade (last year of Polimodal)
Language
Mathematic
Language
Mathematic
2003
2000
2003
2000
2003
2000
2003
2000
Total
Urban Public Schools
Urban Private Schools
Regions
Center
Cuyo
NEA
NOA
South
52.7
48.8
63.8
51.0
47.1
62.3
53.4
50.8
60.8
53.6
50.4
62.8
57.2
54.1
63.1
59.1
54.9
67.2
56.3
52.5
63.6
61.3
57.4
68.7
53.8
52.9
46.1
50.9
56.0
53.2
49.5
43.8
45.3
51.5
54.4
53.1
47.5
52.0
55.7
56.0
52.3
44.5
48.6
53.3
59.3
53.5
49.5
52.2
57.6
61.8
57.0
50.3
52.1
58.0
58.9
53.3
45.7
50.4
56.4
64.2
58.9
50.8
54.2
60.0
26
III. Data and Methodology
III.1. Data
In this study we will use different databases, which we briefly describe here. To analyze
Argentina in particular we will use the following local surveys and national evaluations:
•
EPH (Encuesta Permanente de Hogares): this is a household survey representative
at sub-regional level for 24 urban areas within the country. For the larger urban
area, which corresponds to the city of Buenos Aires and the sub-urban areas
(called Great Buenos Aires or GBA area) where approximately 37% of the total
population lives, the survey started in 1974. For the rest of the regions the survey
started in 1996. The main objective of the survey is to characterize the labor
market and its evolution.
•
Censo Nacional de Maestros y Establecimientos Educativos. This is a national
census collecting information about schools and teachers. The most recent Census
was in 2004 (the previous ones were in 1994, 1980 and 1970).
•
Anuarios de Estadísticas Educativas: A yearly report done by the Ministry of
Education that contains statistics about the national system.
•
ONE Test (Operativo Nacional de Evaluación). ONE is a test administrated by the
“Programa de Promoción y Evaluación de la Calidad Educativa” of the Ministry
of Education of Argentina every year at national level.
ONE test has been administrated since 1993, and the test is complemented with a student,
teacher, and school director/principal survey that permits to link students’ achievements
with students and families’ characteristics as well as schools and teachers’ characteristics.
The test had been increasing the coverage since 1993 to reach a coverage similar to the
Census in the 2000 version. After the 2000 test, the test was administrated only in 2003.
The 2003 version was administrated only to a relatively small random sample of students,
27
who by design are only representative of provincial level but not of the school or city
level. The Ministry of Education only made public the average results at region level, and
the datasets are not publicly available. For these reasons we will focus mainly on the year
2000, when students from 6th grade of Educación General Básica (EGB) level (primary
school) and the last year of Polimodal (5th year secondary level) were tested. Using the
2000 test, in addition to having the advantage of covering all the students in the country,
is also very close in time to the two international tests in which Argentina participated
and that we will analyze in this study. These tests are:
•
PISA tests (to which Argentina was included in PISA 2000 and the recently
released PISA 2006)
•
PIRLS test (Argentina participated in the 2001 round)
With PISA and PIRLS tests we will make a comparative analysis of the situation of
Argentina and of other countries included in these tests.5 Since the tests are based on a
(relatively small) random sample, they are representative of school level (for those
schools included in the tests) and national level, but not of regional level or school type.
PISA 2000 is the most recent test in which Argentina participated. It provides detailed
information on students’ family background (including family structure, the parents’
education level and occupation)m and schools’ functioning conditions, so we can
associate students and schools’ characteristics with students’ performance. In particular,
it is possible to decompose, to some extent, test results into students, schools, and
country’s characteristics, with potential policy implications.
PISA assesses the knowledge of 15-year-old youth, but the emphasis is not on the
curriculum content; instead it focuses on the skills that students will need in their
everyday lives (see OECD 2001).
5
We will not use TIMSS, in which Argentina participated only in 1995, the LLECE test, by
UNESCO/OREALC, which included Argentina in 1997, or the Laboratory test (1997) as PISA and PIRLS
are two more up-to-dated sources.
28
The focus of PISA in 2000 was on students’ reading skills, with mathematics and science
skills treated as minor domains (these two latter tests were administrated only to a minor
sub-sample of students). For this reason, our analysis will be based on the reading test
only.
One limitation with PISA is that the data set is cross-sectional (i.e. it does not include
information on the students’ previous performance); therefore, we cannot apply a panel
data type of analysis, which is useful when students’ unobservable characteristics affect
the achievement, and we cannot identify the value added by the school or teacher. An
additional limitation of PISA 2000 is that it does not include any information at
classroom level: we do not know the teachers’ characteristics or which class the student is
attending (school and grade).
PIRLS assesses the literacy skills of students in their fourth grade and includes a survey
to students, parents, teachers, and school administrators, what provides a very rich set of
controls (see Mullis et al (2003)). As well as ONE test, PIRLS focuses on curricula
contents, which is directly associated with what the student should have learned in that
grade.
In terms of international comparison, a potential inconvenient with PISA and PIRLS tests
is that only a few Latin American or developing countries are included, what limits the
comparison of Argentina within the region and countries with similar characteristics.
III.2. Methodology
The achievement of a student in a standardized test is a function of:
i. the student’s ability (b),
ii. other student’scharacteristics (s), such as family background,
iii. class’ characteristics (c), such as “quality” of the peers and teacher’s
characteristics,
29
iv. school’s characteristics (sc), which are characteristics common to all the
classes in that school
v. regional characteristics (r), which are important in Argentina since education
has been decentralized at provincial level, existing some regional variation in the
main characteristics.
vi. educational system characteristics, which usually varies at country level
For the student i, in class c, at school s, in region j the test score is a function of:
[1]
TSi ,c ,s , j = f (b, s, c, sc, r ) + ε
where ε is a random variable affecting performance. f() might be interpreted as a
production function (see Lazear (2001)), but it has several particular characteristics that
make the empirical analysis quite different from standard production function
estimations. Not only are there endogeneity issues for inputs (as in any production
function), but there are also externalities in production (peer-group effects), sorting of
inputs (not only does the school select inputs, but also inputs select schools), and input
heterogeneity, which in part is unobservable to the econometrician (and affects the
sorting).6 This generates a very complicated framework for an econometrician, with
double-selection bias and endogeneity problems biasing the results of a simple estimation
of the production function [1].
In particular, student and teacher’s ability is not
observable for the econometrician,7 and the distribution of ability among schools (and
perhaps classes) is not random; therefore, selection bias problems can easily emerge
when trying to factor decompose test results.8 This is the main difficulty in trying to
analyze the factors behind the quality of education: the contamination of the simple OLS
results by endogeneity and selection biases.9 10
6
The evidence shows that not only do students select schools but also teachers select schools based on
other school inputs (see Hanushek, Kain and Rivkin (2004)).
7
There are a few approaches to control the unobservable students’ ability, but given the type of data we
have available in Argentina, most of these approaches are not really feasible. For instance, we do not have
IQ tests, none of the tests taken in Argentina follows the student across time (i.e. they are just repeated
cross-section tests and not longitudinal data), and we do not have students’ characteristics at birth (although
the correlation between ability and birth characteristics has not a strong empirical support).
8
In fact, Hechman and Vytlacil (2001) argue that it is not possible to separate the effects of ability and
schooling.
9
An early description of the empirical issues in the estimation of educational production functions can be
found in Hanushek (1979), see also Filmer and Pritchett (1999).
30
In the economics literature, the empirical evidence on school inputs and achievement is
based mainly on three types of econometric exercises: a) naive OLS regressions (omitting
all the technical problems we have just mentioned), b) more sophisticated econometric
models that try to correct biases,11 or c) the identification of an effect of a particular input
based on a natural or controlled experiment.12 On the other hand, much of the literature
on education has developed and refined the use of hierarchical lineal models (HLM) or
multilevel models to factor decompose test score (see Willms and Raudenbush (1989) or
Willms (2006)). HLM models –also known as random coefficient models (Rosenberg,
1973), multilevel linear models (Mason et. al. 1984), and mixed linear models (Goldstein,
1986)- exploit the hierarchical structure of the test scores, since the achievement depends
on individual’s, the class’, and the school’s characteristics as well as on any other cluster
level (such as the characteristics of the city, province or country); these models also try to
decompose the residual variance in different components. Within the same class (school,
or cluster), the measurements from individuals are not independent; students from the
same class (or school) might have a more similar achievement than students from
different classes (or schools). The violation of independence is one of the main reasons
for not using traditional regression models at student level. Note that aggregating at
school level we avoid violating the independence assumption at class level (due to the
possible non-random assignment of student into classes), but still we have the nonindependence at school level, which can be solved aggregating at city or province level.
Although it is still true that there might be reasons even at city level for non-random
assignment, such as Tiebout sorting, self-selection problems are drastically reduced. But
10
Controlling for the endogeneity of school inputs with cross-sectional test scores is still a pending issue in
the literature of the economics of education. The literature of industrial organization has been able to give
an answer to the endogeneity of inputs when estimating production functions, which relies on structural
approaches (see Olley and Pakes, or Levinsohn and Petrin), but in the economics of education, we do not
still have strong results about the structural process behind the production process to develop structural
estimations.
11
An early work in this area is the two-way nested error component model of Montmarquette and
Mahseredjian (1989) to estimate the effect of school characteristics on educational achievement.
12
Some examples of these kinds of papers are: Cullen, Brain, and Levitt (2006), who analyze students’
achievement based on an experiment in Chicago, where certain types of students were able to choose
school, or Altonji and Dunn (1996), who use siblings’ information to estimate the effect of school quality
on wages, or Angrist and Lavy (1999), who identify the effect of class size on achievement using a
Maimonide kind of rule to separate classes.
31
by aggregating we lose information at individual levels; in addition, we might create
problems of aggregation bias and lose precision.
For these reasons, in most of our econometric exercises we will follow the HLM
approach, based on finding the optimal balance of ordinary least square (OLS) and
aggregation approaches, which we briefly describe here.13
HLM Model
For simplicity, consider only two levels. Level-1 data are the individual student’s factors
while Level-2 refers to group characteristics (such as classroom or school).14 To keep the
exposition simple, suppose that at individual level there is just one covariate X (e.g. sex).
Denote i as the ith student (level-1) and j as the jth class (level-2). The Level-1 model is:
Yij = β 0 j + β 1 j X ij + eij
where Yij is the test score of the student i that attends school j; e is normally distributed
with mean 0 and variance σ2.
The Level-2 model (without covariates) is:
β 0 j = γ 00 + u oj
β1 j = γ 11 + u1 j
u0j and u1j are assumed to followed a bivariate normal distribution with mean 0, variances
τ00; τ11 respectively, and covariance τ01. When combining level 2 and 1 we obtained the
following reduced form model:
Yij = γ 00 + γ 10 X ij + u oj + u1 j X ij + eij
The random error now has three components: the random effect of the jth class on the
mean u0j, the random effect of jth class on the slope interacting with the student
characteristic, and the level 1 error.
13
14
For a more advanced treatment, see Raudenbush and Bryk.
This exposition is based on Qu (1997).
32
To estimate the Level-1 coefficients we will use the “shrinkage estimator”. For the oneway ANOVA case, we have the two-level model:
Y j = β 0 j + e j , e j ~ N (0,V j )
β 0 j = γ 00 + u0 j , u0 j ~ N (0,τ 00 )
This model suggests that we could estimate β 0 j by using Y j or γˆ00 . A Bayes estimator,
called β *0 j is an “optimal" weighted combination of Y j or γˆ00 :
β *0 j = λ j Y j + (1 − λ j )γˆ00
where the optimal weight is given by:
λj =
=
τ 00
τ 00 + V j
parameter var
parameter var + error var
If the parameter variance τ 00 is large relative to the error variance Vj, then the weight λ j
is large, what means that we will put less weight to the parameter estimator τ 00 and more
weight to the group mean.
In general, if we have the two-level model with covariates at level 2 we have:
Y j = X j β j + R j , R j ~ N (0,V j )
β j = W jγ + U j ,U j ~ N (0, T )
Then the OLS regression estimator for the first equation is:
βˆ j = (X 'j X j ) X 'jY j
−1
and the second estimator based on group characteristics captured in Wj is:
β~j = W jγˆ
where γˆ is estimated by generalized least square (GLS):
33
γˆ = ∑ (W j' Δ−j1W j
j
) ∑ (W Δ
−1
'
j
−1
j
βˆ j )
j
where:
Δ j = Tj +Vj
The optimal combination of these two estimators is
β *j = Λ j βˆ j + (1 − λ j ) β~j
where
Λ j = T (T + V j ) −1
Since β *j pulls βˆ j towards β~j , the estimator is called a shrinkage estimator. In general,
the more reliable βˆ j is as an estimate of β j (i.e., T is small), the more weight it will
have and the more β *j will look as βˆ j .
Oaxaca-Blinder Decomposition
We complement the HLM analysis with Oaxaca-Blinder decompositions, where we
separate the score gap between Argentina and the benchmark country as:
GAPj , ARG = E (Y j | X j ) − E (YARG | X ARG )
∀ j = 1,..., J
(
)
(
GAPj , ARG = ∑ βˆ ARG ,k ( X j ,k − X ARG ,k ) + ∑ X ARG , k βˆ j ,k − βˆ ARG ,k + ∑ ( X j ,k − X ARG ,k ) βˆ j ,k − βˆ ARG ,k
K
K
k =1
k =1
K
k =1
)
where the subscript j corresponds to the benchmark country selected, and the k the
regresors (k=1 is the intercept).
We defined the score gap as the difference between the predicted OLS score for the
benchmark country and the predicted OLS score for Argentina (first equation). Therefore
the gap is positive if the benchmark country performs better than Argentina, and negative
if performs worse.
34
The score gap (second equation) is decomposed in three effects:
i)
The endowment (or characteristic) effect (first term) is the difference in scores
due to differences in the average for each regresors, weighted by the
Argentine slope. It represents the part of the score gap that can be explained
just because of different average characteristics between both countries.
ii)
The returns effect (second term) represents the proportion of the score gap that
can be explained by differences in the slopes between both countries (given
the average Argentinean characteristics)
iii)
The interaction effect (third term) is the residual part of the decomposition.
Usually, it represents a small value and it captures the leverage produced by
both of the previous effects happening simultaneously.
We complement the Oaxaca-Blinder decomposition with the Juhn Murphy Pierce
decomposition, which is a decomposition for the entire distribution and not only the
mean.
35
IV. International Benchmarking
IV.1. Structure and Coverage
As already mentioned, Argentina has reached high levels of literacy rate and coverage
earlier than other Latin-American countries and even some developed countries. Its
evolution since 1950 has been favorable compared to Latin America, but not so much
compared to fast growing countries.
In this section we compare Argentina with two groups of countries: Latin American
countries and upper-middle income (UMI) countries (as Argentina). Most of the Tables
and Figures are shown in Annex B, here we discuss the main results.
Argentina, compared to both groups of countries, is a top performer in many indicators.
The literacy rate (97%) is well above the average for Latin America (89.9%) and UMI
countries (94.3%). The school life expectancy (or years of schooling) is 15.4 years, again
over the average for Latin-America and UMI countries (13.1 and 13 years respectively).
It is also a top performer in terms of net enrollment ratio in primary school and gross
enrollment ratio at tertiary level. For secondary level, it is a top performer in LatinAmerica, but when compared to UMI, it is close to the average, what shows that the
country has some problems in this level of education. The high enrollment in tertiary
education but with lower (net and gross) enrollment in secondary schools means that an
important group of students do not have access to higher education simply because they
do not finish high school, what implies an unequal situation. In the last years there have
been policy debates about the reasons for the high drop-out rates in secondary education,
the usual suspects are poor quality in the previous levels for some students and the early
entrance to the labor force. As shown before, students in Argentina have a significant
decreasing performance in the national tests as they advance, what, given the high dropout rate, means that the problem of quality could be even bigger than what the tests
suggest. Related with this finding is the unusually high ratio of repeaters to total
36
enrollment in secondary school (11.5%), which is the highest in Latin-America and also
high when compared to UMI countries. This means that Argentina is able to reach a
relatively high school life expectancy but very inefficiently, with a high proportion of
students repeating or finishing school later than expected.
In terms of students per teachers, Argentina has a very low ratio (17.3) for primary level
and close to average at secondary level, but as mentioned before, this should not be taken
as an indicator of quality, because between 20% and 30% of the primary level teachers
are doing administrative tasks rather than teaching.
The high proportion of repeaters in primary school means that Argentina is one of the
worst performers in terms of students’ reaching grade 5 among the UMI countries, where
only 84.3% reach that level on time (only Gabon has a worst performance).
Table 3. Argentina compared to LATAM and Upper-Middle Income countries
2004
indicador
Adult literacy rate (%)
Duration of compulsory schooling
Duration of primary education
Duration of secondary education
Girls as % of total enrolled, primary
Girls as % of total enrolled, secondary
GNI Per capita
Gross enrollment ratio (%), primary
Gross enrollment ratio (%), secondary
Gross enrollment ratio (%), tertiary
Net enrollment ratio (%), primary
Net enrollment ratio (%), secondary
Private enrollments as % of total, primary
Private enrollments as % of total, secondary
Progression to secondary school (%)
Public expenditures on education, as % of GDP
Public expenditures per student (% of p. c. GDP), primary
Public expenditures per student (% of p. c. GDP), secondary
Public expenditures per student (% of p. c. GDP), tertiary
Pupils reaching grade 5 (% of cohort)
Ratio of pupils to teachers, primary
Ratio of pupils to teachers, secondary
Repeaters as % of total enrolled, primary
Repeaters as % of total enrolled, secondary
School life expectancy (years)
Argentina
97.2
10
6
6
49
50.9
3,580.0
112.2
86.4
63.9
99
79.1
20.6
27
92.8
3.5
10.9
14.3
13.1
84.3
17.3
17.3
6.4
11.5
15.4
Latin
America
88.9
8.8
5.8
5.7
48.4
50.8
2915.0
112.2
77.0
32.0
93.3
60.8
16.4
25.8
89.9
3.7
11.0
11.4
25.1
82.9
25.2
19.9
6.9
5.6
12.7
Upper-middle
Income
91.7
9.0
5.7
6.3
48.6
50.3
7177.3
105.0
89.2
38.2
92.5
79.6
14.3
15.0
93.0
4.7
15.5
20.0
33.0
93.7
19.2
14.6
5.9
5.8
13.6
Source: World Bank
37
In public spending per student, Argentina is in the average of the region and well below
the average of the UMI countries, whereas private sector spending in Argentina is slightly
above the mean. The share of the private sector in the total enrollment has been slightly
increasing at country level in the last 30 years; a trend that is observed in most of the
countries of the region. For instance, the share of private schools on total primary and
secondary students increased from 17% in 1974 to 22% in 1988 and 25% in 2005. There
are although strong regional differences, whereas some provinces such as Mendoza still
rely mainly on the public provision, in other regions such as the Great Buenos Aires area
or the City of Buenos Aires, the private sector share on total enrollment has increased
more, reaching 34% and 43% respectively for primary and secondary education in 2005
(the highest ratios for all the regions).
It is interesting to note that Argentina is among the top performers in terms of literacy
rate and tertiary education enrolment in both the region and the group of upper middle
income countries, what shows that the country has a relatively mature system, or in other
words, it has developed earlier in terms of education coverage. Argentina had at the
beginning of the 80s a gross enrolment rate of 70% in secondary school, well above other
Latin-American countries (except for Uruguay) and also among upper-middle income
countries. This ratio has increased since then by 23 percentage points (reaching 86% in
2005). In the same period, Costa Rica, increased from a low gross enrollment ratio of just
40.2% to 79.2% (an increase of 197%), and Brazil from 35.4% to 105.7%. The fact that
Argentina has reached high coverage ratios earlier means that there are not strong
“compositional effects”. In primary school, for instance, Argentina had already reached
almost full coverage in the late 70s. This shows that universalizing the access is not an
issue in Argentina, except for the secondary level, where the problem in fact is high dropout rates after the first years.
If we compare Argentina with similar income level countries, similar culture, or a higher
performance in terms of quality, we find that the most striking difference in Argentina is
the repetition rate, which in primary school is 6%, well above the other countries.
38
Table 4. Argentina and selected comparators, 2004
Argentina
Chile
Czech Republic
Poland
Hungary
Spain
Italy
France
Finland
Population
ages 5-14
Population
Growth
GDP per
capita PPP
Repeaters
primary
School (%)
Survival rate
to grade 5 (%
of cohort)
18.6
19.0
12.4
14.8
12.1
10.5
9.9
12.9
12.5
1.1
1.2
12.147
9.188
15.222
10.548
5.9
2.0
1.2
0.8
2.1
2.4
0.4
4.2
0.5
93.1
99.2
96.6
99.3
N.A.
100
96.5
98
99.4
-0.09
-0.53
-0.26
0.84
0.05
0.3
0.21
12.263
22.313
25.302
25.656
25.912
Source: World Bank, Edstats, IMF
IV.2. Performance in International Tests
Argentina has participated in a few international tests. Table 5 shows the position that
Argentina obtained in the Language test for LAB (1997), PISA (2000), PIRLS (2001) and
PISA (2006).
Laboratory covered 11 Latin American countries, and the test was administrated for a
sample of 3rd and 4th grade students. Argentina ranked second in 3rd grade, after Cuba, and
third in 4th grade, after Cuba and Chile, what shows a relatively good performance
compared to other Latin American countries; this result is consistent with the better
aggregate indicators for years of schooling and enrollment.
Table 5. Performance of Argentina in International Tests (Language)
Ranking Argentina
Total Sample
LATAM sub-sample
LAB 1997 3rd
LAB 1997 4th
PISA 2000
PIRLS 2001
PISA 2006
2 / 11
2 / 11
3 / 11
3 / 11
34 / 39
2/4
31 / 35
2/2
51/57
4/6
If we analyze the entire sample of countries for PISA and PIRLS, the picture is very
different. In PISA 2000 (literacy) Argentina finished 34 out of 39 countries and in PISA
2006 (Science) finished 51 out of 57 countries. Something similar is shown by PIRLS,
Argentina is 31 out of 35 countries in Language.
39
These tests include a larger set of countries, with a high participation of developed
countries. If we restrict the comparison to Latin American countries, Argentina finished
second in PISA right after Mexico and above Brazil and Peru, and second in PIRLS right
after Colombia. But Argentina has not improved over time. Ranking the Latin American
countries that participated in LAB 1997 and PISA 2006, Argentina lost one position in the
ranking, whereas Chile and Mexico show a significant improvement.
Table 6. Evolution of Argentina compared to other Latin American countries
1
2
3
4
5
Lab 1997
(Language)
Chile
286
Argentina
282
Brazil
277
Colombia
265
Mexico
252
PISA 2006 (Science)
Chile
Mexico
Argentina
Brazil
Colombia
438,2
409,7
391,2
390,3
388,0
To have a more homogenous set of comparators, we classify the countries that participated
in PISA 2000 according to the World Bank classification (in terms of 2000 GNI per
capita, Atlas Method). Of the 39 countries we have: 25 high income, 6 upper-middle
income, 7 lower middle income and just 1 low income country. Comparing Argentina with
non-high income countries, we found that it ranks fairly bad, 9 out of 14 countries in
Language, 8 out of 14 in Math, and 10 out of 14 in Science. Argentina was the richest
country in this set; the average score in PISA is well below the expected score according
to its developing level (see Figure 4). In fact Argentina, Mexico, Peru and Brazil, the four
Latin-American countries participating in the Language test of PISA-2000 show a similar
underperforming pattern, but Argentina is the country that is more distant from the
expected level (according to the linear cross-country trend). Eastern European countries
are the top performers in this group.
40
Figure 4. GNI per capita and performance in PISA
550
500
Mean Score in Language, PISA 2000
Czech Republic
Hungary
Poland
Russian Federation
Latvia
450
Bulgaria
Thailand
Mexico
Argentina
400
Brazil
Macedonia, FYR
Indonesia
350
Albania
Peru
300
-
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
GNI per capita (US$), Atlas Method
Unfortunately, there are not large enough datasets covering very different developing
countries to understand better where Argentina (and Latin America) stands in the scope
of educational achievement. To overcome this limitation, we make a very strong
assumption and construct a dataset combining different international tests (basically we
choose a pivotal country which is in two different test, A and B, and assign a score for
countries in B which is not in the sample A following a simple proportional rule, using
PISA as our basis). With this approach we can expand the sample to 58 countries,
Argentina ranking 42.
The next figure plots the performance for this extended sample and the income level. The
results are:
•
There is a positive correlation between development level (GNP per capita) and
average test performance (the coefficient of correlation between GNI per capita
and average test score is 66.8%.) with a relative small set of countries (mainly
from Eastern Europe) somewhat above this relationship (i.e., obtaining on average
41
similar scores as developed countries even though they are upper-middle income
countries).
•
Argentina performs relatively well for Latin American standards, but Latin
America as a region underperforms, most of the countries with the lowest scores
given their income level belong to this region. Other developing countries,
particularly from Eastern Europe, have a much better performance with similar
income level.
Figure 5. GNI per capita and performance in international tests
550
Finland
Canada Germany
Australia Ireland Netherlands
Hong Kong
UK
Singapore
Belgium
Iceland
Austria
France
Denmark
Sweden
Italy
Lithuania
New Zealand
Korea
Slovak Republic
Romania
500
Slovenia
Czech Republic
Hungary
Poland
Moldova
Aveage Test Result
Russian Federation
Latvia
450
Cyprus Spain
Iran
U.S.A
Switzerland
Norway
Greece
Portugal
Israel
Turkey
Bulgaria
Thailand
Japan
Mexico
Argentina
Chile
Brazil
Bolivia
Paraguay
Colom.
400
Kuwait
Macedonia, FYR
Dominican Republic
Indonesia
Ven.
Honduras
350
Albania
Morocco
Peru
Belize
300
-
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
GNI per capita (Atlas Method)
If we compare the educational achievement according to school life expectancy (years of
schooling) we also find a similar picture: Argentina obtains 415 points in the score when,
according to the cross country relationship between both variables, Argentina should have
a score of 483 points (given its school life expectancy), a score similar to countries such as
Hungary (481) or Poland (480), and not far away from Italy (488), The Czech Republic
(492) or Spain (493).
42
Figure 6. School life expectancy and performance in international tests
600
y = 23.973x + 173.13
2
R = 0.3832
550
Finland
Lithuania Germany Netherlands
New Zealand
Korea Japan
UKIreland
Switzerland
Slovakia
Belgium
Romania
Norway
France
U.S.AAustria
Iceland
Slovenia
Sweden
SpainDenmark
Czech
Rep.
Cyprus
Italy
Hungary
Greece
Poland
Portugal
500
Average score
Moldova
Latvia
Israel
450
Bulgaria
Mexico
Argentina
Iran
400
Chile
Colombia
Kuwait Bolivia
Paraguay
Indonesia Dom. Rep.
Venezuela
350
Australia
Brazil
Albania
Morocco
Peru
Belize
300
250
7
9
11
13
15
17
Expected years of schooling
We also compare educational achievement with expenditure in education (as a share of
GNI, as a share of GDP and as a share of total expenditure) finding that Argentina is also
below the expected level of achievement given its expenditure in education. Taking into
account the public expenditure in education only, Argentina should have a score of 460
points instead of the observed 415.
Figure 7. Public Expenditure in Education as a % of GNI
and performance in international tests
600
500
ARG
Average Score
400
300
200
100
-
1
2
3
4
5
6
7
8
9
10
Public Expenditure in Education as % of GNI
43
Figure 8. Public Expenditure in Education as a % of GDP
and performance in international tests
600
500
ARG
Average Score
400
300
200
100
-
1
2
3
4
5
6
7
8
9
10
Public Expenditure in Education as % of GDP
Figure 9. Public Expenditure in Education as a % of total expenditure
and performance in international tests
600
550
Average Score
500
450
ARG
400
350
300
250
200
-
5
10
15
20
25
30
Public Expenditure in Education as % of total expenditure
Finally we analyze the relationship between average score and total public educational
expenditure per pupil as a percentage of GDP per capita in primary school. In doing this,
we control for differences in relative prices, and the number of students who are in the
system. Based on this indicator, the situation of Argentina changes drastically. Given its
expenditure per pupil and according to the international trend, Argentina should have a
score of 436 when the observed score is 415, what shows that its performance is not too
distant from what is expected. An important difference between Argentina and similar
income level countries such as Hungary and Poland is the amount the government is
spending per pupil. Although the three countries spends in education more or less the same
44
proportion of GDP, the spending per student is much lower in Argentina (12.3%)
compared to Hungary (19.2%) and Poland (23.5%), because Argentina has a population
pyramid with a larger base. Argentina has 19% of its population between 5 and 14 years
old, whereas Hungary has 12.1%, Poland 14.8% and the Czech Republic 12.4%.
Therefore, the educational effort Argentina should make in terms of GDP should be higher
to spend per student the same as the comparator countries. The low expenditure per
student due to a large proportion of young population is a common fact in Latin America,
except for Chile and Mexico, where the public expenditure is higher (14.5 and 13.3 per
student respectively).
Figure 10. Public Expenditure in Education per pupil as a % of GDP
per capita and performance in international tests, 2000
600
550
Ireland Hong Kong
Slovak R.
500
Czech R.
Average Score
450
Turkey
Iran
400
Indonesia
350
Finland
Germany
Australia
Netherlands
New Zealand
UK
Korea
Japan
Switzerland
NorwayBelgium
Iceland
Austria
France
U.S.A.
Denmark
Sweden
Cyprus Spain
Italy
Hungary
Poland
Greece
Portugal
Latvia
Israel
Slovenia
Bulgaria Thailand
MEX
ARG
BR
CHI
BOL
COL
PAR
Macedonia, FYR
DOR
Albania
Morocco
PER
Belize
300
250
200
0
5
10
15
20
25
30
Total public educational expenditure per pupil as a percentage of GDP per capita. Primary
The relatively low expenditure per student is not something we observe only in 2000
(where the economy was in a recession period), but also in 1998, when the economy was
booming. It is rather a structural problem. Nevertheless, it is important to point out that
some countries with similar expenditure per student, such as the Czech Republic, have
much better scores than Argentina; therefore, the lack of investment cannot be the only
factor explaining the poor performance.
45
Table 7. Country characteristics and Expenditure in Education
Primary, secondary and
post-secondary non-tertiary education
expenditure as percentage of the GDP
Argentina
Chile
Czech Republic
Poland
Hungary
Spain
Italy
France
Finland
U. K.
Public
Private
Total
2.7
2.7
2.7
3.5
2.9
3.3
3.4
4.1
3.7
3.4
0.3
1.2
0.4
0.1
0.2
0.4
0.0
0.2
3.1
3.9
3.1
3.6
3.1
3.7
3.5
4.4
3.7
3.8
0.4
Expenditure per
Student in
primary school
Ratio of expenditure
per student to GDP
per capita (1998)
Primary School
1389
1500
1645
1496
2028
3267
5653
3752
4641
3329
11.6
17.1
12.7
18.3
19.5
19.2
25.5
17.8
21.3
15.8
Source: World Bank, Edstats
Average test scores are not a measure of quality, since they are influenced by different
student’s characteristics, which are not necessarily related to the educational system.
Even assuming that individual ability is homogeneously distributed across countries,
international comparisons are affected by difference in enrollment ratio, since the test
only covers those students attending a school. PISA tests, for instance, are administrated
to 15-years-old students; while developed countries have almost full coverage at that age
(a net enrollment ratio close to 100%), Argentina has a net enrollment ratio close to 75%.
Basically we do not know the performance of that 25% of Argentineans who are not at
school.
To make the distribution comparable, one can either truncate the distribution of
developed countries or “inflate” the distribution of Argentina, making some assumptions
about those who are not at school (for instance, assuming that those 15-year-old
individuals who are out of the schooling system would be at the bottom of the test
distribution). The alternative we prefer and follow in this work is to control for the
students’ socioeconomic status. We simply regress test score on individual characteristics
and a set of dummies for each country (fixed effects at country level). The coefficient for
46
the dummy is the average quality of education in each country after controlling for
observable household and demographic characteristics.
With this measure of average quality we analyze the cross-country differences,
comparing the means (the estimated dummy coefficients) and standard deviations
(standard deviation of estimated coefficients). For Argentina in particular, controlling for
socioeconomic variables does not change the main result, and for the overall sample the
change was not so large (the coefficient of correlation between both rankings was 73 %.).
Using PIRLS, for instance, Argentina ranks 30 out of 35 countries according to the score
in Math, and it remains in the same position after controlling for student characteristics.
For Argentina, most of the previous conclusions are preserved when we eliminate the
individual factors, therefore we do not show the analysis here.15
IV.3. Quality of Education from a Mincerian Perspective
A measure of quality of education can be obtained comparing the returns to one more year
of schooling for similar workers (controlling by other worker’s observable characteristics)
in the same labor market, but educated in different systems. This is precisely what
Bratsberg and Terrell (2002) do using U.S. Census data (1980 and 1990). Since Hanushek
and Kim (1999) find a strong correlation between the implicit quality index obtained from
Mincer equation for immigrants in the U.S. (who have studied in their countries of origin)
and direct measures of school quality (standardized test), we interpret the difference in
returns to one more year of schooling as differences in the quality of education.16 From the
67 countries analyzed in Bratsberg and Terrell (2002), Argentina ranked 23rd in 1980 and
fell to 27th place in the 1990. Developed countries have better quality than Argentina, but
Argentina is the top performer in Latin America and the Caribbean. In addition, Argentina
in 1990 was relatively worse off compared to its situation in 1980. But the 1990 workers
were educated in the 70s or earlier, and the perception of the deterioration of the public
15
Results are available from the authors.
Since migration is not exogenous, there might be a process of self-selection on unobservable
characteristics, which might be not homogeneous across countries. Hendrix (2002) controls for this
endogeneity, still finding similar rankings.
16
47
education is something relatively new, reflected in the recent growth of the private schools
enrollment share.
Table 8. Ranking by Implicit Quality of Education
Latin America and the Caribbean, 1990 US Census Data
Country
Argentina
Uruguay
Chile
Brazil
Costa Rica
Panama
Jamaica
Colombia
Cuba
Peru
Ecuador
Honduras
El Salvador
Guatemala
Dominican Republic
Mexico
Haiti
Overall Rate of Return of one
Ranking more year of schooling
27
0,0506
36
0,0461
42
0,0438
46
0,0417
50
0,0377
52
0,0364
53
0,035
56
0,0332
57
0,033
58
0,032
60
0,0277
62
0,0234
63
0,0221
64
0,0214
65
0,021
66
0,0203
67
0,0202
Source: Bratsberg and Terrell (2002)
Compared to similar income level countries (see Table 9), Argentina looked relatively
well in 1980 and 1990, above Poland, Hungary and Chile, but below The Czech Republic.
Note that all these countries, except for Hungary, fell in the ranking between 2 and 4
positions, therefore the trend observed for Argentina was also observed for this group of
peer countries.
Table 9. Ranking by Implicit Quality of Education
Similar Income Level countries
Ranking
1980
Country
Ranking
1990
Country
30
29
27
21
19
Poland
Hungary
Chile
Argentina
Czech Republic
32
27
31
24
21
Poland
Hungary
Chile
Argentina
Czech Republic
Source: Bratsberg and Terrell (2002)
IV.4. Recent Evolution in the Quality of Education
The standardized test does not allow us to analyze the recent evolution of Argentina in
terms of quality of education. The perception is that the quality has deteriorated; in part
as a result of the subsequent crisis and fiscal constraints that has affected the economy in
general and the educational system in particular.
48
To analyze the recent performance of Argentina in terms of quality of education we
combine two data sets: the quality of education estimation of Bradsberg and Terrel
(2002), which gives us a ranking for Argentina for workers in 1980 and 1990 (related
with the quality of education when those workers were educated), and PISA (2000). For
PISA we construct two measures: a) the average test score and b) the average quality of
education, what we measure as the mean test score of the country after controlling for
individual characteristics as explained before. We were able to match 35 countries which
are in both databases. Table 10 shows the corresponding rankings and the estimated
measure of quality for each year.
Argentina ranked 19 out of 35 countries in 1980, for workers educated earlier than 1980
(assuming the average worker is 40 years old, it corresponds to students that were 15
years old in 1955). In the US Census of 1990 Argentina fell to the position 23 (again this
corresponds to individuals educated in early years than 1990, for instance 1965.) The
next column shows the ranking according to PISA-2000, i.e. for 15 years old students
who took the test, where Argentina ranked 30 out of 35 countries. Finally, the last column
corresponds to PISA 2006, where Argentina ranks 32 out of 35 countries. This shows a
very significantly decrease in the quality of education, and Argentina ranks among the
worst performers in terms of the evolution, representing the third largest fall in the
ranking between 1980 and 2000. Brazil is the worst relative performer in this period
(with the highest fall in the ranking), but an important difference is that Brazil, in this
period, expanded significantly the coverage, particularly in secondary school, whereas
Argentina had already reached a fairly good coverage (in international terms) by the early
80s, therefore the fall in Argentina is not due to a change in composition (as in Brazil) but
more related with a fall in quality. Similar income level countries like Hungary, Poland
and The Czech Republic are in a much better relative situation in 2000 compared to 1980
and 1990, and they ranked even better in the recent PISA 2006. The increased
performance in the ranking for these countries is related to the growth performance.
Argentina between 1980 and 2000, where the quality of education apparently went down,
grew at an annualized rate of 2.4% (GDP per capita ppp) whereas The Czech Republic,
49
Hungary and Poland grew at 3.2%, 3.4% and 3.4% respectively. What is even more
striking is that after the 2000, when Argentina continued loosing positions, growth was
even more unfavorable for Argentina, 0.5% per year up to 2006 compared to 1.2%, 1.4%
and 1.3% for the East European countries respectively.
Table 10. Relative Evolution of Argentina in terms of Quality of Education
Ranking
1950s (1)
1960s(2)
2000 (3)
2006 (4)
(best to
worst)
Country
Value
Country
Value
Country
Value
Country
Value
1
Norway
0.0632 Japan
0.0822 Netherlands
566.6 Finland
563
2
Switzerland
0.063Norway
0.0789Japan
554.5 Canada
534
3
Denmark
0.059Sweden
0.0739Rep. of Korea
536.8 Japan
531
4
Belgium
0.0584New Zealand
0.0729Finland
533.1 New Zealand
530
5
Australia
0.0566Switzerland
0.0716New Zealand
532.5 Australia
527
6
UK
0.056UK
0.0703Switzerland
527.8 Netherlands
525
7
Canada
0.0555Australia
0.0703Australia
527.1 Korea
522
8
Sweden
0.0543Austria
0.0699UK
526.5 Germany
516
9
Austria
0.0533Denmark
0.0692Canada
522.2 UK
515
10
France
0.0531Belgium
0.069Belgium
521.7 Czech Republic
513
11
Japan
0.0522Canada
0.0685Denmark
514.2 Switzerland
512
12
Netherlands
0.0511Finland
0.0671France
513.6 Austria
511
13
Germany
0.0509Netherlands
0.0654Sweden
509.1 Belgium
510
14
Brazil
0.0496France
0.0645Austria
506
Ireland
508
15
Finland
0.049Germany
0.0635Ireland
501.7 Hungary
504
16
Italy
0.0442Ireland
0.0587Germany
499.8 Sweden
503
17
Czech Republic
0.0442Israel
0.0562Czech Republic
499.1 Poland
498
18
New Zealand
0.044Italy
0.0542Norway
498
Denmark
496
19
Argentina
0.0436Czech Republic
0.0534Hungary
486.1 France
495
20
Portugal
0.0433Macedonia/1
0.0522Spain
480.6 Spain
488
0.0432Spain
0.0518Russian Fed.
479.2 Norway
487
21
Macedonia/1
22
Ireland.
0.0429Indonesia.
0.0508Poland.
464.3 Russian Fed.
479
23
Spain
0.0424Argentina
0.0506Portugal
462.3 Italy
475
24
Romania
0.0414Romania
0.0501Italy
461.7 Portugal
474
25
Chile
0.0406Hungary
0.0482Greece
451.8 Greece
473
26
Indonesia
0.0402Russian Federation
0.045Romania
451
Israel
454
27
Hungary
0.04Republic of Korea
0.0449Israel
448.5 Chile
438
28
Poland
0.0398Portugal
0.0446Thailand
444.1 Thailand
421
29
Israel
0.0386Chile
0.0438Mexico
406.3 Romania
418
30
Russian Fed.
0.0339Poland
0.0431Argentina
404
Mexico
410
31
Republic of Korea 0.0333Greece
0.0429Chile
398.1 Indonesia
393
32
Peru
0.0301Brazil
0.0417Macedonia
391.5 Argentina
391
33
Greece
0.03Thailand
0.0341Indonesia
363.4 Macedonia*
374
34
Thailand
0.0252Peru
0.032Brazil
352.3 Brazil
390
35
Mexico
0.0248 Mexico
0.0203 Peru
324.6 Peru*
327
Notes: (1) Corresponds to the ranking of Bratsberg and Terrrel using US Census 1980 (where we assume the average worker has 40
years old, therefore they were in 4th grade in the 50s)
(2) Corresponds to the ranking of Bratsberg and Terrrel using US Census 1990 (where we assume the average worker has
40 years old, therefore they were in 4th grade in the 60s)
(3) Ranking according to the SES adjusted mean score based on PISA 2000.
(4) Ranking according to average score based on PISA 2006.
1/ Macedonia ranking correspond to the former Yugoslav Republic. * Corresponds to the 2000 score, since 2006 was not available
50
Figure 11. Change in relative ranking between 1980 measure and 2000 measure
Republic of Korea
New Zealand
Netherlands
Finland
Japan
Russian Federation
Hungary
Greece
Ireland
Poland
Thailand
Mexico
Spain
Israel
Czech Republic
Australia
UK
Canada
France
Romania
Germany
Portugal
Peru
Switzerland
Sweden
Austria
Belgium
Chile
Indonesia
Denmark
Italy
Argentina
The former Yugoslav Republic of Macedoni
Norway
Brazil
-25
-15
-5
5
15
25
Combing PISA, PIRLS and LAB, as explained before, we can create an even larger set of
countries, what is shown in the next table. In this expanded sample Argentina was 21 out
of 42 countries in 1980, it fell to the position 24 in 1990, and to the position 33 in 2000,
what again shows a declining trend in terms of relative quality, but not so drastic as the
previous figure.
IV.5. Equity in the Distribution of the Quality of Education
A striking figure for Argentina is the difference in performance between high
socioeconomic level and low socioeconomic level students. According to PISA 2000, the
25% with highest SES in Argentina has an average score which is 104 points higher than
the lowest 25% students. This is the largest difference among the Latin-American
countries included in the sample, and well above the difference for OECD countries. Also
note that the top 25% is closer to OECD average (62 points below) than the lowest 25%
(84 points). We will discuss in more detail later how achievement and quality is
distributed among students and schools.
51
Table 11. Performance in PISA 2000 according the SES quantiles
Argentina
Brazil
Chile
Mexico
Peru
Avg. OECD
<25%
S.E.
25%-50%
S.E.
50%-75%
S.E.
379
368
373
385
283
463
(7.1)
(3.9)
(3.8)
(4.1)
(5.9)
(0.9)
393
387
388
408
317
491
(9.9)
(3.8)
(4.3)
(3.7)
(4.3)
(0.8)
440
413
420
435
338
515
(9.6)
(4.0)
(4.6)
(4.0)
(4.7)
(0.7)
>75%
S.E.
483
435
466
471
383
545
(6.3)
(4.5)
(3.5)
(5.9)
(5.8)
(0.9)
Top 25% vs.
lowest 25%
104
67
93
86
100
82
Source: PISA, Argentina country report
Here, based on PISA (2000), we analyze in more detail how the achievement is
distributed. First we analyze the differences in average score for quintiles based on
student’s wealth. We find that Argentina is the country with the highest difference in terms
of score points between the 20% wealthier and 20% poorest (97 points), followed by the
U.S. (86 points) and Chile (82 points). Latin American countries are in the top of this
ranking.
The ranking is sensible to the socioeconomic variable chosen to make the quintiles. For
instance, some countries which have small differences between the top and lowest 20% in
terms of wealth, such as The Czech Republic (31.8 points) have large differences when we
chose the household socioeconomic index (based on wealth, parents´ education, and
parents´ occupation) as the ranking variable (105 points), and changes from the position
21 to 39 among the 43 countries (ordered from lowest to highest difference). But
Argentina is still among the countries with the highest differences, 105 points.
We estimate quality as the residual of an OLS estimation of test score on student
socioeconomic variables for the entire sample, and then compute the difference in average
quality by quintiles instead of the difference between average scores. The results do not
change much. Argentina is still among the countries with the highest difference between
students with SES in the top 20% and the lowest 20%.
52
Figure 12. Difference in average score between top and lowest 20% according to wealth
PISA 2000
Argentina
United States of America
Chile
Portugal
M exico
Brazil
Israel
Romania
Peru
Luxembourg
Germany
Hungary
France
New Zealand
Indonesia
Greece
Bulgaria
Spain
United Kingdom of Great Britain and Nort
Thailand
Australia
Canada
Czech Republic
Austria
Russian Federation
Ireland
Switzerland
Hong Kong Special Administrative Region
Republic of Korea
Liechtenstein
Italy
Finland
Belgium
Poland
Denmark
Sweden
The former Yugoslav Republic of M acedoni
Latvia
Japan
Norway
Albania
Netherlands
Iceland -13.0
-20
96.9
85.6
82.1
78.5
75.6
74.4
73.1
65.4
65.2
63.1
59.2
52.1
51.2
51.0
44.7
40.3
39.6
39.5
35.5
35.2
33.3
32.7
31.8
30.1
29.8
28.8
26.7
26.0
24.8
24.5
23.5
22.2
21.0
19.5
19.2
18.0
16.5
16.0
7.6
5.3
2.3
-2.2
0
20
40
60
80
100
120
Figure 13. Difference in average score between top and lowest 20% according to household
socioeconomic index
PISA 2000
Germany
Switzerland
Luxembourg
Argentina
Czech Republic
Bulgaria
Belgium
Portugal
Peru
United Kingdom of Great Britain and Nort
Israel
Hungary
Austria
Albania
Poland
Australia
Chile
United States of America
Mexico
New Zealand
Greece
The former Yugoslav Republic of
Liechtenstein
France
Ireland
Denmark
Russian Federation
Sweden
Romania
Spain
Netherlands
Italy
Canada
Brazil
Norway
Latvia
Indonesia
Iceland
Thailand
Finland
Hong Kong Special Administrative Region
Republic of Korea
Japan
119
113
107
105
105
102
99
99
99
98
97
96
95
93
90
89
89
88
88
85
84
84
83
81
76
74
73
72
71
70
69
69
68
68
65
63
59
55
54
49
42
37
19
0
20
40
60
80
100
120
140
53
Figure 14. Difference in average quality between top and lowest 20% according to household
socioeconomic index
PISA 2000
114.3
113.5
109.4
Mexico
Romania
Argentina
Peru
Brazil
Portugal
Chile
United States
Albania
Indonesia
Greece
Hungary
Poland
The former
Latvia
Spain
Israel
New Zealand
Ireland
Luxembourg
France
Switzerland
Germany
Italy
Belgium
United
Thailand
Canada
Australia
Russian
Bulgaria
Finland
Czech Republic
Iceland
Denmark
Netherlands
Sweden
Austria
Liechtenstein
Republic of
Norway
Hong Kong
Japan
99.3
98.1
95.3
94.3
90.8
90.6
89.7
89.3
86.7
86.5
85.2
83.8
83.5
83.3
82.8
82.5
81.7
81.5
80.8
79.7
79.6
79.2
79.0
78.9
77.8
77.1
77.0
76.8
74.2
74.0
73.3
72.9
71.6
71.3
69.8
68.8
68.8
66.6
61.5
59.3
0
20
40
60
80
100
120
The high inequality in Argentina is not only related to socioeconomic variables, but also in
terms of best and lowest performers in this test. If we construct quintiles by test score or
quality, Argentina still ranks as one of the countries with the highest difference between
the highest 20% and lowest 20% in terms of achievement.
In terms of the inequality looking at the entire distribution we find similar results. No
matter what index of inequality we take into account, Argentina is among the countries
with the highest inequality in terms of scores and quality (3
th
and 4th, respectively).
Nevertheless, it is also true that in Argentina the inequality in terms of wealth and SES is
among the highest in the world. According to the Gini coefficient (or Theil coefficient) for
parents’ SES, Argentina ranks 4th among 43 countries. It might look surprising that some
countries which usually rank worse that Argentina in terms of income inequality, such as
Brazil, are better ranked than Argentina in our ranking of parents’ socioeconomic level,
but this is explained by the differences in coverage; since Brazil, for instance, has a much
smaller net enrollment ratio than Argentina at secondary school, and therefore the
54
distribution is truncated due to those who are not in the school at 15 years old, which in
general have lower income and socioeconomic level.
The correlation between the inequality in achievement and the inequality in parents’ SES
is extremely high. The inequality in SES explains almost 60% of the variation in the
achievement inequality in a cross-country regression. What this seems to show that part of
the inequality in education observed in Argentina could be due to a more structural pattern
of inequality. Nevertheless, Argentina is above the cross-country linear relationship, what
means that its level of inequality in education is above the expected level given its
inequality in SES. It is interesting to note that all the countries with similar income level
than Argentina have lower SES inequality, and in general they rank better in terms of SES
inequality than score or quality inequality. For instance, The Czech Republic has the
lowest Gini coefficient for SES, but it ranks 17 according to the Gini coefficient for
scores, or 15 according to the Gini coefficient for quality.
Figure 15. Gini coefficient for test score
PISA 2000
0.160
0.157
Poland
Albania
Argentina
The former
Israel
Mexico
Romania
Bulgaria
Germany
Liechtenstein
Brazil
Chile
Netherlands
Belgium
Portugal
United States of
Republic of Korea
Switzerland
Indonesia
Norway
Peru
Greece
Russian Federation
Hungary
Austria
Denmark
Czech Republic
United Kingdom of
Luxembourg
Australia
Italy
France
Thailand
Iceland
Sweden
Ireland
Canada
Spain
New Zealand
Japan
Finland
Hong Kong Special
Latvia
0.147
0.141
0.138
0.134
0.133
0.131
0.125
0.124
0.122
0.121
0.119
0.118
0.116
0.115
0.115
0.114
0.114
0.114
0.113
0.113
0.112
0.111
0.110
0.110
0.110
0.108
0.108
0.107
0.104
0.103
0.102
0.102
0.101
0.099
0.098
0.097
0.094
0.093
0.091
0.090
0.076
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
55
Figure 16. Gini coefficient for quality
PISA 2000
0.070
Netherlands
Poland
Romania
Argentina
Indonesia
Brazil
Thailand
Chile
Israel
Albania
Liechtenstein
Russian Federation
Republic of Korea
Portugal
The former Yugoslav Republic of Macedoni
United States of America
Hungary
Greece
Norway
Ireland
Spain
Mexico
France
Switzerland
Australia
Bulgaria
United Kingdom of Great Britain and Nort
Belgium
Czech Republic
Germany
Canada
Finland
Italy
Sweden
Iceland
Luxembourg
Denmark
New Zealand
Austria
Peru
Latvia
Hong Kong Special Administrative Region
Japan
0.065
0.065
0.064
0.064
0.062
0.060
0.057
0.054
0.054
0.053
0.052
0.052
0.052
0.050
0.050
0.048
0.048
0.047
0.047
0.046
0.045
0.044
0.044
0.044
0.044
0.044
0.044
0.044
0.043
0.043
0.042
0.042
0.041
0.041
0.041
0.041
0.040
0.039
0.039
0.038
0.038
0.036
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Figure 17. Gini coefficient for parents’ SES
PISA 2000
Indonesia
Albania
Thailand
Argentina
Netherlands
Brazil
Chile
Poland
Greece
Liechtenstein
Mexico
Romania
The former Yugoslav Republic of Macedoni
Spain
Republic of Korea
France
Russian Federation
Belgium
Portugal
Latvia
Switzerland
Italy
Luxembourg
Finland
Denmark
Ireland
Sweden
Norway
New Zealand
Hungary
Germany
Iceland
Australia
United States of America
United Kingdom of Great Britain and Nort
Japan
Canada
Bulgaria
Israel
Peru
Hong Kong Special Administrative Region
Austria
Czech Republic
0.274
0.239
0.232
0.230
0.224
0.220
0.219
0.219
0.214
0.211
0.208
0.207
0.205
0.204
0.202
0.198
0.195
0.193
0.190
0.189
0.188
0.188
0.186
0.186
0.185
0.184
0.182
0.182
0.181
0.180
0.179
0.179
0.179
0.179
0.178
0.174
0.173
0.171
0.164
0.161
0.161
0.160
0.158
0
0.05
0.1
0.15
0.2
0.25
0.3
56
Figure 18. Relationship between inequality in quality and inequality in SES at country level
PISA 2000
0.08
y = 0.2716x - 0.0044
R2 = 0.5666
0.07
AR
Gini coefficient for students' quality
0.06
0.05
0.04
0.03
0.02
0.01
0
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
0.26
0.28
0.3
Gini Coefficient for parents' SES
Finally we analyze the inequality across schools instead of students. First we compute the
average score and average quality at school level, and then we compared the school
averages. Argentina is less unequal between schools than between students in terms of
scores. Its ranking changes from second at student level in terms of highest inequality
(comparing the students who performed at the top 20% and lowest 20%) to ninth when
analyze inequality between schools. Some countries, particularly the Eastern European
countries we have mentioned before (Hungary, Poland and The Czech Republic) have the
opposite result, they are much more unequal at the school level than at the student level.
For instance, Hungary ranks 24 in terms of inequality at the student level, but third in
terms of inequality at school level. Poland ranks 20 at student level but sixth at school
level. Among Latin American countries only Chile shows this pattern of more inequality
at school level. The only way a country can have more inequality at school level than at
the student level is if the country sorts the good students in one school and bad students
in other schools. Argentina, on the other hand, shows that the ability sorting is not so
57
strong, and the mixed of good and bad students is more homogeneous across schools.
Chile is the most extreme example, and there is evidence that the voucher system has
facilitated the segregation (see Auguste and Valenzuela (2004). In Chile 50%, of the
students in primary school attend a private school, which has freedom to admit students
as they want. In Argentina, the share is much smaller (20%) and public schools in general
do not apply any selection in the admission process (there are, though, some exceptions).
We will further explore later in the paper the role that ability tracking and segregation
might have on performance.
IV.6. Comparing the Educational Systems
So far we know now Argentina is underperforming in terms of quality of education, that
part of that underperforming could be explained by the low investment in education, given
the population in school age. We also know that the quality of education is very unequally
distributed within students, and part of this can be explained by the high inequality in
students’ characteristics. Before analyzing these hypotheses in more detail, based on
PISA and PIRLS we will analyze the differences between Argentina and a set of
comparators in the “effective” educational system. That is, we will not compare rules or
the design, but how the system is really working according to the teachers and school
director responses.
To obtain comparable countries we cluster17 countries using a quadratic Euclidean
distance rule18 along the following dimensions: GNI per capita (Atlas Method), life
expectancy, expenditure in education as a share of GDP, Human Development Index.
17
Cluster analysis encompasses a number of different methods for grouping objects into categories on the basis of their
similarity. Cluster analysis is an exploratory tool aimed at sorting country observations into groups in a way that the
association between two observations is high if they belong in the same group and low otherwise. Clearly, cluster
analysis is used to uncover structures in data without a preliminary interpretation, i.e., cluster analysis helps in
discovering structures in data without explaining them.
18
Distance in n-dimensional space is used as a clustering rule for grouping objects. The most straightforward way of
computing distances in n-dimensional space is to calculate Euclidean distances. Standard Euclidean and quadratic
Euclidean distances are the two most common clustering rules; quadratic distance places greater weight on
observations that are further apart. Quadratic Euclidean distance is computed as:
2 .
D ( x, y ) =
∑ (x
i
− yi )
i
58
According to this index from the countries participating in PISA 2000 the most similar to
Argentina, in decreasing order are: The Czech Republic, Mexico, Hungary, Poland and
Chile. The least similar are: Sweden, Denmark, U.S.A., Japan, Norway and Switzerland.
To do the benchmarking with PISA we use the following set of countries:
a) Similar income level and educational effort: The Czech Republic, Mexico,
Hungary, Poland and Chile
b) Developed with similar culture: Spain and Italy
c) Developed with high scores: Australia, Canada, Finland and New Zealand
Some of the conclusions of this simple benchmarking exercise are:
a) Compared to similar countries: Argentina has the highest school life expectancy
(average years of schooling) but it is a poor performer in terms of average score,
even when it has the highest GNI per capita and second highest HDI. The low
score is in line with its expenditure in Educations as share of GDP, since it has the
lowest ratio.
b) Compared to similar countries: Once we control by student SES and compute the
mean score (SES adjusted mean), Argentina is the second country with the highest
increase in average score, second after México, and right before Chile. This shows
that part of the bad performance is due to lower student SES. The difference with
the top performer in this group is 14% in average score and 10% in SES adjusted
scores.
c) In terms of inequality, Argentina is the country with the highest inequality in
individual scores (coefficient of variation), but once we take into account the SES
inequality Argentina is the fifth country with highest inequality (slope of the SES
line) from the 12 countries. Within the similar countries group, Hungary and The
Czech Republic have the highest slopes, what shows high inequality according to
SES. Compared to the entire sample, Argentina has the highest coefficient of
variation but it is 24 of 42 countries in terms of slopes (ranked from lowest to
highest).
59
d) The similar culture countries, Spain and Italy, are also underperformers within
similar countries. The most similar country to Italy according to our clustering
measure is Australia, which has an average score 34 point higher, or a SES
adjusted mean score 28 points higher. For Spain, the most similar country is New
Zealand, which has an average score 39 points higher, and a SES adjusted score 19
points higher. But both, Australia and New Zealand show more inequality
(according to the SES slope) than Italy and Spain (that show the lowest inequality
among the 12 countries), and even higher than Argentina.
e) Argentina, with Mexico and Chile, have more young people to educate (population
age 5 to 14 years old). Computing the expenditure per student in primary school as
a ratio of the GPD per capita, Argentina has the lowest ratio (11.6), even lower
than Chile (17.6) and Mexico (13.3). The differences in this indicator are much
more notorious when compared the Eastern European countries.
Table 12. Country Characteristics
Average
Score
School life
expectancy
GNI per
capita (US$)
HDI
Expenditure in
Life
Education (% Expectancy
of GDP)
(years)
Similar Countries
Argentina
422
12.9
7000
0.860
3.5
74.3
Chile
416
11.1
4600
0.843
3.7
77.9
Czech Republic
491
12.2
5750
0.865
4.6
75.5
Hungary
481
11.7
4750
0.845
6.0
72.6
Mexico
423
10.9
5580
0.811
5.8
74.9
Poland
480
11.9
4650
0.848
5.8
74.3
Similar Culture
Italy
493
12.5
20180
0.924
4.9
80.0
Spain
488
12.9
15050
0.927
4.5
79.5
Australia
527
16.6
20490
0.947
4.8
80.2
Canada
532
12.3
22090
5.2
78.4
Finland
544
13.6
24160
0.938
6.5
79.0
New Zealand
527
14.0
13560
0.925
6.9
78.6
High Score
Source: Own elaboration based on World Bank data
60
Table 13. Performance in PISA
Average Score
Coef. of
Variation
SES adjusted
mean (*)
SES Slope (*)
Similar Countries
Argentina
422
0.279
453.7
41.2
Chile
416
0.216
441.7
40.9
Czech Republic
491
0.200
501.5
49.3
Hungary
481
0.198
486.5
53.9
Mexico
423
0.210
459.5
35.1
Poland
480
0.209
495
37.8
Italy
493
0.186
485.1
31.6
Spain
488
0.173
504.3
31.9
Similar Culture
High Score
Australia
527
0.190
513.4
46.1
Canada
532
0.176
527.3
36.9
Finland
544
0.165
544.4
29.8
New Zealand
527
0.203
523.2
44.9
Sweden
514
0.180
504.0
35.9
(*) Wilms (2006)
Based on PIRLS we create the following set of comparators:
d) Similar income level and educational effort: The Czech Republic, Colombia, and
Hungary
e) Developed with similar culture: Italy
f) Developed with high scores: Sweden, England.
Table 14. Performance in PIRLS
Average
St Dev
Score
Coef. of
Var.
SES adjusted SES Slope
mean (*)
(*)
Similar Counties
Argentina
419.5
90.9
0.2167
446.6
33.3
Colombia
422.4
76.3
0.1807
437.5
21.1
Czech Republic
536.9
60.8
0.1132
542.1
27.4
Hungary
543.2
62.0
0.1141
543.2
37.8
540.7
67.0
0.1240
548.7
22.1
England
552.9
82.7
0.1496
563.7
33.9
Sweden
561.1
61.6
0.1097
548.6
23.1
Similar Culture
Italy
High Performers
(*) Wilms (2006)
61
Since PIRLS test 4th grade students and PISA 3th year secondary school students (15 years
old), we can obtain some interesting conclusions of just comparing the performance in
both tests:
a) Argentina performs relatively better in PISA (secondary school) than in PIRLS
(early primary school). Whereas Argentina increases the average score in 2.5
points (or the mean SES in 7.1 points) when comparing PISA and PIRLS, The
Czech Republic, Hungary, and Poland reduce the mean by 46, 62 and 48
respectively (or 41, 57, and 64 respectively when we compare the mean SES
adjusted score). Since the difference holds (and it is even higher) after controlling
by student SES, there is not a compositional effect biasing the result. Argentina
somehow is able to reduce the gap, although the gap is still large.
b) The SES slope is higher in PISA (secondary school) than in PIRLS (primary
school), but this is true for all the countries in this benchmark group.
c) Argentina has a SES slope lower than Hungary and The Czech Republic in
secondary school, but higher or similar in primary school, what might be explain
by the higher drop out, or the differentiation in secondary schools in these two
countries
compared to Argentina. Hungary, for instance, has two type of
secondary schools: a comprehensive or academic secondary school (Gimnázium)
and a vocational secondary school (Szakközépiiskola), and the admission to each
type depends on previous performance, what might create more sorting and
differences than the secondary school in Argentina, where there is less
differentiation, and usually the admission does not depend on previous
performance, but more on the student choice. Something similar is found in The
Czech Republic, where secondary education comprises three main types of
schools: secondary general schools (gymnasium), secondary technical schools and
secondary vocational schools.
d) The differences in performance between Argentina and The Czech Republic,
Hungary or Italy (benchmark countries included in both tests) is more significant
in primary level (PIRLS) than in secondary school (PISA). For instance, Hungary
obtains 124 point more than Argentina in 4th grade, but just 59 points more in
62
secondary school, what shows that the gap is reduced to less than half. This is not
explained by student composition, since in the SES adjusted average score the
reduction of the gaps is even higher: Hungary obtains 97 points more than
Argentina in primary school but just 33 points more in secondary school, a fall in
the difference of 74%. With Italy this reduction in the gap for the SES adjusted
score is even more marked. Looking at average scores Italy has 121 points more
than Argentina in primary school, and 71 points more in secondary school, what
gives a reduction in the differences of 31 points. But looking at the SES adjusted
average score, Italy has 102 points more than Argentina in primary and 31 points
more in secondary, a reduction in the difference of 71 points.
e) The increase in the SES inequality given by the change in the slope between
primary and secondary education is also more significant in the other countries.
The SES slope increases for Argentina in 24% when comparing secondary with
primary education, but in The Czech Republic it increases 80%, in Hungary 43%
and in Italy 43%.
Table 15. Comparing PISA and PIRLS performance
Difference in points with Argentina
Average
Score
PISA
SES
Adjusted
mean
Czech Republic
-69
Hungary
-59
Italy
-71
-31.4
Sweden
-92
-50.3
PIRLS
SES
Adjusted
mean
SES
Slope
Average
Score
SES
Slope
-47.8
-8.1
-117.4
-95.5
5.9
-32.8
-12.7
-123.7
-96.6
-4.5
9.6
-121.2
-102.1
11.2
5.3
-141.6
-102
10.2
To understand better how Argentina seems to be able to reduce the gap, we compare the
countries in terms of other variables usually associated with quality, such as repetition rate
and drop out, We find that Argentina is underperforming in terms of these indicators too,
with the highest repetition rates. Given its educational attainment, it means many students
finished the school later than expected, what might be seen as an inefficient way to
achieve the same average years of schooling and to reduce the score gap with the before
mentioned countries.
63
Table 16. Other quality related indicators
Repeaters as % of Repeaters as % of
total enrolled,
total enrolled,
primary
secondary
Argentina
6.4
11.5
Chile
2.4
Colombia
5.4
Czech Rep.
1.1
Pupils reaching Progression to
grade 5 (% of
secondary
cohort)
school (%)
84.3
92.8
2.7
99
96.5
4
60.9
89.6
1
98.4
99.2
Hungary
2.2
2.8
n.a.
98.6
Mexico
4.8
2.1
92.6
93.6
Poland
0.6
1.6
99.7
98.5
Italy
0.4
3.2
96.5
99.8
Finland
0.5
0.4
99.4
99.9
Source: Own elaboration based on EdStats, year 2000
To see the role of repetition, we illustrate in the next table the repetition rate for the first
4th grade in the primary school, and the hypothetical survival rate for several of the
benchmark countries. Argentina is the most inefficient country, with the highest repetition
rate in all the grades, what means that at the end of fourth grade it would end up with just
74% of the students compared with more than 90% in all the other countries. Of course
this hypothetical rate is not exact, since students can repeat more than once. In ONE 2000,
for instance, from the entire population which is in sixth grade in 2000, 19% repeated at
least once, what is a very high ratio.
Table 17. Repetition Rate by Grade
Hypothetical
Survival Rate
Hungary
Argentina
Chile
Czech Republic
Italy
91.3%
73.7%
92.5%
95.3%
98.2%
Repetition rate by grade
Grade 1
2.1
5.9
2.0
1.2
0.4
Grade 2
3.8
10.0
0.9
1.5
0.6
Grade 3
1.8
7.2
3.9
1.1
0.5
Grade 4
1.3
6.1
0.8
1.0
0.3
Source: Own elaboration based on EdStats, , year 2000
What are the difference between Argentina and these countries other than the scores? A
first noticeable difference is the difference in the yearly time allocated to instruction. In
Argentina the average number of hours of school per year is 693, well below all the other
countries in the benchmark group.
64
Table 18. Number of school hours per year
Mean
St. Dev.
Coef. Of Variation
Argentina
693.7
133.0
0.192
Colombia
1072.5
425.4
0.397
Czech Rep.
809.2
166.9
0.206
Hungary
n.a.
n.a
n.a
Italy
1038.4
176.6
0.170
England
958.3
68.0
0.071
Sweden
860.0
124.4
0.145
A second difference is the proportion of students who reach grade 4th with reading
problems. According to the teachers, in Argentina 30% of the students need remedial
instructions in reading in grade 4, compared to 20% for the entire sample, and 15% in The
Czech Republic. In the entire sample Argentina ranks 4th in terms of higher ratio of
students with remedial instructions needs (of 34 countries with answers in this question).
The school director, when asked about students from grade 1 to 4 shows a similar pattern,
learning problems are very generalized and its incidence is higher than any other country.
This might be a sign that the problem in Argentina is manifest early in the educational
process.
Table 19. % of Students with Reading Problems. PIRLS
% of students that
need remedial
instruction in reading
according to teachers
Percentage of students from grade 1 to 4 that have learning
disabilities related to reading, according to the school director
0-10%
11-25%
26-50%
more than 50%
Argentina
30.0
21.66
31.23
36.22
10.88
Colombia
25.5
40.51
32.13
20.45
6.91
Czech Rep
14.9
52.43
44.3
3.27
0
Hungary
16.5
52.54
40.88
5.9
0.67
Italy
12.0
62.64
32.05
4.58
0.74
England
18.1
38.06
40.96
19.11
1.87
Sweden
16.1
50.64
43.87
5.49
0
World
20.3
71.07
18.76
7.1
3.07
Even a more striking difference is that although a very high ratio of students in Argentina
have already learning problems in 4th grade, schools do not have reading specialists. Only
4% of the total students attend a school which has a reading specialist, this is the second
65
lowest ratio in the entire sample (Turkey has 3.5% of the students and Italy 4.4%). This
ratio is well below the world average, 44.4%, and the countries with similar students needs
as Argentina. Colombia, for instance, which also has a relatively high ratio of students
with reading problems in 4th grade, has 12.3% of the students attending schools with
reading specialists. In Argentina, only 5% of the students work with reading specialist
compared to 14% in Colombia, 20% in Hungary or 32% in The Czech Republic. Sweden,
the best performer, has 77% of the students working with reading specialist, and England,
another good performer, has 60%, whereas Italy, a low performer for its development
level, has only 10%. Sweden and England have also reading specialist working
specifically with those students who have difficulty in reading (79% and 73%
respectively) , something that all the countries in the sample do more than Argentina. The
Czech Republic, for instance, has 17% of the students in schools with reading specialist
for the students with problems whereas in Argentina only 3.7%.
Some countries, such as Hungary and Italy, do not have a reading specialist but have
teacher-aid or other adults helping with those students with problems. In Argentina, only
7% of the students are in schools that use other kind of help for these students.
Table 20. Use of reading specialists. PIRLS
Argentina
Colombia
Czech Republic
Hungary
Italy
England
Sweden
World
% of students in school that has a reading
specialist available:
Always
Sometimes
Never
1.4
2.7
95.9
2.3
10.0
87.7
21.1
26.9
52.0
6.3
11.8
81.9
0.6
5.1
94.4
12.8
63.7
23.5
17.9
63.7
18.4
11.1
23.7
65.3
% of students
working with a
reading specialist
5.6
13.9
32.4
19.9
9.6
59.4
77.2
30.0
Other interesting aspect of how teachers teach in Argentina is their response to the
teachers needs. If the student begins to fall behind in reading, almost 60% of the students
are in schools where the teacher does not have the student doing other activities, whereas
in Sweden is the opposite, 84% of the teachers respond given the student other activities.
Only Italy has a similar approach to Argentina, where 65% do nothing to compensate.
66
In terms of homework, in Argentina 50% of the students have reading assignment as part
of the homework at least three times a week, 20% every day, which below most of the
countries except for Sweden, where only 26% of the students has this burden (but the
country is working in class, and students do the exercise in the school with the specialist
more often than in any other country).
In addition teachers in Argentina do not seem to involve parents in the process in the same
degree as other countries. It is the country from the group of similar development level
with the highest ratio of students in schools where teachers do not sent to home examples
of the student work (or at most they do it 6 times a year).
Table 21. Parents’ Involvement. PIRLS
% of students attending schools where
the teacher sends to home example of
the student work:
Weekly Monthly 6 times a year or less
Argentina
31.64
31.47
36.89
Colombia
35.21
34.63
30.17
Czech Rep
54.65
28.73
16.63
Hungary
13.54
58.6
27.86
Italy
59.34
24.24
16.42
England
7.21
6.47
86.31
Sweden
14.23
16.77
69
In terms of how teachers teach, all the countries have a higher proportion of students in
schools where reading instruction is a separate subject. Only Colombia and Sweden have
an pattern similar to Argentina. In The Czech Republic and Hungary reading as a separate
subject is more common.
Table 22. Reading Instructions. PIRLS
Reading
Reading
instructions as
instruction as a
part of different
separate subject
curricular areas
Both
equally
Argentina
19.28
6.8
73.92
Colombia
37.02
8.19
54.8
6.5
35.25
58.26
55.23
17.74
27.04
Czech Republic
Hungary
Italy
4.69
28.72
66.59
England
9.58
42.69
47.73
Sweden
27.98
7.19
64.83
67
Argentina is the country with lowest quantity of hours per week used for Language
instruction among the benchmark countries, and one the lowest in the sample: only 8
countries from 35 have less hours per week than Argentina, in general poor performers
too. Argentina is also the country in the benchmark group with the lowest number of hours
per week allocated to reading instruction, and also has differences with other countries in
terms of the time allocation (Sweden allocates proportionally more time to reading than
Argentina). Argentina and Colombia are also the countries with a higher percentage of the
weekly time allocated to teach Language assigned to formal instruction.
Table 23. Teaching Time Allocation. PIRLS
Hours of
Hours of
Ratio of hours of
Percentage of the
Language
Reading
reading instruction
time used for formal
Instruction per Instruction per
to language
reading instruction
week
week
instruction
Argentina
6.86
1.85
26.9%
50.6%
Colombia
8.88
1.87
21.1%
65.2%
Czech Rep.
6.99
2.28
32.7%
26.1%
Hungary
6.98
1.88
26.9%
37.9%
Italy
7.85
2.26
28.7%
35.7%
England
7.27
2.40
33.0%
24.6%
Sweden
7.20
2.35
32.6%
26.0%
It is also interesting to note that students in Argentina receive more often reading activities
as a whole-class activity, which is also observed in Italy, and in a lower degree in
Colombia. Sweden, on the other hand, do not use generalized activities so often, neither
England or Hungary.
Table 24. Teaching methodology
Frequency of reading activities as a wholeclass activity
always or almost
often sometimes never
always
Argentina
57.9
37.6
4.6
0.0
Colombia
40.2
41.4
18.4
0.0
Czech Rep.
38.1
54.0
7.9
0.0
Hungary
10.9
64.7
23.7
0.7
Italy
56.9
40.4
2.7
0.0
England
25.3
48.3
26.5
0.0
Sweden
14.9
22.9
54.3
7.9
68
It is also the case that in Argentina as well as in Colombia (and with the exception of the
Czech Republic) teachers tend to create mixed-ability groups when the students are doing
reading instructions or reading activities in group. Ability grouping within the class is not
common in Argentina, neither in Colombia or Italy, where 25%, 31% and 36% of the
students are never cluster in same ability groups, compared to Hungary where 44% of the
student are grouped in same ability groups often or always.
Table 25. How teachers work with student groups
How often do you create mixed-ability groups?
Sum of 1
often
sometimes
never
and 2
Country
always or
almost always
Argentina
26.8
31.3
Colombia
13.1
Czech Rep.
73.1
Hungary
34.6
7.3
27.4
37.0
4.9
21.2
1.0
14.5
Italy
6.4
England
Sweden
Sum of 3
and 4
58.1
41.9
22.5
40.5
59.5
0.7
78.1
21.9
62.2
22.3
15.5
84.5
19.0
53.0
21.6
25.4
74.6
2.7
10.7
80.3
6.3
13.5
86.5
6.4
10.6
58.6
24.5
16.9
83.1
always or
almost always
How often do you create same-ability groups?
Sum of 1
often
sometimes
never
and 2
Sum of 3
and 4
Argentina
6.84
18.67
48.97
25.52
25.5
74.5
Colombia
3.07
21.29
44.74
30.89
24.4
75.6
Czech Rep.
2.43
13.43
69.9
14.23
15.9
84.1
Hungary
5.63
38.96
50.67
4.74
44.6
55.4
Italy
2.12
8.66
53.39
35.83
10.8
89.2
England
26.54
54.69
16.99
1.78
81.2
18.8
Sweden
4.74
22.65
58.15
14.46
27.4
72.6
Finally, Argentina and Colombia use more often reading material for students with
different reading levels than Hungary and The Czech Republic, but much less than rich
countries. This might be related with the availability of resources.
It is interesting to note that teachers do not complain about the time available for teaching
in Argentina, since 90% think to have adequate time available, similar to Hungary and The
Czech Republic (94% and 97% respectively) and well above what teachers think in Italy
and Sweden (53% and 28% respectively). Similarly 89% of the students are with teachers
69
that think the school offers enough incentives for the teacher development, a ratio similar
to Hungary and The Czech Republic, and well above Italy and Sweden.
Table 26. Use of Reading Instructional Material for students at different reading levels
same material,
same material,
same material,
Different
different reading
same reading
different reading material, different
level, different
level
level, same speed
reading level
speed
Argentina
4.7
73.8
5.3
16.3
Colombia
10.5
66.2
4.3
19.1
Czech Rep.
2.2
85.8
3.1
8.9
Hungary
1.1
90.6
1.2
7.1
Italy
8.2
53.7
6.8
31.4
England
0.0
29.5
1.3
69.2
Sweden
2.4
32.9
0.9
63.9
Next we explore whether there are significant differences in the parents’ characteristics. In
particular we are interested on those variables that can give us an idea of the educational
effort of the parents (expenditure in education, time allocated to their kids educational
process, etc.).
We start comparing expenditure. Among the group of similar countries, in 2000 Argentina
was first in terms of GNI per capita (Atlas Method), third in purchasing power parity, and
second according to the Human Development Index. According to PISA, Argentina was
4th according to SES, first according to self report SES, and third according to the SES of
the father’s occupation. Therefore the relative position of parents seems to be in line with
the aggregate income level variables. The striking figure is the spending of Argentinean
parents on educational resources or related goods. Argentina ranks almost in the bottom
(close to Mexico) in terms of the proportion of households with more than 50 books, and
last with Chile in terms of the PISA Index of Educational Possessions.
Argentina has an income per capita adjusted by ppp very similar to Hungary, and a higher
HDI, but whereas 95% of the Hungarian families have more than 50 books, in Argentina
only 69%. Since Argentina has a more unequal distribution than Hungary (for instance, the
Gini coefficient for parents’ SES is 0.23 for Argentina compared to 0.18 for Hungary), we
70
could think that the decreasing marginal utility of books might be generating this effect
(Argentina has more poor people, and the rich people might not compensate what these
poor people do not expend on books, what results that Hungary with the same average
income has more spending on books). We explore this hypothesis first looking at the top
decile, and then looking to the entire distribution. We find that even in the top decile
Argentinean students have less books than the Hungarian students. The difference between
both countries is larger for low SES families.
This finding is not only for books, but for most of the variables related to educational
expenditure, summarized in the Index of Educational Possessions. The evidence seems to
suggest that in Argentina not only there was a lower expenditure per student in education
(adding public and private expenditure) than in Hungary when both countries were
similarly rich, but also parents showed the same pattern, equally rich parents spend less in
educational related goods in Argentina than Hungary.
Table 27. Income Level and Educational Resources. PISA 2000
GNI per
capita
GDP per
HDI
capita PPP
% of
Mean
Index of home
Mean Mean Self
Households
occupational
educational
SES
SES
with more than
SES
resources
50 books
Argentina
7,000
12,210.31
0.86
43.3
66.7
40.3
69%
-0.858
Chile
4,600
9,239.71
0.843
39.9
60.7
37.3
70%
-0.970
Czech Republic
5,750
15,731.24 0.865
48.3
49.8
42.6
96%
0.078
Hungary
4,750
13,212.19 0.845
49.5
53.2
42.0
95%
0.067
Mexico
5,580
9,037.90
0.811
42.5
66.6
40.0
67%
-0.725
Poland
4,650
10,434.74 0.848
46.0
57.3
40.2
85%
-0.305
Italy
20,180
25,808.90 0.924
47.1
58.2
43.5
91%
0.178
Spain
15,050
21,400.86 0.927
45.0
58.8
42.4
94%
0.183
Australia
20,490
25,422.80 0.947
52.2
57.2
45.6
95%
0.048
Canada
22,090
27,796.65
52.9
61.6
45.7
93%
0.005
New Zealand
13,560
19,574.64 0.925
52.2
57.2
45.2
94%
-0.038
Sweden
26,950
26,122.89 0.949
50.6
55.3
45.4
94%
0.032
71
Figure 19. Distribution of students according to number of books at home. PISA 2000
60
50
40
Argentina
Hungary
30
20
10
0
none
1-10
11-50
51-100
101-250
251-500
> 500
Figure 20. Proportion of students with more than 50 books at home
according to their SES category. PISA 2000
120%
100%
80%
A rgentina
60%
Hungary
40%
20%
0%
SES Index
72
Figure 21. Index of Home Educational Resources according to SES category. PISA 2000
100%
Index of Home Educational Resources
50%
0%
-50%
-100%
Argentina
Hungary
-150%
88
79
83
77
71
69
67
65
61
59
57
55
53
51
49
44
46
42
40
36
38
34
32
30
28
26
24
22
20
16
-200%
SES Index
In fact Latin America as a regions seems to show a lower demand, since the same pattern
is observed for Mexico, Brazil, Peru and Chile, the spend less in books than similar
income level countries.
Figure 22. Income level . PISA 2000
100%
Latvia
Czech Rep
Hungary
95%
Russia
Korea
Israel
Bulgaria
90%
% of Households in PISA with more than 50 books
Iceland
Spain
New Zealand
Australia
Sweden
Canada
Austria
Finland
Germany
UK
Ireland
Italy
Greece
85%
Poland
Norway
Switzerland
Denmark
France
Luxembourg
Japan
USA
Netherlands
Belgium
Portugal
Romania
80%
Macedonia FYR
75%
Thailand
70%
Chile
Argentina
Hong Kong
Mexico
65%
60%
Indonesia
Peru
Brazil
Albania
55%
50%
-
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
GDP per capita
PIRLS shows a similar pattern. In Argentina 66% of the students have less than 25 child
books at home, compared to 23% in Hungary or 18% in The Czech Republic. Only 37%
73
of the students have the minimum level of educational resources at home, compared to
76% in Hungary, or 78% in The Czech Republic.
Figure 23. Proportion of Students with more than 25 Child Books at home . PIRLS 2001
100
87.33
90
81.76
77.17
76.6
80
70
57.09
60
50
40
33.72
29.52
30
20
10
0
Argent ina
Colombia
Czech Republic
Hungary
It aly
England
Sweden
Figure 24. Proportion of Students with minimum educational resources at home . PIRLS 2001
100
92.9
88.0
90
77.7
80
75.6
72.7
70
60
50
40
37.0
30
21.9
20
10
0
Argentina
Colombia
Czech
Republic
Hungary
Italy
England
Sweden
Argentina shows also disadvantages in terms of early home literacy activities (EHLA), as
well as Colombia. For instance, whereas 61% of the students in Hungary show high early
home literacy activities, in Argentina just 50%, but the main difference is in those families
with low EHLA (16% in Argentina compared to 7% in Hungary). This variable shows a
high correlation with the student score, 12% of the student level variation is explained by
this index. The difference between Argentina and Hungary are not so large for high
educated parents but for low educated parents. In Hungary only 14% of parents with less
than lower secondary show low EHLA, whereas in Argentina the ratio for this group of
parents is 23%.
74
Table 28. Index of Early Home Literacy Activities. PIRLS 2001
Argentina
Colombia
Czech Republic
Hungary
Italy
England
Sweden
High
49.9
40.1
51.6
61.3
62.4
83.0
41.4
Medium
34.6
38.9
40.9
31.9
29.6
14.5
44.7
Low
15.5
21.0
7.6
6.8
8.0
2.5
13.9
Table 29. Index of Early Home Literacy Activities
By parents educational level. PIRLS 2001
Argentina
High
Medium
Low
finished univ. or hig
70.8
22.47
6.73
finished upper second
61.6
28.48
9.92
finished lower second
44.49
39.18
16.33
some prim./lower sec.
38.92
38.58
22.51
Hungary
High
Medium
Low
finished univ. or hig
71.41
23.93
4.66
finished upper second
57.72
34.15
8.13
finished lower second
48.72
39.37
11.92
some prim./lower sec.
48.25
38.07
13.68
The importance that parents give to reading is also remarkably lower in Argentina. 74% of
the parents in Hungary have a high attitude compare to 33% in Argentina, a variable that
for the entire sample explains almost 7% of the student variation in test score.
Table 30. Importance given by family to reading. PIRLS 2001
Argentina
Colombia
Czech Republic
Hungary
Italy
England
Sweden
High
32.9
40.9
64.2
74.1
56.0
68.6
71.3
Medium
62.4
52.5
31.9
23.4
37.3
25.7
23.8
Low
4.8
6.7
3.9
2.5
6.7
5.7
4.9
In terms of school climate or resources, Argentina has a higher proportion of students in
schools with high violence, low educational resources, less computers per students and
less libraries (14% of the students in Argentina attend a school without a library compared
to 5% in Hungary or 8% in The Czech Republic).
75
Figure 25. Index of school violence
57.3%
45.9%
42.8%
38.9%
30.1%
23.5%
Argentina
Colombia
Czech
Republic
Hungary
Italy
Sweden
Table 31. Availability of school resources. PIRLS 2001
High Medium
Low
Argentina
35.7
48.6
15.7
Colombia
22.8
48.2
28.9
Czech Republic
67.2
29.9
2.9
Hungary
62.8
27.8
9.4
Italy
36.2
57.3
6.6
Sweden
76.7
17.5
5.8
What this simple benchmarking shows is that Argentina has several differences compared
to the other countries what might be explaining the low performance. A first important
difference is resources, in terms of instructional time, additional resources for low
performers, materials and libraries. A second difference is the teaching approach the
school and teachers have in terms of grouping students in the class, in terms of allocating
the instructional time, and in terms of differentiating the educational process to have low
performers more involved, or to make an effort for the low performer students being able
to catch up the rest of the class. Perhaps this lack of differentiation is the explanation for
the very high repetition rate and drop out rate. There seems to be also differences in terms
76
of what parents contribute to the educational process, again Argentina showing a more
disadvantageous situation.
77
V. Quality of Education with-in Argentina
The average quality of education in Argentina shows some regional disparity that we will
explore in more detail in this section.
Argentina is organized in 24 jurisdictions or provinces, and the primary and secondary
school are administrated at this government level, therefore it seems reasonable to expect
some regional variation, particularly since there are large differences in terms of economic
development across regions and socioeconomic characteristics.
To compare the achievement at province level we use three different measures:
a) Mincerian Quality: measure as the returns to education for migrants to Great
Buenos Aires region who were educated in their province
b) Achievement: average test score in ONE Language Test
c) Test Quality: obtained as the coefficient for a provincial dummy in a regression at
individual level of test score in language against socioeconomic characteristic of
the students and the dummy.
To compute measures b and c we consider only urban basic education 6th grade schools
(i.e. we eliminate from the sample rural and adult schools). We standardized the test score
to have a national mean for this restricted sample of 500 and a standard deviation of 100.
The details of this analysis is explained in Annex D, here we summarize the main results.
Analyzing test scores at individual level we find:
a) Students with more educated parents have higher scores in both tests for all the
parents’ educational levels. A student with a mother who has not finished primary
school obtains on average a score of 467 point in Language if he/she is attending a
public school compared to 514 in a private school. The difference between average
score according to the mother’s education is more notorious in private schools than
public schools. A student with a mother with university degree has on average a higher
78
score than a student with a mother not finishing primary school of 55.5% of one
standard deviation at the sample (55.5% SD) in a private school, whereas in a public
school this difference is 36.7% SD. The gains in terms of test score of having a more
educated mother is more important in the math test than in language (63.2% SD and
41.9% SD for private and public schools).
b) The regional difference in test score is relatively large. The top performer has an
average test of 77.5% SD or 87.1% SD higher than the worse performer in
Mathematics and Language respectively. The difference is much lower for private
schools than for public schools. Private schools who less dispersion in Language,
whereas public schools less dispersion in Mathematics.
d) If we merge CABA and GBA in one region, the average test score is 497.6 and 505
in math and language respectively, what reduces enormously the regional
differences. Now the top performer (which is Santa Fe in math and San Luis in
Language) is only 60% SD and 47% SD over the worse performer (for math and
language scores respectively).
e) In terms of variance decomposition, 38% of the total variance in test score at
individual level is between-schools for math test score, and 36% for Language test
score, and the between-classes variation is slightly higher 43% and 40% for math
and language respectively.
Table 32. Between-classes variation
Mean St. Dev
Mathematics
overall
Max
100.0 215.1
699.8
between
66.2 233.3
698.4
within
76.6 134.0
829.2
Language
overall
500
Min
500
N= 484,008
n= 21,472
T-bar= 23
100.0 179.1
692.8
N= 504,469
between
63.7 237.3
692.8
n= 21,409
within
78.4
809.8
94.5
T-bar= 24
79
Table 33. Between-schools variation
Mean St. Dev
Mathematics
overall
500
between
within
Language
overall
500
between
within
500
Min
Max
100.0
215.1
699.8
N
= 484,008.0
62.0
247.4
691.7
n
=
82.0
121.5
805.3
100.0
179.1
692.8
61.0
254.0
687.1
82.8
89.3
795.1
100.0
215.1
699.8
T-bar =
9,857.0
49.1
N
= 504,469.0
n
=
T-bar =
N
9,847.0
51.2
= 484,008.0
If we merge GBA and CABA as one observation, the between-regions variation
explains only 3.3% and 2.6%, for Math and Language tests respectively, of the
individual test score variation. Not merging these two jurisdictions gives us slightly
higher between-regions variation (4% and 3.8%, respectively) for the reasons already
mentioned. In other words, if we think that most of the regional differences in school
inputs are between provinces (responsible for providing the local public good), then
eliminating the differences between provinces it will only improve achievement at the
student level in just 3 to 4%.
f) To analyze the inequality within each province we compute the coefficient of
variation at province level (ratio of standard deviation of individual test score at
province level to average score at province level). In Math, the coefficient of
variation goes from 17.1% in CABA to 21.4% in Santiago del Estero; for
Language, from 15.8% in CABA to 22.6% in Santiago del Estero.
g) We find a strong positive relationship between the average test score at province
level and the province development level (measure as the GDP per capita in ppp
for the year 2000). This relationship holds for both public and private schools.
h) When we analyze inequality, we find that the richer is a province, the lower the
inequality.
80
Figure 26. Relationship between Average Test Score and GDP per capita (in ppp) at province level)
560
550
Average Literature Test Score at province level
540
530
520
510
500
490
480
470
460
-
5.000,0
10.000,0
15.000,0
20.000,0
25.000,0
GDP per cápita in ppp (year 2000)
Figure 27. Relationship between the Coefficient of Variation in Test Score
and GDP per capita (in ppp) at province level)
Coefficient of Variation, Literature Test Score at province level
0,24
0,22
0,2
0,18
0,16
0,14
0,12
0,1
-
5.000,0
10.000,0
15.000,0
20.000,0
25.000,0
GDP per cápita in ppp (year 2000)
We estimate “test quality” as an OLS regression at student level of standardized test (TS)
as:
TSi = β 0 + β1 X i + β 2 Di + ε i
where D is a vector of dummy variables for the regions, X are socioeconomic
characteristics of each student (parents’ education, index of wealth, index of SES, age,
gender, family size, books at home, and whether he/she has repeated a grade before 6th
grade), and ε is an independent but not identically distributed (i.n.i.d) error term (where
81
White robust standard error estimation was used). We will refer to the coefficients in D,
added to the constant, as Test Quality. The results are:
a) As expected we found: a negative coefficient for age, family size, and whether
he/she has repeated, and we found a positive sign for parents’ education, the
variables related to wealth, and the number of rooms per family member. In terms
of gender, we find that girls perform better in Language and worse in Math than
the boys, another usual result in the literature. Regarding the region dummies,
which are the objective of study here, the following table shows the results for each
concept, where provinces are ranked from top performer to worst performer.
b) The ranking of provinces is affected whether we consider Test Quality or Average
Test Score as the ranking variable, or whether we consider Math or Language Test,
but the differences in general are not so large. Note that once the differences in
socioeconomic level are removed, the disparity across provinces reduces, what
shows that the quality of education is more equally distributed that income (and
other socioeconomic characteristics). Nevertheless, even after controlling for
students family characteristics, there is strong relationship between the average test
quality at province level and GDP per capital (in ppp) at province level as the
following figure shows.
Figure 28. Relationship between Test Quality and GDP per capita (in ppp) at province level)
530,00
Average Test Quality (Literature) at Province Level
520,00
510,00
500,00
490,00
480,00
470,00
460,00
-
5.000,0
10.000,0
15.000,0
20.000,0
25.000,0
GDP per cápita in ppp (year 2000)
82
c) After controlling for student’s family characteristics the within province inequality
reduces substantially, as the coefficient of variation computed for the dummy
coefficient collapses from a range between 15.8% to 22.6% in Literature, for instance,
to 0.6% to 1.1%. What is even more remarkable is that the positive relationship
between income level and test score inequality disappears once we consider test
quality. All the provinces has a relatively similar inequality, around a coefficient of
variation of 0.9% .
a) After controlling for student’s socioeconomic characteristics the difference
between the best province and the worst province reduces 33.4%. In test scores,
the best province has an average score which is 85.6% SD at individual level
score, which reduces to 57% when test quality is considered.
b) The differences between provinces are still large, and the average test quality is
positive correlated with the income level of the province.
c) But the inequality within students in each province reduces significantly. Inside
each province, students receive a relatively homogenous quality, and most of
the variation is between provinces.
Figure 29. Inequality and GDP per capita (in ppp) at province level)
2,0%
Coefficient of Variation in Literature Scores at province level
1,8%
1,6%
1,4%
1,2%
1,0%
0,8%
0,6%
0,4%
0,2%
0,0%
-
5.000,0
10.000,0
15.000,0
20.000,0
25.000,0
GDP per cápita in ppp (year 2000)
Using the EPH household survey we estimate excess returns to education according to the
province where the worker studied as:
83
ln( wi ) = β 0 + β1 X i + β 2YS i + β 3 DiYS i + ε i
where w is the worker wage, X are worker characteristics (except education), YS is years
of schooling, and D is a vector of dummy variable that indicate in which province the
migrant was educated. We excluded from our analysis immigrants from other countries,
and we restrict the sample to those workers that have complete secondary education as the
maximum level. To have enough number of migrants, we pool together the EPH survey
for 8 different times periods during 2000 and 2001, two years that were highly stable (zero
inflation, the economy did not grow, and there were not changes in wages). We estimate
this equation following Heckman two-steps procedure to control for the participation bias.
β3 should be interpreted as the excess return that a worker educated in other than Buenos
Aires province obtains compared to a non-migrant worker. Since we are interested in
ranking provinces according to their quality of education, we focus on β3, which vary by
province of origin. We call the estimated coefficient as Mincerain Quality of education, to
make explicitly that this concept of quality is different than the concept of quality captures
in the test score. A priori we should not expect both measures of quality to give us the
same result or ranking, since they capture different aspects. Test Quality measures the
average achievement of a student in a Language or Math score, after controlling for
individual characteristics, what takes into account the curricula contents in 6th grade. In
some sense, the Test Quality measures one dimension of the multidimensional product
space, Language (or Math). On the other hand the Mincerian Quality measures the
dimensions that the labor market considers relevant, what a priori might not be correlated
with the Language curricula in 6th grade.
The results are:
a) The Mincerian Quality and Test Quality measures are highly correlated,
particularly when Language is considered (0.6 for Language, 0.5 for Math). The
ranking although, shows some changes depending on which measure is considered.
If we classify provinces in high, medium and low quality, 10 out of 18 has the
same classification, and only in 2 cases a province change from low to high quality
depending on the measure.
84
b) The heterogeneity in quality across provinces is moderate. The between-provinces
variation accounts only for 3% of total test score variation. The difference between
the worst and best performer according the Test Quality is only 48% of one
standard deviation at individual level or 4 times the standard deviation between
provincial averages in Test Quality.
Table 34. Ranking of Provinces according to different measures of Quality of Education
Test Quality (Language)
Catamarca
68.73659
Corrientes
70.80002
Chubut
71.05939
Chaco
71.58761
Sgo.del Estero
71.92281
Jujuy
72.45296
Tucumán
73.08728
Misiones
73.20003
San Juan
73.37159
Formosa
73.76364
La Pampa
73.95602
Entre Ríos
74.40442
Córdoba
74.42363
Río Negro-Bs. As
74.51502
Santa Fe
75.46587
Mendoza
75.65468
San Luis
76.0322
Salta
77.18568
Difference between the
best and worst performer
8.449
Average test score (Language)
Catamarca
54.71535
Corrientes
54.90631
Sgo.del Estero
54.97566
Chaco
55.38323
Jujuy
57.12993
Formosa
57.38063
Misiones
58.20464
Tucumán
58.76175
Chubut
60.12679
San Juan
60.89456
Entre Ríos
62.34269
Salta
62.40857
Río Negro-Bs. As
63.00591
Córdoba
63.15547
La Pampa
63.18432
Mendoza
63.4216
Santa Fe
63.6759
San Luis
64.12317
9.408
Mincerian Quality /1
Catamarca
0.01154
Jujuy
0.039132
Chaco
0.0454352
Chubut
0.059195
San Luis
0.0823863
Córdoba
0.0901121
Misiones
0.0911275
La Pampa
0.0924342
Santa Fe
0.0942947
Corrientes
0.0973847
Sgo.del Estero
0.098145
San Juan
0.0990863
Entre Ríos
0.1007615
Río Negro-Bs. As
0.1025249
Salta
0.1103325
Mendoza
0.1128027
Formosa
0.120258
Tucumán
0.1366031
0.125
Notes: 1/ Buenos Aires was not included in this ranking because is the province which is used to measure the returns for migrants (therefore
we do not have a measure for GBA , the city of Buenos Aires (CABA) and the rest of Buenos Aires.
The provinces of La Rioja, Santa Cruz and Tierra del Fuego were not considered in the Mincerian ranking because of the low number of
migrants to Buenos Aires.
Next we analyze the disparity across provinces in both measures and public expenditure in
education. We find that disparity in terms of quality of education and expenditure in
education is much smaller than the GDP disparity. For instance, the richest province has
8.4 times the GDP per capita of the poorest province, what is large compared to developed
countries (Australia 1.5 times, USA 2 times, Canada 1.79 times, Spain 2.1 times), and
Brazil (6 times), but not as large as other developing countries such as Russia (12 times) or
China (17 times).
The public expenditure in education shows also a lower heterogeneity than the GDP. For
instance, the expenditure in education per capita in the province that spends the most is 4.5
times the expenditure of the province that spends less. A similar difference is found when
we analyze the public expenditure per public school student, which is 4.3 times between
the top and the bottom.
85
Next, we analyze the relationship between the quality of education and GDP per capita (or
GDP per capita growth). The natural question is to what extend the disparity in quality is
associated with the disparity in development. A simple analysis between the quality of
education and GDP per capita at province level shows a positive and moderate correlation.
The coefficient of correlation is 15.4% or 35.7% when we compared the GDP per capita
with the Mincerian Quality or Test Quality, respectively. The correlation is much higher
for the Average Test Score: 64.6%, a coefficient very similar to the cross country
correlation between GNI per capita and average test score (66.8%).
Table 35. Expenditure and Quality across provinces
Public Expenditure in Education as:
Per student
Per Teacher
A share
in basic
in Basic
of GDP
school
School
(US$)
(US$)
4.5%
1,042
22,001
8.9%
1,358
20,054
4.0%
1,076
24,646
8.4%
840
17,590
8.9%
968
17,751
4.5%
1,327
21,171
6.2%
1,157
20,609
11.9%
1,019
19,569
9.1%
1,014
15,067
5.8%
1,778
25,160
10.9%
1,614
24,429
4.7%
1,102
21,877
6.5%
796
15,758
5.8%
2,027
28,671
5.5%
1,283
16,822
6.0%
678
14,399
7.8%
1,216
22,749
4.5%
1,365
18,478
7.1%
2,889
34,350
4.3%
1,076
19,993
10.1%
1,024
20,019
5.5%
852
13,633
4.2%
2,143
31,881
1.6%
1,706
23,831
A share of Total
Public
Expenditure
Buenos Aires
Catamarca
Cordoba
Corrientes
Chaco
Chubut
Entre Ríos
Formosa
Jujuy
La Pampa
La Rioja
Mendoza
Misiones
Neuquén
Río Negro
Salta
San Juan
San Luis
Santa Cruz
Santa Fe
S. del Estero
Tucuman
T. del Fuego
C.B.A.
35.6%
26.9%
33.4%
33.8%
26.7%
26.8%
28.1%
23.7%
28.4%
23.9%
21.0%
29.4%
25.9%
26.3%
27.8%
23.4%
23.7%
28.2%
21.7%
33.1%
31.5%
26.7%
19.4%
33.3%
TOTAL
Coef. of Variation
30.8%
0.158
Coef. of correlation
with Mean Quality
22.3%
4.2%
0.385
-39.7%
1,117
0.388
3.2%
21,012
0.244
3.5%
Test Quality
GPD
per
capita
Mean
St.
Dev.
Coef. of
Variation
6,161
4,636
6,693
3,107
3,265
8,392
4,994
2,835
3,557
7,424
4,593
6,033
3,631
10,514
6,847
3,458
4,010
8,278
12,013
6,490
2,828
3,745
17,046
23,639
72.4
68.74
74.42
70.80
71.59
71.06
74.40
73.76
72.45
73.96
68.74
75.65
73.20
0.720
0.782
0.725
0.760
0.748
0.760
0.738
0.779
0.737
0.776
0.794
0.727
0.751
0.99%
1.14%
0.97%
1.07%
1.05%
1.07%
0.99%
1.06%
1.02%
1.05%
1.15%
0.96%
1.03%
74.52
77.19
72.39
71.92
75.47
73.37
76.03
73.09
71.45
79.02
0.754
0.735
0.793
0.762
0.725
0.757
0.768
0.730
0.841
0.728
1.01%
0.95%
1.09%
1.06%
0.96%
1.03%
1.01%
1.00%
1.18%
0.92%
7,093
0.718
73.3
0.034
35.7%
100.0%
The correlation between real per capita GDP growth (between 1993 and 2000) is much
more correlated with the Test Quality (a coefficient of correlation of 48.2%) than GDP. Is
the quality of education restricting economic growth at province level or the differences
are due to more spending on education in those provinces that grew faster? The almost
null correlation between Mincerian Quality and growth (a coefficient of correlation of 86
9%) and the fact that workers can migrate relatively easily seems to support the idea that
causality goes from a healthy regional economy to quality of education. It is important to
take into account that in 1993 school administration was decentralized from the national
level to the province level, what might have increase the disparity in resources and other
relevant variables. To analyze this we compare the ratio of public expenditure in education
to GDP in 1994 and 2000, finding that the heterogeneity fell (the between provinces
standard deviation fell from 0.08 to 0.03, and the coefficient of variation from 1.29 to
0.39). In addition, there is some evidence that the decentralization increased the quality
(see Galiani and Schargrodsky (2002)).
In terms of public expenditure in education and quality of education, we find a negative
relationship between expenditure in education as a ratio of GDP and quality, and a positive
relationship between expenditure in education as a ratio of total expending and quality.
The explanation for this result seems to be the role of private schools, which is different
across region. Unfortunately we do not have the expenditure in private schools to have a
better measure of the amount of resources allocated to education by provinces.
87
VI. A Hierarchical Lineal Model with PISA
In this section we use an HLM model to compare the achievement of Argentinean
students in PISA with other countries and to explore what are the school factors behind
those differences. We base our analysis in Language test, since this was the objective of
PISA in 2000 (and the test that every student had to solve, whereas for math and science
only a random sub-sample of the students had to solve it).
VI.1. Variance Decomposition
A first issue is what percentage of the total variance in test scores is related to school
characteristics and country characteristics, what eventually can be related to educational
policies and educational effort.
To obtain this variance decomposition we estimate a 3-level HL null model. For the
entire sample of 38 countries (Reading Literacy Test), 27% of the variance is at country
level, 30% at school level, and 43% within school variance. This means that a large part
of the test results is related exclusively to student characteristics (43%), but there are
some important differences across countries and schools what might be related to
different policies. Also note the importance of the between country variance (27%)
compared to the very low variation across provinces in Argentina (3%).
Separating the entire sample in OECD and non-OECD countries we find that the country
level variance is much more important in the second group of countries, and the within
variance is much smaller. This means that these countries are not so heterogeneous at
individual level but much more heterogeneous at country level, what can be interpreted
that the differences between these countries is mainly due to different policies rather than
different individual characteristics.
88
Table 36. Percentage of variance in student performance in reading, mathematical and scientific
literacy
Between
Between
Within
countries
schools
Reading literacy
OECD countries
8
35
57
Partner countries
28
35
37
All PISA countries
27
30
43
Mathematical literacy
OECD countries
16
31
54
Partner countries
36
27
38
All PISA countries
35
25
40
Scientific literacy
OECD countries
10
32
59
Partner countries
27
28
45
All PISA countries
26
27
47
Source: Own elaboration based on PISA (2005)
Next we analyze a 2-level HL null model at country level to see what percentage of the
within country variance is related to schools (between schools) and to student
characteristics (within schools). On average, for the entire sample 39% of the variance in
Reading Scores is between schools and 61% within schools. Argentina, although, has a
much higher between school variance (50%), and it ranks 13 out of 38 countries in terms
of higher between-school variance importance (see Figure). This relative high importance
of the between-schools variance is common in Latin America (Peru, Chile and Mexico
rank even higher than Argentina, and Brazil ranks 16), what is usually associated with
high inequality between schools (either because of student composition or because of
high heterogeneity in school quality). This result is not surprising since these countries
have a high level of income and educational inequality.
89
Figure 30. Between and within school variance decomposition with PISA
Between-school variance, as a share on total variance
Within school variance, as share on total variance
Hungary
Poland
Peru
Austria
Belgium
Germany
Bulgaria
Chile
Italy
Czech Republic
Mexico
Netherlands
Greece
Argentina
Hong Kong-China
Brazil
Israel
FYR Macedonia
Indonesia
Switzerland
Albania
SAM PLE AVG
Korea
Portugal
Russian Federation
Thailand
Latvia
United States
United Kingdom
Spain
Canada
Australia
Ireland
New Zealand
Denmark
Norway
Sweden
Iceland
Finland
0
10
20
30
40
50
60
70
80
90
100
Source: Own elaboration based on PISA (2005)
Based on the cluster analysis previously discussed, the countries comparable to Argentina
are: Chile, The Czech Republic, Poland and Hungary. Among this group, only Chile
ranks worse than Argentina, both in average test score and in Quality (after controlling by
student socioeconomic characteristics). Adding the spending per student in primary and
secondary level, Argentina is the country that spends the less. Poland, with the highest
spending, has 1.5 times more spending per student. The Czech Republic, on the other
hand, with a lower public expenditure than Argentina, and with a lower private sector
share, it is able to obtain better quality.
How these countries with similar educational effort and development level are able to
obtain a better performance? The answer seems to be in the between school variation.
The four countries have a much higher proportion of the total variance explained by
between school variance, what shows that the better performance is at the expense of
more inequality across schools. In some sense it seems Argentina is paying a price in
90
terms of aggregate quality for having a more egalitarian (or more equally mediocre)
educational system.
Table 37. Percentage of variance in student performance in reading, mathematical and scientific
Poland
Czech
Republic
5,790
15,163
97.2
Hungary
Argentina
Chile
GNI per capita (2000)
4,570
GDP per capita ppp (2000)
10,061
Net Enrollment ratio primary
96.6
Net Enrollment ratio
90.2
secondary
repetition rate
0.8
school life expectancy
14.7
Exp/GDP
4.8
Current spending per student (% of p.c.GDP)
Primary level
22.5
Secondary level
19.9
Tertiary level
17.3
Private Enrollment Share
Primary
0.8
Secondary
4.5
4,600
12,264
87.8
85.3
7,470
12,148
99.3
79.1
4,840
9,188
89.6
.
2.1
14.2
4.9
5.9
15.6
4.6
2
12.9
3.9
1.1
19
20.2
34.2
12.3
15.7
17.7
14.4
14.8
19.4
11.7
21.6
31
5.1
7.2
20
12.3
45.5
49.7
0.9
5.7
Average Score in Math
St dev of Avg. Score in Math
Quality in Math
464.3
100.3
224.2
486.1
96.8
219.3
404.0
113.4
151.1
398.1
94.8
139.6
499.1
98.2
233.6
Proportion of between-school
variance
62.7
65.5
49.8
55.8
54.0
4
Source: Own elaboration based on PISA (2005)
Next we include student and school characteristics in our two-levels HL model (as
covariates affecting only the intercept), we can see that a large part of the between-school
variance in Argentina is due to a compositional effect, since 55% of the between-school
variance is explained by the average student characteristic in the school. This is close to
the world average and OECD average, but since the between-school variance in
Argentina is relatively more important, it means that 27.5% of the total variance in test
scores is explained just by different school compositions.
91
Table 38. Variance Decomposition. Reading Literacy Achievement
As % of total variance
School Peer
Characteristics
Between-school variance
School
School
Climate,
Context
Policies and
Resources
Unexplained
Within
School
Variance
High Income Countries
Australia
Austria
Belgium
Canada
Denmark
Finland
Germany
Greece
Hong Kong-China
Iceland
Ireland
Israel
Italy
Netherlands
New Zealand
Norway
Spain
Sweden
Switzerland
United Kingdom
Average HI
9.7
33.7
38.4
9.3
8.9
1.1
21.4
27.6
10.0
1.6
7.2
13.1
15.8
37.4
8.4
3.3
9.1
5.3
15.6
9.6
14.3
5.1
11.4
11.4
2.3
1.1
0.2
26.1
5.5
13.4
0.2
5.9
13.6
15.8
8.8
3.9
0.6
3.8
1.0
8.6
10.2
7.4
1.4
4.8
6.0
1.2
1.3
1.2
4.2
3.0
15.3
2.0
0.4
1.8
7.6
1.6
1.1
1.1
1.1
0.7
7.4
3.2
3.3
4.1
10.2
4.2
7.9
4.4
5.2
7.1
14.1
9.1
4.3
4.5
17.2
14.7
3.6
2.7
4.1
7.2
1.9
9.4
6.1
7.1
79.7
39.9
40.0
79.3
84.3
92.3
40.6
49.8
52.2
91.8
82.0
54.8
45.5
48.0
83.9
90.9
78.8
91.2
59.0
71.0
67.8
Upper-Middle Income Countries
Argentina
Brazil
Chile
Czech Republic
Hungary
Korea
Mexico
Poland
Average UMI
27.4
25.7
36.3
32.4
25.6
22.0
28.3
31.4
28.6
13.0
9.4
11.2
13.0
27.5
3.8
14.9
10.0
12.8
4.0
3.3
3.3
2.2
3.3
5.7
2.7
9.4
4.2
6.0
8.9
5.0
5.9
9.2
6.4
7.5
11.9
7.6
50.2
53.2
44.2
46.0
34.5
62.1
46.6
37.3
46.8
Low-Middle Income Countries
Albania
Bulgaria
FYR Macedonia
Latvia
Peru
Russian Federation
Thailand
Average LMI
15.2
12.7
19.6
11.5
32.8
5.2
12.0
15.6
17.2
32.3
12.0
6.0
15.2
10.7
6.3
14.3
1.6
5.2
7.6
2.7
2.4
4.4
1.3
3.6
6.0
7.5
5.3
9.7
10.3
16.6
12.0
9.6
60.0
42.3
55.5
69.8
39.3
63.1
68.3
56.9
Low-Income Country
Indonesia
15.9
7.5
4.9
15.5
55.8
17.97
13.67
7.81
60.94
27.4
25.7
36.3
28.3
32.8
13.0
9.4
11.2
14.9
15.2
4.0
3.3
3.3
2.7
2.4
6.0
8.9
5.0
7.5
10.3
Total Sample
Latin America
Argentina
Brazil
Chile
Mexico
Peru
Source: Own elaboration based on PISA (2005)
50.2
53.2
44.2
46.6
39.3
The effect of the peer characteristics on the achievement of the student has been related
in the literature to the existence of externalities in production and sorting. First, better
92
peers can have a positive impact on the individual student achievement (what is known in
the literature as peer-group effect). Second, it is possible that a better group of students
attracts better teachers (i.e. there is positive assortative matching between teachers and
schools). Nevertheless, part of variation assigned to the peer student characteristics might
be due to the interaction of student characteristics and school (unobservable)
characteristics. Since the allocation of students among schools is not random,
unobservable characteristics of the school that interact with the peer characteristics are
going to be captured, in part, in this variation. We cannot estimate with this exercise the
peer-group effect on achievement, but later in this paper we will use a particular
characteristic of the Argentine system which allows us to identify this effect, and separate
the peer student characteristic variation in peer-group effect and the interaction of peergroup effect and unobservable school characteristics. In terms of policy variables at the
school level, the proportion of the total variation explained by climate, policies and
school resources is just 4% of the total student variation, which is small, but larger than
the group of comparators (except for Poland) what shows that Argentina has some room
to improve its quality of education with policies at the school level.
In terms of the comparators, for all the countries the proportion of the variance explained
by the peer student characteristics and school context is much larger than in Argentina,
what confirms that sorting across schools is higher in these countries, what might be
related with a more differentiation (or an unequal educational system).
Table 39. Decomposition of the between school variation in explained and unexplained factors
Poland
Hungary
Argentina
Chile
Average Score
St dev of Avg. Score
Quality
464.3
100.3
224.2
486.1
96.8
219.3
404.0
113.4
151.1
398.1
94.8
139.6
Czech
Republic
499.1
98.2
233.6
Proportion of between-school
variance
Explained by:
Peer Student Characteristics
School Context
School Climate, Policies and
Resources
Unexplained
62.7
65.5
49.8
55.8
54.0
31.4
10.0
25.6
27.5
27.4
13.0
36.3
11.2
32.4
13.0
9.4
11.9
3.3
9.2
4.0
6.0
3.3
5.0
2.2
5.9
Source: Own elaboration based on PISA (2005)
93
VI.2. School Factors Related to Quality of Education
In Table 40 we show the estimated coefficients (shrinkage estimators) for the variables
included in our HLM model. Most of the coefficients have the expected sign. In terms of
student characteristics, SES is positively related to test score, girls perform significantly
better than boys, and if the student is older or immigrant the score is lower.
Table 40. Estimated coefficients for school level in a 3-levels HLM model
Student characteristics
Grade level (deviation from country mode)
Age
In vocational program (ISCED 2B, 2C, 3B or 3C)
Parents’ occupational status (HISEI)
Female student
Immigrant
School context
School average parents’ occupational status (HISEI)
School type (reference category = public schools)
Independent private schools
Government-dependent private schools
School location (reference category = Town 15.000 – 100.000 inhabitants)
Village, less than 3.000 inhabitants
Small town, 3.000 – 15.000 inhabitants
City, 100.000 – 1.000.000 inhabitants
Large city, more than 1.000.000 inhabitants
School climate
Index of disciplinary climate
Index of teacher support
Index of achievement press
Index of teacher-student relations
Index of students’ sense of belonging at school
Index of principals’ perceptions of teacher-related factors affecting school climate
Index of principals’ perceptions of student-related factors affecting school climate
Index of principals’ perceptions of teachers’ morale and commitment
School policies
Instructional time
Index of monitoring of student progress
Index of school self-evaluation
Student’s performance is considered for school admission
Study program for 15-year-olds is based on students’ academic record
Study program for 15-year-olds is based on students’ placement exams
Transfer of low achievers to another school:
likely
very likely
Performance information is communicated to parents
Performance information is communicated to school principal
Performance information is communicated to local education authorities
Index of school autonomy
Index of teacher autonomy
School resources
School size
Index of the quality of schools’ physical infrastructure
Index of the quality of schools’ educational resources
Proportion of computers available to 15-year-olds
% of teachers with an ISCED 5A qualification in the language of assessment
Index of teacher shortage
Student-teaching staff ratio
Professional development
Source: Own elaboration based on PISA (2005)
Regression
coefficient
Robust
Standard
Deviation
25.77
-1.86
-18.72
13.31
22.4
-22.06
0.22
0.16
0.76
0.17
0.32
0.81
24.6
0.55
-6.88
5.32
2.16
1.19
0.28
0.91
-0.94
-3.39
1.57
1.23
1.22
1.86
7.64
-3.08
0.3
-2.18
7.91
-4.21
9
1.9
0.51
0.61
0.52
0.59
0.52
0.58
0.57
0.45
0.95
1.1
0.6
2.83
-0.15
0.75
0.81
0.71
0.72
0.56
0.59
0.54
5.79
11.98
0.92
-0.4
-0.22
-2.03
0.37
1.2
1.85
0.6
0.57
0.45
0.56
0.43
1.87
-1.53
2.73
-0.3
3.34
0.6
0.62
-0.59
0.57
0.49
0.53
0.46
0.56
0.47
0.67
0.44
94
Based on the international evidence, in which of these variables Argentina is underperforming?
Table 41 analyses the situation of Argentina in those factors related to School Climate
that have a statistically significant impact on quality of education according to the HLM
estimation. We separate the factors in those that have a positive impact and those that
have a negative impact.
In terms of the positive impact factors, Argentina has an average index for both variables
above the world average, what suggests Argentina should have a better than the average
score given these characteristics. Among the countries selected as benchmark Argentina
has also a relative good position, ranking first in terms of Disciplinary Climate and
Second (right after Chile) in Students’ sense of belonging at school.
Table 41. School Climate Factors
POSITIVE IMPACT
Index of disciplinary climate
World
Argentina
Chile
Czech Republic
Hungary
Poland
Mean St. Dev
-0.022 0.995
0.368 0.909
0.325 0.692
-0.143 1.020
-0.230 0.987
0.060 0.805
Index of students’ sense of belonging at school
Mean St. Dev
World
-0.057 0.981
Argentina
0.183 1.067
Chile
0.196 1.096
Czech Republic
-0.290 0.775
Hungary
0.140 0.972
Poland
-0.198 0.988
NEGATIVE IMPACT
Index of teacher support
World
Argentina
Chile
Czech Republic
Hungary
Poland
Mean St. Dev
0.093 0.995
0.205 1.079
0.303 0.976
-0.499 0.798
0.054 0.882
0.300 0.885
Source: Own elaboration based on PISA (2005)
95
In those factors that have a negative impact (teacher support), Argentinean average is
above the world’s average, what suggests its average score should be below the average,
but in line with the average for the set of comparators.
In terms of school resources, Argentina has a qualification of teachers below the world
average and well below the set of comparators, what suggests that this is a problem in the
country. The same is not true for educational resources, what shows that Argentina is in a
relatively good position in terms of this variable.
Table 42. School Resources Factors
% of teachers with an ISCED 5A qualification in the language
of assessment
Mean St. Dev
World
0.818 0.314
Argentina
0.281 0.361
Chile
0.647 0.444
Czech Republic
0.883 0.226
Hungary
0.984 0.109
Poland
0.783 0.253
Index of the quality of schools’ educational resources
Mean St. Dev
World
0.332 1.171
Argentina
0.521 1.115
Chile
0.289 1.049
Czech Republic
-0.216 1.004
Hungary
-0.503 0.911
Poland
1.287 1.118
VI.3. Oaxaca-Blinder Decomposition
In this section we decompose the explained score gap between Argentina and a set of
benchmark economies. The group of benchmark countries similar to Argentina in terms
of development and educational effort is: Chile, the Czech Republic, Mexico, and Poland.
The group of countries with similar culture includes: Spain and Italy. And the group of
high quality countries is: Finland, Great Britain and Sweden.
The results for the similar countries are shown in the next table.
96
Table 43. Oaxaca-Blinder decomposition for Similar development countries
Czech Republic
Mexico
Endow Returns Inter
Endow Returns Inter
Poland
Chile
Endow
Returns Inter
-135.2 0.6
Endow
Returns Inter
Student characteristics
Age
-1.2
Gender
-1.8
-188.8 0.8
-5.5
-2.5
-127.0 1.1
-1.2
0.6
-2.8
-10.3
1.6
-3.1
-1.0
1 if foreign
-0.1
# of siblings
5.0
0.2
-142.8
-0.1
0.2
-1.7
-9.3
0.9
0.1
0.1
-0.8
-0.1
-0.3
0.0
8.3
-3.1
-3.5
0.4
0.1
3.5
-0.1
0.0
-0.3
0.2
0.3
5.0
-1.3
0.8
8.3
-0.5
Position of the child within brothers and sisters
Oldest (excl.)
Middle
-0.7
-1.9
-0.6
0.3
1.3
-0.1
-0.5
0.5
0.1
0.2
2.1
-0.2
Youngest
1.7
-1.1
0.6
-1.3
1.6
0.7
1.0
1.9
-0.6
-0.2
0.6
0.0
Primary complete
4.4
32.4 -31.5
-0.8
6.6
1.2
4.5
4.6
-4.5
2.4
16.8
-8.8
Secondary incomplete
0.3
17.3 -12.8
-0.1
2.2
0.4
0.2
13.0
-6.3
-0.1
7.4
2.5
Secondary complete
0.0
0.0
38.8
0.0
0.0
2.4
0.0
0.0
21.3
0.0
0.0
10.3
Mother’s education
Primary incomplete (excl.)
Higher Ed. incomplete
3.7
30.4
22.1
-3.9
2.8
-2.1
4.1
22.0
17.6
0.3
15.2
1.0
Higher Ed. complete
-1.8
27.9
-8.9
-2.4
0.9
-0.4
-0.5
21.2
-1.8
-1.2
13.9
-3.0
0 if single parent household
0.4
-0.8
-0.1
-0.2
-2.8
0.1
0.5
-4.8
-0.6
-0.4
4.8
-0.4
-0.3
-4.1
0.3
2.7
-0.5
-0.3
0.6
5.2
0.8
4.0
8.8
8.8
7.8
-0.9
-0.9
16.2
-2.5
0.5
5.6
0.5
0.6
8.1
0.9
-49.8 -3.2
-4.8
29.2
6.1
0.5
86.3
-1.8
-6.6
94.7
27.2
2.7
-0.6
-4.8
18.4
0.5
25.9
15.1
-0.1
-4.5
School characteristics
School size (# of st.)
1 if Big town
-0.6
Hours of teaching
-1.5
Teacher qualification
17.5
0.4
21.0
# of computers per student
10.3
-8.3
-7.2
-0.9
0.8
-0.1
13.5
-11.7 -13.2
-3.6
0.5
-0.2
Index of Student behavior
-4.2
-8.5
-2.6
-11.0
8.5
6.6
-8.9
-29.8 -18.9
-7.9
-10.7
-6.0
Index of shortage of teachers
-5.0
-17.7 11.4
1.4
-6.5
-1.2
-3.7
-9.5
4.6
-2.1
-9.8
2.7
Index of building quality
Index of educational materials
availability
6.9
30.6 -18.8
-3.5
14.1
4.4
-1.9
33.7
5.8
2.8
12.7
-3.2
46.7 -14.4
-8.3
18.3
3.4
6.3
24.0
-3.4
4.7
24.8
-2.6
13.8
Intercept
Total
103.6
46.7
19.1
44.1
-8.4
-40.3
-0.7
-45.1
16.4
33.9
-13.8 25.0
-104.4
7.0
-58.2
25.2
From the student level characteristics we are particularly interested in the mother’s
education, which is a proxy of the socioeconomic level of the students. In terms of the
endowment effect Argentina has: a) lower endowment in terms of student’s SES
compared to the Czech Republic and Poland, b) higher than Mexico, and c) relatively
similar to Chile (with more students than Chile in the low level, and more students in the
high level). The policy related variable is the return effect, that shows how efficient is
each country compared to Argentina when resources are identical. A positive number
indicates that the benchmark country has a slope larger than Argentina, what is
interpreted that the student with that level of parent’s education is able to obtain better
97
scores than Argentina. Note that for all the countries and all the educational levels the
numbers are all positive. The differences in the slopes are higher in the extremes (lowest
and highest level of mother education). The Czech Republic and Poland show the
highest differences, followed by Chile. Mexico has slopes which are not significantly
different than Argentina. For instance, the Czech Republic is able to obtain on average
4.4 points more than Argentina because they have a lower proportion of students with
mothers’ education with primary complete, and 32.4 points because Poland has a higher
slope than Argentina for those students. For the highest level of mother’s education,
Argentina obtains 1.8 point more because it has a larger proportion of students in this
category than the Czech Republic, but the Czech Republic obtains 28 points more
because it has a higher slope. In all the cases the interaction term is high; if we add the
three effects, all the countries, except for Mexico, have a positive difference with
Argentina, what means that it does not matter what SES has the student, these countries
are able to obtain better results. The overall effect is larger for more educated mothers.
For instance, adding the three effects the Czech Republic has 5 points more than
Argentina for low educated mothers (primary complete and secondary incomplete), but
56 point more for mothers with higher education studies. Poland shows a similar pattern,
between 5 and 7 points more for low educated mothers, but 44 points more for mothers
with higher education. Chile is the only country with a relatively homogenous positive
overall difference (between 10 and 17 points for all the educational levels). These results
confirm our previous hypothesis, that the Czech Republic and Poland have better results
for all the students, it does not matter their SES, but the significant difference are in the
more advantageous students. Argentina, therefore, as a more mediocre overall
performance, but with less heterogeneity by SES than these countries. Compared to
Chile, both countries have the same pattern of inequality on performance by SES, but
Chile performs better than Argentina, mainly because it is able to achieve a higher
efficiency (slope).
In terms of school variables, hours of effective teaching per year seems to be the most
important variable to explain the differences, but more than the stock of hours the
difference is due to a slope or return effect. All the countries, except for the Czech
98
Republic, have a larger slope that Argentina, what means that they are able to obtain
better scores with the same number of hours (in other words, they are more efficient to
transform hours of teaching in scores). Poland has 86 points more than Argentina, and
Chile 95 points more just because they have a higher slope. This difference is very large,
is almost one standard deviation in the Argentinean score at the student level. If
Argentina is as efficient as Poland in terms of transforming hours of teaching in scores, it
would have the same average score. And if Argentina is as efficient as Chile in terms of
transforming hours of teaching in scores, it would have almost 100 points more than
Chile in its average score. Note that Argentina has an expected average score 26 points
larger than Chile, what means that even if Argentina is much less efficient than Chile in
transforming hours of teaching in scores, it is able to catch up in the final score with other
aspects that are favorable to Argentina. Unfortunately, we do not how Argentina is able
to match Chile, since Chile has better endowment and slopes for all the variables
included, and the favorable difference for Argentina is in the unobservable variables,
captured by the intercept, which is 104 points larger in Argentina.
A similar result is found for school infrastructure (an index variable representing the
quality of the building infrastructure). Argentina has not large difference in terms of
endowments, but the difference is in how that infrastructure is used (the slopes). The
Czech Republic and Poland obtain 31 and 34 points more than Argentina just because
they are able to transform the relatively similar school infrastructure in scores.
In terms of educational materials (an index measuring the availability of materials such as
books, boards, furniture, etc.), all the countries except for Mexico, have better
endowment, but again the difference is in how those educational materials are used. For
instance, the Czech Republic has 14 points more than Argentina because they have more
educational material on average, but it has 47 points more because they are more efficient
using that material.
For computers per student, another variable related to school resources, Argentina has a
positive difference (i.e. less endowment) compared to the Czech Republic and Poland,
99
but is able to use those poorer resources more efficiently (the return effect is negative),
perhaps the only positive aspect of Argentina.
In addition to school resources variables (educational material and computers per student)
the other variable that has a very important endowment effect is teacher qualification.
Argentina has on average less qualification (the endowment effect is positive) but it uses
the resource in a similar way than the other countries (the slopes are not significantly
different). Improving the teacher qualification Argentina can reduce the score gap with
the Czech Republic and Poland in approximately 18 points, which is approximately 31%
and 40%, respectively, of the explained differences between these countries and
Argentina. It is important to point out that the shortage of teachers in Argentina is not a
problem (the endowment effect is negative, what shows that on average it has a lower
shortage of teachers), and neither how Argentina deals with its shortage. This has a clear
policy implication: Argentina does not need more teachers, but it needs more highly
qualified teachers. Qualified teachers in Argentina do their job as well as qualified
teachers in other countries, but Argentina has less qualified teachers. Argentina needs to
make the teaching job more attractive and increase the level of qualification, probably
making teaching a university degree (instead of tertiary level instruction as it is now).
The teachers in Argentina that have a high qualification is because they choose to do
higher degree studies (masters or professional careers), not because the state is requiring
the teachers to have a high minimum level of qualification.
We think is important to pay attention to the student behavior variable, what represents
how bad the students behave according to the school director (and the standards these
directors demand). Argentina has on average a better student behavior (negative
endowment effect) and also has a better slope (return effect), what shows that Argentina
is able to better transform the relatively better behavior in more score points. It is not
easy to interpret this result, but if we think that the student behavior at the school is
related to how important the students consider their educational process, it shows that the
problem in Argentina is not the lack of interest of the students, on the contrary, it seems
100
the good behavior of the students and their effort is partially compensating the poor
quality of education that there are receiving at the school.
Finally, the intercept presents large values for all the countries, reflecting an important
degree of unobservable differences explaining the gaps, but the sign depends on which
country we choose to do the benchmarking, what suggests that these hidden unobservable
elements are probably different in each case.
The results of the Oaxaca-Blinder decomposition for the second benchmarking group, the
similar culture countries, are shown in the next table.
Table 44. Oaxaca-Blinder decomposition for Similar Culture countries
Endow
Student Characteristics
Age
Gender
1 if foreign
# of siblings
-1.2
-1.6
-0.5
6.2
Position of the child within brothers and sisters
Oldest (excl.)
-0.7
Middle
2.0
Youngest
Italy
Returns
Inter
Endow
Spain
Returns
Inter
-82.8
-3.4
0.0
-11.4
0.4
0.3
0.1
5.1
0.1
-2.6
-0.6
4.8
68.9
-5.8
0.0
4.9
0.0
0.9
0.1
-1.7
0.1
-2.2
-0.3
-1.1
1.8
0.0
1.3
1.1
7.2
0.6
-0.6
-0.1
0 if single parent household
-0.2
0.5
3.4
5.9
Mother’s education
Primary incomplete (excl.)
Primary complete
Secondary incomplete
Secondary complete
Higher Ed. incomplete
Higher Ed. complete
3.1
-0.4
0.0
1.0
-1.7
14.3
6.0
0.0
8.8
6.0
-9.8
6.0
4.3
1.8
-1.8
-0.7
0.0
0.0
-1.8
-0.8
20.2
11.0
0.0
14.2
12.8
3.2
-0.4
3.5
-5.0
-1.7
1.2
0.6
-3.1
0.4
-7.7
13.8
-0.4
-1.6
14.9
-6.7
-83.5
0.7
-21.4
-13.4
-3.3
5.3
4.5
-0.7
-11.4
0.8
-11.7
-15.4
0.2
0.8
1.5
0.0
-2.0
0.4
-10.0
3.9
-4.7
1.5
-0.3
0.3
64.8
-1.5
2.2
-11.1
-5.6
9.8
-0.1
0.0
5.8
-1.7
1.6
-3.6
3.4
-1.3
12.9
0.0
22.2
34.8
192.1
66.9
-10.1
0.0
-39.2
13.1
0.0
2.7
40.5
-177.5
57.2
-11.9
0.0
-9.2
School Characteristics
School Size
1 if big town
# of teaching hours per year
Teacher Qualification
Student Behavior
# of Computer per student
Index of shortage of teachers
Index of building quality
Index of educational materials
availability
Intercept
Total
101
In terms of student SES, Italy and Spain have a relatively similar endowment than
Argentina, but the important difference is in the efficiency (return effect) particularly for
the students with lowest SES.
In terms of school characteristics, the most important variable is the availability of
educational material, in terms of endowments but more importantly on how that
endowment is used. For the number of computers per student, we find a similar result as
before, Argentina has a poorer endowment, but it uses that endowment much more
efficiently. For the quality of the building, as before we found no large difference in
terms of endowments but large differences in terms of how those endowments are used:
both Italy and Spain are remarkably more efficient. Since this variable only measures
“availability”, part of the slope effect might be related to differences in quality. Since
Italy and Spain are richer, it is expectable that they have more and better quality
materials.
The number of teaching hours per year is not so different between these countries, but
Argentina is much more efficient than Italy in using those hours, but much less efficient
than Spain.
It is unexpected that teaching qualification has not significant differences for both
countries compared to Argentina, neither in terms of endowment nor in terms of how that
endowment is used. For the rest of the variables we find a similar result, there are no
large differences.
Finally, the intercept is larger here than when we compared Argentina with similar
development level countries, large and positive for Italy, and large and negative for
Spain.
Surprisingly, Argentina is much more similar to these countries in terms of the
observable variables, both in endowments and how the endowments are used, at least
according to our own prior beliefs. Is it the case that the strong cultural influence of both
102
Italy and Spain is reflected in the Argentinean educational system? Since Italy and Spain
are underperformers when compared to similar income level countries, if this hypothesis
is true, Argentina inherited some of this underperformance as part of the overall cultural
baggage.
The comparison with the third group of countries is shown in the next table. In general
we find the same qualitative results as when we compare Argentina with the Czech
Republic or Poland. For instance, the high-score countries have better qualified teachers
(endowment effect) but with the same efficiency as Argentina (almost null return effect).
They have more computers per student than Argentina, but Argentina is more efficient in
using the computers available. In terms of building quality and availability of materials,
the differences are in the slope more than in the endowment. The three countries have a
better slope for low SES student and high SES students. And a very large proportion of
the difference is represented by the intercept, what shows that unobservability is an issue.
103
Table 45. Oaxaca-Blinder decomposition for High performing countries
Age
Sex
1 if Foreign
# of Siblings
Finland
Endow
Returns Inter
-3.4 -37.5 0.4
-2.2
10.3 -1.2
-0.4
-0.1 -0.2
2.3
6.6 -1.1
Great Britain
Endow Returns Inter
-3.2
-76.4
0.9
-2.3
-4.4
0.6
-1.5
0.2
1.2
2.2
-0.6
0.1
Sweden
Endow Returns Inter
-1.2
67.2 -0.3
-3.1
3.1 -0.5
-2.4
-0.1 -1.0
1.1
6.4 -0.5
Position of the child within brothers and sisters
Oldest (excl.)
Middle
Youngest
0 if single parent
household
-0.5
0.5
-1.3 -0.3
0.8 -0.1
-0.2
0.6
0.1
1.8
0.0
-0.3
-0.3
0.1
-1.9
-0.1
-0.2
0.0
12.8
0.0
-0.2
19.1
-0.7
0.0
9.1
-0.1
2.7
0.1
0.0
-1.9
1.4
9.3 -5.0
2.4 -0.2
0.0 6.6
7.3 -2.8
3.5 1.0
4.3
0.1
0.0
-3.7
5.3
20.6 -19.6
10.6 -1.5
0.0 27.6
25.9 -18.6
14.6 14.0
4.3
0.1
0.0
-0.2
7.6
-1.0
-2.0
1.9 -0.6
-11.6 2.3
3.4
-0.4
-8.6
-5.9
-7.3
0.5
-0.5
-1.8
2.9
0.1
-0.4
0.0
20.7
-13.2
0.6 34.1
7.8 7.6
-1.6
17.7
-8.7
8.4
0.6
-0.3 -14.9
-27.7 -17.1
0.1
19.1
-10.4
41.4
-0.2
3.3
-0.2
-9.1
2.5
24.4
-13.7 -25.9
24.4
-16.9 -34.2
23.4
-3.0
-14.5
3.9
0.8
-5.0
-0.5
0.4
-10.2
-0.5
-1.1
18.0
1.8
-3.5
20.2
6.3
-0.3
9.2
0.3
6.3
46.2 -6.9
3.5
23.5
-1.9
9.1
39.1
-7.9
115.1
0.0
0.0
-139.9
0.0
29.6
58.3 13.3
114.1 -64.9
45.0
0.0
Mother’s education
Primary incomplete (excl.)
Primary complete
Secondary incomplete
Secondary complete
Higher Ed. incomplete
Higher Ed. complete
School Characteristics
School Size
1 if big town
# of teaching hours per
year
Teacher Qualification
Student Behavior
# of Computer per
student
Index of shortage of
teachers
Index of building
quality
Index of educational
materials availability
Intercept
Total
9.4
0.0
37.0
16.4 -15.5
8.2 -1.7
0.0
6.3
13.0 -0.5
8.2 11.2
-10.0 -19.6
65.3 -37.7
Grouping Variables
In an attempt to summarize the effects already discussed we group variables and compute
the overall effect for each group. The most important observable differences are:
i) Although Argentina tends to have a poorer endowment in terms of students SES
(except for Mexico), the important difference for this variable is in the slope
effect. Argentina has a similar slope as developed countries, but similar income
level countries, particularly the Czech Republic and Poland, have a much better
104
slope than Argentina. This explains how these two countries, poorer than Finland,
Sweden or Great Britain, are able to obtain similar average score: they are more
efficient in producing high score with low level SES students.
ii) Argentina shows a lack of qualified teachers compared to all the countries.
iii) Argentina has poorer school resources, poorer quality, and uses them more
inefficiently.
Table 46. Oaxaca-Blinder decomposition
Czech Republic
Endow Returns
Mexico
Inter
Endow Returns
Poland
Inter
Chile
Endow Returns
Inter
Endow Returns
Inter
Student charact.
3.2
-189.6
-1.6
-10.7
-136.9
3.2
0.2
-133.7
-1.6
-1.4
-136.1
0.1
Student SES
6.7
108.1
7.6
-7.2
12.5
1.6
8.4
60.7
26.4
1.4
53.4
2.0
School charact.
-0.9
3.7
-0.6
1.9
15.8
-2.8
1.1
10.8
1.3
4.6
16.9
9.6
Instructional time
-1.5
-49.8
-3.2
-4.8
29.2
6.1
0.5
86.3
-1.8
-6.6
94.7
27.2
Teacher Qualification
17.5
0.4
21.0
2.7
-0.6
-4.8
18.4
0.5
25.9
15.1
-0.1
-4.5
School Climate
-4.2
-8.5
-2.6
-11.0
8.5
6.6
-8.9
-29.8
-18.9
-7.9
-10.7
-6.0
School Resources
26.0
51.2
-29.0
-11.2
26.7
6.5
14.1
36.5
-6.2
1.8
28.2
-3.3
Constant
103.6
Total
46.7
19.1
44.1
-8.4
-40.3
-45.1
-0.7
16.4
33.9
-13.8
Italy
Returns
Student SES
School charact.
Instructional time
Teacher
Qualification
School Climate
School Resources
Inter
School charact.
Instructional time
Teacher
Qualification
School Climate
School Resources
3.5
2.3
77.6
-0.8
3.8
1.5
-0.1
-0.1
-3.1
-83.5
-11.4
-2
64.8
5.8
0.4
0.7
0.8
0.4
-1.5
-1.7
-7.7
-21.4
-11.7
-10
2.2
1.6
2.1
35.1
0.4
-3.3
58.1
-0.5
24.7
23.4
-24.5
13.8
33.6
-13.4
-39.2
2.7
66.9
-177.5
57.2
-9.2
Great Britain
Inter
Endow
Returns
Sweden
Inter
Endow
Returns
Inter
-8.4
22.6
-9.7
-2.5
-0.5
1.7
-4.6
6.0
3.0
-1.6
-60.3
71.6
-14.5
8.4
1.7
2.0
-6.8
0.6
-5.8
11.8
-2.3
0.1
20.7
-13.2
26.5
0.6
7.8
36.1
34.1
7.6
-27.1
17.7
-8.7
25.2
-0.3
-27.7
21.8
-14.9
-17.1
-30.3
19.1
-10.4
32.6
-64.9
45.0
9.4
29.6
58.3
25.2
Inter
8.2
192.1
Returns
-58.2
-3.6
2.3
-3.0
Constant
Total
Returns
-87.8
22.2
Endow
Endow
4
Spain
Student char.
Student SES
7.0
1.8
Constant
Total
25.0
Spain
Endow
Student charact.
-104.4
115.1
13.3
37.0
114.1
105
VII. A Hierarchical Lineal Model with PIRLS
A drawback of PISA is that it does not include the teacher level. PIRLS instead have a
rich set of variables about the teaching methodology obtained from a survey to teachers.
With PIRLS we can analyze three levels: students, teachers/class, and school.
Argentina ranked relatively bad in PIRLS too, it is well below the average, close to
Colombia (the other Latin-American country included in the study) and far away from
similar income level countries such as Hungary and the Czech Republic (see Table 47). .
It ranks bad in term of the overall reading achievement test, and in each of the blocks
considered (reading for Literacy and Informational Purposes, see Table 48).
Comparing the percentage of students that reaches PIRLS international benchmark, only
17% of the Argentine students taking the exam were above the median (see Table
49).This is above Colombia (14%) which has a slightly better overall tests, what shows
that in Argentina quality is more unequally distributed when compared to Colombia.
Similar income level countries such as Hungary and the Czech Republic have a much
larger proportion of students over median (71% and 68% respectively).
In Argentina even the top students are underperforming. In general students show big
problems to understand basic texts, and the achievement is well below the expected level
for a country with its income level and overall educational indicators.
106
Table 47. Distribution of Reading Achievement
Source: PIRLS 2001 International Report
107
Table 48. Achievement in Reading for Literacy and for Informational Purposes
Source: PIRLS 2001 International Report
108
Table 49. Percentages of Students Reaching PIRLS International Benchmarks in
Reading Achievement
Source: PIRLS 2001 International Report
The poor performance in understanding basic text is surprising since other related
indicators are not so bad. For instance, 50% of the parents qualify as doing high Early
Home Literacy Activities, a ratio above Norway, Germany or Sweden. Most of the
countries with similar scores in the Reading Test as Argentina have a ratio well below
(for instance, Colombia 40% and Turkey 26%), but it is also the case that the countries in
the benchmark group that outperform Argentina as Hungary and the Czech Republic have
a better index.
109
Argentina has a very low proportion of students where Spanish is not the mother
Language, and a very small proportion of the students’ parents are foreigners, what this is
not a disadvantage neither.
A significant difference between Argentina and the average is the high proportion of
students with poor educational resources at home (Index of Home Educational
Resources). According to this index 40% of the students have low resources, a ratio only
worse in Turkey (41%), Colombia (48%), and Morocco (76%). Something similar is
observed for the number of books at home, where only 9% of the students have more
than 100 books, compared to the world average of 35%. Similar income countries such as
Hungary and the Czech Republic have 62% and 54%, respectively. These two countries
have also a very low proportion of students with low Home Educational Resources (4%
and 2%, respectively). Comparing the children books, in Argentina only 6% of the
students have more than 50, whereas in Hungary and the Czech Republic 44% and 46%,
respectively. This evidence seems to show that Argentina has been not putting the same
educational effort at home than other similar income level countries did. To what extend
the household decisions and conditions are affecting the student performance?
600
Average Test Score for LOW
EHLA students
Average Test Score of HIGH
EHLA students
Figure 31. Average Reading Achievement and Early Home Literacy Activities
550
500
450
AR
400
350
0
20
40
60
80
600
550
500
450
400
AR
350
100
0
% of students that have HIGH Early Home Literacy Activities
20
30
40
50
700
600
Average Test Score
ARG
Average Test Score
10
% of students that have LOW Early Home Literacy Activities
550
500
450
400
650
ARG
600
550
500
450
400
350
350
0
30
40
50
60
70
% of foreing parents
80
90
100
10
20
30
40
% students with Index of Home Educational Resources
(HER)
110
To understand the role of socioeconomic characteristics we analyze “quality of
education” (i.e. after controlling for student’s characteristics).
The first important result is that after controlling for the student socioeconomic
characteristics the differences across countries reduces significantly. The average quality
for the entire sample increases to 542 and the standard deviation falls to 67.7 (instead of a
mean of 500 and a standard deviation of 100 for the test score). Argentina changes from
31 out of 35 countries (when ranked by average test score) to 26 out of 35 (when ranked
by quality), showing that a more disadvantage socioeconomic level is affecting its
ranking. Its position now is very close to Hungary (24) and some developed countries
such as New Zealand (28) or Singapore (25). For Colombia, instead, its ranking did not
change (still is 30th), the same for Belize (35) and Kuwait (33). Eliminating household
income and parents’ education (which might be related with private school enrollment)
the ranking does not change much, Argentina is 27 out of 35.
It is interesting to note that in PISA we did not find a change in the ranking after
controlling by SES but we do find an improvement in the ranking in PIRLS. This is
related with our previous finding that SES adjusted average score changes more in
PIRLS than in PISA. The SES slope was also higher in PIRLS than in PISA, what
suggest that SES and difference among the students’ SES have a more important role in
the early educational process.
Another important result is related to the heterogeneity or dispersion of quality of
education between students (what we measure as the coefficient of variation). According
to the test score, Argentina ranks 4 out of 35 in terms of lower coefficient of variation,
showing low heterogeneity, but after controlling for socioeconomic characteristics
Argentina ranks 30 out of 35 (29 if we exclude income and parents’ education), what
shows that Argentina not only has a relatively poor quality of education but quality is
very heterogeneously distributed and this is not due to differences in income level.
111
VII.1. Variance Decomposition
Decomposing the total variance in between-schools and within school by country,
Argentina is 7 out of 35 countries in terms of highest between-school variation. For all
the countries, the between-school variation share on total variation is, on average, 36.3%
but for Argentina it is 49.4%. This level of between-schools variation is similar to the one
found in PISA (50.4%). But in PISA we found that the benchmark countries have a
higher between-school variation than Argentina, and now we find the opposite. In fact
Hungary and the Czech Republic are now among the countries with lowest betweenschool variation (3th and 7th respectively). Colombia, the only Latin-American country in
the benchmark group in PIRLS has also a very high between-school variation, higher
than Argentina.
How should we interpret these results? Argentina is providing a more unequal quality at
primary school, but less differentiation in secondary school. In this sense, the inequality
in primary school is definitely a negative aspect of Argentina. But the low inequality in
secondary school is not necessarily a positive aspect. We are measuring a differentiated
product in just one dimension: reading, and probably the students in Hungary and the
Czech Republic attending technical schools (preparing students for the job market), are
developing other dimensions which are not measured here. In Argentina secondary
schools are not as differentiated, what might be reasonable to expect less between-school
variation. The important result is that Hungary and the Czech Republic, even with more
differentiation, have an overall performance in this unique dimension much better than
Argentina, even for the students who are in technical schools.
112
Figure 32. Between and within school variation
As % of Total Variance
Between-School Variation
Within School Variation
Russian
Romania
Macedonia,
Colombia
Norway
Belize
Iceland
Argentina
Slovenia
Lithuania
Iran, Islamic
Morocco
Bulgaria
New Zealand
Sweden
Italy
Greece
Slovak Republic
Russian
Kuwait
Latvia
England
Canada
Germany
Hong Kong,
Israel
Moldova, Rep.
United States
Turkey
Czech Republic
France
Cyprus
Singapore
Hungary
Netherlands
Scotland
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
In PIRLS we can decompose the overall variation in schools and classes. We estimate a
four-level HLM model for the entire sample with the following levels: country, school,
class and student. Once these levels are added to the analysis, the between-schools
variation reduces a lot. The previously found between-schools variation for the entire
sample (36.3%) now is broken in: 18% between-countries, 5% between-schools and 77%
between classes. This means that the policies with more impact in terms of improving
quality should be looked for at the country level differences (such as spending in
education, organization of the overall system) and the class level (mainly teachers,
student composition, and resources at the class level).
At the school level, 52% of the variation is explained by school resources, 29% by school
climate, 7% by school funding, and 7% by peer characteristics. At school level, only 5%
is unexplained. At the class level, on the other hand, much of the variation is unexplained
(88%), 7% is explained by peer characteristics, 4% by the methodologies that the teacher
uses, and less than 1% by climate and class infrastructure.
113
VII.2. School Factors Related to Quality of Education
In the next table we show the estimated (Bayesian) coefficients for our more general
HLM model for the entire sample, in order to try to identify which variables are
statistically significantly affecting the quality of education. We only show the school and
class level variables, the one we are interested in. The dependent variable (student score
in PIRLS) was standardized to have a zero mean and a standard deviation of 1.
As with PISA, most of the variables are not statistically significant, although most of the
variables have the expected sign. There are only three variables that are statistically
significant at 1%: i) the proportion of students that come from disadvantaged homes (a
proxy for school SES), ii) the proportion of the students that need remedial instructions in
reading at 4th grade (a proxy of how the students arrives to grade 4th)19, and iii) the
proportion of time that the teacher allocates to formal reading instructions. From these
three variables the larger effect is observed for the last one, which is a class level
variable, and it depends on how the teacher allocates the time internally (there is a lot of
variation across schools and countries). An increase in one standard deviation in the
proportion of time allocated to formal teaching reduces the score in 40% of its standard
deviation, what definitely is a very large effect.
The index of availability of resources at the school is significant at 10% only for the
lowest level, which has a negative sign as expected. Something similar happens for the
index of teacher shortage at school level, which has a negative sign only for the extreme
case with a lot of shortage. In the case of school resources, a school with low availability
of resources, compared to a school with high availability, has a lower score in
approximately 15% of the standard deviation of scores. For teacher shortage, we find a
similar effect, a difference of 16% of the standard deviation of test scores between the
schools with no shortage and the schools with a lot of shortage.
19
We also estimate the model excluding this variable and the results do not change much.
114
Table 50. HLM Estimations results
School Level Variables
% of students from disadvantaged home
1 if the school has a library
Index of Availability of school resources:
High (excluded dummy)
Medium
Low
Shortage of books in the school library
Not at all (excluded dummy)
A little
Some
A lot
Shortage of teachers in the school
Not at all (excluded dummy)
A little
Some
A lot
Lenght the student stay with same teacher:
Varies greatly (excluded dummy)
1 year or less
two years
three years
four or more years
Parents’ influence in school decisions
A lot (excluded dummy)
Some
Little
Not aplicable
Students’ influence in school decisions
A lot (excluded dummy)
Some
Little
Not aplicable
Proportion of the students that attended pre-primary school
Less than 25%
25% - 50%
50% - 75%
More than 75%
% of students that need remedial instruction in reading
1 whether the school has a reading specialist available
Teacher level variables
% of hs in reading instructions
% of hs in formal learning
% of hs with reading instruction
Teaching time
1 if the teacher approach is to give general lectures
1 if the teacher gives the same material for all students
Coef.
St Error
-0.049
0.027
0.016
0.048
-0.042
-0.150
0.048
0.090
0.072
0.059
0.044
0.045
0.058
0.076
-0.015
0.057
-0.161
0.046
0.068
0.078
0.090
0.080
0.093
0.161
0.066
0.074
0.077
0.068
-0.042
-0.064
0.014
0.073
0.082
0.106
0.042
0.028
0.034
0.062
0.071
0.102
0.014
0.017
0.071
-0.033
0.092
0.047
0.054
0.074
0.004
0.055
0.018
-0.411
0.045
0.018
-0.065
-0.095
0.035
0.067
0.055
0.044
0.041
0.075
***
*
**
**
***
*
***
The availability of a reading specialist in the school has a positive and statistically
significant (at 10%) effect on the student scores. Since the dependent variable is
standardized to have zero mean and a standard deviation of 1, the availability of the
reading specialist increase the score in 9.2% of the standard deviation in test scores at the
student level, what is a relatively important effect.
115
In terms of for how long the teacher stays with the same student, we find that there are
not differences from 1 to three years, but there is a significant effect for those who are
fourth years or more with the same teacher. Most of the countries in this category are
Eastern European countries. For instance, in Argentina only 0.75% of the school
surveyed fall in this category, whereas in Hungary 55%, Bulgaria 89%, and Romania
90% (see table). Therefore, some caution should be taken when interpreting this result,
since this variable might be capturing other effects that are common to this countries. In
fact when we include country fixed effect, the estimated coefficient for this variable is
reversed, it goes from positive to negative as the teacher stay more years with the student.
Using a country fixed effect model do not change qualitatively the rest of the results,
although even less variables remains significant (because a large part of the school
variation is eliminated once we include the country level).
Table 51. Length the students stay with the same teacher
Country
Argentina
Belize
Bulgaria
Canada
Colombia
Cyprus
Czech Republic
England
France
Germany
Greece
Hong Kong, SAR
Hungary
Iceland
Iran, Islamic Rep. Of
Israel
Italy
Kuwait
Latvia
Lithuania
Macedonia, Rep. Of
Moldova, Rep. of
Morocco
Netherlands
New Zealand
Norway
Romania
Russian Federation
Scotland
Singapore
Slovak Republic
Slovenia
Sweden
Turkey
United States
World
varies greatly
4.5
23.4
0.6
2.0
13.3
32.1
10.1
1.6
20.0
5.5
34.1
54.6
0.9
31.4
12.0
14.8
6.0
65.0
2.9
1.4
0.7
4.1
26.4
7.7
3.4
11.9
3.6
4.4
5.6
2.0
30.2
6.9
2.1
2.6
1.8
11.4
one or less
46.3
57.0
0.0
89.5
35.7
42.8
1.4
86.4
47.4
0.0
14.1
31.2
0.0
0.0
77.1
17.0
0.6
27.6
2.2
0.0
0.0
2.0
60.0
71.8
74.2
0.0
0.7
0.0
69.4
50.0
4.7
20.1
2.1
8.5
95.1
30.5
two years
35.1
16.8
3.0
8.5
21.7
24.4
40.3
11.2
21.5
29.2
41.5
14.2
27.4
17.0
6.0
65.2
1.1
6.5
2.2
0.0
0.7
0.0
10.0
17.1
16.3
5.2
2.9
0.0
23.2
48.0
33.6
68.1
28.5
11.8
2.4
18.4
three years
13.4
1.9
7.7
0.0
10.5
0.8
33.8
0.0
8.2
10.6
9.6
0.0
17.0
43.2
1.1
2.2
13.1
0.0
19.0
0.0
0.0
4.1
1.8
3.4
3.4
44.0
2.9
52.9
1.9
0.0
10.1
4.9
63.2
19.6
0.0
11.6
four or more
0.8
0.9
88.7
0.0
18.9
0.0
14.4
0.8
3.0
54.8
0.7
0.0
54.7
8.5
3.8
0.7
79.2
0.8
73.7
98.6
98.5
89.8
1.8
0.0
2.7
38.8
90.0
42.7
0.0
0.0
21.5
0.0
4.2
57.5
0.6
28.1
116
VII.3. Oaxaca-Blinder Decomposition
In the next table we present the Oaxaca-Blinder decompositions for similar income level
countries: Colombia, the Czech Republic and Hungary. The results are qualitatively
similar to those found in PISA, but here we have a richer set of variables. The
disadvantage of PIRLS is the high frequency of missing variables, what might affect the
results. Here we assume the missing is ignorable, and proceed to do the analysis
following the complete case approach.
The complete case approach has some problems. For instance, for the entire sample
Colombia has 1072 school hours per year and Argentina 693, but in our regressions the
differences are not so large. This limitation of PIRLS must be consider when interpreting
the results.
We start our analysis with those variables that were also included in PISA, showing that
the results are similar. First we find that difference in term of school hours per year is
more in the slopes that in the endowment (average number of hours), where Argentina is
less efficient in transforming school hours in score points. This is true even when we
compare Argentina with Colombia. Second, Argentina does not show large differences
compared to Colombia and the Czech Republic in terms of the proportion of with low
SES, and more students with low SES when compared to Hungary. The estimated
coefficient for all the countries is negative, therefore a negative return effect means that
these countries have a larger estimated coefficient, what means they are able to obtain
better results with these socioeconomically less advantageous students. We find this
result for Hungary and Colombia, and a negligible difference with the Czech Republic,
what shows that the quality of education in Argentina is more unequal.
117
Table 52. Oaxaca-Blinder Decomposition, similar income level countries
Endow
School Level
Number of school hours per year
-28.8
% of students from disadvantageous families
0.3
Index of Availability of school resources:
High (excluded dummy)
Medium
2.1
Low
-16.1
Shortage of books in the school library
Not at all (excluded dummy)
A little
-1.9
Some
2.8
A lot
12.3
1 whether the school has a reading specialist available
0.1
Shortage of teachers in the school
Not at all (excluded dummy)
A little
-2
Some
2.9
A lot
0.6
1 if the school has a library
-9.1
Length the student stay with same teacher:
Varies greatly (excluded dummy)
1 year or less
-4.5
two years
-6.7
three years
0.8
four or more years
-37.7
Parents’ influence in school decisions
A lot (excluded dummy)
Some
0.6
Little
-0.2
Not applicable
-1.8
Students’ influence in school decisions
A lot (excluded dummy)
Some
0.4
Little
0
Not applicable
-1.6
Proportion of the students that attended pre-primary school
Less than 25%
25% - 50%
-3.2
50% - 75%
0.2
More than 75%
3
Class Level
% of time allocated to teaching
-0.4
% of time allocated to formal reading instructions
-12.3
% of time allocated to reading instruction
-3.1
Frequency of reading classes
-1.3
1 if the teacher approach is to give general lectures
-2.9
1 if the teacher gives the same material for all students
2.5
% of students that need remedial instruction in reading
1.6
Intercept
Total
-103.4
Colombia
Czech Republic
Hungary
Returns Inter. Endow Returns Inter. Endow Returns Inter.
53.7
-16.5
31.2
-1.2
-5.5
-1
45.9
3.5
5.1
-0.8
8.7
-27.7
9.1
17.2
5
-1.6
11.2
8.6
4.2
18.6
7.2
-7.1
-4.2
10.2
4.6
14.1
10.3
-6.3
-3.9
-25.6
-13.5
-33.4
1.1
3
-13.9
-25.1
-0.3
6.2
1.5
-10.7
-5.2
-14.4
-3.8
-18.1
0.4
-5.5
-2.1
11.8
5.3
-0.4
2.4
-5
0.7
-7.4
-1.8
-3.6
-0.5
0.3
-1.6
2.1
-0.6
-7.2
0.6
-1.1
-41.9
2.1
1.1
-0.3
7
-3.5
1.2
-2.1
9.6
-5.4
-2.9
-2.1
-69.5
2.7
-2.3
2.1
-12.3
-1.8
0.1
1.7
8.2
-1.6
0
-1.6
-52.4
0.6
0
-1.4
-7.6
-43.6
-23.1
-3.2
2.9
8.5
13.4
-2.1
36.3
-22.3
1.6
3.5
-22.8
-22.3
-18.3
-1.9
2.5
21.7
-2.5
-5.3
18.4
-20.1
-3.2
1.5
-75.1
-20.1
-0.7
-0.4
2.3
20.1
0.2
-0.3
78.6
12.7
21.4
6
1.4
9.8
-2.6
2.5
0
-0.2
-7.8
-1.4
-2.7
-3.3
0
0.2
2.5
0.9
-2
-9.8
-4.2
-4.2
-3.2
-2
2.3
-11.6
-13
-2
-2.6
0.6
-9.3
0.6
0.2
-0.9
7.5
0.4
0.3
2.4
0.4
0.9
0
-6.4
-5.3
-7.8
1.3
0.6
0.1
3
4.2
-11.2
-1.3
-4.2
2.9
0.2
-2.9
-5.9
1.2
-4.3
-5.8
1.2
-4.3
2.8
-1.2
4.3
-2.5
0.2
-7.2
0
0.7
-7.2
0
-0.5
7.2
-19.9
39.2
-7.4
-54.5
32
7.9
32.5
141.6
39.5
0.9
10.8
3.4
1.5
-8.9
-4.5
-1.7
-3.9
22.7
36.6
-16.2
-0.9
1.7
16.6
8
-23.9
-44.7
18.6
1.8
-1.7
-5.4
-2.6
4.2
8.7
-15.7
-4
2
11.6
68.3
-21.6
46.8
-8.3
-52.5
-20.2
4.3
9.8
255.1
13.1
120.1
-15.7
-19
27.7
-5.9
-27.7
-31.6
4.7
2.6
272.2
-83.2
101.3
4.1
-4.7
-8.4
15.6
7.2
-3.5
-1.4
109.4
In terms of school resources, Argentina has more resources than Colombia (negative
endowment effect) and less resources than the Czech Republic and Hungary, but the
return effect is positive for all the countries, what means that these countries are more
efficient than Argentina to transform those resources in score points. For the dummy
taking 1 if the school has a library (another resource-related variable) we find the same
118
pattern in terms of endowment (Argentina better than Colombia but worse than the Czech
Republic and Hungary); but in this case the return effect is negative, what shows that
Argentina is more efficient in using those more limited resources. Also related, the lack
of books in the library favors Argentina, but since the coefficient is negative it reflects the
fact that Argentinean scores are less affected by books shortages than Colombian, Czech
or Hungarian scores.
Having or not a reading specialist in the school has a relatively small effect,
In terms of the length the student is with the same teacher, the interpretation is not neat,
since Argentina has a negative coefficient when the length is larger (this is related with
the fact that in Argentina the teacher is more than one year with the student only in small
schools or in schools in small towns). In terms of endowment, the proportion of students
that stay more than one year with the same teacher is much larger in Colombia, the Czech
Republic and Hungary than in Argentina.
In terms of shortage of teachers, there are not large differences in terms of endowments
neither in terms of slope, although Argentina shows a low severe shortage since it has a
negative endowment effect for the first category (low shortage).
The variable “Parents’ influence in school decisions” reflects the degree of attention paid
to parents’ opinions. It is important only for Colombia in the return effect, indicating that
Colombians best exploit listening to parents than Argentina. Variable “Students’
influence in school decisions” reflects the degree of listening to students which does not
presents particularly large differences among the countries.
Finally, preprimary school variables are including presenting small differential effects
over the scores for both countries.
Next we analyze the class-level variables, mainly related with the approach the teacher
uses to teach. First we find that in Argentina using the same material for all the students,
119
even those who show reading problems, is more common that in the other countries, but
the effect of this endowment differences is relatively small. The slope coefficient, related
to how the students with teachers that use the same reading material for everybody
perform in the test is positive for all the countries, what shows that these countries can
obtain better scores even when they use the same material.
Second, the proportion of time that teachers spend in reading instruction relative to the
whole number of hours in language instruction, the results depend on which country we
choose as a comparator.
Third, the proportion of time allocated to formal instruction has a positive and strong
return effect, what shows that Argentinean teachers seem to exploit formal hours of
language not as well as Colombian, Czech or Hungarian teachers. Notice that the
endowments and interaction effects are smaller and tend to cancel each other because
Argentina has a different sign coefficient for this variable as formal hours do not improve
scores in Argentina. This could be a side effect of bad teaching quality or lack of
preparation from teachers, hypothesis reinforced by the return effect.
Fourth, the difference between spending a large amount of time in language compared to
spending the least amount of time suggested that the passage from very little time in
language instruction to a lot of time is reflected in around 50 score points more for both
Colombia and Czech Republic classes and 30 more points for Hungary than for
Argentineans. Over a total of 500 points, this is a very important effect which signals
again a problem in the marginal return of extra hours of teaching.
We now turn to analyze the degree of personalization in teaching. The variable reflects
whether classes are directed to all students at the same time or personalized. Here the
effect is very different between Colombia and the Czech Republic. For Colombia, this
variable tends to contribute to a positive gap reflecting that global teaching improves
scores more in Colombia than in Argentina. However, in the cases of the Czech Republic
and Hungary, the variable presents a negative coefficient reflecting that teaching to all the
120
students together is negative for scores. As for Argentina this effect is the oppositve, the
return effect is very large and negative when compared to the Czech Republic and
Hungary.
As a summary of this first subset of countries, in different variables we find important
return effects that suggest a worse or better utilization of the resources available in
Argentina compared to the other two countries. In some cases this is due to the fact that
the effect of variables does not have the same sign in both countries and therefore the
returns effect does not reflect a higher impact of the variable over scores but simply a
qualitatively different effect, possibly due to cultural or institutional differences.
121
VII. A Hierarchical Lineal Model with ONE
In this section we decompose test score using a hierarchical lineal model but at subregional level (instead of cross country). Neither PISA nor PIRLS can help to analyze the
heterogeneity in the quality of education by economic regions in Argentina. We base this
analysis on ONE 2000, which identifies the class level and it is a census type data. We
can extend the HLM to 4 levels: student, class, school, and province.
ONE 2000 tested 6th grade students in primary school (the last year of EGB2) and
students in 5th grade of secondary school (last year of Polimodal). Because teachers are
uniquely assigned to each class only in primary school, whereas in secondary level the
same teacher teaches the same class to different classrooms, and we are interested in the
factors associated to teachers explaining the between classroom variation, the analysis
will be performed only with the students in 6th grade.
IX.1. Variance Decomposition
Classroom and School Variation
First we estimate a null HLM model with 2 levels for the entire sample: students and
schools. This includes all the students, even in rural schools and adult education. The
percentage of the total variance explained by the between school variation is 37.4% (and
62.6% is explained by the within school variation). This results is significantly different
from the variance decomposition result found with PISA, where the within variation
represented 50% of total variation. An explanation for this is that we are considering now
a much larger sample, what increase the heterogeneity at the student level, although we
have to take into account that the tests are measuring different aspects.
Next we estimate a 2 levels HLM null model with students and classes (instead of
schools), finding a slight reduction in the proportion explained by the student level,
which now is 60.6%. This is somewhat a surprising result, what shows that there is not
122
much variation between different classes inside each school. Even restricting the sample
to schools with more than one class we find that class-variation is much smaller than
school variation.
To understand better the relationship between classes and schools, we restrict the analysis
to non adult urban schools with more than two classes in 6th grade. First we run a 3-level
HLM with: students, classes and schools. The within variation (student level) represents
64% of the total score variation. In terms of between schools and between classes
variation, we find that school variation is much more important than class variation,
explaining 26.5% of the total variance, compared to the 9.5% that explains the betweenclasses variation. This result is in contradiction to what Hill and Rowe (1996) claims, that
when classrooms are included as a level between the students and the school, the
between-classroom variation in achievement is larger than the between-school variation,
and the latter is often reduced to a very small value. Based on that result the authors
claimed that school effectiveness research should be focus more at classroom level than
at school level. Our result does not necessary mean that the teacher characteristics or
peer-group effects at the classroom does not matter. Rather it is showing that here is a
lack of variation in classes within schools, what might be a result of a school selection of
teacher which is relatively similar, and to the random assignment of students into classes
(something we will discuss further later.)
4-levels HL model
Next we extend our model including a fourth level, provinces, finding that the between
provinces variation explains only 3% of the total variance, mainly due to differences
reflected at the school level. This relatively low importance of the between–provinces
variation shows that despite the large across-provinces disparity in development level, the
educational system (as a global system) is very homogeneous across regions, and the
differences are mainly the school and classroom inputs more than differences in inputs at
province level (such as public expenditure, years of schooling, etc.). This is related with
123
the results found earlier that across provinces there is not much variation in the public
expenditure per public student, in teachers wages and other systemic variables.
Provinces
School
3.03%
26.45%
23.44%
Class
Student
9.50%
9.48%
64.05%
64.05%
Next we decompose the variance in each level including covariates.
At classroom level we include the following block of variables:
a) Class Resources: which includes two variables:
- class infrastructure (a factor index constructed with variables related to
the class situation in terms of infrastructure)
- class material (a factor index constructed with variables related to
material available at the class level, such as books, TV, computers, etc.)
b) Teaching Methodology, which includes the following variables:
- Tests frequency
- Task frequency: An index constructed with variables related to the
frequency in which the teacher gives homework assignments, oral lessons,
class presentations, etc.
- Special classes: frequency in which the teacher gives special classes
targeted to those students with more difficulties in the learning process.
- A dummy variables for the categorical variable of “importance put on
passing the grade” (that takes one if promoting is highly or moderately
important when grading)
- A dummy variable for grading importance, which takes 1 if the teacher
believes grading is highly o moderately important.
c) Class Climate, which includes the following variables:
- Involvement: which is a simple average of Degree of parent involvement
(a factor index with several variables related to the extend that parents
124
collaborate with the teacher, meeting with parents, etc.) and Degree of
student involvement (a factor index constructed with variables related to
the degree of the student participation in classroom activities)
- Class Environment
d) Peer characteristics at the classroom: a set of variables related with the
classroom average student characteristics (such as wealth, SES, age, family size,
etc.)
At school level we include the following block of variables:
a) Funding, a set of dummies variables indicating the proportion of public
funding on the total school resources
b) School Infrastructure: a factor index constructed with variables such as ratio
of computers per student in the school; whether there is a library in the school;
whether there is a computer lab in the school; proportion of computers used
exclusively for administrative task; whether the school has access to
electricity, water, telecommunications, internet; the state of the building;
whether the director thing the building structure is appropriated for teaching;
etc.
c) School climate: a factor index related to school environment (violence,
whether some student has been robbed, whether some student has been
attacked, frequency of violent acts, etc.)
d) Peer characteristics at the School: school average of wealth, SES, age, family
size, etc.
The main results in terms of variance decomposition are shown in the next table. The
most important variable in both levels (classroom and school) is the peer characteristics,
which explains 10.8% of the total variance at classroom level and 64.7% of the variance
at school level. At classroom level, the climate explains 7.9% of the total, whereas the
remaining policy variables included explains a very small proportion of the classroom
variance. Note the high proportion of classroom variance not explained by any of the
variables included, what means that unobservable variables at classroom level, what
125
presumably are related to teacher unobservable characteristics (since we include several
student characteristic variables). At school level, the most important policy variable is
infrastructure related variables, what explains 4.5% of the school level variance. Public
funding and school climate explains a negligible part of the total between-school
variation.
It is important to point out the gains in terms of understanding policy issues of using
classroom and school levels. If we assume that the main effect of the peer group effect is
at the classroom level, what does not seem to be a strong assumption, then the proportion
of the total variance explained by peer characteristics at the school level should be more
related with other school variables correlated with the student characteristics at the
school, such as the school resources (private contribution of parents in public schools,
what are very common in Argentina, or the fee in the case of private schools). In the
ONE survey to school directors it was asked the proportion of total school funding that
parents contribute though the “Cooperadoras” (which is a parallel to the school institution
formed by parents that through private contributions and activities rise funding for the
school) but it does not ask the total funding available. Since in private schools the fund
raising is through the school fees and not the Cooperadoras, we have zeros or small
proportion for these schools even though they probably have more resources. The teacher
wage was neither asked in the teacher survey. Therefore we do not know how important
are the unobservable differences in funding, which probably are related to peer
characteristics. We will explore this hypothesis further in this study limiting the analysis
to public schools, to see whether among public schools the proportion of total funding
contributed by the Cooperadoras matters, and this reduce the proportion of the variance
explained by school peer characteristics.
126
Table 53. Variance Decomposition for between school and classrooms variation
Unexplained
Peers
Climate
Teaching Methodology
Class Resources/ School
Infrastructure
Funding
Total variance at the level
Classroom Level
As a % of
As a % of
between
total variance
classroom
variance
8.16%
80.65%
1.09%
10.76%
0.80%
7.90%
0.01%
0.14%
0.06%
0.55%
10.11%
School Level
As a % of
As a % of
between
total variance
school
variance
6.54%
30.67%
13.78%
64.65%
0.01%
0.03%
100.00%
0.97%
4.53%
0.02%
21.31%
0.11%
100.00%
In terms of comparing these results with those found in PISA, we should remember than
in PISA, 55% of the between-school variance in Argentina was explained by peer
characteristics, and 26.1% by school context. This means that 81.1% of the between
school variance according to PISA and 64.7% according to ONE was explained by
variables which are not related to specific school policies (other than the admission).
Infrastructure and resources, according to PISA explained 8% of the between-school
variance, and here we find that similar variables explain only 4%.
How should we interpret these results? Does school and classroom policy matters? The
fact that socioeconomic variables unrelated with classroom policies and school policies
(other than admission) explained a moderate part of the total variance at each level, when
we include a rich set of controls for the peer characteristics, means that the high
proportion of unexplained variance should be related with other variables, which
presumably are associated with teacher unobservable characteristics and school policies.
In this sense, the set of controls included for these variables has not been rich enough to
capture these characteristics, what still leaves open the question of what factors explains
the quality of education (if we can ever observed those factors). It is important to note
that some of the school and classroom variables can have an effect interacting with
student characteristics, what we will explore next in this work.
We think the importance of peer group effects following this approach cannot be
quantified. For this reason an important part of this paper is devoted to the identification
127
of peer group effects. Once we identify this effect we can ask the opposite questions,
what proportion of the variance remains to be explained at classroom and school level
once these effects are eliminated? If we correctly identify peer group effects, even if we
cannot precisely identify the factors, we know what proportion of the student
achievement is really related to policy variables (other than the admission policy).
Table 54. Estimated Coefficients for Classroom and School Levels
4-Levels HLM model
Variable
CLASSROOM LEVEL
Class Resources
Infrastructure
Materials
Class Climate
Environment
Involvement
Teaching
Test Frequency
Special Classes
Task intensity
Grade importance
Promotion Importance
SCHOOL LEVEL
Infrastructure
Funding
Climate
Participation
Coefficient
Standard
Error
***
Robust Standard
Error
-0.0194
-0.1839
0.005869
0.017099
0.0019
-0.0987
0.002792
0.003192
0.0025
-0.0102
0.0022
0.0039
-0.0090
0.001431
0.002966
0.004893
0.005936
0.006113
*
0.0095
-0.0007
0.003673
0.002017
***
0.00803
0.00438
-0.0580
0.0008
0.008614
0.000311
***
0.01987
0.00070
***
***
***
***
0.01319
0.03938
0.00575
0.00701
***
***
0.00313
0.00613
0.01042
0.01247
0.01276
***
Notes: ***, **, * denotes statistical significance at 1%, 5% and 10% respectively
128
IX. Peer-Group Effects in the Classroom
The peer group effect is in the core of the discussion. First because previous HLM
estimates might overestimate the peer group effect at school level, if these characteristics
are associated with unobservable school or city characteristics.
Second, if the peer group effect is non-linear, the increasing segregation of students and
schools observed in Argentina might have an effect on educational achievement. In
Argentina, the increasing share of private schools in total enrollment has increased the
segregation by socioeconomic level, particularly in the richest cities and neighborhoods.
The fact that private schools share is growing in richer communities, might be a
reflection of a higher demand for education (spending and quality) and since public
schools are administrated at province level, the local demands are not easily satisfied,
what gives room to the emergence of private schools. If the peer group effect is linear, the
resorting has a more neutral effect on aggregate school quality at city level, because what
the high socioeconomic level student gains in terms of achievement is similar to what the
low socioeconomic level student losses. Of course the inequality increases with the
resorting. If the effect is concave, the resorting of students according to socioeconomic
level might have increased the overall quality of the city (and country) and if it is convex,
it should have had a negative impact.
The usual assumption in the literature is that better peers increase the student
performance, but the peer group effect might have a more complex effect. For instance, it
could be that the student gains with more similar classmates rather that when better peers,
for instance because teachers are more effective with a more homogeneous classroom. If
this is the case, the resorting observed in Argentina is welfare improving (Pareto
efficient).
In this section we estimate the peer group effect in the classroom using the fact that in
Argentina, the typical primary school assigns students to classrooms in the first grade and
129
they preserve this assignment up to the end of the primary school. Ability tracking is not
a common practice (reallocating student across classes according to their performance)
neither the students attend different classes (there is a single class and a single teacher for
each group). In addition, schools do not take tests to incoming students; therefore, the
school has not enough information about the student type to assign students in the first
grade based on the student ability. The allocation, therefore, works as a natural
experiment, with a random allocation of students.
The details are shown in the appendix, here we discuss the main results.
The first result is in terms of the direction of the bias. We find that the coefficients for all
the peer group variables included have an upward bias in a simple OLS estimation,
compared to the school fixed effect model. The bias is not small, for instance the
estimated coefficient for parents’ education is 65% higher in the OLS estimation.
Table 55. Estimated coefficients for the peer-group effects variables
Robust Standard Errors in Parenthesis
Estimated
OLS
Estimated
Coefficient
upwards
Coefficient
Fixed
bias
OLS
Effects
Parent's education
1.218
0.737
65.3%
Wealth
0.091
0.051
78.4%
Family size
-2.024
-1.083
86.9%
Rooms per family
4.959
3.07
61.5%
member
Second we analyze non-linearities in the peer-group effect using different specifications.
We include the Parents’ Education Squared, to see whether the function is locally
concave or convex. We find a negative sign for the squared term, suggesting a concave
relationship, i.e. decreasing returns to peer characteristics. This result has been found by
McEwan (2003) or Auguste (2004) in Chile. Next we include an interaction term between
parents education at the student level and parents’ education at classroom level, finding a
positive effect, suggesting that when the student has a more educated family the gains
from better peers are larger. Finally, we find that students who are very different from
their peers suffer in terms of score.
130
Table 56. Exploring the functional form of the peer group effect
Estimated coefficients for the peer-group effects variables
Robust Standard Errors in Parenthesis
Parents’ Education
Parents’ Education Squared
Interaction between parents
education and peers parent’s
education
Disparity between parents
education of the students
and the peers
OLS
(I)
(II)
(III)
Fixed Effects
(IV)
2.078
(0.188)
1.184
(0.028)
1.187
(0.028)
-0.028
(0.002)
0.156
(0.052)
(V)
(VI)
1.183
(0.028)
0.647
(0.055)
0.093
0.048
(0.004)
(0.004)
-0.067
(0.002)
-0.058
(0.002)
These results show:
a) The high importance of the socioeconomic variables at the classroom and school
level in our HLM estimations was overestimating the true peer-group effect, and
therefore they are capturing other unobserved variables (such as school
resources).
b) The estimated peer-group effect function shows: i) decreasing returns, ii) good
students gain more with better peers than bad students, and iii) better peers
matters but also matters the difference between the student characteristic and the
peer characteristics, the more different is the student from his/her peers, the worse
is his/her achievement.
131
X. Teachers and the Quality of Education
Some specialists argue that the problem in Argentina is that the teacher quality has fallen,
and the studies to become a teacher (Magisterio) is providing only pedagogical courses,
without paying attention to the contents the teacher would have to teach in the future. The
argument follows: teachers are not motivated enough, and teachers themselves are
educated in the worst schools, because school quality is highly correlated with the
socioeconomic level of the family.
In this section we explore who are the teachers now and how teacher composition has
changed in the last 30 years. Here we summarize the main results, and a detail analysis is
shown in the Annex.
X.1. Who are the teachers now?
According to the last National Teacher Census (2004), teachers in Argentina is mostly a
public sector job. At national level, almost 69.8% of the teachers in basic education work
exclusively in public schools, 23.4% only in private schools, and 6.8% in both. The
private sector share varies a lot across jurisdictions. Buenos Aires City has the highest
ratio with 51.7% of the teachers working in private schools, whereas the lowest ratio is in
Formosa, 7.2%. Only 84.9% of the public school teachers are really working, whereas in
private schools this ratio is almost 90%. 80% of the teachers are females, although 24
years ago this ratio was even higher (85%).
Table 57. Evolution of the Teacher Composition
% of Female teachers
% of public teachers
1980
85%
83.3%
1991
82.2%
78.4%
2004
79.5%
69.8%
132
Most of the teachers in activity are young, 65% are between 25 and 44 years old. There
are not large differences between public and private schools teachers in terms of age
composition.
In 1994, 24% of the teachers were younger than 30 years old (new teachers) whereas in
2004 only 14.3%, even though the increase in the number of years of mandatory
schooling has increased from 7 years to 10 years, and more teachers are needed. In fact
the total number of teacher fell between both Census, in 1994 there were 833,391
teachers and in 2004, 825,250, when the total number of students in the system increased.
The average age for a teacher increased from 32 years old in 1994 to 40 in 2004.
On the other hand, 51% of the teachers have less than 15 years of working experience in
2004 compared to 69% in 1994. The average teacher had 11 years of experience in 1994
and now 15 years.
These distributions show that a high proportion of teachers leave their job earlier than
expected, and that less young people is interested in the job.
In terms of education, 77% of teachers have a higher education degree (although 13.7%
of the teachers did not answer this question), and almost 9% of those who answer do not
have a teaching degree. The ratio of teachers without teaching degree is higher in
secondary schools, and higher among private schools than public schools. Only 6% of the
teachers have a university degree, in a country where teaching is a tertiary education
degree.
The situation for primary school and secondary schools is relatively similar, therefore we
focus the rest of the analysis in primary school teachers (EGB 1 and 2).
In primary school, only 10% of the teachers have a Master degree or Doctorate, and there
are not differences in this ratio between public and private schools. 75% of the teachers
133
affirm to have done a training course in the last 5 years, ratio that is higher among public
school teachers (78.5% vs 63.6%).
In terms of the level of education for the teachers’ parents, we find that 70% did not
finish the secondary school in public schools, compared to only 45% in private schools.
The heterogeneity across provinces is very high, and since most of the private education
is in the richest provinces the difference is in part due to a compositional effect.
To establish whether the educational level for teachers’ parents is high or low, we
compute the distribution for the education of young teachers (between 20 and 25 years
old), and compared the distribution with the level of education for individuals between 40
and 60 years old according to the household survey EPH. We find that teachers’ parents
have more education than the average adult in their similar age range. For instance,
whereas 9% of the adults between 40 and 60 years old did not finished the primary
school, for young teachers’ parents this ratio is 5.2%. Unfortunately the EPH does not
have information for emancipated children, therefore we cannot compare the educational
level of the parents of those young people who finished the secondary school (which is
the right comparison group) with young teachers’ parents, but the levels shown do not
seem to be very high. We check this with the IDEO survey recently collected by FIEL.
This survey was collected in Buenos Aires City and Great Buenos Aires area in June
2007 to study intergenerational mobility. It collects information about the household,
emancipated sons and daughters, parents and brother and sisters. We first select only
those households with son and daughters between 23 and 29 years old (living or not in
the household) that have some higher education study (complete or incomplete) and then
tabulate the education level for these parents. We find that the educational level of this
group of parents is very similar to the educational level of the teachers´ parents, with
small differences. Whereas in the control group 19.5% of the parents have up to primary
complete, for the teachers´ parents this proportion is 23.1%. Similarly, in the control
group 63.4% of the parents have secondary complete or more, whereas for teachers’
parents this proportion is 62.3%. Although the difference goes in the direction that
134
teachers’ parents are relatively less educated than the parents of other individuals in the
same age range with more than secondary education, the differences are not so large.
Table 58. Teachers’ parents education
Individuals
Parents for
Parents for individuals
between 40 teachers between
between 20 and 25 years old
and 60
20 and 25 years
(in 2004) with more than
years old
old
secondary education studies
no instruction
1.18
0.2
0.3
primary incomplete
8.19
5.0
2.6
primary complete
29.42
17.9
16.7
secondary incomplete
14.52
14.6
17.1
secondary complete
21.69
21.5
23.3
higher education incomplete
5.24
9.8
10.2
higher education or more
19.75
31.0
29.9
Another interesting result is that only 70% of the teachers work in just one school, with
large heterogeneity across provinces (from 90% in Chaco to just 50% in Tierra del
Fuego). In terms of the job positions, only 61% have a regular position (20% are
substitute teachers, 10% temporary covering the chare, and the rest some combination
among these options). There is a significant difference between public and private school
teachers. In public schools, only 58% have a regular position compared to 73% in private
schools. In the Buenos Aires city, the jurisdiction with the largest private sector share,
only 56% of the public school teachers have a regular position compared to 78.4% in
private schools. The differences are not only large between public and private schools but
also across provinces. In Santa Cruz, only 7% of the teachers have a regular position in
public schools compared to 57% in private schools; only 52% of the teachers work in just
one school, and 48% have to work in two or more schools, what explains why this
province has had recently so many labor conflicts and strikes.
X.2. International Benchmarking
In most of the countries included in PISA 2000 teaching is a university degree
qualification, but not in Argentina. According to the ISCED 5A qualification of language
teachers, Argentina ranks extremely bad (see next table), even below other Latin
American countries, and in the last position among all the countries in the sample.
135
Table 59. % of teachers with an ISCED 5A qualification in the language of assessment
Mean
St. Dev
World
81.8%
0.314
Argentina
28.1%
0.361
Chile
64.7%
0.444
Czech Republic
88.3%
0.226
Hungary
98.4%
0.109
Poland
78.3%
0.253
In terms of shortage of teachers, Argentina ranks relatively well; in fact Argentina is the
second country with the lowest ratio of students per teacher (8.4), very low compared to
the world average (18.1). This is explains for the high proportion of teachers actually not
working (between 20% and 30%) or working partially (what is called “passive tasks”).
This shows: a) shortage of teachers is not a problem, but rather their qualification, b) the
low ratio of students per teacher more than a sign of quality is a sign of high inefficiency.
Figure 33. Ratio of students per teacher, PISA 2000
Brazil
Chile
Mexico
Peru
Thailand
Republic of Korea
The f ormer Yugoslav Republic of
Albania
Hong Kong Special Administ rat ive
Indonesia
Germany
Unit ed Kingdom of Great Brit ain and Nort
Russian Federat ion
Net herlands
Czech Republic
Romania
Japan
Ireland
Unit ed St at es of America
Spain
Aust ralia
Iceland
Sweden
Aust ria
Poland
New Zealand
France
Israel
Bulgaria
Swit zerland
Lat via
Denmark
Finland
Hungary
Greece
Belgium
Luxembourg
It aly
Norway
Port ugal
Argent ina
Liecht enst ein
31.5
29.1
27.4
22.9
22.2
21.0
19.8
19.6
18.5
18.2
17.8
16.1
15.7
15.5
15.5
15.2
15.1
15.0
15.0
14.8
13.7
13.0
12.6
12.6
12.6
12.5
12.5
12.5
12.3
12.1
12.0
11.4
11.3
10.7
10.3
9.7
9.6
9.2
9.0
8.9
8.4
7.7
0
5
10
15
20
25
30
35
136
Teachers’ participation in the school decisions in Argentina are above the world average,
but below Chile and Hungary.
Table 60. Index of teacher participation in school decisions. PISA 2000
Mean
St. Dev
World
-0.05
1.01
Argentina
0.108
0.83
Chile
0.212
0.997
Czech Republic
-0.223
0.889
Hungary
0.284
0.919
Based on the World Bank EdStats database and UNESCO we compare teacher
characteristics and salaries. Argentina is the country with the largest proportion of young
teachers. 30% of the total teachers are 30 years old or younger, compared to 9% in Chile.
Argentina is also the country with the lowest proportion of teachers in the more than 60
age range. Argentina also shows a very high participation of teachers and other teaching
stuff in the total labor force, twice as much as Chile, and only Hungary has a ratio similar
to Argentina, what confirms our previous result that Argentina does not have a lack of
teachers.
In terms of salaries, Argentina is the country with the lowest salary per teaching hour in
the benchmark group; nevertheless, teachers in Argentina are able to obtain monthly
salaries (in terms of GDP per capita) in line with the international standards, below Chile
but above Hungary and Czech Republic. This shows that teachers in Argentina have more
teaching hours than the other countries, something we show before given the high
proportion of teachers teaching in more than one school. More teaching hours means that
teachers in Argentina have less time to allocate to other activities such as grading and
preparing classes, what might reduce the quality of their job. The extremely low
137
Table 61. Teacher Characteristics
%
% Labor force (25-64 YO) of
Student - Teacher
% of
Teachers
teachers and other related staff
Ratio in Primary % Teachers Between 30 Teachers
in Primary and Secondary
School
over 60
Under 30
and 60
Education
20.73
30.29
58.72
10.06
4.4
33.43
8.72
60.94
25.52
2.1
23.41
15.08
51.60
28.80
2.7
17.42
13.83
60.93
24.62
3.0
19.62
12.64
66.30
20.86
3.3
10.89
0.00
4.6
11.33
4.70
66.69
24.69
3.6
0.00
15.45
0.00
3.8
22.45
20.51
56.95
21.82
3.1
Argentina
Chile
Czech Rep.
Finland
France
Hungary
Italy
Poland
Spain
UK
Starting
salary
/minimum
training
Salary after
15 years' Salary at top
of scale
experience
/minimum /minimum
training
training
Argentina
8906
Chile
9067
Czech Rep.
6,806
Hungary
5,763
Mexico
10,465
Australia
25,661
England
19,999
Finland
18,110
France
19,761
Italy
19,188
Norway
22,194
New Zealand
16,678
Spain
24,464
Sweden
18,581
United States
25,707
OECD mean
20,358
Source: OECD/UNESCO WEI
12377
10476
9,032
8,252
13,294
36,971
33,540
24,799
26,599
23,137
25,854
32,573
28,614
24,364
34,705
27,597
14697
14043
12,103
11,105
22,345
37,502
33,540
25,615
39,271
28,038
27,453
32,573
37,317
n.a
43,094
33,752
Ratio of
starting
salary to
GDP per
capita
0.8
1.1
0.5
0.5
1.2
1.0
0.9
0.8
0.9
0.9
0.8
0.9
1.3
0.8
0.8
1.0
Ratio of
Ratio of salary salary after
after 15 years'
15 years'
Years
experience experience to from
starting
(min. train.) to
starting to
salary
GDP per capita
top salary
1.1
1.2
0.7
0.7
1.5
1.5
1.5
1.1
1.2
1.0
0.9
1.8
1.6
1.1
1.0
1.3
1.4
1.2
1.3
1.4
1.3
1.4
1.7
1.4
1.3
1.2
1.2
2.0
1.2
1.3
1.4
1.4
21-24
30
32
40
11
9
9
20
34
35
28
8
42
n.a
30
25
X.3. Recent Evolution
It seems that the teaching activity has lost social recognition in the last 25 years. In
addition, now women have better access to the labor market and to higher education than
25 years ago. Both factors should have affected the composition of teachers. If teaching is
a less attractive option, we should observe that those who can exercise the option would
not opt for teaching.
To investigate this hypothesis we analyze who are the teachers using the EPH household
survey for the Great Buenos Aires region for 1980 and 2006, which is the only region
that we can track back in time up to 1980 (the EPH for other regions started in the 90s).
138
We focus on the characteristics of the teacher husband’s since the teacher characteristics
are endogenous. Under the assumption that there is assortative matching (higher
socioeconomic level women match higher socioeconomic level men, or in a more drastic
characterization more able women match with more able men) we can induce who are
choosing to be teachers today compared to 27 years ago based on their husband
characteristics.
According to the occupation code we can identify who is working in education as the
main job, but we do not know if they are working in primary and secondary level or
tertiary and university level. Since we are interested in the former, we restrict our analysis
to those workers who do not have a university degree. This degree is necessary to teach at
tertiary or university level, so restricting our sample we eliminate higher education
professors. It could be also the case that some university level workers are teaching
secondary level schools (particularly private schools), although most likely teaching
would not be their main job and therefore the sample bias should be small.
Husband’s income Distribution
We first analyze the position of the teacher’s husband according to income deciles. The
deciles are computed for the entire population. Then we characterize each teacher
according to her husband’s income. If her husband is in the second decile we impute this
decile to the teacher. Next we compute the accumulated distribution for teachers
according to their decile. The results are shown in the next figure.
In 1980, 12.4% of teachers are in the top 4 deciles according to her husband’s income
level, in 2006 only 9.1% of the teachers were in these deciles. For the overall distribution,
the 1980 distribution (first order) stochastically dominates the 2006 distribution what
means the teachers’ husbands are not worse.
139
Figure 34. Accumulated Distribution of teachers by deciles
according to husband's income
GBA Region
100%
90%
1980
2006
Accumulated Frecuency
80%
70%
60%
50%
40%
30%
20%
10%
0%
1
2
3
4
5
6
7
8
9
10
Decile according to individual income level (entire population)
Comparing 1980 with 2006 we found that husband’s education is much higher now. This
is not related to the overall increase in years of schooling at country level. The proportion
of persons from the total population in the GBA region with at most complete secondary
level fell between 1980 and 2006 by 14.5%, whereas the proportion of teachers’
husbands with at most complete secondary fell 49.2%. The increase in education for
teacher’s husband is important among public and private schools.
Figure 35. Teacher Distribution according to husband educational level
GBA
140
100%
90%
1980
2006 public schools
2006 private schools
Accumulated Frecueny
80%
70%
60%
50%
40%
30%
20%
10%
0%
Sin educación
Primaria Incomp Primaria Comp
Secundaria
Incomp
Secundaria
Comp
Universitaria
Incomp
Universitaria
Comp
Husband's Educational Level
This shows a pattern seems to be in contradiction of the hypothesis that teaching activity
has today a relatively worse position. To asses whether the growth of the private sector in
this city is affecting our results we analyze the composition differentiating between
private and public school teachers. We do not have information regarding the public or
private sector affiliation for 1980, but we have that information for 2006. Since in 1980
most of the teachers were in public schools, we compared 2006 distribution across public
and private schools independently with the 1980 distribution, what is shown in the next
figure. The results show that there is an important difference between public and private
school teachers, and the effect found earlier was due entirely to the increase in the private
school share, since private school teachers belong mostly to top income deciles. When we
compare public school teachers in 2006 with teachers in 1980 we found that the first
distribution (first order) stochastically dominates the second one, what means that public
school teachers’ husbands are now more biased to low income levels, what shows a
change in composition in line with the hypothesis of a downward change.
141
Figure 36. Accumulated Distribution of teachers by deciles
according to husband's income for private and public schools
GBA Region
Accumulated Distribution of teachers by deciles according to husband's incomel GBA Region
100%
Public 2006
90%
Private 2006
Total 1980
80%
Accumulated Frecuency
70%
60%
50%
40%
30%
20%
10%
0%
1
2
3
4
5
6
7
8
9
10
Decile according to individual income level (entire population)
Are public school teachers similar to private school teachers?
Private school teachers’ husbands are mostly highly educated workers, 45% of them have
a university degree level, whereas among public school teachers only 37% of the
husbands finished this level. Something similar happens with the income level, whereas
80% of private school teachers’ husbands are in the top 3 deciles, this ratio is just 66%
for public teachers. Surprisingly there are not large differences in term of teacher
observable characteristics such as gender, age or experience. There are neither
statistically significant differences between average wages, although the teachers in
private schools, since they usually have a tuition waiver for their son and daughters, have
a much higher real wage, what could even double or triple, depending on how expensive
is the private school and how many siblings have.
142
XI. Conclusions and Policy Implications
Argentina is definitely ranking very bad in terms of quality of education. It has an
average score or a SES adjusted average score which is well below the expected level for
its income level. In addition the distribution of the quality according to the student SES is
very unequally distributed. Part of this inequality is caused by the high inequality of its
income distribution, in terms of SES adjusted scores, although shows high inequality, it is
not anymore in the top of the ranking.
It seems that the quality of education has a declining trend. This trend is perceived even
in the very short period of time of the last 10 years. If we compare Chile with Argentina,
Chile was able to improve between 1997 (LAB) and 2006 (PISA), but Argentina has
been loosing positions.
The explanation for this poor performance is in part explained by lack of resources
allocated to education, as our econometrics results and the benchmark analysis show, but
this is not the only explanation. Argentina spends in a relatively similar way in terms of
its GDP than countries with similar income level, but it has more students to educate
since its population is relatively younger. Per student, thus, Argentina is spending less
than other similar income level countries. Argentina has lower endowments in almost
every dimension: computer per students, educational material, quality of the
infrastructure, etc., but also the slope or the efficiency in which those resources are used
is in general lower than comparable countries.
We find that in addition to the lack of investment in this sector there are several factors
showing that the Argentine educational system is very inefficient. We find differences in
both, in how much effort do the parents and how the school is working. For instance, we
find that in Argentina parents buy less child books and read less to their kids than in
similar income level countries outperforming Argentina, what helps to explain the gap.
143
This might show a lower demand for quality of education from the parents point of view,
therefore it is not surprising that the state is also spending less per student.
In terms of the variables related to how teachers teach, we find that in Argentina there is
not enough effort to increase the performance of those students that are falling behind in
terms of learning. It is not common in Argentina to have a reading specialists, to have
different material for students with problems, or the teacher to educate in a differentiated
way, giving different reading, or trying to motivate low performers. In Argentina the
policy seems to be to make the student repeat the grade, what might not be the most
efficient way to accumulate human capital. Argentina should study the trade offs between
the cost per student that repeat vs. the cost of allocating more resources at the school
level in order to avoid the repetition (for instance, generalizing the availability of reading
specialists). Part of this could be explained by the teacher wage, teachers need to work
more hours to match international standards in terms of monthly salaries, what leaves the
teacher with less time to spend preparing classes, grading homework, or to have a more
effort-intensive teaching. But also we find that teachers in Argentina are lower qualified
than in other comparable countries. The high qualified teacher produces in a relatively
similar way than in these countries, but the large difference is in the stock of highly
qualified teachers that Argentina has. The data shows that Argentina does not need more
teachers, but highly qualified teachers, motivated and involved with the teaching process.
The Eastern European countries analyzed in this study as benchmark economies have a
system which is more stressful, since the performance of the students in the early stages
conditions their posterior possibilities (for instance, having access to Academic
Secondary schools and therefore to university degree). This higher stress might be
reflected in more effort from the parents and the students (a higher demand for quality of
education). On the contrary, free access to all the levels in Argentina might be seen as
more flexible, or equalizing the opportunities more, but the fact is that if the quality of
education in the early stages is poor, the students are self-restricted even if there is free
access. Low SES students in Argentina have a score gap much larger than high SES
students when compare to similar income level countries, particularly in primary school,
144
what might imply that low SES students, even with free access, do not have equal
opportunities.
In terms of policies, there are several things Argentina can do to improve the situation,
and this is something the specialists know, but first the country has to convince itself that
education is a priority, in a country led by the short-run needs imposed by the recurrent
crisis.
Among the policies, this study provide some favorable evidence in terms of: improving
the qualification of teachers, making the teacher instruction process a university degree,
improving the hourly wage so they do not need to work so many hours, increasing the
ratio of teachers actually teaching, spending resources in the less advantageous students
in order to improve their performance (such as having a reading specialist in every school
with a high proportion of low performers, or motivating the teachers to have a more
personalized teaching).
Argentina needs to involve the families in the process, and needs to develop a system
specifically designed to deal with, on average, less disadvantageous students. Argentina
is now in a vicious circle of declining quality and a very unequal system, what means that
the students with low SES are condemned to have a poor quality of education, and
aggravating the future problem. Argentina needs to break this cycle allocating enough
resources and effort to educate its future labor force properly, something the country has
not been doing for several years.
145
References
Aitkin, M. & N. Longford (1986) “Statistical modelling in school Effectiveness studies.”
Journal of the Royal Statistical Society, A., 149: 1-43.
Altonji, Joseph G. and Thomas A. Dunn (1996) “Using Siblings to Estimate the Effect of
School Quality on Wages” The Review of Economics and Statistics, Vol. 78, No. 4.
(Nov., 1996), pp. 665-671.
Angrist; Joshua D. and Victor Lavy (1999) “Using Maimonides' Rule to Estimate the
Effect of Class Size on Scholastic Achievement”, The Quarterly Journal of Economics,
Vol. 114, No. 2. (May, 1999), pp. 533-575.
Arredondo, Inés ed. (2006) Educación en la Argentina. Qué pasó en los 90s. Colección
Educación, Papers Editores.
Auguste, Sebastian (2004) "Product Differentiation and School Competition", mimeo,
2005.
Auguste, Sebastian (2004) "School Entry and Coordination Failures in a Voucher
System”, mimeo, 2004.
Auguste, Sebastian (2004) “An Empirical Model of School Formation. Evidence from
Chile”, mimeo, 2004.
Auguste, Sebastian and Juan Pablo Valenzuela (2006) “Is It Just Cream-Skimming?
School Vouchers in Chile” mimeo.
Barro, R. (1991) "Economic Growth in a Cross Section of Countries." Quarterly Journal
of Economics 106(2), pp. 407-44.
Benhabib, J. and Spiegel, M. (1994) “The Role of Human Capital in Economic
Development: Evidence from Aggregate Cross-Country Data”, Journal of Monetary
Economics, 34 (2), pp. 143-173.
Berry Cullen, Julie; Brian A. Jacob and Steven D. Levitt; (2006) "The Effect of School
Choice on Participants: Evidence from Randomized Lotteries." Econometrica, 2006,
74(5), pp. 1191-230.
Bratsberg, Bernt, and Dek Terrell (2002), “School Quality and Returns to Education of
U.S. Immigrants”, Economic Inquiry 40(2), pp. 177-198.
Bryk, A. S. & Raudenbush, S. W. (1992). Hierarchical Linear Models. Newbury Park,
California. Sage.
Caucutt, E., 2002. Educational Vouchers when there are Peer Group Effects. Size
Matters. International Economic Review 43(1), pp. 195-222.
146
Coleman, J., Campbell, E., Hobson, C., McPartland, J., Mood, A., Weinfield, F. & York,
R. (1966). Equality of educational opportunity. Washington, DC, US Government
Printing Office.
Elchanan Cohn, Richard A. Rossmiller (1987) “Research on Effective Schools:
Implications for Less Developed Countries”, Comparative Education Review, Vol. 31,
No. 3, pp. 377-399 Goldstein H. Multilevel Statistical Models. Kendall’s Library of
Statistics 3
Epple D., Figlio, D., Romano, R., 2004. Competition between Private and Public
Schools: Testing Stratification and Pricing Predictions. Journal of Public Economics
88(7), pp. 1215-1245.
Epple, D., Romano, R., 1998. Competition between Private and Public Schools,
Vouchers, and Peer-Group Effects. American Economic Review, 88 (1), pp 33-62.
FIEL (2006), Las Familias Argentinas y sus Decisiones de Consumo, Ahorro e Inversión
en Capital Humano, Manantial editores.
FIEL/PREAL (2004) “¿Las Políticas Compensatorias Mejoran la Equidad? Argentina:
Plan Social Educativo. Educación Primaria.” (“Do Social Policies Impact on Schooling
Inequality? Agentina: Educational Social Plan. Elementary School.” FIEL, PREAL,
Buenos Aires, Julio de 2004.
Filmer, D., Pritchett, L., (1999). “What education production functions really show: A
positive theory of education expenditures”. Economics of Education Review, 18(2), pp.
223-39
Friedman, M., (1962). Capitalism and Freedom. The University of Chicago Press.
Galiani S and Shargrodsky E. “Evaluating the Impact of School Decentralization on
Educational Quality”, Economía LACEA (2002), Vol. 2 (2) pp. 275-314
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item
Response Theory. Newbury Park, CA: Sage Press
Hanushek Eric A. and Jin-Yeong Kim (1999) “School Quality around the World:
Evidence from U.S. Immigrant Earnings”
Hanushek, Eric A. (1979) “Conceptual and Empirical Issues in the Estimation of
Educational Production Functions”, The Journal of Human Resources, vol 14 No. 13,
Summer 1979, pp. 351-388,
Hanushek, Eric A. (1995) “Interpreting Recent Research on Schooling in Developing
Countries”. World Bank Research Observer, 10(2).
Hanushek, Eric A. (1998) “Conclusions and Controversies about the Effectiveness of
School Resources”, FRB New York Economic Policy Review 4 (1), pp. 11-27.
147
Hanushek, Eric A. and Ludger Woessmann (2007). The Role of Education Quality in
Economic Growth. World Bank Policy Research Working Paper, No. 4122.
Hill, P.W. and Rowe, K. J. (1996) “Multilevel modelling in school effectiveness
research” School Effectiveness and School Improvement, 7, pp. 1-34.
Jencks, C., Smith, M., Ackland, H., Bane, M., Cohen, D. Gintis, H., Heyns, B. and
Michelson, S. (1972). Inequality: A reassessment of the effects of family and schooling in
America. New York: Basic Books.
Lazear Edward P. (2001) “Educational Production” The Quarterly Journal of
Economics, Vol. 116, No. 3. (Aug., 2001), pp. 777-803.
Lockheed, ME & Hanushek, E (1988) Improving educational efficiency in developing
countries: what do we know? Compare 18,1,21-38
Loeb, S., and Bound, J., 1996. “The Effect of Measured School Inputs on Academic
Achievement: Evidence from the 1920s, 1930s and 1940 birth Cohorts”. The Review of
Economics and Statistics, 78(4), pp. 653-664.
Maddison, Angus (2001). The World Economy: A Millennial Perspective. OECD
Development Centre.
McEwan, Patrick J. (2003) "Peer Effects on Student Achievement: Evidence from Chile,"
Economics of Education Review 22, 131-141.
Mortimore, P., Sammons, P., Stoll, L., Lewis, D. & Ecob, R. (1988), School Matters.
Wells, Open Books.
Mullis, I.V.S., M.O. Martin, E.J. Gonzalez and A.M. Kennedy (2003). PIRLS 2001
International Report: IEA’s Study of Reading Literacy Achievement in Primary Schools.
Chesnut Hill, MA: Boston College.
Organisation for Economic Co-operation and Development (2001). Knowledge and skills
for life: First results from the OECD programme for international student assessment
(PISA) 2000. Paris: OECD.
PISA (2005) School Factors Related To Quality And Equity. Results from PISA 2000,
OECD.
Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests.
(Copenhagen, Danish Institute for Educational Research), expanded edition (1980) with
foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.
Sala- i-Martin, X. (1997) “I Just Ran Four Million Regressions” Working Paper No.
6252, National Bureau of Economic Research, November 1997.
Scheerens, J. (1992). Effective Schooling: research, theory and practice. London, Cassell.
148
Scheerens, J. (1993) “School effectiveness in developing countries - A review of the
research evidence”, Education Research Paper No. 01, 1993, 25 p.
Scheerens, J. (2000) “Improving School Effectiveness.” Fundamentals of Educational
Planning N68, United Nations Educational, Scientific, and Cultural Organization, Paris
(France).
Willms Douglas (2006) “Learning Divides: Ten Policy Questions about the Performance
and equity of schools and schooling systems” UIS Working Paper No. 5, UNESCO
Institute for Statistics.
Willms, J.D. and S.W. Raudenbush (1989). “A longitudinal hierarchical linear model for
estimating school effects and their stability.” Journal of Educational Measurement,
26(3): 209-232.
Wiñar, D. (1974) “Aspectos sociales del desarrollo educativo Argentino, 1900-1970”,
Revista del Centro de Estudios Educativos vol 4, México.
149