EXECUTIVE SUMMARY
Student achievement in
Latin America and the Caribbean
Student achievement in
Latin America and the Caribbean
Results of the Second Regional Comparative
and Explanatory Study (SERCE)
organización de las naciones unidas para la educación, la ciencia y la cultura
united nations educational, scientific and cultural organization
organisation des nations unies pour l’éducation, la science et la culture
Regional Bureau for Education in Latin América and the Caribbean
Published by the Regional Bureau for Education in Latin America Latina and the Caribbean OREALC/UNESCO Santiago.
LLECE Team
Héctor Valdés (Coordinator), Ernesto Treviño, Carmen Gloria Acevedo, Mauricio Castro, Sandra Carrillo, Roy Costilla,
Daniel Bogoya, Carlos Pardo.
Thematic Areas
Beatriz Macedo, Liliana Bronzina and Ana Atorresi.
Administrative Staff
Silvia Ortiz
Our special thanks go to Rosa Blanco and Ana Luiza Machado, a.i. Director and former Director of OREALC/UNESCO
Santiago, respectively, and to all LLECE members who collaborated to make SERCE possible. We are particularly indebted
to Javier Murillo and Marcela Román for their help in drafting the preliminary versions of this report.
Design and Lay-out
Ana María Baraona, Ximena Milosevic, Julia Salazar and Alejandro Urbán
English Translation
Ernesto Leigh
The members of the working team are responsible for the contents of this report. The opinions expressed herein
are theirs alone and not necessarily those of UNESCO.
The place names and maps used in this publication do not imply on the part of UNESCO any opinion or position in
regard to the legal status of countries, cities, territories, or zones; nor regarding their authorities or the drawing of
their borders. This publication may be reproduced in its entirety or in part provided that explicit reference always
be made to the source.
ISBN: 978-956-8302-94-8
Santiago, Chile. June, 2008
Table of contents
PRESENTATION
7
SECOND REGIONAL COMPARATIVE AND EXPLANATORY STUDY
8
STUDENT ACHIEVEMENT
12
FACTORS ASSOCIATED WITH ACHIEVEMENT
45
FINAL REFLECTIONS
47
UNESCO has been called upon to generate, through its mandated fields of action, conditions
that guarantee individuals and communities the benefits of a genuine peace and opportunity
for development. Considering that poverty and inequality in the region continue to pose an important threat to the dignity and safety of the population, the international community should
adopt a humanised vision of development based on respect for human rights, intercultural
dialogue and the pursuit of justice. In the field of education, UNESCO has embraced three major
objectives, namely: promoting education as a fundamental human right; furthering educational
quality and innovation; and generating knowledge to inform educational policy-making.
In recent years, the Latin American and Caribbean countries have made important inroads
in the region in terms of expanding compulsory education and increasing the system’s coverage, designing new curricula, improving the provision of didactic materials and strengthening
school infrastructure, all of this accompanied by substantial investments in teacher training
initiatives. Nevertheless, quality of education and its equitable distribution across social groups
still remains an unresolved issue.
UNESCO’s Regional Bureau for Education in Latin America and the Caribbean proposes, from
a human-rights approach, a concept of quality that integrates five dimensions: relevance, fostering learning that takes into account the developmental needs of individuals and societies;
pertinence, the need for education to be meaningful for people of different social and cultural
strata; equity, giving to all persons the aid and support that will guarantee equal opportunity to
access and complete their education, and fully develop their potential; efficacy, ensuring that
relevance, pertinence and equity-related goals translate into concrete actions; and efficiency,
the proper assignation and use of resources in the quest of the proposed objectives.
One of the central activities of the Regional Bureau is generating and disseminating
knowledge to inform decision-making on initiatives that promote educational policies and
practices aimed at strengthening the quality of education in the various countries. Within this
framework, the Latin American Laboratory for Assessment of the Quality of Education (LLECE),
founded in Mexico City in 1994, and under the co-ordination of OREALC/UNESCO Santiago,
represents a regional network of education evaluation systems committed to provide technical
support to the countries of the region. The LLECE launched its First Regional Comparative and
Explanatory Study (PERCE) during the 1995 - 1997 period releasing its results in December
1998. Subsequently, seven countries participated in a qualitative research study of the highperforming schools identified in the First Study. The major findings of SERCE’s Second Regional
Comparative and Explanatory Study (SERCE, 2002-2008), are presented in this report. We hope
they will facilitate decision-making and foster the implementation of educational policies and
practices that make possible a faster and more assertive transition to quality education without exclusion in the region. Rosa Blanco
a.i. Director,
Regional Bureau for Education in Latin America and the Caribbean
UNESCO Santiago
Executive summary
Presentation
Improving the quality of education is still the major challenge confronted by the education
systems of Latin America and the Caribbean. Governments strive to implement policies
that foster quality education, ensure it is made available to all, and equitably distributed,
in an attempt to break away from the social determinism that keeps the lower income
sectors –and the minorities within them– at a permanent disadvantage.
The information generated by evaluations on educational quality of national education
systems has allowed the technical and political authorities to review and analyse what is
being taught, how is being taught and, obviously, what are primary school girls and boys
learning in Latin American and Caribbean schools.
In late 2002, member countries of UNESCO’s Latin American Laboratory for Assessment
of the Quality of Education (LLECE) led by the OREALC Santiago Office, launched the Second
Regional Comparative and Explanatory Study (SERCE) which, drawing on the experience and
lessons learnt in a first such study (PERCE, 1998), took relevant steps aimed at expanding the
analysis so as to include a higher number of countries, grades and areas in its evaluations.
The main objective of the SERCE is to gather valid, accurate, and reliable data on what
are primary students actually learning, as well as relevant information on associated factors. The extent to which these results are discussed and integrated into educational and
social actions/policies aimed at enhancing and strengthening the quality of public education in participating countries, will provide a measure of its success.
This document summarizes SERCE’s process and application, its findings and results. It
offers an outline of its purposes, the conceptual perspective used in evaluating performance
in the areas of Primary Education Mathematics, Reading and Science of students who during
the period 2005 /2006 attended third and sixth grades, major factors associated with these
results, and their implications and recommendations to social and educational policies.
The SERCE is the result of the effort and commitment of many teams, organisations
and national and regional authorities. We owe a special debt of gratitude to the persons
who headed the Organisation during the various stages of this Study, namely, Ana Luiza
Machado former OREALC/UNESCO Director and Rosa Blanco a. i. Director of the Organisation; to the World Bank, the Inter-American Development Bank, and Ford Foundation,
major donors, for their support in each one of the phases; to National LLECE Coordinators;
and to the national delegates and their teams. Our most sincere thanks to the principals,
teachers, fathers and mothers, boys and girls of participating schools, without whose
collaboration and commitment this research would have not been possible. These are
ultimately the main actors and beneficiaries of SERCE’s findings.
Based on the school calendar of the surveyed countries.
Executive summary
Second Regional Comparative and
Explanatory Study
The Second Regional Comparative and Explanatory Study (SERCE) represents the most
important and ambitious student performance evaluation project ever launched in Latin
America and the Caribbean. Under the direction and coordination of the Laboratory for
Assessment of the Quality of Education (LLECE), this Study forms part of UNESCO Regional Bureau for Education in Latin America and the Caribbean (OREALC/UNESCO Santiago)
global actions aimed at guaranteeing the right to quality education to all Latin American
and Caribbean students.
Its objective is to give insight into the learning acquired by Latin American and
Caribbean Third and Sixth Grade Primary Students in the areas of Mathematics, Language
(Reading and Writing) and Natural Science during their school trajectory.
Student achievement in Latin America and the Caribbean
In addition to identifying what girls and boys have learned, the results obtained are
analysed and explained keyed to factors related to students, classrooms, and schools, placing special emphasis on those factors that may be changed through the implementation of
relevant programmes and policies.
SERCE represents a collective effort on the part of participating countries, duly articulated by a central LLECE team, and supported by a Technical Advisory Committee and
panels of experts in every area. The Study was launched in February 2004 and its main
stages will extend through the second semester of 2008.
Graph 1
Entities participating in SERCE
Sixteen countries and the Mexican State of Nuevo Leon are taking part in this survey.
Third and Sixth Grade Primary Students of all participating countries were evaluated in
Mathematics and Language, while sixth graders of nine countries and the State of Nuevo
Leon were evaluated in Natural Science. A total of 3.065 schools – encompassing 4.627
Executive summary
Third Grade classrooms and 4.227 Sixth Grade classrooms – were surveyed. This represents
a total of 100.752 Third Grade and 95.288 Sixth Grade Primary school students. This
sample is representative of approximately eleven million third graders and ten million
sixth graders in the region.
In terms of evaluating performance and associated factors, SERCE uses a set of instruments specially designed for this purpose.
Each of the evaluated students took the Mathematics, Reading and Science tests on
different days and was allotted a time consistent with the nature of the tests.
Contextual, socio-demographic, family and personal data, in addition to information associated with school processes and dynamics, were captured through the direct administration of questionnaires to students, teachers, principals, and parents of the sampled schools.
The objectives of each of these instruments are summarised in the following table.
Table 1
Actor
Instrument
Objective
Students
Student
Questionnaire
Inquire about the family and socio-cultural environment,
classroom dynamics and interaction, degree of
satisfaction with the school, classmates and teachers,
among other topics.
Teachers
Teacher
Questionnaire
Inquire about socio-demographic aspects, professional
training, labour conditions, teaching experience and degree
of satisfaction with the school, among other topics.
Questionnaire on
teaching practices
Look into pedagogical practices at the corresponding
grade and area, such as time management, availability of
educational resources, expectations teacher form of their
students, types of activities, curricular implementation,
evaluation strategies, among other topics.
Questionnaire for
Principals
Capture data relative to personals traits, professional
profile and trajectory, management model adopted,
expectations, degree of satisfaction with the school and
co-workers, in addition to other aspects of school life.
Questionnaire
on School
Characteristics
Collect information on school location, equipment and
infrastructure.
Family
Questionnaire
Inquire about the socio-demographic characteristics
of the family, the availability of services and physical
amenities in the home, involvement in and support
of the educational process of their children, degree of
satisfaction with the school, among other aspects.
Principals
Parents
10
Synthesis of SERCE’s data collection instruments
Student sample data correspond to the total number of students who took at least one of the tests. This total
differs from the total number of students evaluated in each area.
Student achievement in Latin America and the Caribbean
The Study shows student performance results from two different perspectives.
• On the one hand, it presents mean scores of students’ and their variability by
country, areas and grades. Also, the relationship between average scores, national
per capita income and Gini Index, for each country.
• On the other, it shows results based on student distribution at each national level
of performance. This information gives a clear idea of the percentage of students
who have similar performance profiles in each country.
The First Report on SERCE’s Results includes a progress report on achievement-associated factors that provides a preliminary insight into the variables that have an impact on
student learning.
Executive summary
11
Student achievement
Evaluation of Learning - Approaches
In order to evaluate student performance, SERCE used tests based on curricular elements
known to be common to the region, fashioned after the life-skills approach propounded
by UNESCO.
The creation of a common and consensuated curricular framework for Latin America
and the Caribbean implied reviewing, systematising and analysing contents prescribed by
the curricula for the different areas to be evaluated in the region, in order to determine
12
Student achievement in Latin America and the Caribbean
which conceptual domains were common to the primary education students of participating countries.
The identification of common contents, the approaches used by participating countries
to evaluate their students’ performance, and the organisation of this performance, were
the criteria guiding the curricular analysis on which the elaboration of tests was based.
For its part, the life-skills approach establishes the abilities, principles, values and
attitudes that Latin American students should learn to develop in order to ensure their
full and active participation in society, both as actors and citizens. This means, dealing
with situations, making decisions based on available information, solving problems, and
supportting their points of view, among others.
Designing tests that inspired on a common curricular framework also place emphasis
on life-skills, challenges education to go beyond academic success by offering students
learning spaces that promote and ensure a better quality of personal and social life.
The tests administered by SERCE evaluate not only the knowledge acquired by Third
and Sixth Grade primary education students, but also how these students use – or are
capable of using – such knowledge to understand and interpret the world under various
daily-life circumstances and contexts.
The questions administered during the tests were distributed into six different booklets, thus ensuring coverage of all domains contained in the test reference framework.
The inclusion of open-ended questions, allowed students to construct their own responses and, based on that construction, the strategies used by the students to respond
could be inferred. This type of question also provided insight into the degree to which the
students have acquired attitudes, values and procedures, and developed their own ways
of thinking.
The questions asked vary substantially in terms of how the information is presented:
• Some questions present information as written texts, whereas in others, the information is contained in tables, narratives, graphs or drawings.
• Content is also presented in everyday contexts that are familiar to the students, as
a way of highlighting the functionality and usefulness of this learning.
In order to determine what do Latin American and Caribbean students know, two dimensions were conceived: conceptual domains or area-specific knowledge, and cognitive
processes, understood as the mental operations students use to establish relationships
with and among objects, situations and phenomena.
Guatemala joined the study after the curricular analysis had been completed. Therefore, there is no guarantee
that Guatemala’s curricular content will fully agree with the contents selected for the tests.
Executive summary
13
Table 2
Conceptual Domains and processes involved in each SERCE test
Area
Mathematics
Conceptual Domains
Numerical
Geometrical
Processes
•Recognition of elements and objects
•Solution of simple problems
•Solution of complex problems
Measurement-based
Information handling
skills
Variational
Reading
Length of the tested
text
Text type and genre
Natural science
Health and living beings
Earth and environment
•General processes
•Processes related to specific texts
•Metalinguistic processes
•Concept recognition
•Interpretation and application of concepts
•Problem solving
Matter and energy
The following are some examples of the items used:
14
Student achievement in Latin America and the Caribbean
Level I - 3rd Grade Mathematics
Example 1. Books sold per month
Summary card for example 1
3rd Grade
I
Information handling skills
Recognition of objects and elements
Interpreting direct information presented in a
bar graph
Key
A: January
Difficulty level
412.02
Percentage of correct responses
75.64%
Percentage of responses involving distractors
B: 9.01%
C: 6.16%
D: 6.02%
Percentage of non-valid responses
3.17%
Grade
Performance level
Domain
Process
Required action /task
Executive summary
15
Level IV - 3rd Grade Mathematics
Example 2. Number sequence
Summary card for example 2
3rd Grade
IV
Variational
Solving complex problems
Recognise an additive numerical sequence rule
by its definition
Key
C: 300 units were added each time
Difficulty level
629.06
Percentage of correct responses
30.45%
Percentage of responses involving distractors
A: 24.95%
B: 21.19%
D: 15.98%
Percentage of non-valid responses
7.43%
Grade
Performance level
Domain
Process
Required action /task
16
Student achievement in Latin America and the Caribbean
Level II - 6th Grade Reading
Example 3. The perfect horse
Summary card for example 3
6th Grade
II
Length: Complete text
Type of text and genre: Narrative; introduction-climaxresolution
Process
General: Identifying secondary information
Specific: Identifying “voices” in the narrative
Metalinguistic: None
Required action /task
Recognising a character’s attribute based on the saying of
a third party
Key
B: smart
Difficulty level
436.69
Percentage of correct responses
74.95%
Percentage of responses involving
A: 5.45%
distractors
C: 6.09%
D: 11.28%
Percentage of non-valid responses
2.23%
Grade
Performance level
Domain
Executive summary
17
Level IV- 6th Grade Reading
Example 4. Title and passages of a narrative
Summary card for example 4
6th Grade
IV
Length: A relatively lengthy text.
Type of text and genre: Explanatory/narrative: legend
Process
General: Associating a synthesis with that synthesised
Specific: Identifying which part of the narrative text is
synthesised in the title
Metalinguistic: Knowing the meaning of “title” and the
different names of the passages
Required action /task
Identifying which part of the text is synthesised in the
title, distinguishing them from other passages through the
use of metalanguage
Key
B: The conflict
Difficulty level
599.623
Percentage of correct responses
35.77%
Percentage of responses involving
A: 22.99%
distractors
C: 18.67%
D: 18.95%
Percentage of non-valid responses
3.62%
Grade
Performance level
Domain
18
Student achievement in Latin America and the Caribbean
Level II - 6th Grade Science
Example 5. Balanced breakfast
Summary card for example 5
6th Grade
II
Health and living beings
Recognising and applying concepts
The student should be able to recognise the concepts
involved and apply them to a familiar and daily situation
Key
A: Fruit, milk and bread
Difficulty level
495.60
Percentage of correct responses
56.18%
Percentage of responses involving
B: 31.29%
distractors
C: 5.73%
D: 5.77%
Percentage of non-valid responses
1.03%
Grade
Performance level
Domain
Process
Required action /task
Executive summary
19
Level IV - 6th Grade Science
Example 6. The Moon
Summary card for example 6
6th Grade
IV
Earth and environment
Problem Solving
Handling concepts related to the force of gravity and their
correct application to solve the problem at hand
Key
D: there is little gravitional force
Difficulty level
822.45
Percentage of correct responses
18.37%
Percentage of responses involving
A: 30.39%
distractors
B: 13.93%
C: 33.97%
Percentage of non-valid responses
3.34%
Grade
Performance level
Domain
Process
Required action /task
20
Student achievement in Latin America and the Caribbean
Presentation of Results
Results are presented, by grade and area, as follows:
• Average scores and variability for each of the countries, based on an arbitrary scale
with a mean of 500 and standard deviation of 100. This is meaningless in terms of
approving/not approving grade.
• Performance levels which classify students on the basis of what they are capable
of doing.
• Comparisons of students in urban and rural contexts, and gender-based analysis.
• Relationship between learning results, per capita gross domestic product of each
country and income distribution, using the Gini Index.
Learning in Third Grade
Mathematics
Mathematics results for Third Grade students reveal significant differences among
countries. Countries situated at the high and low ends of the performance scale are separated from each other by more than 250 points, equivalent to more than 2.5 standard
deviations. However, a comparison between the second and next to last countries reveals
a difference of approximately one standard deviation. This implies that there is greater
homogeneity among countries occupying mid-positions.
Based on a global analysis of results, countries may be classified in five groups according to their difference with the countries’ average:
• Countries that exhibit mean scores in Mathematics, significantly higher than the
regional average (more than one standard deviation). This, however, is only true
for Cuba.
• Countries that exhibit average scores higher than the regional average (but less
than one standard deviation): Chile, Costa Rica, Mexico and Uruguay, and the
Mexican State of Nuevo Leon.
• Countries matching the regional average, that is, cases where no statistically significant differences are evident. This group is comprised of Argentina, Brazil and
Colombia.
• Countries that exhibit mean scores in Third Grade Mathematics lower than the
regional average (less than one standard deviation): Guatemala, Ecuador, El Salvador, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic*.
*
Significant differences (5% error) based on a t test for median comparison.
Executive summary
21
At the regional level, the difference in performance in Third Grade Mathematics between
10 and 90th percentiles is 241 points, with extreme values of 165 points (Nicaragua) and
341 points (Cuba).
th
Graph 2
Mean and variability of Third Grade Mathematics scores in each
surveyed country
LAC total: Latin American and Caribbean countries’ total.
CILL: Confidence interval lower limit (a = 0.05).
CIUL: Confidence interval upper limit (a = 0.05).
Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in
each country. That is to say, the far-right segment of each bar represents the scores of students
in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability.
The white vertical line running through the centre of each bar identifies the mean, while the
confidence interval is shown as a dark area around it. The width of this darkened area illustrates
its possible values.
Along with Cuba, Paraguay and Brazil exhibit the greatest differences between their
10 and 90th percentiles, with 258 and 245 points, respectively.
For their part, Panama, El Salvador and Guatemala exhibit the smallest differences
(fluctuating around 180 points) between their 10th and 90th percentiles.
th
22
Student achievement in Latin America and the Caribbean
Table 3
Level
Cut-off
score
IV
Description of third grade mathematics performance level and
percentage of students occupying each level
% Students
Description
11.23%
•Students recognise a numerical sequence rule and identify it.
•Students solve multiplication problems with one unknown
or problems which require the use of equivalences between
commonly used measures of length.
•Students identify an element on a bi-dimensional plane and the
properties of the sides of a square or rectangle in order to solve a
problem.
14.30%
•Students solve multiplication problems or problems which require
the use of an addition equation or two separate operations.
•Students solve addition problems involving measurement units
and their equivalences or problems which require using common
fractions.
•Students must identify the graphic or addition numerical sequence rule being used in order to continue it.
•Students identify the elements of unusual geometrical shapes and
interpret different types of graphs in order to retrieve information
and solve problems that involve operating with the data.
28.26%
•Students recognise the organisation of the decimal-positional numeral system and identify the constituent elements of geometrical
shapes.
•Students identify a trajectory on a plane and the most suitable
measurement unit or instrument, in order to measure a known
object’s attribute.
•Students interpret tables and charts in order to obtain information and compare data.
•Students solve addition or multiplication problems involving
proportional relationships, using natural numbers.
36.03%
•Students recognise the relationship between natural numbers and
common bi-dimensional geometric shapes in simple drawings.
•Students locate relative positions of an object in a spatial representation.
•Students interpret tables and graphs in order to obtain direct
information.
10.19%
•Students at this level have not been able to acquire the abilities
required in Level I.
621.68
III
558.54
II
489.01
I
391.50
Below I
As shown in Table 3, 10.2% of all students are not capable of completing the tasks designed for the lowest level. This group of girls and boys – which total over a million for all the
countries surveyed – demands urgent and appropriate help given their low levels of learning.
Table 4 shows student distribution by performance levels for each participating country.
Cuba exhibits the highest performance levels, with 54.36% of its students occupying
Level IV.
In Chile, Costa Rica, Mexico, Uruguay and Nuevo Leon, over a third of their students
occupy Levels III and IV.
Executive summary
23
In Brazil and Argentina one fourth of their students occupy Levels III and IV. In the
rest of the countries less than one fourth of the surveyed students placed at these levels.
In the case of the Dominican Republic, 41.28% of the country’s students are below
Level I, a figure that is considerably lower (between 14% and 16%) for students of Ecuador, Panama, Paraguay and Peru.
Table 4
Percentage of Third Grade students by performance level in
Mathematics in each surveyed country
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Below I
10.46
10.32
5.10
8.57
2.62
1.09
14.34
10.31
17.34
5.15
12.10
15.98
15.87
15.24
41.28
5.78
2.34
10.19
I
32.77
36.55
27.90
38.60
24.44
10.19
45.48
45.00
50.06
28.85
47.95
49.69
37.88
45.42
49.27
25.95
18.45
36.03
II
31.13
26.74
33.60
33.19
37.00
16.95
28.12
31.80
25.07
30.70
30.50
25.15
25.50
25.95
8.49
30.03
31.69
28.26
III
15.17
14.32
19.37
12.97
22.30
17.41
7.91
9.25
5.46
19.71
7.49
6.42
11.56
8.61
0.84
19.29
24.41
14.30
IV
10.47
12.07
14.02
6.67
13.65
54.36
4.14
3.64
2.08
15.59
1.97
2.75
9.20
4.77
0.13
18.95
23.11
11.23
Note: Below I students are those who cannot attain level I.
School location is also responsible for the differences in student performance levels
observed in the region. Table 5 shows that Latin American and Caribbean girls and boys
attending rural schools perform at lower levels when compared to their counterparts attending urban schools4. In this sense, Peru, Brazil, and Mexico exhibit the largest urbanrural gaps. Cuba, Nicaragua, and Paraguay do not reveal statistically significant differences
in the averages obtained by urban and rural students.
4
24
The definition of “rural area” is not exactly comparable among countries. The identification of
rural schools was based on the definition provided by each country. Consequently, totals for Latin
America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Student achievement in Latin America and the Caribbean
Table 5
Average score differences between urban and rural schools, and
by gender. Third Grade Mathematics
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Urban/ Rural Difference
40.09*
62.17*
33.29*
26.29*
29.25*
7.79
20.70*
39.92*
40.07*
43.01*
-1.15
22.41*
17.91
69.88*
17.60*
31.72*
28.68*
-
Girl/ Boy Difference
-1.42
1.91
-13.37*
-8.26*
-10.80*
4.47
0.55
-10.90*
-6.98*
0.09
-12.72*
5.53
2.31
-9.20*
12.66*
0.28
-3.92
-1.25
* Significant at a 5% confidence level.
In terms of gender, at the regional level, median scores for Third Grade Mathematics
do not reveal significant differences. However, this overall result conceals important differences among countries:
• In Argentina, Brazil, Cuba, Ecuador, Mexico, Panama, Paraguay, Uruguay and the
Mexican State of Nuevo Leon, gender-based differences are not significant.
• Chile, Colombia, Costa Rica, El Salvador, Guatemala, Nicaragua and Peru exhibit
significant differences which would seem to indicate that boys outperform girls in
Mathematics.
• Exceptionally, in the Dominican Republic the opposite is true.
Analyses of the existing relationship between performance and gross national product
and income distribution reveal interesting differences among countries.
There is a correlation between student average score in Third Grade Mathematics and
their national per capita GDP. In fact, this economic indicator accounts for 28.37% of the
countries’ average performance variance.
The relationship between results and the Gini Index –as income distribution indicator– is equally significant, although inverse. In other words, the greater the inequality
Executive summary
25
the lower the results obtained in Third Grade Mathematics. The Gini Index can account for
17.06% of the countries’ average performance variance in Mathematics.
Reading
As with Mathematics, important differences are evident among participating countries.
Thus, the difference between the countries with highest and lowest performances is 2.3
standard deviations, that is to say, about 230 points. However, the difference between the
second and next to last country is 1.15 standard deviations, which reveals a somewhat
more homogeneous distribution of results.
Based on the high disparity and internal dispersion of national averages, five groups of
countries were identified relative to the average performance of their students.
• Countries where average performance is markedly higher than the median of SERCE
participants by more than one standard deviation. This case is illustrated by
Cuba.
• Countries where average performance is higher than the average of SERCE participants by less than one standard deviation. This group is comprised of Argentina,
Chile, Colombia, Costa Rica, Mexico, Uruguay, and the Mexican State of Nuevo
Leon.
• Countries where performance exhibits a mean score statistically identical to the
regional average: Brazil and El Salvador.
• Countries where performance exhibits a lower score than the average of SERCE
participants of less than one standard deviation. This is the case of Ecuador, Guatemala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic*.
* Significant differences (5% error) based on a t test for median comparison.
26
Student achievement in Latin America and the Caribbean
Graph 3mean and variability of Third Grade Reading scores in each
surveyed country
LAC total: Latin American and Caribbean countries’ total.
CILL: Confidence interval lower limit (a = 0.05).
CIUL: Confidence interval upper limit (a = 0.05).
Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in
each country. That is to say, the far-right segment of each bar represents the scores of students
in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability.
The white vertical line running through the centre of each bar identifies the mean, while the
confidence interval is shown as a dark area around it. The width of this darkened area illustrates
its possible values.
In terms of Third Grade Reading tests, performance differences between percentile
10th and percentile 90th students in each country fluctuate between 208 and 242 points,
with the exceptions of Cuba and Nicaragua.
• Cuba shows the greatest dispersion of results, since the distance between students
of the percentiles under comparison is 295 points. However, the lower performing
Cuban students obtain scores that are similar to the countries’ average.
• For its part, Nicaragua shows a low dispersion of results with differences between
students at both extremes that scarcely exceed 183 points.
• Guatemala, Peru and El Salvador, show a moderate dispersion with differences
between extremes fluctuating in the 208 - 220 point range.
• The twelve remaining countries reveal differences between their first and last
deciles in the 224 and 241 point range.
Executive summary
27
Table 6
Level
Cohort
score
IV
Description of Reading performance levels of Third Grade students
% students
8.41%
•Integrate and generalise information given in a paragraph or in
the verbal codes and graph;
•Replace non-explicit information;
•Read the text identifying new information;
•Translate from one code to another (from numeric to verbal, and
verbal to graphic)
21.63%
•Locate information discriminating it from adjacent information;
•Interpret reformulations that synthesise several data;
•Infer information based on knowledge about the world;
•Discriminate, based on the text, the meaning of words that have
several other meanings.
37.74%
•Locate information in a brief text that must no be distinguished
from other conceptually similar information;
•Discriminate words with a single meaning;
•Recognise simple sentence reformulations;
•Recognise redundancies between graphic and verbal codes
25.51%
•Locate information with a single meaning, in a prominent part
of the text, repeated literally or synonymously, and isolated from
other information.
6.71%
•Students at this level have not been able to acquire the abilities
required in Level I.
637.49
III
552.14
II
461.32
I
367.36
Below I
Description
In terms of reading achievement, 6.7% of the total number of Third Grade Primary
Education students in the region scored below Level I. This means that students failed
to locate information, with a single meaning, which is repeated in the text and isolated
from other information. Table 7 shows performance levels by country, and confirms the
fact that:
• 44.3% of Third Grade Cuban students scored the highest in Reading, followed by
students of Nuevo Leon (18.4%), Costa Rica (18.2%), and Chile (17.8%).
• 31.4% of the students of Dominican Republic scored below Level I, similarly to
more than 14% of the ones of Ecuador and Guatemala, and approximately 11% of
those of Panama and Paraguay.
In terms of Reading, rural school students participating in SERCE obtained lower scores
than their counterparts attending urban schools5. This is shown in Table 8 where the differences
in the results obtained by urban school students versus rural school students are described.
5
28
The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was
based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean
represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Student achievement in Latin America and the Caribbean
Table 7
Percentage of third grade students by Reading performance level
in each surveyed country
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Below I
6.26
6.29
1.60
4.94
1.46
0.56
14.62
5.34
14.37
3.65
6.95
11.21
11.47
9.24
31.38
4.69
1.70
6.71
I
22.01
25.25
9.97
23.61
10.40
6.48
37.47
29.05
43.18
19.64
37.29
37.24
37.85
36.18
46.73
19.96
12.71
25.51
II
39.73
39.84
34.46
41.78
34.20
21.09
34.20
41.05
32.04
37.09
43.38
35.29
32.27
35.79
18.04
39.02
34.82
37.74
III
23.63
21.54
36.22
21.16
35.73
27.61
11.61
19.15
8.51
27.52
10.69
12.35
12.92
15.13
3.29
24.94
32.40
21.63
IV
8.37
7.07
17.76
8.52
18.22
44.27
2.10
5.40
1.91
12.09
1.70
3.91
5.49
3.65
0.56
11.39
18.38
8.41
Significant differences in Reading results obtained by Third Grade students attending
urban and rural schools are evident in Latin America and the Caribbean.
• Peru exhibits the greatest differences –over 79 points– in terms of rural versus
urban school results. The country is followed by Guatemala, Brazil and Mexico with
differences that fluctuate between 62 and 64 points.
• Cuba and the Dominican Republic show the smallest differences between rural and
urban schools – 16 and 19 points, respectively.
Reading results also reveal marked differences in gender. Overall, among SERCE participants, girls obtained the highest Third Grade Reading scores. In fact, girls outperformed
boys by an average of 12.7 points.
• Argentina, Brazil, Cuba, Mexico, Panama, Paraguay, the Dominican Republic, Uruguay and the Mexican State of Nuevo Leon, show significant differences between
boys and girls in terms of Reading scores.
• The rest of the countries show no statistically significant differences when making
gender-based comparisons.
Reading performance of Third Grade Primary Education students shows a direct correlation with the gross national product of each country. In particular, differences in national
per capita GDP, account for one third of the variability observed in national performance
averages.
Executive summary
29
The greater the income distribution inequality, the lower the average Reading performance observed among Third Grade students. The Gini Index, for its part, accounts for
12.6% of the variability detected in the national performance median.
Table 8
Difference in average scores between urban and rural schools, by
gender. Third Grade Reading
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Urban/ Rural Difference
34.53*
62.67*
34.68*
50.92*
41.24*
15.94*
42.83*
57.29*
64.07*
62.47*
29.42*
54.70*
36.45*
79.30*
19.45*
28.56*
37.24*
-
* Stands for 5% confidence level.
30
Student achievement in Latin America and the Caribbean
Girl/ Boy Difference
17.74
18.57
2.46
4.64
4.69
13.34
9.02
1.39
1.67
13.20
1.77
14.94
15.69
0.76
13.05
12.73
9.05
12.74
Learning in Sixth Grade
Mathematics
The analysis of Sixth Grade Mathematics mean scores shows marked differences. The
difference between the average scores of highest and lowest performing countries (Cuba
and the Dominican Republic, respectively) reaches 220 points, that is to say, more than 2
standard deviations. However, the difference between the second and next to last countries is 1.26 standard deviations.
Graph 4mean and variability of Sixth Grade Mathematics median scores in
each surveyed country
LAC total: Latin American and Caribbean countries’ total.
CILL: Confidence interval lower limit (a = 0.05).
CIUL: Confidence interval upper limit (a = 0.05).
Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in
each country. That is to say, the far-right segment of each bar represents the scores of students
in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability.
The white vertical line running through the centre of each bar identifies the mean, while the
confidence interval is shown as a dark area around it. The width of this darkened area illustrates
its possible values.
Executive summary
31
Based on an overall analysis of results, countries may be divided into four groups,
relative to their difference with the countries’ average:
• Countries where Mathematics Sixth Grade students exhibit a higher average performance than the regional average, at one standard deviation above this average.
Cuba, with an average of 637 points, is part of this first group.
• Countries that exhibit mean scores above the regional average, but less than one
standard deviation. Uruguay, the Mexican State of Nuevo Leon, Argentina, Chile,
Costa Rica and Mexico form part of this group.
• Countries with average performances equal to the average of all participating
countries, in other words, where no statistically significant differences between
these two average values are evident. Brazil, Colombia and Peru belong in this
group.
• Countries that exhibit mean scores below the countries’ average (less than one
standard deviation): Ecuador, El Salvador, Guatemala, Nicaragua, Panama, Paraguay and the Dominican Republic*.
An analysis of student performance variability can shed light on the inequality of
education. Within the region, the difference between average scores in the 10th and 90th
percentiles represents 242.6 points. Disaggregating by countries, we find differences between the 10th and 90th percentiles that fluctuate between 182 and 385 points. On this
basis, four groups of nations can be established:
• In the Dominican Republic, Nicaragua, El Salvador, Panama and Guatemala, the
range of dispersion between these percentiles is less than 200 points.
• In Colombia, Paraguay, Brazil, Costa Rica, Argentina, Ecuador, Chile and the Mexican
State of Nuevo Leon, variability between the 10th and 90th percentiles fluctuates
between 200 and 250.
• Mexico, Peru and Uruguay exhibit a performance dispersion range above 250 points
but below 300 points.
• Cuba’s internal variability exceeds 300 points.
*
32
Significant differences (5% error) based on a t test for median comparison.
Student achievement in Latin America and the Caribbean
Table 9
Level
Cut-off
Score
IV
Description of Mathematics performance levels of Sixth Grade students
% students
Description
11.44%
•Students find averages and do calculations using the four basic
operations in the filed of natural numbers.
•Students identify paralleIism and perpendicularity in a real situation
and the graphic images of a percentage.
•Students solve problems involving properties of angles, triangles and
quadrilaterals as part of different shapes, or involving operations
with two decimal number
•Students solve problems involving fractions.
•Students make generalisations in order to continue a complex
graphic sequence pattern.
32.35%
•Students compare fractions, use the concept of percentages when
analysing information and solving problems that require this type of
calculation.
•Students identify parallelism and perpendicularity on a plane, as well
as bodies and their elements without the benefit of graphic support.
•Students solve problems that require interpreting the constituent
elements of a division or equivalent measures.
•Students recognise central angles and commonly used geometrical
shapes, such as circles, and resort to their properties for solving
problems.
•Students solve problems involving areas and perimeters of triangles
and quadrilaterals.
•Students make generalisations in order to continue a graphic
sequence or find the numerical sequence rule that applies to a relatively complex pattern.
40.82%
•Students analyse and identify the structure of the positional decimal
number system, estimate weight (mass) expressing it in units consistent with the attribute being measured.
•Students recognise commonly used geometrical shapes and their
properties in order to solve problems.
•Students interpret, compare and work with information presented
through various graphic images.
•Students identify the regularity of a simple pattern sequence.
•Students solve addition problems in different numerical fields
(natural numbers, decimals) including commonly used fractions or
equivalent measures.
•Students solve multiplication or division problems, or two natural
number operations, or operations that include direct proportionality
relations.
13.91%
•Students arrange natural numbers (up to 5 digits) and decimals (up
to thousands) in sequence.
•Students recognise common geometrical shapes and the unit consistent with the attribute being measured.
•Students interpret information presented in graphic images in order
to compare it and change it to a different form of representation.
•Students solve problems involving a single addition using natural
numbers.
624.60
III
514.41
II
413.58
I
309.64
Below I
1.48%
•Students at this level have not been able to acquire the abilities
required in Level I.
Executive summary
33
Table 10, shows that nearly 75% of Cuban and Uruguayan students are placed in Levels
III and IV, exhibiting the highest performances for Mathematics achievement. More than
50% of Sixth Grade students in Nuevo Leon, Costa Rica, Mexico and Chile, attained the
highest performance levels in Mathematics. On the other hand, between 50% and 60%
of all surveyed students in Argentina, Brazil, Peru, Colombia and Paraguay, performed at
Levels I and II. This is also true for more than 70% of the surveyed students in Ecuador,
El Salvador, Guatemala, Nicaragua, Panama and the Dominican Republic.
Table 10
Percentage of sixth grade students by Mathematics performance
level in each surveyed country
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Below I
1.53
1.46
1.40
1.02
0.09
0.19
4.24
1.95
2.78
0.51
2.25
3.32
3.85
2.41
5.69
0.67
0.34
1.48
I
11.89
14.00
9.84
13.29
4.55
4.43
24.86
19.18
24.94
8.38
23.88
27.16
21.00
19.58
41.79
4.26
6.29
13.91
II
37.99
44.09
37.85
47.64
32.71
17.93
45.15
51.61
50.80
32.41
52.69
49.55
46.50
39.82
45.43
22.36
29.35
40.82
III
36.26
31.65
37.39
32.60
43.70
26.33
21.41
23.81
19.52
39.10
19.41
17.64
23.91
28.90
6.85
40.41
40.66
32.35
IV
12.34
8.80
13.52
5.46
18.95
51.13
4.34
3.45
1.96
19.60
1.76
2.33
4.74
9.29
0.24
32.31
23.36
11.44
As shown in Table 11, Sixth Grade rural school students participating in SERCE obtain
lower scores in Mathematics than their counterparts attending urban schools.
34
Student achievement in Latin America and the Caribbean
Table 11
Difference in average scores between urban and rural schools, by
gender. Sixth Grade Mathematics
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Urban/ Rural Difference
40.21*
42.74*
36.51*
29.03*
23.34*
4.98
42.81*
44.76*
38.39*
51.42*
10.24*
37.33*
31.18*
87.03*
9.01*
52.45*
35.79*
-
Girl/ Boy Difference
-5.79
-10.02*
-6.84*
-14.53*
-20.67*
8.24*
0.29
-9.48*
-6.91*
6.35
-10.16*
2.81
-0.59
-18.94*
0.96
0.18
0.27
-6.17*
* Stands for 5% confidence level
Latin American and Caribbean Sixth Grade students attending urban schools outperform rural school students in Mathematics.
• In terms of urban versus rural school results, Peru shows the largest gaps exceeding an 87 point difference, on average. Uruguay and Mexico follow with differences
in the neighbourhood of 52 points.
• By contrast, Cuba and the Dominican Republic show the smallest differences between rural and urban schools (5 and 9 points, respectively).
A SERCE’s gender-based analysis reveals that, at the regional level, boys score some 6
points higher than girls in Sixth Grade Reading tests. Furthermore, based on other important differences detected among countries, three groups can be established:
• A first group is comprised of Argentina, Ecuador, Mexico, Panama, Paraguay, the
Dominican Republic, Uruguay, and the Mexican State of Nuevo Leon, where no
statistically significant differences in terms of performance of girls and boys were
detected.
• A second group includes Cuba, where girls obtained significantly higher scores
than boys.
Executive summary
35
• Finally, a group of countries where average performance is skewed in favour of
boys. These countries are: Brazil, Chile, Colombia, Costa Rica, El Salvador, Guatemala, Nicaragua and Peru.
On the other hand, national per capita income is strongly associated with student
performance in Mathematics. Differences in GDP account for 41% of the average score
variability observed in Sixth Grade Mathematics tests.
These results reveal that, in any one country, the greater the inequality the lower its
average performance. Similarly, differences in the Gini Index among countries account for
32% of the average score variance.
Reading
An overall analysis of average Sixth Grade Reading scores and their distribution provides insight into the inequalities within and across countries. The difference between
countries located at both extremes is 1.75 standard deviations. However, between the
second and next to last countries this difference is reduced to only 1.16 standard deviations.
Based on students’ average performance countries may be classified in five groups:
1. Countries where students’ scores exceed the average of the countries participating in SERCE (less than one standard deviation). Cuba, Costa Rica, Brazil, Chile,
Colombia, Mexico, Uruguay and the Mexican State of Nuevo Leon are part of this
group.
2. Countries where students’ mean scores are equal to the regional average. Argentina
illustrates the only case.
3. Countries where students’ scores are lower than SERCE’s regional average, and less
than one standard deviation: Ecuador, El Salvador, Guatemala, Nicaragua, Panama,
Paraguay, Peru and the Dominican Republic*.
*
36
Significant differences (5% error) based on a t test for median comparison.
Student achievement in Latin America and the Caribbean
Graph 5mean and variability of Reading scores obtained by Sixth Grade
students in each surveyed country
LAC total: Latin American and Caribbean countries’ total.
CILL: Confidence interval lower limit (a = 0.05).
CIUL: Confidence interval upper limit (a = 0.05).
Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in
each country. That is to say, the far-right segment of each bar represents the scores of students
in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability.
The white vertical line running through the centre of each bar identifies the mean, while the
confidence interval is shown as a dark area around it. The width of this darkened area illustrates
its possible values.
Looking at the variability across student performance facilitates an analysis of the
learning inequalities that characterise each country. Performance differences between
students in the 10th and 90th percentiles of the various countries fluctuate in the 182
to 294 point range (244 points at the regional level). In El Salvador, Nicaragua and the
Dominican Republic, students in these percentiles are separated by less than 200 points.
This is not the case of Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, Guatemala,
Mexico, Panama, Paraguay, Peru, Uruguay and the Mexican State of Nuevo Leon, where the
difference between the 10th and 90th percentiles fall in the 206 to 259 point range. Lastly,
Cuba exhibits the greatest difference in scores, with a 294 point gap between 10th and
90th percentiles.
Executive summary
37
Table 12
Level
Cut-off
Score
IV
Description of Reading performance levels of Sixth Grade students
% students
Description
20.30%
•Integrate, rank and generalise information distributed across the
text;
•Establish equivalences among more than two codes (verbal, numerical
and graphic);
•Reinstate implicit information associated with the entire text;
•Recognise the possible meanings of technical terms or figurative
language;
•Distinguish various tenses and nuances (certainty, doubt) used in a
text
26.79%
•Locate information and separate it from other near-by information;
•Interpret reformulations and synthesis;
•Integrate data distributed across a paragraph;
•Reinstate implicit information in the paragraph;
•Re-read in search of specific data;
•Identify a single meaning in words that have several meanings;
•Recognise the meaning of parts of words (affixes) using the text as a
reference
35.46%
•Locate information in the middle of a text that must be distinguished
from a different piece of information found in a different segment;
•Identify words with a single meaning
16.51%
•Locate information with a single meaning in a prominent or central
part of the text (beginning or end), that is repeated literally or
synonimously and is isolated from other information.
593.59
III
513.66
II
424.54
I
299.59
Below I
0.93%
•Students at this level have not been able to acquire the abilities
required in Level I.
Table 13 shows how students are distributed in each of the performance levels, by
country. In terms of Reading achievement, 50% of Cuba’s Sixth Grade students can be
found at Level IV, followed by Costa Rica with slightly over a third of its students occupying this level.
For their part, the percentage of students performing at Level IV in Uruguay, Chile, the
Mexican State of Nuevo Leon, Mexico and Brazil, fluctuate between 20% and 30%.
At the other extreme, 47.8% of the Dominican Republic’s Sixth Grade students performed at Level I, followed by Ecuador, Guatemala, Panama and Paraguay with slightly over
one third of their students at this level.
38
Student achievement in Latin America and the Caribbean
Table 13
Percentage of sixth grade students by Reading performance level
in each surveyed country
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Below I
1.78
0.57
0.30
0.39
0.22
0.30
4.47
0.95
2.86
0.23
1.02
1.95
3.90
2.24
4.08
0.47
0.21
0.93
I
17.93
14.85
8.02
13.17
5.00
5.26
33.69
21.49
33.06
12.23
22.08
28.97
33.46
24.08
47.84
9.60
9.12
16.51
II
35.59
34.65
30.06
38.25
23.45
19.57
39.48
44.02
43.36
33.40
50.58
38.76
36.81
41.65
37.50
30.80
29.99
35.46
III
25.48
27.47
32.37
30.40
36.73
24.20
16.63
23.99
15.73
29.75
21.10
20.77
18.60
22.57
9.19
29.68
32.37
26.79
IV
19.22
22.46
29.26
17.80
34.59
50.68
5.73
9.54
4.99
24.39
5.22
9.55
7.23
9.46
1.39
29.45
28.31
20.30
Data on performance by school type reveal marked differences between the learning
acquired by students in urban and rural areas6 10as shown in Table 14.
Latin American and Caribbean Sixth Grade students attending urban schools outperform rural school students in Reading.
• Cuba is the only country that does not show significant performance differences
between urban and rural school students.
• In terms of school location, Nicaragua and the Dominican Republic show the smallest differences - 21 and 24 points, respectively.
• By contrast, Peru shows the greatest differences –around 80 points– between
urban and rural school students, followed by Mexico, Panama and Paraguay with
differences approaching 57 points.
6
The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was
based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean
represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Executive summary
39
Table 14
Difference in average scores between urban and rural schools, by
gender. Sixth Grade Reading
Country
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
El Salvador
Guatemala
Mexico
Nicaragua
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Urban/ Rural Difference
43.55*
49.35*
35.66*
41.74*
34.37*
12.75
46.22*
54.31*
53.75*
57.71*
21.42*
56.67*
56.32*
78.96*
23.75*
49.10*
39.23*
-
Girl/ Boy Difference
11.05*
15.69*
6.89*
-4.43
-0.75
15.21*
6.39
-0.19
-2.44
13.32*
-0.61
15.89*
11.14*
-1.87
15.09*
19.64*
7.98
10.44*
* Stands for 5% confidence level
A gender-based analysis reveals that in Latin America and the Caribbean Sixth Grade
girls outperform boys in Reading. The regional gap between genders is 10.4 points.
Girls also obtain significantly higher scores in Argentina, Brazil, Chile, Cuba, Mexico,
Panama, Paraguay, the Dominican Republic and Uruguay.
Per capita GDP bears a direct correlation with students’ average learning. Differences
in national wealth account for 44.4% of the variation detected in Sixth Grade Reading
national averages.
The greater the Gini Index the lower the Reading average performance among Sixth
Grade students. Differences in the Gini Index account for 11% of the variation observed
across national Reading averages.
Natural Science
The Natural Science test was administered to Sixth grade Primary Education students
exclusively, with the participation of only 10 national entities: Argentina, Colombia, Cuba,
El Salvador, Panama, Paraguay, Peru, the Dominican Republic, Uruguay, and Nuevo Leon.
The difference separating countries located at the upper and lower ends of the performance scale was calculated at 2.35 standard deviations. However, the difference between
the second highest and next to last country is only 0.68 standard deviations, which im40
Student achievement in Latin America and the Caribbean
plies that there is greater homogeneity among countries occupying mid-positions in the
distribution.
Overall, both national averages and distribution of scores show differences in each
country. Relative to performance in Science, four groups can be identified:
• The first group is made up of countries with mean scores markedly higher than
the regional average (more than one standard deviation, that is, over 650 points).
Cuba is the only case.
• The second group consists of countries with scores higher than the Latin American
and Caribbean average (less than one standard deviation): Uruguay and the Mexican State of Nuevo Leon.
• Colombia is the only country in this third group, characterised by a national mean
that does not show significant differences versus the regional media.
• Countries that exhibit lower scores than the Latin American and Caribbean average
(less than one standard deviation) are part of a fourth group. These countries are:
Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic1*.
Graph 6mean and variability of Science scores obtained by Sixth Grade
students in each surveyed country
LAC total: Latin American and Caribbean countries’ total.
CILL: Confidence interval lower limit (a = 0.05).
CIUL: Confidence interval upper limit (a = 0.05).
Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in
each country. That is to say, the far-right segment of each bar represents the scores of students
in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability.
The white vertical line running through the centre of each bar identifies the mean, while the
confidence interval is shown as a dark area around it. The width of this darkened area illustrates
its possible values.
*1 Significant differences (5% error) based on a t test for median comparison.
Executive summary
41
The differences detected in learning results are reflected in the students’ scores dispersion. Three scenarios characterise the region:
• In most countries the distance separating the 10th from the 90th percentiles fluctuates between 200 and 230 points. This is the case of Argentina, Colombia, Panama,
Paraguay, Peru, Uruguay and the Mexican State of Nuevo Leon.
• El Salvador and the Dominican Republic, with less than 200 points separating the
10th and 90th percentiles, exhibit the smallest dispersion of results.
• Cuba, in addition to showing the highest average score, also shows the greatest dispersion of results – 386 points between students in the 10th and 90th percentiles.
Table 15
Level
Cut-off
score
IV
Description of Science performance levels of Sixth Grade students
% students
Description
2.46%
•At this level, students use and transfer scientific knowledge, which
requires a high degree of formalisation and abstraction, to diverse
types of situations.
•Students are capable of identifying the scientific knowledge involved
in the problem at hand. These problems are more formally stated and
may relate to aspects, dimensions or analyses that may be detached
from the immediate setting.
11.40%
•At this level, students explain everyday situations on the basis of
scientific evidence; use simple descriptive models to interpret natural
phenomena, and draw conclusions from the description of experimental activities.
42.24%
•At this level, students apply school-acquired scientific knowledge:
compare, organise and interpret information presented in various
formats (tables, charts, graphs, pictures); identify causality relations
and classify living beings according to a given criterion.
•In connection with Level I, it should be noted that these students
are capable of accessing information presented in different formats,
which requires the use of much more complex skills.
38.72%
•At this level, students relate scientific knowledge to daily situations
that are of common occurrence in their context.
•Students are capable of explaining their immediate world based on
their own experiences and observations, and establish a simple and
lineal relation with previously acquired scientific knowledge.
•Students describe concrete and simple events involving cognitive
processes such as remembering, evoking and identifying.
704.75
III
590.29
II
472.06
I
351.31
Below I
42
5.18%
•Students at this level have not been able to acquire the abilities
required in Level I.
Student achievement in Latin America and the Caribbean
Data on Science performance levels provide the grounds for grouping countries around
three possible scenarios:
• In Cuba, 65% of its students perform at Levels III and IV.
• In Colombia, Uruguay and the Mexican State of Nuevo Leon, practically half their
students perform at Level II.
• In Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic,
over 40% of their students perform at Level I or below.
Table 16
Percentage of sixth grade students by Science performance level
in each surveyed country
Country
Argentina
Colombia
Cuba
El Salvador
Panama
Paraguay
Peru
Dominican Rp.
Uruguay
Nuevo Leon
Total
Below I
5.32
2.62
0.26
3.78
6.34
7.20
6.97
14.29
1.69
2.59
5.18
I
37.73
31.68
8.78
44.73
44.60
46.18
46.93
62.82
22.76
30.98
38.72
II
43.04
51.09
25.92
42.55
39.89
38.11
39.36
21.50
48.47
47.78
42.24
III
12.73
13.59
30.31
8.23
8.40
7.52
6.37
1.37
24.01
16.38
11.40
IV
1.17
1.02
34.73
0.71
0.77
0.99
0.36
0.03
3.06
2.28
2.46
In terms of Science, students attending urban schools outperform their rural school
counterparts712.
Peru exhibits the greatest difference –in excess of 57 points– in Science performance
between urban and rural schools. El Salvador and Panama follow with an approximate 40
point difference.
Located at the opposite extreme is Cuba where no significant performance differences
between urban and rural school students are evident. For its part, the Dominican Republic
shows minimal differences that fluctuate around 11 points.
7
The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was
based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean
represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Executive summary
43
Table 17
Difference in average scores between urban and rural schools, by
gender. Sixth Grade Science
Country
Argentina
Colombia
Cuba
El Salvador
Panama
Paraguay
Peru
Dominican Rep.
Uruguay
Nuevo Leon
Total
Urban/ Rural Difference
19.74*
22.83*
11.36
41.91*
38.27*
30.23*
56.18*
11.14*
29.28*
26.65*
-
Girl/Boy Difference
-5.06
-18.93*
7.41
-10.16*
1.26
1.88
-16.12*
-0.65
-4.44
-12.77*
-11.52*
* Significant (5% confidence level)
Gender-based comparisons in the region reveal that boys have a marked advantage
over girls, obtaining average scores that are 11.5 points higher.
• In Colombia, El Salvador, Peru and the Mexican State of Nuevo Leon, boys’ Science
scores are significantly higher than girls’.
• By contrast, in Argentina, Cuba, Panama, Paraguay, the Dominican Republic and
Uruguay, no statistically significant differences between girls and boys were detected.
Student performance and the internal production of a country are directly related.
National per capita GDP accounts for 11.57% of the variations observed in Science performance.
Data seem to indicate that there is an inverse relationship between the learning of
Science and income distribution inequalities. In fact, the Gini Index accounts for 30.68%
of the national mean variances observed in Science performance.
44
Student achievement in Latin America and the Caribbean
UNESCO/G.Tealdi
foto pendiente
Factors associated with achievement
It will probably come as a ray of hope to all educational systems, that through the
study of associated factors, SERCE has been able to corroborate the fact that schools are
in a position to contribute importantly to student performance. While the socioeconomic
dimension has a strong influence on performance, school-related variables can help significantly to reduce the learning inequalities associated with social inequity.
In line with PERCE’s conclusions, the school climate variable was confirmed to have
the greatest impact on student performance. It follows that, in order to promote learning
among students, it is essential to provide a welcoming and warm environment based on
mutual respect.
Executive summary
45
Collectively, the school resources variable also contributes to performance. While it is
entirely possible that variables such as school infrastructure, basic services, the number
of books in the school library, and the work experience of teachers, can only make modest
individual contributions, as a whole, they can help substantially to encourage learning.
The clear message behind this assertion is that resources are necessary elements to drive
performance.
School segregation based on the socioeconomic and/or cultural status of the student is
the second most important variable that explains performance. Segregation seems to have
a stronger impact on Reading than on Mathematics or Science. And, while this is not an
education-related variable per se, any progress in this area will translate into important
advances in students’ learning.
46
Student achievement in Latin America and the Caribbean
UNESCO/D.Roger
foto pendiente
Final reflections
Quality education must be seen as a right of all girls and boys. Attaining it represents
a solid base for sustainable development, democratic progress and social equality. The
SERCE embodies joint efforts undertaken by the Latin American and Caribbean countries
and OREALC/UNESCO, aimed at enhancing educational opportunities for all students and,
ultimately, promoting development in the region.
The evaluations conducted within the framework of this Study, attempt to provide an
analysis of what students learn, the inequalities that affect learning, and the factors that
determine differential achievement.
Executive summary
47
On Primary Education student learning
In terms of academic performance, quality education is expected to lead to high levels
of learning among all students, without exclusions of any kind. From SERCE’s perspective,
equity is transversal since it focuses on social conditions that prevent from fully exercising
the right to education, and on the way schools ensure a balanced provision of learning
opportunities to their students.
Significant differences in the quality of student learning are evident in the region.
This can be observed across all areas and grades, as reflected by the dispersion of results within countries, and by the gaps in scores detected among participating countries.
Thus, in connection with Third Grade education, the differences observed between
the highest and lowest performing countries exceed 230 points, both in Reading and
Mathematics. In terms of Sixth Grade, the differences although somewhat smaller, still
exceed two standard deviations in Science and Mathematics, and rise to 174.5 points in
Reading.
This diversity affecting quality of learning can also be presented graphically by dividing participating countries into four groups, on the basis of their average test results.
Table 18
Comparison of third grade school results
Difference relative to the
regional mean
Higher than the mean– more
than one standard deviation
Higher than the mean– less
than one standard deviation
Identical to the regional mean
Lower than the mean– less
than one standard deviation
48
Mathematics
Reading
Cuba
Cuba
Chile, Costa Rica, Mexico,
Uruguay and Nuevo Leon
Argentina, Chile, Colombia,
Costa Rica, Mexico, Uruguay
and Nuevo Leon
Brazil and El Salvador
Ecuador, Guatemala, Nicaragua,
Panama, Paraguay, Peru and the
Dominican Republic
Argentina, Brazil and Colombia
Guatemala, Ecuador, El
Salvador, Nicaragua, Panama,
Paraguay, Peru and the
Dominican Republic
Student achievement in Latin America and the Caribbean
Table 19
Comparison of sixth grade school results
Difference relative to
the regional mean
Higher than the
mean– more than one
standard deviation
Higher than the
mean– less than one
standard deviation
Identical to the
regional media
Lower than the
mean– less than one
standard deviation
Matemática
Reading
Science
Cuba
Cuba
Argentina, Chile, Costa Costa Rica, Cuba,
Rica, Mexico, Uruguay Brazil, Chile, Colombia,
and Nuevo Leon
Mexico, Uruguay and
Nuevo Leon
Brazil, Colombia and
Argentina
Peru
Ecuador, El Salvador,
Ecuador, El Salvador,
Guatemala, Nicaragua, Guatemala, Nicaragua,
Panama, Paraguay
Panama, Paraguay, Peru
and the Dominican
and the Dominican
Republic
Republic
Uruguay and Nuevo
Leon
Colombia
Argentina, El Salvador,
Panama, Paraguay, Peru
and the Dominican
Republic
It should be noted that in countries occupying the second and next to last position in
the distribution scale, in practically all cases, mean scores differences are slightly above
one standard deviation. This would point to a greater homogeneity among countries occupying mid-positions on the performance scale. Science constitutes a special case, since
here standard deviations between the upper and lower extremes rise to 2.35 points, while
intermediate results show a standard deviation of 0.68, indicative of greater homogeneity
in this segment, and a substantial difference versus the extremes.
This diversity within countries is also made evident when comparing differences between students in the 10th and 90th percentiles. On this basis, four country categories may
be established, both for Third and Sixth Grade Primary Education students, namely:
1) Countries where the dispersion range between highest and lowest performance
levels is less than 200 points;
2) Countries that exhibit variability between 10th and 90th percentiles in the 200 - 250
point range;
3) Countries with a performance dispersion range of more than 250 points but less
than 300 points, and
4) Countries that exhibit an internal variability in excess of 300 points
In connection with scores obtained by Third Grade students, differences fluctuate
between 165 and 341 points in Mathematics, and between 183 and 296 in Reading. Cuba,
Uruguay and Paraguay exhibit the highest internal dispersions in Mathematics and Reading, while Nicaragua shows the lowest.
Executive summary
49
Table 20
Comparison of school results dispersion for third grade students,
by country
Difference between
90th and 10th
Percentiles
Less than 200 points
Between 200 and 250
points
Between 251 and 299
points
300 and over
Mathematics
Reading
Colombia, Ecuador, the Dominican
Rep., Guatemala, El Salvador,
Panama and Nicaragua
Brazil, Uruguay, Argentina, Mexico,
Chile, Costa Rica, Peru and the
Mexican State of Nuevo Leon
Paraguay
Nicaragua
Paraguay, Mexico, Uruguay, Argentina, Brazil, the Dominican Rep.,
Costa Rica, Chile, Colombia, Panama,
Ecuador, El Salvador, Peru, Guatemala
and the Mexican State of Nuevo Leon
Cuba
Cuba
In connection with Sixth Grade students, average performance differences between
students in the 10th and 90th percentiles fluctuate between 182 and 385 points in Mathematics, and 176 and 387 points in the case of Science. Once again, Cuba shows the highest dispersion in all three areas, while the Dominican Republic exhibits the lowest internal
dispersion in the aforementioned areas and grade.
Table 21
Comparison of school results dispersion for sixth grade students,
by country
Difference between
90th and 10th
Percentiles
Less than 200 points
Mathematics
Reading
Between 200 and 250
points
The Dominican Rep.,
Nicaragua, El Salvador,
Panama and Guatemala
Colombia, Paraguay,
Brazil; Costa Rica, Argentina; Ecuador, Chile,
and the Mexican State
of Nuevo Leon
Between 251 and 299
points
300 and over
Mexico, Peru and
Uruguay
Cuba
El Salvador, Nicaragua
and the Dominican
Rep.
Uruguay, Mexico, Brazil,
Chile, Paraguay, Costa
Rica, Peru, Panama,
Ecuador, Guatemala, Colombia, and the Mexican
State of Nuevo Leon
Argentina and Cuba
Science
The Dominican Rep.,
and El Salvador
Argentina, Colombia,
Uruguay and the
Mexican State of Nuevo
Leon
Cuba
While Cuba shows the highest dispersion and the Dominican Republic the lowest, these
findings should be carefully interpreted. On the one hand, scores obtained by the lower
50
Student achievement in Latin America and the Caribbean
performing Cuban students are similar to those of the average Latin American and Caribbean students. This fact places lower performing Cuban students much farther ahead than
the rest of the region’s student population.
On the other hand, the Dominican Republic exhibits the lowest results of the surveyed
countries while the minimal score dispersion would seem to indicate that the results
obtained by these students are generally low. In short, the results yielded by these two
countries illustrate that, on the one hand, high general performance and high variability
are not mutually exclusive and, on the other, that there are cases where results may be
equally distributed but nevertheless learning levels remain low.
In qualitative terms, this diversity affecting the quality of learning of Latin American
and Caribbean students is reflected in the distribution of performance levels. Analyses
based on performance levels give an in-depth view of what students are capable of doing
in each of the surveyed grades and areas. SERCE classifies students’ achievements into four
performance levels (I through IV) of increasing complexity. Each level is made up of a set
of tasks the students will be tested on. In order to solve these tasks students must master
specific contents and apply distinct cognitive process.
An ideal distribution would show that most students perform at the higher levels.
However, results do not generally conform to this pattern and while over 20% of the
students in the region do, in fact, perform at the higher levels in practically all areas and
grades (except in Science, where the percentage drops to 13.8%), there is an important
number of students who are unable to perform beyond Level I: more than 40% in Third
Grade Mathematics and Science; 32% in Third Grade Reading; and more than 15% in Sixth
Grade Mathematics and Reading. The implication is that these students can only attempt
the tasks that SERCE has defined as having the lowest levels of complexity.
For instance, while more than 40% of Cuban students perform at the highest level, in
every area and grade, there are other countries where approximately 50% of their students
perform at or below Level I, in every area and grade. These results give insight into the
region’s learning gaps and underline the importance of going beyond average scores when
discussing educational quality, in order to identify and understand what students know
and are capable of accomplishing. These findings shed light on the challenges that must
be surmounted to enhance the quality of instruction imparted in Primary Education, and
highlight the serious learning inequities that persist in the region.
The equitable distribution of learning across different social strata remains a
pending task
a) The economic conditions of countries, particularly income generation and distribution, have a bearing on Primary Education student learning.
In order to explain this assertion, SERCE has analysed the existing link between student average performance, per capita Gross Domestic Product, and the Gini Index for each
Executive summary
51
country. Due to the unavailability of relevant data, Cuba and the Mexican State of Nuevo
Leon have not been included in this analysis.
Data confirm the existence of a positive correlation between the average scores of
a given country and its per capita GDP. However, many countries obtain results beyond
what their internal production would have predicted, which indicates that while resources are important they are not the only factors that determine student performance.
Country by country analysis of average performance versus the Gini Index, shows an
equally significant but inverse relation. In other words, the higher the income distribution inequality the lower the average student performance exhibited by Latin American
and Caribbean students.
b) Student gender has an impact on SERCE’s results
Consistent with other studies on gender-based student performance, the present
Study corroborates differences in most countries favouring girls in Reading and boys in
Mathematics. Exceptions can be found in the Dominican Republic and Cuba where girls
outperform boys in Third Grade and Sixth Grade Mathematics. In terms of Science, four
participating countries show differences skewed in favour of boys, while in the remaining six countries no significant gender-based differences are evident.
c) School location influences student achievement
Within the region, the location of schools is also responsible for generating differences in student performance. In Latin America and the Caribbean, rural school boys
and girls show lower levels of performance when compared to their urban school counterparts.
These inequalities become sharper in some countries. The greatest differences in
performance favoring urban school students –in both areas and grades surveyed– can
be found in Peru, while the smallest differences attributable to the geographic location
of schools were evident in the Dominican Republic and Cuba. In terms of Science, the
greatest inequalities related to location are found in El Salvador and Panama. Conversely, Peru and the Dominican Republic, show the smallest differences.
An analysis of student distribution by performance levels corroborates the existence
of these gaps. There are clear differences, both at the regional level and within the
countries, relative to the percentage of students occupying each of these performance
levels that depend almost exclusively on whether the student is attending an urban or
rural school. Moreover, in urban schools, performance distribution seems to have shifted
to the next upper level, vis-à-vis rural schools. As a result, the percentage of students
performing at Levels II, II and IV is systematically higher in urban schools, while at the
lower levels (I and below I) there is a larger percentage of rural students represented.
52
Student achievement in Latin America and the Caribbean
The school does make a difference
In what undoubtedly constitutes an encouraging message to all education systems, SERCE
has been able to corroborate through its study of associated factors that schools can, in
fact, contribute importantly to student performance. While socio-economic factors are
known to have a significant effect on performance, school-related variables can have a
substantial impact on reducing the learning inequalities associated with social disparities.
In line with PERCE’s findings, school climate was found to be the single most important variable conditioning student performance. Hence, generating a friendly and positive
environment based on mutual respect becomes an essential strategy to foster student
learning.
As a whole, school resources variables also contribute positively to student performance.
While the contributions made by school infrastructure, availability of basic services, number of books comprising the school library, and the teaching experience of educators is,
at best, modest when taken individually, collectively these variables represent a valuable
help to student learning. The key message derived from this finding is that resources are
indeed necessary to drive performance.
School segregation based on the students’ socio-economic and cultural status is the
second most important performance conditioning variable. Segregation has been shown to
have a stronger impact on Reading as opposed to Mathematics and Science, and although
this is not an educational variable per se, any efforts aimed at reducing it will greatly
influence student learning and achievement.
Clearly, the Second Regional Comparative and Explanatory Study conducted by the
Latin American Laboratory for Assessment of the Quality of Education, has contributed
important information and knowledge to inform the decision-making process in matters
concerning social and educational policies in Latin America and the Caribbean. Each of
the countries participating in the Study must now retrieve the main lessons derived from
this important inquiry.
Executive summary
53