DIME BRIEF
Making the grade: Assessing literacy and
numeracy in African countries
The Development Impact
Evaluation Initiative is a
broad-based World Bank
program to generate
knowledge on the
effectiveness of
government programs. It
supports government
agencies adopt a culture
of real time evidencebased policy-making on
the basis of rigorous
impact evaluation. By
testing how to make
policies work, it
contributes to improving
Making the grade
Access to and completion of primary school has significantly expanded in recent years
but can yield disappointing results if accompanied by reduction in education quality.
The extent of student learning and skills acquisition measured through test scores is
crucial to assess the effectiveness of any investments in education. Early grade reading
and math skills are the fundamental building blocks. Children who fail to develop basic
skills early in their education are more likely to lag behind in the future.
Because tests of basic literacy and numeracy are not implemented on a wide scale at an
early stage of schooling and with greater scarcity of test results in Africa, a series of
student assessments in Sub-Saharan Africa, known collectively as “Making the Grade”
(MTG), was undertaken in 2009-2010 by the Africa Program of Education Impact
Evaluation. MTG was designed to provide a simple and consistent overview of basic
reading and math skills in a range of African countries, to characterize the current state
of education quality in those countries, and to provoke further discussions and research.
policy performance.
Literacy and numeracy assessments
DIME works with 300
The assessments targeted primary school pupils and consisted of basic mathematics and
reading skills test administered by trained enumerators. The results of the assessments
conducted in sub-Saharan Africa are reported for The Gambia, Ghana, Liberia, Sierra
Leone (numeracy only), and Senegal. In each country, schools and students from two
grade levels were randomly selected to participate in the assessment. The sample size
varies from 38 schools (Senegal) to 178 schools (the Gambia) and from 835 students
(Liberia) to 16113 students (Sierra Leone).
agencies in 72 countries
across 15 thematic
programs to generate
knowledge, improve
quality of operations and
strengthen country
capacity for evidencebased policy- making.
This study is part of the
Africa Program for
Education Impact
The literacy assessment was administered orally to one child at a time. It included
elements of the Early Grade Reading Assessment (developed by RTI International) and
comprised of three sections:
1.
Students were given one minute to read as many letters as possible on a chart containing a
total of 98 or 100 letters arranged in several rows.
Evaluation.
APEIE supports 14 country-
2.
Passage reading
3.
Reading comprehension
specific impact
evaluations and
convenes regular crosscountry meetings to build
capacity and discuss
policy lessons. APEIE is fully
integrated into the Africa
Education unit and
mainstreamed into the
operations. For more
information, visit:
http://go.worldbank.org/3YJY
7TIQD0
Letter name knowledge
Students were then given a written passage to read, including between 9 and 12 short
sentences.
To gauge the extent to which children understood the passage read, they were then asked
to answer four or five reading comprehension questions.
The written numeracy test lasted 25 minutes and comprised a standard battery of 32 to
38 questions covering the basic areas of arithmetic (addition, subtraction,
multiplication, and division).
Literacy and numeracy results
Even though test instruments used were very similar, absolute results are not fully
comparable across countries because cultural differences are reflected in the literacy
test (e.g. names of places and characters used) and protocols for implementation were
DIME Brief on Making the Grade 1
not identical. Nevertheless, some standards can be used to gauge country
results. US benchmarks for these assessments suggest that pupils who can
name less than 40 letters per minute in kindergarten and/or read less than
110, 118, 124, 125 words per minutes in grades 3, 4, 5 and 6 should be
considered at risk. Furthermore, in cognitive neuroscience it is considered
that, to understand a text, the brain must process words at a minimum rate
of 35 to 60 words per minute.
Literacy Mean Scores - Ghana
80
60
40
20
0
3
Average scores on the letter naming and passage reading tests are 77 and 90
letters per minute and 53 and 66 words per minute for 3rd and 5th graders
respectively. Reading fluency stands at half the US benchmarks but within
the minimum range for comprehension. The 45 words per minute threshold
is attained by 85 percent of 5th graders and 64 percent of 3rd graders and
very few students cannot read a single word: less than 1 percent in grade 3
and none in grade 5. This results in good comprehension test scores with
averages of 69 percent (grade 3) and 79 percent (grade 5) and a majority of
students scoring at least 80 percent. Boys significantly outperform girls only
in grade 5 passage reading and grade 3 reading comprehension.
On average, pupils score 74 percent (grade 3) and 78 percent (grade 5) on
additions and 44 percent (grade 3) and 53 percent (grade 5) on
multiplications. Scores of zero are almost as rare for math as for reading: 5
percent of 3rd graders and 1 percent of 5th graders could not solve a single
subtraction problem. Numeracy skills of boys and girls are similar except for
additions in grade 5 and subtractions in grade 3 where boys perform
significantly better.
5
3
Average math scores are 60 and 83 percent (grade 3 and 5) on additions and
23 and 53 percent (grade 3 and 5) on multiplications.
36 37
60 56
3
Words/min
5
Comp.
Girls
Numeracy Mean Scores - Ghana
100
50
7067
8279
0
3
5749
5
7469
3
Add.
5
Sub.
3525 6154 1613 3730
3
5
3
Mult.
Boys
5
Div.
Girls
Literacy Mean Scores - Liberia
100
50
78 76
91 89
71 67 81 79
54 52 69 63
0
3
5
3
Letters/min
5
3
Words/min
Boys
5
Comp.
Girls
Numeracy Mean Scores - Liberia
100
50
7474 8077
0
3
6165 6969 4543 5353 2928 3538
5
3
Add.
5
Sub.
Boys
The Gambia
In grade 4 and 6 respectively, average performances on letter naming and
reading fluency are 56 and 77 letters per minute and 30 and 65 words per
minute. Reading fluency is about one fourth (grade4) and half (grade 6) of
the US standards and falls slightly below the neuroscience lower bound in
grade 4. Among tested students, 25 percent (grade 4) and 56 percent (grade
6) could read more than 45 words per minute and 20 percent (grade 4) and
6 percent (grade 6) could not read a single word. Average comprehension
test scores are therefore low at 36 percent (grade 4) and 59 percent
(grade6). On all literacy assessments, boys outperform girls significantly.
67 66
5
Boys
Numeracy average scores are 68 percent and 81 percent on additions and 30
percent and 57 percent on multiplications for grade 3 and 5 respectively.
Furthermore, 17 percent of 3rd graders and 5 percent of 5th graders could not
resolve a single subtraction. Performance gaps between genders are
pervasive. Boys do consistently better than girls in all types of numerical
problems.
Liberia
33 38
Letters/min
Ghana
In grade 3 and 5, average pupils name 50 and 70 letters per minute and read
35 and 67 words per minute. Average reading skills are about half the US
standards, but sufficient for text comprehension. Among assessed pupils in
grade 3 and 5, 32 percent and 63 percent can read at least 45 words per
minute while 22 percent and 6 percent of them could not read a single word.
The average reading comprehension test scores are 37 percent in grade 3
and 58 percent in grade 5. Girls and boys perform equally well except in the
letter naming assessments where boys in grade 5 score higher.
72 68
49 50
3
5
3
Mult.
5
Div.
Girls
Literacy Mean Scores - Gambia
100
50
0
61 51
4
83 71
6
Letters/min
34 26
4
73
57 40 32 63 55
6
Words/min
Boys
4
6
Comp.
Girls
DIME Brief on Making the Grade 2
There are a significant numbers of low and high performers on subtraction
problems: 37 percent (grade 3) and 18 percent (grade5) could not solve a
single problem and 8 percent (grade 3) and 26 percent (grade 5) obtained a
score higher or equal to 87.5 percent. As for literacy skills, boys’
performance exceeds girls’ in all arithmetic areas.
Numeracy Mean Scores Gambia
100
50
6258
8481
3
Senegal
Average grade 3 and 5 students name letters at a rate of 29 and 64 letters
per minute and read the text at a rate of 14 and 45 words per minute.
Reading fluency rates are thus one fourth (grade 3) and half (grade 5) of the
US standards and 3rd graders’ fluency falls considerably short of the
neuroscience lower bound. Besides, the proportions of grade 3 and 5 pupils
reading more than the 45 words per minute are 10 percent and 65 percent
and zero scores are common (20 percent) in grade 3 but rare in grade 5 (1
percent). Average comprehension test scores are correlated with reading
fluency. Mean 3rd and 5th graders respectively score 27 percent and 57
percent.
Average math scores in grade 3 and 4 respectively are 55 percent and 66
percent on additions and 25 percent and 38 percent on multiplications.
Subtraction scores distribution reveals few average students and a
considerable proportion of extremes. In grade 3, 21 percent and 2 percent
of students have null and perfect scores and in grade 4, 10 percent and 4
percent of them do. Boys score significantly higher than girls on all types of
problems and the score gap widens from grade 3 to 4.
3
5
3
Sub.
Boys
5
3
Mult.
5
Div.
Girls
Literacy Mean Scores - Senegal
80
60
40
20
0
65 62
29 29
3
14 14 47 44 28 27
5
3
Letters/min
5
59 56
3
Words/min
Boys
5
Comp.
Girls
Numeracy Mean Scores - Senegal
100
50
0
8783
5251
3330
3
5
8076
3
Add.
5
2017
7872
3
Sub.
Boys
Policy recommendations
Most students in the five countries under observation do not master basic
literacy and numeracy skills. Systematic skills assessment is only the first
step: it points out the problem but not the solutions needed to enhance the
quality of education and learning achievement. The Africa Program of
Education Impact Evaluation (APEIE) is working in 14 Africa countries to
test the range of education policies that may help increase education
quality. The interventions that are being tested include: (i) improving
accountability structures within schools through greater parental and
community participation, performance–based contracts and monitoring
practices; (ii) releasing resource constraints and increasing decentralization
of budget decisions through school grants; (iii) improving availability of
learning materials and reducing class size through double shift; (iv)
introducing e-assisted learning and books; and (v) improving teacher
performance through distance learning and incentives. The results are
starting to provide much needed guidance to improve education policy.
5
Add.
Average numeracy scores are 51 percent (grade 3) and 85 percent (grade 5)
on additions and 19 percent (grade 3) and 75 percent (grade 5) on
multiplications. Akin to reading fluency, an important proportion of zeros,
38 percent, are observed in grade 3 for subtractions. To the contrary, in
grade 5, there are 3 percent of null and 33 percent of perfect subtraction
scores. Gender gaps are only significant in grade 5 and favors boys’.
Sierra Leone
4136 6460 2520 5651 1412 3835
0
7 7 5343
5
3
Mult.
5
Div.
Girls
Numeracy Mean Scores - SL
80
60
40
20
0
5653
3
6864
4
Add.
3632
5347
3
4
Sub.
Boys
2723 4234 1714 2721
3
4
Mult.
3
4
Div.
Girls
www.worldbank.org/dime
To contact the authors email [email protected] , [email protected] and
[email protected]
DIME Brief on Making the Grade 3