Literacy Boost
Kenya Baseline
June 2013
Noah Mann, Margaret Kamau, and Paul Titomett
With special thanks to:
Jarret Guajardo, Cameron Ryall, Lucy Githinji, Edwin Opiyo, John Mutisya, Mary
Mugo, Lucy Mutheu, WVK support staff, John Kingori and Peter Nyororo (WVK
IPA managers from Kalawa and Mutomo respectively), Ronald Ngetich, Adome
Joseph, Bernard Mulei, pupils, teachers and head teachers of participating schools
in the Kalawa and Mutomo communities;
and the team of assessors:
Festus M. Mwau, Daisy Maseki, Agnes Muthama, Mary Nzau, Stellamaria Mutua,
Marrieta Makundi, Samuel Muthama, Esther Mutula, Erick Kithuka, Joseph Kimuyu,
Steve Mwanzia, Dennis Muteti, Saidi Titus, Josephine Matuku, Ancent Kithuka,
Joyce Kavinya, Alice Maithya, Kennedy Musilu, Abednego Kyusya, Patrick Kisilu,
Moki Daniel, Alex Mutinda, Henry Nyamai, Isaac Wambua, Lucy Mutua, Ravasco
Nzioki, and Samson M Ikuthu.
© 2013 Save the Children
Executive Summary
This report examines the results of a primary school learner background survey and reading
assessment conducted in early 2013. The survey and reading assessment covered 944 grade 2
learners throughout 51 schools in World Vision Kenya's Kalawa ADP in Mbooni East District,
Makueni County and Mutomo ADP in Mutomo and Ikutha Districts of the broader Kitui
District, Kitui County. The 51 schools are randomly split by clusters into 26 primary schools
designated to receive Save the Children (SC)’s Literacy Boost program in partnership with
World Vision Malawi, and 25 comparison primary schools to receive no intervention this
school year. The Literacy Boost program includes teacher training, community reading
activities, and age-appropriate local language material creation to support emergent literacy
skills among early-grade children. As part of Literacy Boost, learners are periodically assessed in
each of these skills through an adaptable assessment tool to inform programming and estimate
program impact. The data gathered from these schools is analyzed to present a snapshot of the
emergent literacy skills of grade 2 learners in these schools and to inform the adaptation of
SC’s Literacy Boost program to this context.
This report has three purposes. The first purpose is to provide basic information on the
students’ reading skills. The second is to analyze how reading skills vary by student background
characteristics and the home literacy environment. The final purpose is to test the extent to
which the comparison students and schools are similar to Literacy Boost students and schools
to ensure that they will provide an appropriate counterfactual at endline.
Results from reading skill assessments administered to Literacy Boost and comparison students
showed that students fairly high concepts about print and letter knowledge. However, students
were able to read on average less than 50% of the 20 most used words in a government
textbook. Most disturbing, less than 25% of students could read at least 5 words correctly in 30
seconds in Kikamba or Kiswahili.
In the listening comprehension subtest, learners generally struggled most with inferential
questions that required the learners to think critically. This presents an area where even nonreading family and community members can help the learners. When reading stories or telling
stories to learners, teachers, family and community members should be encouraged to engage
the learners in critical thinking. Asking the learners questions like "What do you think will come
next?" and "Why do you think the character did that?" will help learners engage in the stories
and gain critical thinking skills.
Literacy Boost students for the most part reported strong home literacy environments. The
vast majority of students reported both positive habits and abundant reading materials in the
home. It is important to note however, that the students of low socio-economic status were
most likely to be in the minority of students who did not report strong learning environments.
Regression analysis showed a particularly strong correlation between basic reading skills and
both the number of reading materials in the home and the number of household members who
read to learners. These relationships provide further evidence of the importance of the home
literacy environment in developing basic reading skills. Furthermore, there is some evidence
that both positive home learning environments habits and extra types of reading materials may
1
help students from poorer families close the gap in reading skills with students from more welloff families.
Interestingly, female learners had slightly higher basic reading scores than male learners, but
male learners had slightly higher reading and listening comprehension scores. It is difficult to
find the cause of these differences, although different kinds of household responsibilities could
partially account for them.
Finally, there were few significant differences between Literacy Boost and comparison students
in their basic reading skills and their home literacy environments. In addition, the random
assignments of clusters of schools to be Literacy Boost and comparison schools means that we
can expect comparison schools to be equal to Literacy Boost schools in unobserved
characteristics as well. Thus, comparison students are likely to be a good indicator of how well
Literacy Boost learners have progressed over the course of the year.
2
Contents
Introduction............................................................................................................................................................4
Context ...................................................................................................................................................................4
Methods ...................................................................................................................................................................9
Sampling ...........................................................................................................................................,....................9
Measurement .......................................................................................................................................................9
Analysis .................................................................................................................................................................10
Children’s Background ………………………………….......................................................................10
Children’s Reading Skills ……………………………………...............................................................12
Concepts about Print ......................................................................................................................................12
Letter Identification .........................................................................................................................................12
Word Recognition: Most Used Words .......................................................................................................13
Fluency and Accuracy .....................................................................................................................................13
Comprehension .................................................................................................................................................13
Children’s Home Literacy Environment................................................................................................14
Literacy Boost Site Profiles: Skills and the Home Literacy Environment…........................15
Relationship between reading skills and the HLE……………………………………………17
Gender Differences in the HLE and Reading Skills…………………………….........................19
School Context ..................................................................................................................................................21
Conclusion ………………………………………………………………………………………….22
Appendix A: Inter-rater Reliability ............................................................................................…….....24
Appendix B: Regression Results ………………………………………………………………...25
Appendix C: Skill and HLE Profiles by ADP …………………...…………………………......26
3
1. Introduction and context
Introduction
This report examines the results of a learner background survey and reading assessment
conducted in early 2013. The survey and reading assessment covered 944 grade 2 learners
throughout 51 schools in Kenya. The 51 schools are split into 26 primary schools designated to
receive Save the Children's (SC) Literacy Boost program, and 25 comparison primary schools
that will not participate in Literacy Boost.
The Literacy Boost program includes teacher training, community reading activities, and ageappropriate local language material creation to support emergent literacy skills among earlygrade children. These skills include concepts about print, letter awareness, single word reading
of most used words, reading fluency, reading accuracy, and reading comprehension. As part of
Literacy Boost, learners are periodically assessed in each of these skills through an adaptable
assessment tool to inform programming and estimate program impact. The data gathered from
these schools is analyzed to present a snapshot of the emergent literacy skills of grade 2
learners in these schools and to inform the adaptation of Save the Children’s Literacy Boost
program to this context.
The key research questions to be explored in this report include the following:
1. How comparable are learners in Literacy Boost schools versus comparison schools in
terms of background characteristics, home literacy environment (HLE), school
environment, and reading skills?
2. What can the baseline tell us about learners’ emergent reading skills? What does this
mean for Literacy Boost programming?
3. How do learners’ reading skills vary by student background, school environment, and
home literacy environment? What does this mean for targeting Literacy Boost’s two
strands of intervention?
To investigate these questions, this report will first describe the research methods used;
including sampling, measurement, and analysis. Next, in order to see if groups are statistically
similar, the comparability of learners at Literacy Boost and comparison schools will be
examined through clustered t-tests. The comparability of Literacy Boost and comparison
learners’ scores for each of the emergent literacy skills and learners’ strengths and weaknesses
in each skill will also be examined. The following section will investigate any correlations with
student background, school environment, or home literacy practices and environment variables
using multilevel regression analysis. Finally, the report will examine differences between female
and male learners and the comparability of LB and comparison schools.
Context
Kenya lies on the East Coast of Africa with a total land area of 582,648 square kilometers.
About 75-80% of the population lives in the rural areas. The population in absolute poverty was
estimated to be 46% in 2006. The country has a new two level system of governance- the
national government and the devolved county government. The government’s socio-economic
4
development strategy is anchored in the Kenya Vision 2013, the new development blueprint
covering the period 2008-2030.
Kenya is a signatory to various International Conventions and Commitments aligned to the
well-being of children. Every child in Kenya has a right to free and compulsory basic education,
a provision under the Constitution of Kenya 2010 and stated in The Basic Education Bill of
2012.
Education is one of the social sectors under the social pillar of the Vision 2030 1. The Kenyan
government recognizes education as a basic tool that has a major responsibility of facilitating
the process of developing manpower necessary for transforming into a globally competitive
country 2. Education is also viewed as significant in creating a knowledge-based society able to
uphold justice, democracy, accountability, cultivating nationalistic values and results-oriented
political engagement3. This is hoped to be realized through enhanced access, equity, quality and
relevance to education. Education priorities are guided by national development strategies in
the Vision 2013 and the Constitution 2010. Kenya subscribes to the global commitment of
Education for All (EFA) which pledges the provision of quality basic education for all children,
youth and adults including those from ethnic minorities and other marginalized groups. The
Kitui District Development Plan 2008-2012 states that the most important resource for a
country is a competitive, effective, healthy and well-educated people. The plan also notes that
education improves the quality of the available labor force, which results in economic growth
and poverty reduction.
The current education structure in Kenya consists of 8 years of primary education (age 6 to
13), 4 years of Secondary education (age 14 to 17) and 4 years of Higher education (age 18 to
21 and above). The Early Childhood Development and Education (ECDE) level is for children
age 0 to 5 years. While the structure is applicable in both private and public institutions, school
attendance indicators shows that children attend school even when they are outside of these
age brackets. Pupils not able to transition to secondary education are accommodated in youth
polytechnics and in tertiary level institutions such as Technical, Industrial and Vocational
Educational Institutions (TIVETs). Pupils exiting secondary education but unable to proceed to
the university or colleges may start small-scale income-generating activities. The Government
intends to promote retention of learners in Alternative Basic Education Training (ABET)
institutions.
1
Kenya Vision 2030
Education Sector 2013/14- 2015/16 Medium Term Expenditure Framework
3
Kenya Vision 2030.
2
5
Figure 1: Structure of Education and Training in Kenya 4
In Kenya, Free Primary Education (FPE) started in 2003 while Free Day Secondary Education
(FDSE) began in 2008 5 . This means that all fees are waived in all government (public) primary
schools. The FPE policy has resulted in an influx of learners into schools, resulting in high pupilteacher ratios, insufficient learning materials and congestion in classes 6.
According to the Education Sector 2013/14- 2015/16 Medium Term Expenditure Framework,
the ECDE Gross Enrollment Rate (GER) stands at 65.5% in 2011. The number of pupils in
formal primary schools in 2010 was 9.86 million (4.98 boys and 4.86 girls) with a Net
Enrollment Rate (NER) of 95.6% in 2011. Other vital statistics for the period are: The primary
Gender Parity Index improved from 0.98 in 2009 to 1:02 in 2010 in favor for girls, the
Textbook Pupil Ratio for 2009 was 1:2. There was an observed decline in the pupil completion
rate from 83.2 per cent in 2009 to 74.6 per cent in 2011, a factor attributed to school dropouts
and repetitions. The transition rate from primary to secondary has been increasing over the
years, standing at 73.3 per cent (68.9 per cent for male and 75.3 per cent for female) in 2011.
The UWEZO and NASMLA reports of 2010 and 2011 indicated that most children are
progressing to upper classes without numeracy and literacy competencies as required at grade
2, 3 and 6 7. The Government of Kenya (GOK) Education sector report of 2012 notes that the
national mean score in the primary examination declined from 54.3% in 2010 to 53.0% in 2011
.The report also shows that the country faces regional and gender disparities in education that
need to be reduced.
4
Basic education sector analysis report 2012- JICA
Basic education sector analysis report 2012- JICA
6
SACMEQ policy brief No. 2 on quality of primary school inputs in Kenya
7
UWEZO is a civil society organisation which employs household based learning and competency assessment
methodologies based on grade 2 curriculum while NASMLA is a national assessment of monitoring of learning
achievement which is school based and administered to pupils at grade 3 and 6 (quoted in the WVK Education
sector strategy 2013-2015).
5
6
The government has identified free Primary Education and Early Child Development and
Education as some of the priority programs within education. This prioritization has been based
on criteria that include: linkage with the objectives of the Government Medium Term
Development Plans (2008-2012 and 2013-2017), the degree to which the program addresses
core poverty interventions, cost effectiveness, sustainability of the program and immediate
response to the requirements of the implementation of the Constitution. The government
policy now is to mainstream ECDE in the primary school set up.
Only 61.5% of the adult population has attained minimum literacy level and only 29.6% of the
Kenyan adult population has attained the desired mastery literacy competency. This is likely to
affect the education of their children 8.
Gaps in education activities due to resource constraints include: inadequate financing of free
day secondary schools, a shortage of teachers, inadequate responses to psychosocial challenges
of youth through life skills education, inadequate education opportunities for marginalized
groups including those excluded from education due to poverty and HIV, a lack of access and
equity in special needs education, inadequate recruitment of additional teachers, inadequate
learning facilities, The Ministry of Education is in the process of developing the National
Education Sector Support Programme (NESSP) which is a five-year program for delivering the
reforms earmarked for the education sector in Kenya 9. One emerging issue in the education
sector is devolution of education to the newly created county-level governance under the
Constitution 2010 which will impact teacher services, ECDE and technical level education and
quality assurance 10. The Government recognizes the role different stakeholders play in meeting
education sector goals. Stakeholders include local authorizes, communities, political leaders,
civil societies, NGOs, donors and other Government Ministries and agencies.
Site Profile
Literacy Boost Project is being implemented in two areas in Kenya: Mutha Mutomo Area
Development Program (ADP), situated in Mutomo and Ikutha District of the broader Kitui
District, Kitui County and in Kalawa ADP, situated in Kalawa Division (Kathulumbi and Kalawa
location), Mbooni East District, Makueni County.
The Districts are mainly occupied by the Akamba community with other communities living in
the major towns of the districts. The majority of the population lives in the rural areas. The
land use and settlement patterns are based on agro-ecological zones and they are also
influenced by soil fertility and rainfall.
The following is a profile of the two districts 11 .
Kalawa lies within the arid and semi-arid zones of the country and is prone to frequent drought.
Agricultural production varies over the years depending on the amount of rainfall received per
8
EFA Global Monitoring Report 2006
World Vision Kenya Education sector strategy 2013-2015.
10
Republic of Kenya Education Sector Report 2012.
11
Government of Kenya, Kitui and Makueni District Development Plan 2008-2012
9
7
year. A low rainfall ranging from 300mm to 400mm is hardly enough to sustain any kind of
crops. Local people are therefore engaged in small stock rearing for economic activities. About
three-quarters of the people live below the poverty line in the district. Drought is a major
cause of poverty in the district and the most vulnerable are women, children, the aged and
disabled. Kalawa division lies at the lowest end of the newly created district and has had very
limited interaction with development partners besides GOK departments.
Kalawa Division of Mbooni East District has 135 teachers. The division has 19 primary schools
with a total population of 4941 (Boys 2530 Girls 2411) in 2013. A total of 84% of schools are
single streams 12. The schools in Kalawa Division have an average of 31 pupils in Standard 1, 2
and 313. The broader Kalawa Division (comprising of Kathulumbi and Kalawa locations) has 36
public primary schools and 13 public secondary schools. There are four village youth
polytechnics. Mbooni East district has 107 primary schools and 42 secondary schools 14. The
teacher/ pupil ratio in primary school is 1:33 while the secondary teacher/ student ratio is 1:32.
The District Development Plan identifies inadequate education facilities, inadequate staffing,
rising dropout rates and low transition rates as issues that need immediate intervention in
order to increase literacy rates in the district.
Mutomo District experiences high temperatures throughout the year. The unreliability of the
rainfall and perennial droughts limit intensive and meaningful land use. The district experiences
crop failure and water shortage in most years which affects family income and the wellbeing of
children. According to the Central Bureau of Statistics, the district has 67.2 percent of the
population below the poverty line making residents in this area among Kenya’s poorest.
Absolute poverty in the area is manifested in various ways such as hunger, ill health, lack of
access to quality education and even lack of participation in decision making.
According to statistics gathered during the inception of the program, Mutomo ADP area has 60
primary schools with a total enrolment of 14,899 (7602 boys 7297 girls). There are 286 fully
trained teachers employed by the government and 228 untrained and supported by the Parents
and Teachers Association (PTA). The primary teacher/pupil ratio stands at 1:52 which is far
below the recommended standard of 1:25. Mutomo district has a primary school transition rate
of 39.7% 15. The district has received increased government investment in school infrastructure
but this has not been commensurate to performance in academics. The district is ranked among
the poor performers in the country, a state that is attributed to low enrolment, transition,
retention and completion rates, poor curriculum implementation, low literacy levels among
parents leading to complacency, understaffing, lack of motivation and poor learning/teaching
materials. The community members stress that many school-aged children within the ADP are
not in schools; those who are have learning that is characterized by irregular attendance due to
absenteeism and eventual dropping out for both girls and boys. Cultural factors, such as early
marriage and a preference for educating boys, as well as the heavy productive demands placed
on girls, create significant gender differentials in educational outcomes.
12
Source: Area Education office Kalawa Division
Ibid
14
Mbooni East District education office
15
Mutomo district education office
1313
8
The transition rates from primary level to secondary level and from secondary to tertiary level
is low. Many children have dropped out of school due to lack of school fees and this hinders
transition. The most affected are the OVCs, children from poor families and those under the
care of elderly guardians who are economically disempowered. The competitive job market and
hard economic times in the county exposes these children to uncertainty as they have not
acquired quality education to enable them to find gainful employment opportunities.
Community members also observe that while children accessed primary and secondary
education, the majority of them lacked skills and knowledge required for either self or formal
employment. The education curriculum centres more on classroom work rather than equipping
the children with marketable and self-employable skills as they approach adolescence. Youth in
the area are therefore unable to free themselves from the cycle of poverty. An additional
concern is the lack of quality data, and specifically the lack of baseline information against which
progress towards key education goals and outcomes can be monitored.
II. Methods:
Sampling
The sample for this baseline assessment encompasses 944 standard 2 learners, divided between
26 schools set to receive the Literacy Boost intervention (n of learners = 484) and 25
comparison schools (n of learners = 460). Clusters of schools were randomly assigned to either
receive the Literacy Boost intervention or serve as comparison schools. Within each school, 10
boys in and 10 girls in standard 2 were randomly selected to be assessed at baseline and again
at endline. Some of the classes were less than 20 – due to the overall enrolment in the school
being small because they are remote or have small feeder communities. Because we were able
to randomly assign schools, we expect Literacy Boost and comparison students to be equal in
both observable and unobservable characteristics. Thus, we will be able to attribute any
differences between Literacy Boost and comparison schools at endline to the Literacy Boost
intervention itself.
Measurement
School profile data were collected via direct observation and a survey of school head teachers
at every school in the sample. This data includes information on enrolment & class size,
availability of textbooks, school facilities & construction, and teacher backgrounds. For the
student assessment, all learners in the sample were asked about their background
characteristics (age, household possessions, household building materials, etc.). Learners also
were asked about their family members and reading habits in their home (who they had seen
reading in the week prior to the assessment, who had read to them, etc).
After collecting this background data, all learners were also given an emergent literacy test
composed of six components administered through five sub-tests: concepts about print, letter
awareness, single word recognition (reading of most used words), reading fluency & accuracy
(words per minute read correctly and total percentage of passage read correctly; both within
the same sub-test), and a set of comprehension questions about the fluency & accuracy passage.
The same set of comprehension questions were administered for both those learners who
9
could read independently (reading comprehension) and those who could not and thus had the
assessor read to them (oral comprehension). All instructions were given in Kikamba, and
children were assessed on letter identification, most used words, reading fluency, reading
accuracy, and listening/reading comprehension in both Kiswahili and Kikamba.
To test inter-rater reliability, approximately 10 percent of learners (99 out of 944) were
assessed by two enumerators simultaneously. All of the skills tests had excellent or good
reliability; full results can be found in Appendix A.
Analysis:
The critical purpose of this analysis is to test whether the Literacy Boost learners and the
comparison learners are equal in terms of background and skills. That is, do these learners
possess the same resources and capabilities? This question is important so that at end-line, we
can know how much Literacy Boost has, or has not, contributed to learners' accelerated
reading development.
To test the comparability of learners in the Literacy Boost and comparison samples, this report
will use comparison of means through t-tests, with clustered standard errors to account for the
grouping of student-level data within schools. Summary statistics, accompanied by clustered ttests, will be used to analyze learners’ performance in each of the reading sub-tests. Finally, this
report will look to multilevel regression models to explore relationships between literacy skills
and student background characteristics, school environment, and home literacy environment.
A Note about Benchmarks
While Save the Children has used this approach to reading assessment and intervention in Malawi,
Nepal, Mali, Pakistan, Ethiopia, Uganda, Zimbabwe, Bangladesh, and Mozambique, comparison
across countries and languages is less helpful than more detailed contextual information for setting
expectations of impact. For each measure used in these assessments, the upper end of the range of
scores can be used to consider what is currently possible among these children. Save the Children
usually presents the scores at the 75th percentile of each measure from this baseline assessment to
constitute the benchmarks. However, due to the extremely low reading skills of both the LB and
comparison groups, benchmarks are set high, often in the 95th or even the 99th percentile. These
scores will constitute the benchmarks which the Literacy Boost program will attempt to help
learners achieve by the time of end-line assessment.
III. Children’s Background
Students attending Literacy Boost schools are on average eight years old, speak Kikamba as
their primary language, and live with four other people in their households. Nearly all Literacy
Boost students have attended an early childhood development (ECD) programs, for an average
of one to two years. Despite near universal ECD attendance, nearly a third of LB students
repeated Primary 1, and nearly a fifth repeated Primary 2.
10
Table 1:
Selected student characteristics:
N
Literacy Boost
Female
944
49.6%
Age
944
7.8
Attended ECD
944
97.7%
N of years of ECD
829
1.5
Repeated Primary 1
942
30.6%
Repeated Primary 2
944
19.0%
Kikamba primary language
944
98.8%
N of household members
944
5.0
Height
942
120.4 cm
Weight
944
20.5 kg
Student ate breakfast previous morning
944
80.0%
*** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05
Comparison
47.6%
7.8
98.0%
1.6
38.6%
17.4%
99.3%
4.8
120.7 cm
20.4 kg
79.6%
Sig. Diff.
-
*
-
Students in comparison schools tended to be similar to students in LB schools, although they
attended ECD programs for slightly, though statistically significantly, fewer years. It is not
unusual to find a small number of statistically significant differences in background variables;
these minor differences are unlikely to indicate that the randomization was compromised.
Table 2 shows the household possessions and livestock present in the households of LB and
comparison students as a proxy for socio-economic status. About three quarters of LB
students reported the presence of radios, cellphones, and toilets in their households, but very
few reported having electricity or televisions. Most LB students reported owning livestock;
over 80% reported owning chickens and goats, the most frequently reported types of livestock.
Table 2:
Socio-economic Status
N
Literacy Boost
Radio
944
73.1%
Electricity
944
6.2%
Television
944
5.8%
Cell Phone
944
80.2%
Toilet
944
83.5%
Bicycle
944
45.7%
Bed
944
92.4%
Number of types of household possessions
944
3.9
Sheep
944
16.5%
Goats
944
81.4%
Cattle
944
65.1%
Donkeys
944
73.1%
Chickens
944
82.9%
Number of types of livestock
944
3.2
Number of types of assets
944
7.1
*** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05
Comparison
72.4%
5.4%
6.7%
75.4%
79.1%
51.7%
92.2%
3.8
16.7%
84.1%
61.3%
73.5%
87.0%
3.2
7.1
Sig. Diff
-
LB and comparison students appear have relatively similar socio-economic status, further
indicating that the randomization procedure was correctly carried out.
11
Table 3:
Selected Health Characteristics
N
Literacy Boost
Mosquito net –last night
944
43.8%
Mosquito net – always
944
47.1%
Ever had malaria
941
83.2%
Had diarrhea recently
944
6.8%
Healthy
942
87.0%
*** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05
Comparison
46.5%
49.6%
83.9%
6.5%
91.0%
Sig. Diff.
-
Most children in both LB and comparison schools reported having had malaria at least once in
their lifetime. The extent to which learners are still affected by malaria is unclear. About half of
learners reported sleeping under mosquito nets, although paradoxically, more learners
reported always sleeping under a mosquito net than sleeping under a mosquito net the previous
night. This finding indicates that these data may not be completely reliable. Around 90% of
learners reported feeling healthy in both LB and comparison schools the day of the assessment.
IV. Children’s Reading Skills
Table 4:
Reading Skills:
N
LB
Comparison
CAP
944
74.8%
75.4%
Kikamba Letter Identification (ID)
944
82.7%
79.3%
Kiswahili Letter Identification (ID)
944
64.3%
60.2%
Kikamba Most Used Words
944
31.1%
29.1%
Kiswahili Most Used Words
944
36.0%
32.3%
Non-readers - Fewer than 10 Kikamba wpm
944
77.7%
79.6%
Non-readers - Fewer than 10 Kiswahili wpm 944
79.1%
82.4%
Kikamba Fluency – sample average
944
4.9 wpm
4.9 wpm
Kiswahili Fluency – sample average
944
4.5 wpm
5.3 wpm
Kikamba Accuracy – sample average
944
20.8%
19.0%
Kiswahili Accuracy – sample average
944
19.7%
16.2%
Kikamba Listening Comprehension
742
49.5%
48.5%
Kiswahili Listening Comprehension
762
39.8%
36.9%
Kikamba Reading Comprehension
202
59.6%
54.3%
Kiswahili Reading Comprehension
182
50.8%
46.0%
*** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05
Sig.
Diff.
*
75th
percentile
91.7%
95.2%
75.0%
70.0%
73.8%
0
0
0
0
60%
60%
70%
70%
Suggested
Benchmark
100%
100%
100%
50%
50%
10 wpm
10 wpm
50%
50%
100%
100%
100%
100%
Concepts about Print
The first sub-test of the reading assessment consisted of 12 ‘concepts about print’ (CAP)
questions. These questions concern familiarity with books – where to start, which way to read,
what is a letter, what is a word, etc. LB learners correctly answered approximately 75% of
concepts about print assessed. Learners at Literacy Boost schools and at comparison schools
had statistically indistinguishable scores.
12
Identifying the book’s title (45% correct), identifying the end of the story (50% correct), and
following along while the assessor read (49% correct) were the concepts about print with
which LB students had the most difficultly. Each of the other concepts about print assessed was
understood by more than 65% of LB students.
Letter Identification
The next sub-test examined learners’ letter awareness. Learners were shown a chart of letters
and asked to name the letter, pronounce the letter sound, or give a word that begins with that
letter. The average LB learner could identify over 80% of tested upper and lower case Kikamba
letters and over 60% of Kiswahili letters. Differences between students at LB and comparison
schools on this measure were small and not statistically significant.
The only Kikamba letter that fewer than 60% of LB learners were able to identify was a
lowercase l as only 22% of LB learners correctly identified it. Learners likely confused a
lowercase ‘l’ with an uppercase ‘I’. Literacy Boost should instruct teachers to take
special care to point out the difference between the lowercase ‘l’ and uppercase ‘I.’
There are seven digraphs in the Kiswahili language, including for example ‘dh’, ‘sh’, and ‘ch.’ No
more than 15% of LB learners were able to identify any of the digraphs, while most learners
were able to identify the other letters, which are in large part similar to Kikamba letters.
Perhaps learners’ familiarity with Kikamba helped them to identify similar letters in Kiswahili,
but did not did not translate to being able to identify the seven digraphs. Literacy Boost
should instruct teachers to help students learn digraph letters in Kiswahili.
Word Recognition: Most Used Words
The most used words (MUW) sub-test consists of a chart of 10 words that the student is asked
to read. These 10 words were identified as ‘most used’ by tabulating the number of times a
word appeared in learners’ language arts textbooks. Interestingly, LB readers were able to read
a slightly higher proportion of Kiswahili most used words (36%) than Kikamba most used words
(31%) despite identifying a higher proportion of Kikamba letters. This suggests that LB learners
may have greater access to printed materials in Swahili. Literacy Boost should provide
access to printed materials in Kikamba so that they are able to practice reading in
their first language. Comparison students had similar results as LB students.
Fluency and Accuracy
Fluency (words per minute read correctly) and accuracy (percent of the passage read correctly)
are presented together here because they are measured together in a single sub-test in which
learners read a passage aloud. The number of words learners read correctly in 60 seconds is
tracked for fluency. As the student continues to read after the first 60 seconds, the total
number of words read correctly from the passage as a whole, no matter how long it takes the
student, is computed for accuracy. If a student is not able to read at least five words correctly
in 30 seconds, then the assessor stops the student and reads the rest of the passage aloud.
Students who are read the passage are labeled non-readers.
The majority of LB students were unable to read at least five words in 30 seconds in Kikamba
(77%) and Kiswahili (79%), and thus were labeled non-readers. On average, including those
13
students who were labeled non-readers, LB learners were able to read approximately five
words per minute correctly in both languages. LB learners only correctly read 20% of Kiswahili
and Kikamba words.
Comprehension
The final sub-test quizzed learners on a series of ten comprehension questions related to the
reading passage. Learners who were unable to read more than five words correctly were read
the passage by the assessors before being asked the comprehension questions orally. Learners
who read the entire passage were also asked the comprehension questions orally. Six of the
comprehension questions were factual, two were inferential, one each were summative and
evaluative.
Unsurprisingly, both readers and non-readers were able to answer a higher proportion of
Kikamba questions than Kiswahili questions. The inferential questions were the most difficult
types of questions for readers and listeners of the Kikamba passage, while readers and listeners
of the Kiswahili passage had equally low scores on the factual and inferential questions. Thus,
when reading stories to learners, LB participants should be supported to engage
the learners in critical thinking. Asking the learners questions like "What do you
think will come next?" and "Why do you think the character did that?" will help
learners engage in the stories and gain critical thinking skills.
Literacy Boost and comparison students had statistically different scores in Kikamba reading
comprehension. This finding may indicate that students in Literacy Boost schools as a whole
have higher reading comprehension scores than students in comparison schools.
V. Children’s Home Literacy Environment
Table 5:
HLE
N
Literacy Boost
Number of types of reading materials
944
2.0
Has books at home
944
86.8%
Textbooks
944
55.2%
Religious books
944
68.6%
Magazines
944
7.0%
Newspapers
944
25.8%
Stories
944
30.8%
Coloring books
944
15.3%
Kids’ books
944
37.4%
*** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05
Comparison
2.0
83.2%
53.9%
69.1%
5.4%
26.1%
31.3%
13.9%
37.6%
Sig. Diff.
-
Over 85% of LB learners reported that there were books in their households, while religious
books were the most commonly reported type of book. Only one third of LB students
reported having a type of childrens’ book, either story books or coloring books, in their
household. Furthermore, LB and comparison students of higher socio-economic status were
significantly more likely to report the presence of kids’ books in their households. Learners
with kids’ books reported having 4.3 types of household possessions on average, while learners
without kids’ books reported 3.6 types of household possessions on average. Literacy Boost
14
should focus on improving learners’ access to recreational reading material
designed for children, with particular attention paid to learners of low socioeconomic status. LB students and comparison students reported having statistically similar
types of books in their households.
Table 6:
HLE (past week)
N
Literacy Boost
Comparison
Seen reading
944
89.5%
90.0%
Helped or told learner to study
944
87.6%
90.7%
Read to learner
944
83.9%
82.8%
Told a story to learner
944
84.5%
78.0%
Played a game with learner
944
79.8%
77.0%
*** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05
Sig. Dif.
*
-
Given that nearly all students reported the presence of reading materials in their homes, it’s
unsurprising that nearly 90% of learners sampled reported observing at least one person
reading in their household over the past week. It is encouraging that this high rate of adult
literacy appears to translate to child-focused HLE behaviors as well; over three quarters of
learners reported that a household member helped or told them to study, read to them, told
them a story, and played a game with them.
However, learners of higher socio-economic status were significantly more likely to report
each of these HLE behaviors (with the exception of being helped or encouraged to study, which
was not significantly related to socio-economic status). It is thus important that Literacy
Boost focus its community action efforts on the poorest, most marginalized
households.
A significantly higher proportion of Literacy Boost students reported that at least one member
of their household told them a story over the previous week than comparison students.
VI. Literacy Boost Site Profiles: Skills and the Home Literacy
Environment
This section summarizes the reading skills and HLE data for Literacy Boost students. It includes
a chart summarizing CAP, letter identification, MUW, fluency, accuracy, and comprehension
scores as well as a chart summarizing students’ home literacy environments. Figure 2 displays
Literacy Boost students’ average scores on all literacy sub-skill assessments, and Figure 3 shows
students’ average home literacy environment. Appendix C shows the skill and HLE profiles for
each ADP.
15
Figure 2:
Percent Correct
Overall Kenya Skills Profile
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
75%
83%
78% 79%
64%
60%
50%
31%
51%
40%
36%
21% 20%
5 wpm5 wpm
Reading Skill
Figure 3:
Percent of households
Kenya HLE Profile
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
90%
88%
84%
85%
80%
87%
37%
Seen reading Helped or
told learner
to study
Read to
learner
Told a story
to learner
Played a Has books at
game with
home
learner
Has kids'
books at
home
HLE Dimension
16
VII. Relationship between reading skills and the HLE
Each of the HLE habits, along with the availability of reading materials in the home, is
significantly associated with at least one of learners’ concepts about print scores, letter
identification scores, and most used words scores, when controlling for age, gender, socioeconomic status, and household size and adjusting standard errors for clustering at the school
level. These associations however do not prove that any of these behaviors cause learners to
improve their reading skills. For example, it is plausible that learners with a greater
understanding of concepts about print are more likely to notice and report the presence of
reading materials in the household.
That said, it is still useful look at which HLE habits seem to be most often and most strongly
correlated with reading skills. By doing so, we will be able to verify that the kinds of activities
promoted by Literacy Boost have the potential to increase children’s reading scores.
Average number of Kikamba letters
correct
The number of people in household who read to the learner and the number of types of
reading materials in the household are frequently and strongly correlated with reading skills,
when controlling for age, gender socio-economic status, and household size. One reading skill
for which these relationships are particularly evident is letter identification. Learners who
reported that more than one household member read to them over the past week were able
to identify approximately 35 Kikamba letters on average, while learners who reported that zero
or one household member read to them over the past week were only able to identify 33
Kikamba letters on average. As is evident in figure 4, this relationship holds for the number of
reading materials in the
household. Regression
Figure 4. Relationship between
analysis confirms that this
observed difference is
HLE and Letter ID
statistically significant when
40
controlling for age, gender
35
socio-economic status, and
2.0
2.5
30
household size and
25
adjusting standard errors
20
for clustering at the school
32.8
32.5
15
level. For every additional
10
household member who
5
was reported to have read
0
to a learner over the past
Household members who Types of reading materials in
week and for each
read to learner
household
additional type of reading
HLE Habits and Materials
material in the household,
the learner was able to
0-1 Member or Material Gain if More than 1 Member or Material
identify half of an additional
Kikamba letter.
17
Number of Kiwhahili letters identified
There is also some evidence that it is more important that learners are read to in poorer
households than in less poor households; similarly, the number of reading materials available
appear to be more important in the poorest households. Figure 5 shows an interaction in which
the number of household members
who read to learners is especially
Poverty and Reading to
important for learners of the lowest
socio-economic status. This trend is
Learners
echoed for the number of types of
44
reading materials. These effects
suggest that the gap between
42
learners of high and low socio40
economic status can be
38
significantly reduced simply be
1 possession
36
ensuring that all learners have
2 possessions
access to multiple types of print
34
in their household and that all
3 possessions
32
learners are read to as much as
4 possessions
30
possible in their homes. These
0
1
2
3
4
effects were not significant on the
number of Kikamba letters identified,
Number of household members who read
but the coefficients were of the same
to learner
direction.
VIII. Gender Differences
There are a number of notable differences between male and female learners, chief among
them that female learners appear to have significantly better basic reading skills than male
learners. More specifically, female learners identified a higher proportion of both Kiswahili
letters and were able to read more Kikamba and Kiswahili most used words than male learners.
However, male learners answered a higher proportion of Kiswahili reading and listening
comprehension questions correctly than female learners.
18
Figure 6:
Reading Skills by Gender
100%
90%
Percent Correct
80%
70%
60%
50%
40%
Male
30%
Female
20%
10%
0%
CAP
Kikamba Kiswahili Kikamba
Letters Letters
MUW
Swahili
MUW
Kikamba Kikamba Kiswahili Kiswahili
Listening Reading Listening Reading
Comp.
Comp.
Comp.
Comp.
Reading Skill
*** Significant at p<.01, ** Significant at p<.05, * Significant at p<.1
Is it difficult to pinpoint one cause for these differences, but these results generate some
interesting hypotheses. One is that boys could be more actively engaged both inside and
outside of the classroom. If boys are encouraged to be active participants in storytelling, playing
games, or reading aloud more frequently in school or at home, then they may develop basic
comprehension skills more quickly than girls. There is some very weak evidence that these
activities may play a role in the comprehension gap evident between girls and boys; on average,
boys reported that more members of their household help them study, read to them, tell
stories to them, and play games with them than girls do. However, it is important to note that
these differences are not statistically significant, meaning that they cannot be generalized to the
broader population with confidence. The only statistically significant difference between girls
and boys in the HLE is that a higher percentage of boys reported that at least one household
member played a game with them over the previous week. Despite the lack of firm evidence
that girls are disadvantaged in the HLE, Literacy Boost should train teachers and
sensitize members of the community on the importance of actively engaging both
girls and boys in storytelling, games, reading aloud, and studying.
19
Table 7:
HLE Variable
Male
Female
N Seen reading
2.7
2.4
N Helped or encouraged learner to study
3.0
2.8
N Read to learner
2.3
2.1
N Told story to learner
2.5
2.3
N Played game with learner
2.1
1.8
Seen reading
91%
89%
Helped or encouraged learner to study
90%
89%
Read to learner
83%
84%
Told story to learner
83%
80%
Played game with learner
82%
74%
*** Significant at p<.01, ** Significant at p<.05, * Significant at p<.1
N
944
944
944
944
944
944
944
944
944
944
Sig
**
Another possible hypothesis as that could partially explain why girls have higher basic reading
skills but lower comprehension skills than boys could be that the different household chores
frequently conducted by girls and boys could be related to their learning. Girls were more likely
than boys to report regularly doing chores within the household, such as sweeping, washing,
cooking, and caring for children, while boys were more likely to report regularly doing chores
outside of the household, like farming and herding. Performing chores inside of the home may
give girls more opportunities to interact with print in the home than are available to boys as
they perform chores in the fields. In addition, boys reported spending significantly more time
doing chores than girls did in the afternoon, perhaps further impeding their opportunities to
study. However, no significant differences were detected between girls and boys in reported
time spent studying. Literacy Boost should emphasize the importance of both girls
and boys having sufficient time and opportunity to interact with printed materials
on a daily basis.
Now that we’ve hypothesized that girls may more frequently perform chores than enable them
greater opportunity to interact with printed materials, we turn to a possible hypothesis as to
why boys have higher comprehension scores. By virtue of doing a higher proportion of their
chores outside of the home, boys may have more opportunities to interact with more
members of the community and to exercise critical thinking skills on a regular basis. Boys also
reported being more likely to work than girls, which may also contribute to the development of
critical thinking skills, but restrict the amount of time they have to become familiar with print in
the home. Literacy Boost should sensitize members of the community on the
importance of engaging girls in opportunities to exercise their critical thinking skills
IX. School Context
Since clusters of schools were randomly assigned to either be Literacy Boost or comparison
schools, we can assume that they have similar observable and unobservable characteristics. It is
also not necessary to test for significant differences in the population of LB and comparison
schools, as we have collected these data from all of the LB and comparison schools.
20
Table 8:
Selected School Level Variables
Distance from center
Has electricity
Has library
N
51
51
51
Comparison
26.2 km
8.7%
8.5%
LB
31.6 km
4.1%
6.2%
Literacy Boost schools are on average 32 km from the regional center, the farthest is over 100
km away. Very few schools have electricity, and equally few have libraries. This means that if
learners do not have reading materials in their households, they may be unable to spend time
becoming familiar with print outside of school. A key component of Literacy Boost should be to
provide all learners access to printed materials, not to rely on existing materials in households
or schools.
X. Conclusion
Clusters of schools were randomly assigned to receive the Literacy Boost intervention or serve
as comparison schools in the Lower Eastern region of Kenya. In total, 26 schools were assigned
to receive LB, while 25 were designated as control schools. Since clusters of schools were
randomly assigned to receive the intervention, we expect Literacy Boost and comparison
schools to be equivalent in both observable and unobservable characteristics. As expected,
descriptive statistics of sampled students within LB and comparison schools were similar, with a
only a few minor differences emerging. These minor differences do not indicate that the
randomization procedure was compromised, but may need to be controlled for at endline.
Learners in Literacy Boost schools tended to have strong scores on basic reading skills tests,
including concepts about print and letter identification, but had weak scores on more advanced
reading skills tests, including reading most used words, fluency, and accuracy. These results
suggest that while most students are able to recognize letters, they are largely unable to read
words or reading passages. On average, learners had similar reading skills in Kikamba and
Swahili, with the very notable exception of comprehension. Learners had higher listening and
reading comprehension in Kikamba, likely due to Kikamba being their first language.
Learners in Literacy Boost schools have fairly rich Home Literacy Environments (HLEs); most
reported the presence of books at home and that at least one member of the household read
to them, told a story to them, and help or encouraged them to study over the past week.
However, children of low socio-economic status were less likely to report these positive HLE
habits. It is thus important that Literacy Boost focus its community action efforts on
the poorest, most marginalized households. In addition, less than half of all learners
reported having kids’ books at home, with learners of low socio-economic status again the
most disadvantaged. Literacy Boost should focus on improving learners’ access to
recreational reading material designed for children, with particular attention paid
to learners of low socio-economic status.
Regression analysis confirmed that learners who reported more household members reading to
them and more types of reading materials in their household scored higher on basic reading
skills tests. The regression analysis also showed that HLE habits and materials appear to close
21
the gap between learners of high and low socio-economic status. Thus Literacy Boost
should emphasize the importance of HLE habits and HLE materials, especially in
the poorest households.
Finally, female learners scored slightly higher on basic reading skills tests than male learners, but
the relationship was reversed for comprehension. It is difficult to determine what might be
causing this divergence, but it is possible that the different types of chores and work that girls
and boy regularly perform could contribute to this difference. Girls are more likely to engage in
household work, like sweeping, washing, and childcare, while boys are more likely to work
outside of the home, farming, herding. It is possible that girls may have more time and
opportunity to interact with print, while boy may have more opportunities to exercise and
develop their critical thinking skills. Whatever the reasons for this divergence, Literacy Boost
should emphasize the importance of both girls and boys having sufficient time and
opportunity to interact with printed materials on a daily basis and exercise their
critical thinking skills through games and stories.
22
Appendix A: Inter-rater reliability
To test inter-rater reliability, approximately 10% of students (n=99) were assessed by two
enumerators simultaneously. Long one-way ANOVA techniques were used to calculate the
intra-class correlation within pairs of assessors for a measure of reliability. Table A presents the
results below. Using Fleiss’ benchmarks for excellent (ICC>0.75), good or fair
(0.75>=ICCA>0.4), and poor (0.4>=ICC); all of the literacy outcome variables exhibited
excellent or good or fair inter-rater reliability overall.
Table A1:
Measure
CAP
Kikamba Letters
Kiswahili Letters
Kikamba MUW
Kiswahili MUW
Kikamba Fluency
Kiswahili Fluency
Kikamba Accuracy
Kiswahili Accuracy
Kikamba Reading Comp.
Kiswahili Reading Comp.
Kikamba Listening Comp.
Kiswahili Listening Comp.
Kikamba Reader/Non-reader
Kiswahili Reader/Non-reader
ICC
.93
.99
.91
.89
.96
.77
.65
.93
.89
.96
.92
.93
.98
.89
.87
Rating
Excellent
Excellent
Excellent
Excellent
Excellent
Excellent
Good or Fair
Excellent
Excellent
Excellent
Excellent
Excellent
Excellent
Excellent
Excellent
23
Appendix B: Regression Results
VARIABLES
(1)
Total
Kikamba
Letters
(2)
Total
Kiswahili
Letters
Age in years
-0.469
-0.264
(0.295)
(0.374)
Sex (1=Female)
2.045***
3.437***
(0.657)
(0.751)
N of household possessions
0.729**
2.072***
(0.305)
(0.572)
N of household members
0.0485
-0.357
(0.250)
(0.406)
N of reading materials at home
0.482**
1.534**
(0.220)
(0.714)
0.461**
1.889***
N of hhold members reading to student
(0.199)
N of possessions * N reading materials at home
(0.649)
-0.321*
(0.177)
N of possessions * N of hhold members reading to student
-0.259*
(0.147)
Constant
Observations
31.67***
29.20***
(3.892)
(5.125)
944
R-squared
0.051
Robust standard errors adjusted for clustering in schools in parentheses
*** p<0.01, ** p<0.05, * p<0.1
944
0.050
24
Appendix C: Skill and HLE Profiles by ADP
Figure C1:
Kalawa and Mutomo Reading Skills Profiles
100%
90%
80%
Percent Correct
70%
60%
89%
78%
75%
73%
84%
72%
70%
30%
74%
60%
57%
50%
40%
86%
40%
23%
51%
48%
45%
51%
46%
40%
37%
27%
26%
20%
10%
54%
15%
46
25%
13%
46
0%
Reading Skills
25
Mutom
o
Kalawa
Figure C2:
Kalawa and Mutomo HLE Profiles
100%
90%
93%
93%
87%
79%
Percent of households
80%
91%
90%
86%
85%
78%
84%
80%
74%
70%
Mutom
o
Kalawa
60%
50%
43%
40%
33%
30%
20%
10%
0%
Seen reading
Helped or
told learner
to study
Read to
learner
Told a story Played a game Has books at
to learner with learner
home
Has kids'
books at
home
HLE Dimension
26