Literacy Boost Kenya Baseline June 2013 Noah Mann, Margaret Kamau, and Paul Titomett With special thanks to: Jarret Guajardo, Cameron Ryall, Lucy Githinji, Edwin Opiyo, John Mutisya, Mary Mugo, Lucy Mutheu, WVK support staff, John Kingori and Peter Nyororo (WVK IPA managers from Kalawa and Mutomo respectively), Ronald Ngetich, Adome Joseph, Bernard Mulei, pupils, teachers and head teachers of participating schools in the Kalawa and Mutomo communities; and the team of assessors: Festus M. Mwau, Daisy Maseki, Agnes Muthama, Mary Nzau, Stellamaria Mutua, Marrieta Makundi, Samuel Muthama, Esther Mutula, Erick Kithuka, Joseph Kimuyu, Steve Mwanzia, Dennis Muteti, Saidi Titus, Josephine Matuku, Ancent Kithuka, Joyce Kavinya, Alice Maithya, Kennedy Musilu, Abednego Kyusya, Patrick Kisilu, Moki Daniel, Alex Mutinda, Henry Nyamai, Isaac Wambua, Lucy Mutua, Ravasco Nzioki, and Samson M Ikuthu. © 2013 Save the Children Executive Summary This report examines the results of a primary school learner background survey and reading assessment conducted in early 2013. The survey and reading assessment covered 944 grade 2 learners throughout 51 schools in World Vision Kenya's Kalawa ADP in Mbooni East District, Makueni County and Mutomo ADP in Mutomo and Ikutha Districts of the broader Kitui District, Kitui County. The 51 schools are randomly split by clusters into 26 primary schools designated to receive Save the Children (SC)’s Literacy Boost program in partnership with World Vision Malawi, and 25 comparison primary schools to receive no intervention this school year. The Literacy Boost program includes teacher training, community reading activities, and age-appropriate local language material creation to support emergent literacy skills among early-grade children. As part of Literacy Boost, learners are periodically assessed in each of these skills through an adaptable assessment tool to inform programming and estimate program impact. The data gathered from these schools is analyzed to present a snapshot of the emergent literacy skills of grade 2 learners in these schools and to inform the adaptation of SC’s Literacy Boost program to this context. This report has three purposes. The first purpose is to provide basic information on the students’ reading skills. The second is to analyze how reading skills vary by student background characteristics and the home literacy environment. The final purpose is to test the extent to which the comparison students and schools are similar to Literacy Boost students and schools to ensure that they will provide an appropriate counterfactual at endline. Results from reading skill assessments administered to Literacy Boost and comparison students showed that students fairly high concepts about print and letter knowledge. However, students were able to read on average less than 50% of the 20 most used words in a government textbook. Most disturbing, less than 25% of students could read at least 5 words correctly in 30 seconds in Kikamba or Kiswahili. In the listening comprehension subtest, learners generally struggled most with inferential questions that required the learners to think critically. This presents an area where even nonreading family and community members can help the learners. When reading stories or telling stories to learners, teachers, family and community members should be encouraged to engage the learners in critical thinking. Asking the learners questions like "What do you think will come next?" and "Why do you think the character did that?" will help learners engage in the stories and gain critical thinking skills. Literacy Boost students for the most part reported strong home literacy environments. The vast majority of students reported both positive habits and abundant reading materials in the home. It is important to note however, that the students of low socio-economic status were most likely to be in the minority of students who did not report strong learning environments. Regression analysis showed a particularly strong correlation between basic reading skills and both the number of reading materials in the home and the number of household members who read to learners. These relationships provide further evidence of the importance of the home literacy environment in developing basic reading skills. Furthermore, there is some evidence that both positive home learning environments habits and extra types of reading materials may 1 help students from poorer families close the gap in reading skills with students from more welloff families. Interestingly, female learners had slightly higher basic reading scores than male learners, but male learners had slightly higher reading and listening comprehension scores. It is difficult to find the cause of these differences, although different kinds of household responsibilities could partially account for them. Finally, there were few significant differences between Literacy Boost and comparison students in their basic reading skills and their home literacy environments. In addition, the random assignments of clusters of schools to be Literacy Boost and comparison schools means that we can expect comparison schools to be equal to Literacy Boost schools in unobserved characteristics as well. Thus, comparison students are likely to be a good indicator of how well Literacy Boost learners have progressed over the course of the year. 2 Contents Introduction............................................................................................................................................................4 Context ...................................................................................................................................................................4 Methods ...................................................................................................................................................................9 Sampling ...........................................................................................................................................,....................9 Measurement .......................................................................................................................................................9 Analysis .................................................................................................................................................................10 Children’s Background ………………………………….......................................................................10 Children’s Reading Skills ……………………………………...............................................................12 Concepts about Print ......................................................................................................................................12 Letter Identification .........................................................................................................................................12 Word Recognition: Most Used Words .......................................................................................................13 Fluency and Accuracy .....................................................................................................................................13 Comprehension .................................................................................................................................................13 Children’s Home Literacy Environment................................................................................................14 Literacy Boost Site Profiles: Skills and the Home Literacy Environment…........................15 Relationship between reading skills and the HLE……………………………………………17 Gender Differences in the HLE and Reading Skills…………………………….........................19 School Context ..................................................................................................................................................21 Conclusion ………………………………………………………………………………………….22 Appendix A: Inter-rater Reliability ............................................................................................…….....24 Appendix B: Regression Results ………………………………………………………………...25 Appendix C: Skill and HLE Profiles by ADP …………………...…………………………......26 3 1. Introduction and context Introduction This report examines the results of a learner background survey and reading assessment conducted in early 2013. The survey and reading assessment covered 944 grade 2 learners throughout 51 schools in Kenya. The 51 schools are split into 26 primary schools designated to receive Save the Children's (SC) Literacy Boost program, and 25 comparison primary schools that will not participate in Literacy Boost. The Literacy Boost program includes teacher training, community reading activities, and ageappropriate local language material creation to support emergent literacy skills among earlygrade children. These skills include concepts about print, letter awareness, single word reading of most used words, reading fluency, reading accuracy, and reading comprehension. As part of Literacy Boost, learners are periodically assessed in each of these skills through an adaptable assessment tool to inform programming and estimate program impact. The data gathered from these schools is analyzed to present a snapshot of the emergent literacy skills of grade 2 learners in these schools and to inform the adaptation of Save the Children’s Literacy Boost program to this context. The key research questions to be explored in this report include the following: 1. How comparable are learners in Literacy Boost schools versus comparison schools in terms of background characteristics, home literacy environment (HLE), school environment, and reading skills? 2. What can the baseline tell us about learners’ emergent reading skills? What does this mean for Literacy Boost programming? 3. How do learners’ reading skills vary by student background, school environment, and home literacy environment? What does this mean for targeting Literacy Boost’s two strands of intervention? To investigate these questions, this report will first describe the research methods used; including sampling, measurement, and analysis. Next, in order to see if groups are statistically similar, the comparability of learners at Literacy Boost and comparison schools will be examined through clustered t-tests. The comparability of Literacy Boost and comparison learners’ scores for each of the emergent literacy skills and learners’ strengths and weaknesses in each skill will also be examined. The following section will investigate any correlations with student background, school environment, or home literacy practices and environment variables using multilevel regression analysis. Finally, the report will examine differences between female and male learners and the comparability of LB and comparison schools. Context Kenya lies on the East Coast of Africa with a total land area of 582,648 square kilometers. About 75-80% of the population lives in the rural areas. The population in absolute poverty was estimated to be 46% in 2006. The country has a new two level system of governance- the national government and the devolved county government. The government’s socio-economic 4 development strategy is anchored in the Kenya Vision 2013, the new development blueprint covering the period 2008-2030. Kenya is a signatory to various International Conventions and Commitments aligned to the well-being of children. Every child in Kenya has a right to free and compulsory basic education, a provision under the Constitution of Kenya 2010 and stated in The Basic Education Bill of 2012. Education is one of the social sectors under the social pillar of the Vision 2030 1. The Kenyan government recognizes education as a basic tool that has a major responsibility of facilitating the process of developing manpower necessary for transforming into a globally competitive country 2. Education is also viewed as significant in creating a knowledge-based society able to uphold justice, democracy, accountability, cultivating nationalistic values and results-oriented political engagement3. This is hoped to be realized through enhanced access, equity, quality and relevance to education. Education priorities are guided by national development strategies in the Vision 2013 and the Constitution 2010. Kenya subscribes to the global commitment of Education for All (EFA) which pledges the provision of quality basic education for all children, youth and adults including those from ethnic minorities and other marginalized groups. The Kitui District Development Plan 2008-2012 states that the most important resource for a country is a competitive, effective, healthy and well-educated people. The plan also notes that education improves the quality of the available labor force, which results in economic growth and poverty reduction. The current education structure in Kenya consists of 8 years of primary education (age 6 to 13), 4 years of Secondary education (age 14 to 17) and 4 years of Higher education (age 18 to 21 and above). The Early Childhood Development and Education (ECDE) level is for children age 0 to 5 years. While the structure is applicable in both private and public institutions, school attendance indicators shows that children attend school even when they are outside of these age brackets. Pupils not able to transition to secondary education are accommodated in youth polytechnics and in tertiary level institutions such as Technical, Industrial and Vocational Educational Institutions (TIVETs). Pupils exiting secondary education but unable to proceed to the university or colleges may start small-scale income-generating activities. The Government intends to promote retention of learners in Alternative Basic Education Training (ABET) institutions. 1 Kenya Vision 2030 Education Sector 2013/14- 2015/16 Medium Term Expenditure Framework 3 Kenya Vision 2030. 2 5 Figure 1: Structure of Education and Training in Kenya 4 In Kenya, Free Primary Education (FPE) started in 2003 while Free Day Secondary Education (FDSE) began in 2008 5 . This means that all fees are waived in all government (public) primary schools. The FPE policy has resulted in an influx of learners into schools, resulting in high pupilteacher ratios, insufficient learning materials and congestion in classes 6. According to the Education Sector 2013/14- 2015/16 Medium Term Expenditure Framework, the ECDE Gross Enrollment Rate (GER) stands at 65.5% in 2011. The number of pupils in formal primary schools in 2010 was 9.86 million (4.98 boys and 4.86 girls) with a Net Enrollment Rate (NER) of 95.6% in 2011. Other vital statistics for the period are: The primary Gender Parity Index improved from 0.98 in 2009 to 1:02 in 2010 in favor for girls, the Textbook Pupil Ratio for 2009 was 1:2. There was an observed decline in the pupil completion rate from 83.2 per cent in 2009 to 74.6 per cent in 2011, a factor attributed to school dropouts and repetitions. The transition rate from primary to secondary has been increasing over the years, standing at 73.3 per cent (68.9 per cent for male and 75.3 per cent for female) in 2011. The UWEZO and NASMLA reports of 2010 and 2011 indicated that most children are progressing to upper classes without numeracy and literacy competencies as required at grade 2, 3 and 6 7. The Government of Kenya (GOK) Education sector report of 2012 notes that the national mean score in the primary examination declined from 54.3% in 2010 to 53.0% in 2011 .The report also shows that the country faces regional and gender disparities in education that need to be reduced. 4 Basic education sector analysis report 2012- JICA Basic education sector analysis report 2012- JICA 6 SACMEQ policy brief No. 2 on quality of primary school inputs in Kenya 7 UWEZO is a civil society organisation which employs household based learning and competency assessment methodologies based on grade 2 curriculum while NASMLA is a national assessment of monitoring of learning achievement which is school based and administered to pupils at grade 3 and 6 (quoted in the WVK Education sector strategy 2013-2015). 5 6 The government has identified free Primary Education and Early Child Development and Education as some of the priority programs within education. This prioritization has been based on criteria that include: linkage with the objectives of the Government Medium Term Development Plans (2008-2012 and 2013-2017), the degree to which the program addresses core poverty interventions, cost effectiveness, sustainability of the program and immediate response to the requirements of the implementation of the Constitution. The government policy now is to mainstream ECDE in the primary school set up. Only 61.5% of the adult population has attained minimum literacy level and only 29.6% of the Kenyan adult population has attained the desired mastery literacy competency. This is likely to affect the education of their children 8. Gaps in education activities due to resource constraints include: inadequate financing of free day secondary schools, a shortage of teachers, inadequate responses to psychosocial challenges of youth through life skills education, inadequate education opportunities for marginalized groups including those excluded from education due to poverty and HIV, a lack of access and equity in special needs education, inadequate recruitment of additional teachers, inadequate learning facilities, The Ministry of Education is in the process of developing the National Education Sector Support Programme (NESSP) which is a five-year program for delivering the reforms earmarked for the education sector in Kenya 9. One emerging issue in the education sector is devolution of education to the newly created county-level governance under the Constitution 2010 which will impact teacher services, ECDE and technical level education and quality assurance 10. The Government recognizes the role different stakeholders play in meeting education sector goals. Stakeholders include local authorizes, communities, political leaders, civil societies, NGOs, donors and other Government Ministries and agencies. Site Profile Literacy Boost Project is being implemented in two areas in Kenya: Mutha Mutomo Area Development Program (ADP), situated in Mutomo and Ikutha District of the broader Kitui District, Kitui County and in Kalawa ADP, situated in Kalawa Division (Kathulumbi and Kalawa location), Mbooni East District, Makueni County. The Districts are mainly occupied by the Akamba community with other communities living in the major towns of the districts. The majority of the population lives in the rural areas. The land use and settlement patterns are based on agro-ecological zones and they are also influenced by soil fertility and rainfall. The following is a profile of the two districts 11 . Kalawa lies within the arid and semi-arid zones of the country and is prone to frequent drought. Agricultural production varies over the years depending on the amount of rainfall received per 8 EFA Global Monitoring Report 2006 World Vision Kenya Education sector strategy 2013-2015. 10 Republic of Kenya Education Sector Report 2012. 11 Government of Kenya, Kitui and Makueni District Development Plan 2008-2012 9 7 year. A low rainfall ranging from 300mm to 400mm is hardly enough to sustain any kind of crops. Local people are therefore engaged in small stock rearing for economic activities. About three-quarters of the people live below the poverty line in the district. Drought is a major cause of poverty in the district and the most vulnerable are women, children, the aged and disabled. Kalawa division lies at the lowest end of the newly created district and has had very limited interaction with development partners besides GOK departments. Kalawa Division of Mbooni East District has 135 teachers. The division has 19 primary schools with a total population of 4941 (Boys 2530 Girls 2411) in 2013. A total of 84% of schools are single streams 12. The schools in Kalawa Division have an average of 31 pupils in Standard 1, 2 and 313. The broader Kalawa Division (comprising of Kathulumbi and Kalawa locations) has 36 public primary schools and 13 public secondary schools. There are four village youth polytechnics. Mbooni East district has 107 primary schools and 42 secondary schools 14. The teacher/ pupil ratio in primary school is 1:33 while the secondary teacher/ student ratio is 1:32. The District Development Plan identifies inadequate education facilities, inadequate staffing, rising dropout rates and low transition rates as issues that need immediate intervention in order to increase literacy rates in the district. Mutomo District experiences high temperatures throughout the year. The unreliability of the rainfall and perennial droughts limit intensive and meaningful land use. The district experiences crop failure and water shortage in most years which affects family income and the wellbeing of children. According to the Central Bureau of Statistics, the district has 67.2 percent of the population below the poverty line making residents in this area among Kenya’s poorest. Absolute poverty in the area is manifested in various ways such as hunger, ill health, lack of access to quality education and even lack of participation in decision making. According to statistics gathered during the inception of the program, Mutomo ADP area has 60 primary schools with a total enrolment of 14,899 (7602 boys 7297 girls). There are 286 fully trained teachers employed by the government and 228 untrained and supported by the Parents and Teachers Association (PTA). The primary teacher/pupil ratio stands at 1:52 which is far below the recommended standard of 1:25. Mutomo district has a primary school transition rate of 39.7% 15. The district has received increased government investment in school infrastructure but this has not been commensurate to performance in academics. The district is ranked among the poor performers in the country, a state that is attributed to low enrolment, transition, retention and completion rates, poor curriculum implementation, low literacy levels among parents leading to complacency, understaffing, lack of motivation and poor learning/teaching materials. The community members stress that many school-aged children within the ADP are not in schools; those who are have learning that is characterized by irregular attendance due to absenteeism and eventual dropping out for both girls and boys. Cultural factors, such as early marriage and a preference for educating boys, as well as the heavy productive demands placed on girls, create significant gender differentials in educational outcomes. 12 Source: Area Education office Kalawa Division Ibid 14 Mbooni East District education office 15 Mutomo district education office 1313 8 The transition rates from primary level to secondary level and from secondary to tertiary level is low. Many children have dropped out of school due to lack of school fees and this hinders transition. The most affected are the OVCs, children from poor families and those under the care of elderly guardians who are economically disempowered. The competitive job market and hard economic times in the county exposes these children to uncertainty as they have not acquired quality education to enable them to find gainful employment opportunities. Community members also observe that while children accessed primary and secondary education, the majority of them lacked skills and knowledge required for either self or formal employment. The education curriculum centres more on classroom work rather than equipping the children with marketable and self-employable skills as they approach adolescence. Youth in the area are therefore unable to free themselves from the cycle of poverty. An additional concern is the lack of quality data, and specifically the lack of baseline information against which progress towards key education goals and outcomes can be monitored. II. Methods: Sampling The sample for this baseline assessment encompasses 944 standard 2 learners, divided between 26 schools set to receive the Literacy Boost intervention (n of learners = 484) and 25 comparison schools (n of learners = 460). Clusters of schools were randomly assigned to either receive the Literacy Boost intervention or serve as comparison schools. Within each school, 10 boys in and 10 girls in standard 2 were randomly selected to be assessed at baseline and again at endline. Some of the classes were less than 20 – due to the overall enrolment in the school being small because they are remote or have small feeder communities. Because we were able to randomly assign schools, we expect Literacy Boost and comparison students to be equal in both observable and unobservable characteristics. Thus, we will be able to attribute any differences between Literacy Boost and comparison schools at endline to the Literacy Boost intervention itself. Measurement School profile data were collected via direct observation and a survey of school head teachers at every school in the sample. This data includes information on enrolment & class size, availability of textbooks, school facilities & construction, and teacher backgrounds. For the student assessment, all learners in the sample were asked about their background characteristics (age, household possessions, household building materials, etc.). Learners also were asked about their family members and reading habits in their home (who they had seen reading in the week prior to the assessment, who had read to them, etc). After collecting this background data, all learners were also given an emergent literacy test composed of six components administered through five sub-tests: concepts about print, letter awareness, single word recognition (reading of most used words), reading fluency & accuracy (words per minute read correctly and total percentage of passage read correctly; both within the same sub-test), and a set of comprehension questions about the fluency & accuracy passage. The same set of comprehension questions were administered for both those learners who 9 could read independently (reading comprehension) and those who could not and thus had the assessor read to them (oral comprehension). All instructions were given in Kikamba, and children were assessed on letter identification, most used words, reading fluency, reading accuracy, and listening/reading comprehension in both Kiswahili and Kikamba. To test inter-rater reliability, approximately 10 percent of learners (99 out of 944) were assessed by two enumerators simultaneously. All of the skills tests had excellent or good reliability; full results can be found in Appendix A. Analysis: The critical purpose of this analysis is to test whether the Literacy Boost learners and the comparison learners are equal in terms of background and skills. That is, do these learners possess the same resources and capabilities? This question is important so that at end-line, we can know how much Literacy Boost has, or has not, contributed to learners' accelerated reading development. To test the comparability of learners in the Literacy Boost and comparison samples, this report will use comparison of means through t-tests, with clustered standard errors to account for the grouping of student-level data within schools. Summary statistics, accompanied by clustered ttests, will be used to analyze learners’ performance in each of the reading sub-tests. Finally, this report will look to multilevel regression models to explore relationships between literacy skills and student background characteristics, school environment, and home literacy environment. A Note about Benchmarks While Save the Children has used this approach to reading assessment and intervention in Malawi, Nepal, Mali, Pakistan, Ethiopia, Uganda, Zimbabwe, Bangladesh, and Mozambique, comparison across countries and languages is less helpful than more detailed contextual information for setting expectations of impact. For each measure used in these assessments, the upper end of the range of scores can be used to consider what is currently possible among these children. Save the Children usually presents the scores at the 75th percentile of each measure from this baseline assessment to constitute the benchmarks. However, due to the extremely low reading skills of both the LB and comparison groups, benchmarks are set high, often in the 95th or even the 99th percentile. These scores will constitute the benchmarks which the Literacy Boost program will attempt to help learners achieve by the time of end-line assessment. III. Children’s Background Students attending Literacy Boost schools are on average eight years old, speak Kikamba as their primary language, and live with four other people in their households. Nearly all Literacy Boost students have attended an early childhood development (ECD) programs, for an average of one to two years. Despite near universal ECD attendance, nearly a third of LB students repeated Primary 1, and nearly a fifth repeated Primary 2. 10 Table 1: Selected student characteristics: N Literacy Boost Female 944 49.6% Age 944 7.8 Attended ECD 944 97.7% N of years of ECD 829 1.5 Repeated Primary 1 942 30.6% Repeated Primary 2 944 19.0% Kikamba primary language 944 98.8% N of household members 944 5.0 Height 942 120.4 cm Weight 944 20.5 kg Student ate breakfast previous morning 944 80.0% *** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05 Comparison 47.6% 7.8 98.0% 1.6 38.6% 17.4% 99.3% 4.8 120.7 cm 20.4 kg 79.6% Sig. Diff. - * - Students in comparison schools tended to be similar to students in LB schools, although they attended ECD programs for slightly, though statistically significantly, fewer years. It is not unusual to find a small number of statistically significant differences in background variables; these minor differences are unlikely to indicate that the randomization was compromised. Table 2 shows the household possessions and livestock present in the households of LB and comparison students as a proxy for socio-economic status. About three quarters of LB students reported the presence of radios, cellphones, and toilets in their households, but very few reported having electricity or televisions. Most LB students reported owning livestock; over 80% reported owning chickens and goats, the most frequently reported types of livestock. Table 2: Socio-economic Status N Literacy Boost Radio 944 73.1% Electricity 944 6.2% Television 944 5.8% Cell Phone 944 80.2% Toilet 944 83.5% Bicycle 944 45.7% Bed 944 92.4% Number of types of household possessions 944 3.9 Sheep 944 16.5% Goats 944 81.4% Cattle 944 65.1% Donkeys 944 73.1% Chickens 944 82.9% Number of types of livestock 944 3.2 Number of types of assets 944 7.1 *** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05 Comparison 72.4% 5.4% 6.7% 75.4% 79.1% 51.7% 92.2% 3.8 16.7% 84.1% 61.3% 73.5% 87.0% 3.2 7.1 Sig. Diff - LB and comparison students appear have relatively similar socio-economic status, further indicating that the randomization procedure was correctly carried out. 11 Table 3: Selected Health Characteristics N Literacy Boost Mosquito net –last night 944 43.8% Mosquito net – always 944 47.1% Ever had malaria 941 83.2% Had diarrhea recently 944 6.8% Healthy 942 87.0% *** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05 Comparison 46.5% 49.6% 83.9% 6.5% 91.0% Sig. Diff. - Most children in both LB and comparison schools reported having had malaria at least once in their lifetime. The extent to which learners are still affected by malaria is unclear. About half of learners reported sleeping under mosquito nets, although paradoxically, more learners reported always sleeping under a mosquito net than sleeping under a mosquito net the previous night. This finding indicates that these data may not be completely reliable. Around 90% of learners reported feeling healthy in both LB and comparison schools the day of the assessment. IV. Children’s Reading Skills Table 4: Reading Skills: N LB Comparison CAP 944 74.8% 75.4% Kikamba Letter Identification (ID) 944 82.7% 79.3% Kiswahili Letter Identification (ID) 944 64.3% 60.2% Kikamba Most Used Words 944 31.1% 29.1% Kiswahili Most Used Words 944 36.0% 32.3% Non-readers - Fewer than 10 Kikamba wpm 944 77.7% 79.6% Non-readers - Fewer than 10 Kiswahili wpm 944 79.1% 82.4% Kikamba Fluency – sample average 944 4.9 wpm 4.9 wpm Kiswahili Fluency – sample average 944 4.5 wpm 5.3 wpm Kikamba Accuracy – sample average 944 20.8% 19.0% Kiswahili Accuracy – sample average 944 19.7% 16.2% Kikamba Listening Comprehension 742 49.5% 48.5% Kiswahili Listening Comprehension 762 39.8% 36.9% Kikamba Reading Comprehension 202 59.6% 54.3% Kiswahili Reading Comprehension 182 50.8% 46.0% *** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05 Sig. Diff. * 75th percentile 91.7% 95.2% 75.0% 70.0% 73.8% 0 0 0 0 60% 60% 70% 70% Suggested Benchmark 100% 100% 100% 50% 50% 10 wpm 10 wpm 50% 50% 100% 100% 100% 100% Concepts about Print The first sub-test of the reading assessment consisted of 12 ‘concepts about print’ (CAP) questions. These questions concern familiarity with books – where to start, which way to read, what is a letter, what is a word, etc. LB learners correctly answered approximately 75% of concepts about print assessed. Learners at Literacy Boost schools and at comparison schools had statistically indistinguishable scores. 12 Identifying the book’s title (45% correct), identifying the end of the story (50% correct), and following along while the assessor read (49% correct) were the concepts about print with which LB students had the most difficultly. Each of the other concepts about print assessed was understood by more than 65% of LB students. Letter Identification The next sub-test examined learners’ letter awareness. Learners were shown a chart of letters and asked to name the letter, pronounce the letter sound, or give a word that begins with that letter. The average LB learner could identify over 80% of tested upper and lower case Kikamba letters and over 60% of Kiswahili letters. Differences between students at LB and comparison schools on this measure were small and not statistically significant. The only Kikamba letter that fewer than 60% of LB learners were able to identify was a lowercase l as only 22% of LB learners correctly identified it. Learners likely confused a lowercase ‘l’ with an uppercase ‘I’. Literacy Boost should instruct teachers to take special care to point out the difference between the lowercase ‘l’ and uppercase ‘I.’ There are seven digraphs in the Kiswahili language, including for example ‘dh’, ‘sh’, and ‘ch.’ No more than 15% of LB learners were able to identify any of the digraphs, while most learners were able to identify the other letters, which are in large part similar to Kikamba letters. Perhaps learners’ familiarity with Kikamba helped them to identify similar letters in Kiswahili, but did not did not translate to being able to identify the seven digraphs. Literacy Boost should instruct teachers to help students learn digraph letters in Kiswahili. Word Recognition: Most Used Words The most used words (MUW) sub-test consists of a chart of 10 words that the student is asked to read. These 10 words were identified as ‘most used’ by tabulating the number of times a word appeared in learners’ language arts textbooks. Interestingly, LB readers were able to read a slightly higher proportion of Kiswahili most used words (36%) than Kikamba most used words (31%) despite identifying a higher proportion of Kikamba letters. This suggests that LB learners may have greater access to printed materials in Swahili. Literacy Boost should provide access to printed materials in Kikamba so that they are able to practice reading in their first language. Comparison students had similar results as LB students. Fluency and Accuracy Fluency (words per minute read correctly) and accuracy (percent of the passage read correctly) are presented together here because they are measured together in a single sub-test in which learners read a passage aloud. The number of words learners read correctly in 60 seconds is tracked for fluency. As the student continues to read after the first 60 seconds, the total number of words read correctly from the passage as a whole, no matter how long it takes the student, is computed for accuracy. If a student is not able to read at least five words correctly in 30 seconds, then the assessor stops the student and reads the rest of the passage aloud. Students who are read the passage are labeled non-readers. The majority of LB students were unable to read at least five words in 30 seconds in Kikamba (77%) and Kiswahili (79%), and thus were labeled non-readers. On average, including those 13 students who were labeled non-readers, LB learners were able to read approximately five words per minute correctly in both languages. LB learners only correctly read 20% of Kiswahili and Kikamba words. Comprehension The final sub-test quizzed learners on a series of ten comprehension questions related to the reading passage. Learners who were unable to read more than five words correctly were read the passage by the assessors before being asked the comprehension questions orally. Learners who read the entire passage were also asked the comprehension questions orally. Six of the comprehension questions were factual, two were inferential, one each were summative and evaluative. Unsurprisingly, both readers and non-readers were able to answer a higher proportion of Kikamba questions than Kiswahili questions. The inferential questions were the most difficult types of questions for readers and listeners of the Kikamba passage, while readers and listeners of the Kiswahili passage had equally low scores on the factual and inferential questions. Thus, when reading stories to learners, LB participants should be supported to engage the learners in critical thinking. Asking the learners questions like "What do you think will come next?" and "Why do you think the character did that?" will help learners engage in the stories and gain critical thinking skills. Literacy Boost and comparison students had statistically different scores in Kikamba reading comprehension. This finding may indicate that students in Literacy Boost schools as a whole have higher reading comprehension scores than students in comparison schools. V. Children’s Home Literacy Environment Table 5: HLE N Literacy Boost Number of types of reading materials 944 2.0 Has books at home 944 86.8% Textbooks 944 55.2% Religious books 944 68.6% Magazines 944 7.0% Newspapers 944 25.8% Stories 944 30.8% Coloring books 944 15.3% Kids’ books 944 37.4% *** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05 Comparison 2.0 83.2% 53.9% 69.1% 5.4% 26.1% 31.3% 13.9% 37.6% Sig. Diff. - Over 85% of LB learners reported that there were books in their households, while religious books were the most commonly reported type of book. Only one third of LB students reported having a type of childrens’ book, either story books or coloring books, in their household. Furthermore, LB and comparison students of higher socio-economic status were significantly more likely to report the presence of kids’ books in their households. Learners with kids’ books reported having 4.3 types of household possessions on average, while learners without kids’ books reported 3.6 types of household possessions on average. Literacy Boost 14 should focus on improving learners’ access to recreational reading material designed for children, with particular attention paid to learners of low socioeconomic status. LB students and comparison students reported having statistically similar types of books in their households. Table 6: HLE (past week) N Literacy Boost Comparison Seen reading 944 89.5% 90.0% Helped or told learner to study 944 87.6% 90.7% Read to learner 944 83.9% 82.8% Told a story to learner 944 84.5% 78.0% Played a game with learner 944 79.8% 77.0% *** Significant at p<.001, ** Significant at p<.01, * Significant at p<.05 Sig. Dif. * - Given that nearly all students reported the presence of reading materials in their homes, it’s unsurprising that nearly 90% of learners sampled reported observing at least one person reading in their household over the past week. It is encouraging that this high rate of adult literacy appears to translate to child-focused HLE behaviors as well; over three quarters of learners reported that a household member helped or told them to study, read to them, told them a story, and played a game with them. However, learners of higher socio-economic status were significantly more likely to report each of these HLE behaviors (with the exception of being helped or encouraged to study, which was not significantly related to socio-economic status). It is thus important that Literacy Boost focus its community action efforts on the poorest, most marginalized households. A significantly higher proportion of Literacy Boost students reported that at least one member of their household told them a story over the previous week than comparison students. VI. Literacy Boost Site Profiles: Skills and the Home Literacy Environment This section summarizes the reading skills and HLE data for Literacy Boost students. It includes a chart summarizing CAP, letter identification, MUW, fluency, accuracy, and comprehension scores as well as a chart summarizing students’ home literacy environments. Figure 2 displays Literacy Boost students’ average scores on all literacy sub-skill assessments, and Figure 3 shows students’ average home literacy environment. Appendix C shows the skill and HLE profiles for each ADP. 15 Figure 2: Percent Correct Overall Kenya Skills Profile 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 75% 83% 78% 79% 64% 60% 50% 31% 51% 40% 36% 21% 20% 5 wpm5 wpm Reading Skill Figure 3: Percent of households Kenya HLE Profile 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 90% 88% 84% 85% 80% 87% 37% Seen reading Helped or told learner to study Read to learner Told a story to learner Played a Has books at game with home learner Has kids' books at home HLE Dimension 16 VII. Relationship between reading skills and the HLE Each of the HLE habits, along with the availability of reading materials in the home, is significantly associated with at least one of learners’ concepts about print scores, letter identification scores, and most used words scores, when controlling for age, gender, socioeconomic status, and household size and adjusting standard errors for clustering at the school level. These associations however do not prove that any of these behaviors cause learners to improve their reading skills. For example, it is plausible that learners with a greater understanding of concepts about print are more likely to notice and report the presence of reading materials in the household. That said, it is still useful look at which HLE habits seem to be most often and most strongly correlated with reading skills. By doing so, we will be able to verify that the kinds of activities promoted by Literacy Boost have the potential to increase children’s reading scores. Average number of Kikamba letters correct The number of people in household who read to the learner and the number of types of reading materials in the household are frequently and strongly correlated with reading skills, when controlling for age, gender socio-economic status, and household size. One reading skill for which these relationships are particularly evident is letter identification. Learners who reported that more than one household member read to them over the past week were able to identify approximately 35 Kikamba letters on average, while learners who reported that zero or one household member read to them over the past week were only able to identify 33 Kikamba letters on average. As is evident in figure 4, this relationship holds for the number of reading materials in the household. Regression Figure 4. Relationship between analysis confirms that this observed difference is HLE and Letter ID statistically significant when 40 controlling for age, gender 35 socio-economic status, and 2.0 2.5 30 household size and 25 adjusting standard errors 20 for clustering at the school 32.8 32.5 15 level. For every additional 10 household member who 5 was reported to have read 0 to a learner over the past Household members who Types of reading materials in week and for each read to learner household additional type of reading HLE Habits and Materials material in the household, the learner was able to 0-1 Member or Material Gain if More than 1 Member or Material identify half of an additional Kikamba letter. 17 Number of Kiwhahili letters identified There is also some evidence that it is more important that learners are read to in poorer households than in less poor households; similarly, the number of reading materials available appear to be more important in the poorest households. Figure 5 shows an interaction in which the number of household members who read to learners is especially Poverty and Reading to important for learners of the lowest socio-economic status. This trend is Learners echoed for the number of types of 44 reading materials. These effects suggest that the gap between 42 learners of high and low socio40 economic status can be 38 significantly reduced simply be 1 possession 36 ensuring that all learners have 2 possessions access to multiple types of print 34 in their household and that all 3 possessions 32 learners are read to as much as 4 possessions 30 possible in their homes. These 0 1 2 3 4 effects were not significant on the number of Kikamba letters identified, Number of household members who read but the coefficients were of the same to learner direction. VIII. Gender Differences There are a number of notable differences between male and female learners, chief among them that female learners appear to have significantly better basic reading skills than male learners. More specifically, female learners identified a higher proportion of both Kiswahili letters and were able to read more Kikamba and Kiswahili most used words than male learners. However, male learners answered a higher proportion of Kiswahili reading and listening comprehension questions correctly than female learners. 18 Figure 6: Reading Skills by Gender 100% 90% Percent Correct 80% 70% 60% 50% 40% Male 30% Female 20% 10% 0% CAP Kikamba Kiswahili Kikamba Letters Letters MUW Swahili MUW Kikamba Kikamba Kiswahili Kiswahili Listening Reading Listening Reading Comp. Comp. Comp. Comp. Reading Skill *** Significant at p<.01, ** Significant at p<.05, * Significant at p<.1 Is it difficult to pinpoint one cause for these differences, but these results generate some interesting hypotheses. One is that boys could be more actively engaged both inside and outside of the classroom. If boys are encouraged to be active participants in storytelling, playing games, or reading aloud more frequently in school or at home, then they may develop basic comprehension skills more quickly than girls. There is some very weak evidence that these activities may play a role in the comprehension gap evident between girls and boys; on average, boys reported that more members of their household help them study, read to them, tell stories to them, and play games with them than girls do. However, it is important to note that these differences are not statistically significant, meaning that they cannot be generalized to the broader population with confidence. The only statistically significant difference between girls and boys in the HLE is that a higher percentage of boys reported that at least one household member played a game with them over the previous week. Despite the lack of firm evidence that girls are disadvantaged in the HLE, Literacy Boost should train teachers and sensitize members of the community on the importance of actively engaging both girls and boys in storytelling, games, reading aloud, and studying. 19 Table 7: HLE Variable Male Female N Seen reading 2.7 2.4 N Helped or encouraged learner to study 3.0 2.8 N Read to learner 2.3 2.1 N Told story to learner 2.5 2.3 N Played game with learner 2.1 1.8 Seen reading 91% 89% Helped or encouraged learner to study 90% 89% Read to learner 83% 84% Told story to learner 83% 80% Played game with learner 82% 74% *** Significant at p<.01, ** Significant at p<.05, * Significant at p<.1 N 944 944 944 944 944 944 944 944 944 944 Sig ** Another possible hypothesis as that could partially explain why girls have higher basic reading skills but lower comprehension skills than boys could be that the different household chores frequently conducted by girls and boys could be related to their learning. Girls were more likely than boys to report regularly doing chores within the household, such as sweeping, washing, cooking, and caring for children, while boys were more likely to report regularly doing chores outside of the household, like farming and herding. Performing chores inside of the home may give girls more opportunities to interact with print in the home than are available to boys as they perform chores in the fields. In addition, boys reported spending significantly more time doing chores than girls did in the afternoon, perhaps further impeding their opportunities to study. However, no significant differences were detected between girls and boys in reported time spent studying. Literacy Boost should emphasize the importance of both girls and boys having sufficient time and opportunity to interact with printed materials on a daily basis. Now that we’ve hypothesized that girls may more frequently perform chores than enable them greater opportunity to interact with printed materials, we turn to a possible hypothesis as to why boys have higher comprehension scores. By virtue of doing a higher proportion of their chores outside of the home, boys may have more opportunities to interact with more members of the community and to exercise critical thinking skills on a regular basis. Boys also reported being more likely to work than girls, which may also contribute to the development of critical thinking skills, but restrict the amount of time they have to become familiar with print in the home. Literacy Boost should sensitize members of the community on the importance of engaging girls in opportunities to exercise their critical thinking skills IX. School Context Since clusters of schools were randomly assigned to either be Literacy Boost or comparison schools, we can assume that they have similar observable and unobservable characteristics. It is also not necessary to test for significant differences in the population of LB and comparison schools, as we have collected these data from all of the LB and comparison schools. 20 Table 8: Selected School Level Variables Distance from center Has electricity Has library N 51 51 51 Comparison 26.2 km 8.7% 8.5% LB 31.6 km 4.1% 6.2% Literacy Boost schools are on average 32 km from the regional center, the farthest is over 100 km away. Very few schools have electricity, and equally few have libraries. This means that if learners do not have reading materials in their households, they may be unable to spend time becoming familiar with print outside of school. A key component of Literacy Boost should be to provide all learners access to printed materials, not to rely on existing materials in households or schools. X. Conclusion Clusters of schools were randomly assigned to receive the Literacy Boost intervention or serve as comparison schools in the Lower Eastern region of Kenya. In total, 26 schools were assigned to receive LB, while 25 were designated as control schools. Since clusters of schools were randomly assigned to receive the intervention, we expect Literacy Boost and comparison schools to be equivalent in both observable and unobservable characteristics. As expected, descriptive statistics of sampled students within LB and comparison schools were similar, with a only a few minor differences emerging. These minor differences do not indicate that the randomization procedure was compromised, but may need to be controlled for at endline. Learners in Literacy Boost schools tended to have strong scores on basic reading skills tests, including concepts about print and letter identification, but had weak scores on more advanced reading skills tests, including reading most used words, fluency, and accuracy. These results suggest that while most students are able to recognize letters, they are largely unable to read words or reading passages. On average, learners had similar reading skills in Kikamba and Swahili, with the very notable exception of comprehension. Learners had higher listening and reading comprehension in Kikamba, likely due to Kikamba being their first language. Learners in Literacy Boost schools have fairly rich Home Literacy Environments (HLEs); most reported the presence of books at home and that at least one member of the household read to them, told a story to them, and help or encouraged them to study over the past week. However, children of low socio-economic status were less likely to report these positive HLE habits. It is thus important that Literacy Boost focus its community action efforts on the poorest, most marginalized households. In addition, less than half of all learners reported having kids’ books at home, with learners of low socio-economic status again the most disadvantaged. Literacy Boost should focus on improving learners’ access to recreational reading material designed for children, with particular attention paid to learners of low socio-economic status. Regression analysis confirmed that learners who reported more household members reading to them and more types of reading materials in their household scored higher on basic reading skills tests. The regression analysis also showed that HLE habits and materials appear to close 21 the gap between learners of high and low socio-economic status. Thus Literacy Boost should emphasize the importance of HLE habits and HLE materials, especially in the poorest households. Finally, female learners scored slightly higher on basic reading skills tests than male learners, but the relationship was reversed for comprehension. It is difficult to determine what might be causing this divergence, but it is possible that the different types of chores and work that girls and boy regularly perform could contribute to this difference. Girls are more likely to engage in household work, like sweeping, washing, and childcare, while boys are more likely to work outside of the home, farming, herding. It is possible that girls may have more time and opportunity to interact with print, while boy may have more opportunities to exercise and develop their critical thinking skills. Whatever the reasons for this divergence, Literacy Boost should emphasize the importance of both girls and boys having sufficient time and opportunity to interact with printed materials on a daily basis and exercise their critical thinking skills through games and stories. 22 Appendix A: Inter-rater reliability To test inter-rater reliability, approximately 10% of students (n=99) were assessed by two enumerators simultaneously. Long one-way ANOVA techniques were used to calculate the intra-class correlation within pairs of assessors for a measure of reliability. Table A presents the results below. Using Fleiss’ benchmarks for excellent (ICC>0.75), good or fair (0.75>=ICCA>0.4), and poor (0.4>=ICC); all of the literacy outcome variables exhibited excellent or good or fair inter-rater reliability overall. Table A1: Measure CAP Kikamba Letters Kiswahili Letters Kikamba MUW Kiswahili MUW Kikamba Fluency Kiswahili Fluency Kikamba Accuracy Kiswahili Accuracy Kikamba Reading Comp. Kiswahili Reading Comp. Kikamba Listening Comp. Kiswahili Listening Comp. Kikamba Reader/Non-reader Kiswahili Reader/Non-reader ICC .93 .99 .91 .89 .96 .77 .65 .93 .89 .96 .92 .93 .98 .89 .87 Rating Excellent Excellent Excellent Excellent Excellent Excellent Good or Fair Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent 23 Appendix B: Regression Results VARIABLES (1) Total Kikamba Letters (2) Total Kiswahili Letters Age in years -0.469 -0.264 (0.295) (0.374) Sex (1=Female) 2.045*** 3.437*** (0.657) (0.751) N of household possessions 0.729** 2.072*** (0.305) (0.572) N of household members 0.0485 -0.357 (0.250) (0.406) N of reading materials at home 0.482** 1.534** (0.220) (0.714) 0.461** 1.889*** N of hhold members reading to student (0.199) N of possessions * N reading materials at home (0.649) -0.321* (0.177) N of possessions * N of hhold members reading to student -0.259* (0.147) Constant Observations 31.67*** 29.20*** (3.892) (5.125) 944 R-squared 0.051 Robust standard errors adjusted for clustering in schools in parentheses *** p<0.01, ** p<0.05, * p<0.1 944 0.050 24 Appendix C: Skill and HLE Profiles by ADP Figure C1: Kalawa and Mutomo Reading Skills Profiles 100% 90% 80% Percent Correct 70% 60% 89% 78% 75% 73% 84% 72% 70% 30% 74% 60% 57% 50% 40% 86% 40% 23% 51% 48% 45% 51% 46% 40% 37% 27% 26% 20% 10% 54% 15% 46 25% 13% 46 0% Reading Skills 25 Mutom o Kalawa Figure C2: Kalawa and Mutomo HLE Profiles 100% 90% 93% 93% 87% 79% Percent of households 80% 91% 90% 86% 85% 78% 84% 80% 74% 70% Mutom o Kalawa 60% 50% 43% 40% 33% 30% 20% 10% 0% Seen reading Helped or told learner to study Read to learner Told a story Played a game Has books at to learner with learner home Has kids' books at home HLE Dimension 26
© Copyright 2024 Paperzz