NATIONAL EDUCATION EVALUATION AND DEVELOPMENT UNIT
A PRESENTATION TO
Portfolio Committee of Basic Education
15 November 2011
Cape Town
SCHOOLS VISITED
• Between February and September, NEEDU team (five members)
visited schools in five provinces:
– Mpumalanga, Eastern Cape, KwaZulu-Natal, Northern Cape, and
Western Cape
– A total of 74 schools were visited, 42 primary and 32 high
schools
– No systematic selection of schools. Data mainly anecdotal—a variety of
evaluation and observation instruments were used
– Purpose of school visits vary from province to province:
• To provide some indication as to why high schools and their feeder primary
schools were underperforming;
• To identify good practices from good performing primary schools;
• To evaluate the extent to which schools implement the national initiatives
introduced by the Minister to improve learning outcomes, incl. ANA,
learner workbooks, and the Foundations for Learning Programme
2
PURPOSE OF SCHOOL VISITS
•Purpose of school visits vary from province to
province:
– To conduct a pre-evaluation survey of school landscape in
various provinces
– To provide some indication as to why high schools and
their feeder primary schools were underperforming
– To identify good practices from good performing
primary schools
– To evaluate the extent to which schools implement the
national initiatives introduced by the Minister to
improve learning outcomes, incl. ANA, learner
workbooks, and the Foundations for Learning
Programme
3
APPROACH TO SCHOOL EVALUATIONS
• Schools were selected based on their performance in ANA and Grade
12 national exams
• Data collection processes included:
– Discussion with the principals, school management teams, and selected
teachers
– School records, incl. teacher and learner attendance registers, logbooks, asset
registers, learner assessment records, school improvement plans, etc.
– Review of teachers’ work, incl. their work schedules, daily lesson preparations,
classroom and personal timetables, assessment plans and tasks, and other
records
– Review learners’ work, incl. class work, homework, assessment books, and
learner workbooks provided by the Department
– Selecting learners randomly and asking them to read (especially in primary
schools) and doing mental mathematics
– Entering classrooms and observing teachers teaching—focusing on language
and mathematics (refer to the pack for the instrument that was used)
4
POST SCHOOL EVALUATION
 Provide feedback to teachers after classroom
observations
 Provide feedback schools and district officials on
the findings and recommendations
 It is expected that the latter will provide the
necessary support to schools
 A full report will be submitted to the Minister for
her consideration
5
GUIDING PRINCIPLES
•
Informed by the Sector Plan (Action Plan to 2014 Towards the
realisation of Schooling 2025) and the Minister’s Delivery
Agreement, NEEDU is guided by the following principles when
evaluating schools:
–
–
–
–
–
–
The quality of teaching and learning is central in improving low
levels of learner performance
School evaluation initiatives must lead to improved quality of
teaching and learning;
School improvement will be enhanced if there is an improvement in
the quality of support provided to schools by the relevant structures
There must be visible improvement in the functionality of schools;
Empowered school principals are able to provide necessary
leadership, particularly where it matters the most, i.e., curriculum
delivery; and
There must be sustainable learner performance improvement
annually, as reflected in the set targets in the Sector Plan
FOCUS AREAS FOR EVALUATION
• While NEEDU looked at other contextual issues (refer to
the pack) that have a bearing on curriculum delivery, it
mainly focused key essential drivers of quality to improve
learner outcomes:
– Monitoring student learning
– Instructional leadership of the principal;
– Curriculum quality;
– Opportunity to learn (OTL); and
– Professional development and collaboration
• The latest research published in 2011 (Carnoy et al;
Shepherd; Spaul; Taylor; and Van der Berg) consistently
identify the above as essential drivers of quality
FINDINGS: Monitoring student learning
• Output 2, sub-output 1 in the Minister’s delivery
agreement talks about establishing a world class system of
standardised national assessments
• NEEDU examined how student learning was monitored or
assessed in schools and how results from such
assessments were used (refer to the pack for details).
NEEDU observed that:
– In all schools, teachers were in full compliance with regard to
keeping the prescribed number of formal learner performance or
assessment records.
– The main challenges in schools included over utilisation of a single
form of assessment, quality and use of assessment results,
whether school-based of external, e.g. ANA:
FINDINGS: Monitoring student learning
• Specifically, NEEDU found that:
– In most schools, teachers focus on formal written assessments. Other
equally important forms of assessments are hardly used (doing the
minimum)
– Most school-based assessment tasks, particularly in primary schools, are
of low quality and are not comparable with external assessments. They
test low level skills. High order skills such as problem solving,
independent thinking are not tested
– School-based assessments do not adequately cover the curriculum
– Only in few schools did NEEDU find that assessment results are analysed
to identify problematic areas in the curriculum
– Except in isolated cases, no evidence could be found that either schoolbased assessments or external assessments were used to inform
planning, teaching strategies and school-based or district-based
interventions
FINDINGS: Principals as instructional leaders
• Output 4, sub-output 1 in the Minister’s delivery agreement
refers to the strengthening of school management and
promotion of functional schools.
• NEEDU observed instructional leadership practices of
principals (refer to the pack for the practices) and found that
the majority of principals have abdicated their curriculum
management responsibilities, i.e.
– Principals spend most of their time on other tasks but curriculum
matters
– Most principals do not have systems and procedures to monitor
curriculum delivery, incl. controlling classroom practices and
monitoring teachers’ work
FINDINGS: Principals as instructional leaders
• NEEDU also found that:
– Little evidence to suggest that principals initiate and facilitate
staff development (based on teacher professional development
needs established through IQMS) to improve learning outcomes
– Only a handful of primary school principals know performance
levels in language and mathematics in their schools
– In very few cases did primary school principals use evidence from
ANA to identify appropriate school intervention programs, to
develop a School Improvement Plan (as required by WSE policy)
or to develop an Academic Performance Improvement Pan (as
required in terms of the Education Laws Amendment Act)
FINDINGS: Curriculum Quality
• Output 1, sub-outputs 1 and 2 in the Minister’s delivery
agreement refer to the improvement of teacher capacity
and practices, and the increase of access to high quality
materials.
• NEEDU observed teacher practices in and outside class (refer to the
pack for the practices) and observed the following:
– While most teachers kept files with lesson plans, NEEDU observed
that some teachers did not have clear daily lesson plans
– In almost all schools visited, work schedules and lesson planning
were not informed by findings in ANA
– Most primary school teachers have very low expectations for their
learners--resulting in them focusing on inappropriate level of
cognitive demand, e.g., spending more time on single words and
not extended writing, and in mathematics completely ignoring
word sums
FINDINGS: Curriculum Quality
• NEEDU also found that:
– Except in very few cases, most teachers did not seem to have a
variety of strategies to teach reading. The five key components of
teaching reading ( phonemic awareness, word recognition [sight
words and phonics], comprehension, vocabulary and fluency)
were completely ignored in many schools
– While there were pockets of good practice, in many classrooms
visited there was no differentiated teaching to cater for learners
with different learning styles and learners with barriers to
learning, e.g., learners with foetal alcohol syndrome mostly in the
Northern Cape
– Most schools do not have sufficient reading materials incl. graded
readers. Learner workbooks supplied by the Department were not
used appropriately in most schools.
– Learners (in some cases, teachers) have a challenge using English
as a language of teaching and learning
FINDINGS: Opportunity to learn
• Output 4, sub-output 1 in the Minister’s delivery
agreement acknowledges that there has been insufficient
attention paid to whether teachers complete the year’s
learning programme within the year.
• When evaluating whether learners are provided with
adequate opportunities to learn, NEEDU observed three
elements: the effective use of time, curriculum coverage
and homework (refer to the pack for details). Following
were NEEDU observations:
– Evidence in many schools indicates that there is a problem with
school timetables. The prescribed time for teaching reading (incl.
time for shared reading, guided reading, independent reading or
reading for pleasure) and mathematics is not observed in a good
number of schools. Extreme cases included learners being taught
three hours of mathematics a week instead of a minimum of six.
FINDINGS: Opportunity to learn
• Other findings:
– When observing lessons in class, NEEDU focused on the amount of time
spent specifically on instruction and found that instructional time was
further lost on non-instructional matters, such as maintaining order, etc.
– There was a problem with curriculum coverage and pacing. While work
schedule clearly indicated coverage of the curriculum, there was usually
no congruence between work schedules, daily lesson plans (where they
existed) and exercises in learner workbooks or homework books.
– In many schools, learners were not given sufficient written work in
language or mathematics in class or as homework. Where written work
was provided, NEEDU found no evidence to indicate that enough time
and energy were put on addressing problematic areas in the curriculum
as revealed by ANA
– Many schools did not have a homework policy. Thus, many schools did
not give homework; where homework was given it was done
haphazardly with no clear purpose
FINDINGS: Professional development
• Output 1, sub-output 1 in the Minister’s delivery
agreement acknowledges that there is a need for the
development of new training packages for schools, largely
through distance education and e-Education.
• In terms of professional development and collaboration,
NEEDU observed that:
– Following the identification of individual teacher development
needs through the IQMS processes, these needs are seldom
addressed
– Almost all school improvement plans do not indicate how
professional development needs of teachers will be addressed
FINAL REPORT ON 100 SCHOOLS
• A more controlled scientific study, involving an stratified
random sample of 100 schools, will be conducted between
January and March 2012
• The purpose of this study is to:
– Establish the causes of poor performance in language and
mathematics schools,
– Examine the extent to which the national initiatives introduced
by the Minister (incl. ANA, learner workbooks, CAPS) are making
a difference
– Test NEEDU’s school evaluation instruments
• The study with focus on in-school factors, particularly the
quality of teaching and learning
• Procurement processes are being followed to appoint a
service provider that will provide technical assistance to
NEEDU to conduct this study
MILESTONES:
• NEEDU has developed instruments that will be used to
evaluate the quality of teaching and learning in a sample 100
schools across the provinces
• A service provider has been appointed to quality-assure
instruments developed by NEEDU before they are used to
ensure their credibility
• NEEDU interviewed over 100 applicants for 21 NEEDU
evaluators' posts across the nine provinces. It is anticipated
that all 21 evaluators will assume duties in January 2012 and
will be involved in collecting data in 100 schools.
• The appointed service provider will train NEEDU evaluators in
January 2012
• Data collection in 100 schools will commence in
February/March 2012
• The final report will be available in April 2012