Primary Pupil Profile
Final Evaluation of InCAS and the
Annual Pupil Profile Report
October 2008
Final Evaluation Report on the implementation of
the InCAS computer-based assessments and the
Annual Pupil Profile Report in Primary Schools
during the transitional year 2007/2008
Prepared by:
CCEA Research & Statistics Unit
Copyright 2008 CCEA. All rights reserved.
This report is available at: www.nicurriculum.org.uk
This evaluation was conducted from May 2007 through to August 2008. For further
information about this evaluation please contact Karen Wiltshire:
CCEA Research & Statistics Unit
29 Clarendon Road
Clarendon Dock, Belfast
BT1 3BG, or alternatively email
[email protected]
Prepared by Research & Statistics
1
KW2031/0809
Scope of the Report
This report is the final of three evaluation reports on the implementation of the revised
assessment and reporting arrangements in Primary schools in Northern Ireland. All
reports are available at www.nicurriculum.org.uk.
The first two reports, released in December 2007 and May 2008, were designed to collect
Year 5 teachers’ views on the;
•
Training provided for the InCAS computer-based assessments;
•
Administration of InCAS;
•
Feedback produced from InCAS; and the
•
Usefulness of the InCAS feedback for teachers, parents and pupils.
Year 5 pupils’ opinions on the InCAS assessments were also collected via a questionnaire
that was distributed to all Primary schools in November 2007.
This final report provides a robust evaluation of the views and opinions of all stakeholders
involved in this year’s implementation of
1 InCAS
•
•
•
•
Principals
Year 5 Teachers
Year 5 pupils
Parents
2 Annual Pupil Profile Report
•
•
•
•
Principals
Year 1 and 5 Teachers
Parents with children in Year 1 and 5
Year 5 pupils
Such stakeholders have provided CCEA with constructive information on all aspects of
InCAS assessments and the Annual Pupil Profile Report which will be used to inform
planning, development and the next phase of implementation.
A full breakdown of this year’s consultation for this academic year (2007-08) can be found
in the Annex of this report.
Prepared by Research & Statistics
2
KW2031/0809
Acknowledgements
CCEA would like to thank all Principals, teachers, pupils and parents who contributed to
this evaluation. We are extremely grateful to all participants for taking the time to share
their experiences of the InCAS computer-based assessments and the Annual Pupil Profile
Report. This cooperation and involvement allows CCEA to have a detailed and collective
understanding of the implementation of the new assessment and reporting arrangements
in Primary schools in Northern Ireland.
This consultation has provided schools, parents and pupils with an opportunity to inform
future planning and development of these new arrangements.
Prepared by Research & Statistics
3
KW2031/0809
Contents
Section
1.0
Page
Executive Summary
5
(Contains Conclusion & Recommendations)
2.0
Introduction
14
3.0
Method
16
4.0
Results
4.1
Principal
17
4.2
Year 5 Teacher
40
4.3
Year 5 Parent
57
4.4
Year 5 Pupil
70
4.5
Year 1 Teacher
74
4.6
Year 1 Parent
83
5.0
Conclusion
89
6.0
Recommendations
93
7.0
Annex
95
Prepared by Research & Statistics
4
KW2031/0809
1.0 Executive Summary
This is the final evaluation report on the first year of the implementation of the InCAS
computer-based assessments in Year 5 and the revised reporting arrangements in Years
1 and 5. The purpose of this evaluation is to provide all stakeholders (Principals, teachers,
pupils and parents) involved in this implementation with an opportunity to present their
views and opinions of these new assessment and reporting arrangements. Information
collected from this evaluation will be used to inform the future planning and development.
Summary of Findings
InCAS Computer-based Assessments
Overall feedback on the InCAS computer-based assessments has been largely positive.
The majority of teachers who participated in evaluations in December 2007 believed pupils
found InCAS user-friendly (94.5%, n = 517). 91% of teachers also deemed the feedback
consistent with their professional judgement (n = 477).
Consultation with stakeholders (Principals, teachers, pupils and parents) in June 2008 has
also been encouraging. Over 70% of Principals (71.7%, n = 220) and Year 5 teachers
(70.9%, n = 251) felt the feedback from the InCAS assessments informed teacher
planning. Furthermore, although not a statutory requirement this transition year a sizeable
number of schools (>24%, n=219) opted to share InCAS feedback with parents. 76%
(n = 136) of Principals and 58% (n = 201) of teachers from such schools deemed this
feedback to be meaningful to parents. 57.5% (n = 1097) of parents who received their
child’s results to InCAS believed it enhanced the information shared at the parents
meetings.
Each phase of evaluation has highlighted a number of challenges schools have
encountered in implementing InCAS assessments for the first time. Many of such
difficulties were a result of teachers using the software for the first time (e.g. difficulties
setting up). However, stakeholders have also identified a number of areas of improvement
within the assessment tool (e.g. volume control); all of which have been shared with
CEM/C2K and modifications made for the 2008/09 academic year.
Prepared by Research & Statistics
5
KW2031/0809
Feedback from the Principal and teacher workshops and parent focus groups raised a
number of issues that had not been expressed as strongly within earlier evaluations.
Despite the fact that the Department of Education funded additional laptops within the
Primary sector a large number of Principals still believe that Primary Schools do not have
the capacity to deliver computer-based assessments, particularly those with composite
classes or those without computer suites. Teachers’ feedback on the reliability of InCAS
has been very high within previous phases of evaluation. However, within all workshops
and focus groups facilitated in June 2008 a sizeable number of Principals, teachers and
parents questioned the reliability and validity of InCAS feedback, contradicting responses
made at the time of delivery.
Capita Profiles 7
Despite the majority of Principals and teachers (>92%) rating the Profiles 7 training highly,
over 40% of both Principals and teachers indicated that it was not adequate preparation
for using the software. Principals were asked by C2K to nominate two representatives
from their school to attend the training. It was evident from the teacher and Principal
workshops that a variety of staff attended, many of whom were not the Year 1 or 5
teachers who were required to complete the Annual Pupil Profile Reports. A large number
of Principals and teachers commented that sub-cover should have been provided for this
training and all teachers planning to use the Profiles 7 to complete reports should have
been allocated a place at the training.
Over 70% of Principals and teachers indicated that their school used the Profiles 7
software to produce Annual Pupil Profile Reports. Over 62% of such Principals and
teachers experienced difficulties with this software. Many respondents felt the software
was not user-friendly and was time consuming. Furthermore, the majority of Principals and
teachers (>68%) deemed the Profiles 7 software to be less manageable than other
methods they have used to compile school reports, such as word documents. Participants
at the Principal and teacher workshops expressed a high level of dissatisfaction with the
Profiles 7 software, stating that it was not user-friendly and had significantly increased the
amount of time teachers take to produce pupil reports.
Prepared by Research & Statistics
6
KW2031/0809
Comment Banks
Despite the majority of Principals and teachers indicating that the comment banks were
useful, over 91% of both respondents felt they could be improved upon. A large number of
both Principals and teachers stated at the workshops that comments need to be provided
for all areas of the curriculum. Principals and teachers also stated that they require
exemplary comments for areas in which pupils need to develop.
Annual Pupil Profile Reports
Over 70% of Principals and teachers rated the format of the Annual Pupil Profile Reports
as either excellent or good. The majority of parents with children in Year 1
(96.6%, n = 3467) and Year 5 (86.8%, n = 3007) also rated the format of the Annual Pupil
Profile Report highly.
However, at the workshops and focus groups a large number of attendees (Principals,
teachers and parents) did not rate the content of the Annual Pupil Profile Reports highly.
Such stakeholders felt that grades/marks should have been included within the Annual
Pupil Profile Reports. Furthermore, a large number of Principals, teachers and parents
described the Annual Pupil Profile Reports as “hyper-positive”, bland and impersonal. A
large number of parents with children in Year 1 and 5 stated that it was evident teachers
had used comment banks to write their child’s report.
Attendees at the Principal and teacher workshops suggested that communications
regarding the statutory requirements of the Annual Pupil Profile Report have not been
communicated effectively from the Government agencies. Furthermore, findings from the
parent focus groups indicate that the majority of parents had no knowledge of the purpose
of InCAS computer-based assessments or the Annual Pupil Profile Report.
Prepared by Research & Statistics
7
KW2031/0809
Conclusion
This evaluation has identified a number of issues within the Primary Education sector.
Many Primary schools felt pressured from a combination of demands from Government
agencies and parents. In some cases this is exacerbated by the perception that the new
assessment and reporting arrangements are being introduced at a demanding pace.
Feedback collected from earlier phases of evaluation on InCAS has been very positive.
The majority of teachers who participated within previous evaluations felt confident
interpreting feedback and deemed it to be consistent with their professional judgement.
Feedback collected from all stakeholders in June 2008 has been somewhat more
negative. However, problems teachers encountered completing the Annual Pupil Profile
Reports is likely to have escalated their criticisms and concerns about InCAS.
Furthermore the deployment and use of InCAS took place at the same time as a major
transition to a newly contracted C2K managed service in all primary schools. The majority
of concerns raised have been identified in previous evaluations and modifications made for
the 2008/09 academic year. However, one manageability aspect of InCAS that has not
been highlighted as strongly within previous research was that many respondents do not
believe that Primary Schools have the computer capacity to manage the delivery of InCAS
as it involves more year groups. 8,000 additional laptops have been installed in primary
schools across Northern Ireland. These together with the 5,000 assessment laptops
distributed in 2007 mean there are now over 28,000 devices available for use with InCAS.
However, despite this investment respondents stated that teachers with composite classes
and schools without computer suites have found delivering the InCAS assessments
extremely difficult. Many Principals felt that further ICT investment is required for such
schools and teachers need further support to administer InCAS.
Feedback from this evaluation suggests that some Principals and teachers are unclear
about the nature and status of the Annual Pupil Profile Report. The most widely held
misconceptions were that schools were required to use the Profiles 7 software to complete
reports and that the use of comments in the bank were mandatory. As a consequence
some teachers reported high levels of stress. In some cases comments were used well,
with parents reporting a positive reaction to the information they received about their
children. However less careful use of comments from the comment banks resulted in
reports which were considered by parents as overly positive, general and impersonal.
Prepared by Research & Statistics
8
KW2031/0809
Some schools report receiving different messages about requirements of revised
assessment and reporting arrangements from education stakeholders.
The implementation of new electronic assessment and reporting arrangements represents
significant change for many Primary Schools in Northern Ireland. CCEA provided schools
with a variety of support and guidance in the form of; training materials, telephone
helpdesk, a dedicated section on the Northern Ireland Curriculum website, online tutorials
and an InCAS pupil demonstration. Feedback from this year’s consultation indicates that
some schools did not use the materials or follow published instructions. Some teachers
within the teacher focus groups felt that they did not have time to make use of the
resources provided.
Both Principals and teachers expressed high levels of dissatisfaction with the Profiles 7
report writing software. Feedback indicates that teachers and Principals believe this
software has significantly increased the amount of time teachers take to produce Annual
Pupil Profile Reports whilst reducing the quality of information. The majority of Principals
and teachers deemed this software to be extremely time consuming to use, underdeveloped and have questioned the degree of piloting prior to this year’s implementation.
Only one participant at the Principal and teacher workshops indicated that they intend to
use this software next year.
This evaluation indicates that parents are not entirely satisfied with the new assessment
and reporting arrangements. However, it is important to note that parent’s feelings have
been heightened by the current uncertainty over transfer arrangements and their lack of
knowledge of InCAS and the revised Annual Report. Some parents indicated that they did
not know that their children were being assessed at the beginning of the school year.
A large number of parents, Principals and teachers expressed concerns that INCAS
assessment outcomes may be used as a method of selection at a future date.
A large number of parents were dissatisfied with the format and content of their child’s
Annual Pupil Profile Report. Although many parents recognised the work required by
teachers to produce this new report there were a number of aspects they criticised.
Parents’ main criticism was that it did not contain grades/marks or a comparison with their
child’s peers which they deemed to be an essential component of any school report.
Prepared by Research & Statistics
9
KW2031/0809
Parents described the reports as “impersonal, vague, over generalised” and as a result
were unclear about how their child was progressing. There was also widespread concern
among parents on how ‘hyper-positive’ the reports were. Many parents felt that the report
did not provide them with a true account of their child’s progress. One parent stated:
“[The Annual Pupil Profile Report] was written in politically correct language which
told me nothing. Please encourage teachers to write what they feel and not what
they think ought to be written. More teacher freedom please”.
Some teachers felt uneasy about writing what might be considered less than positive
comments in the reports.
Teachers also have requested advice on how to share InCAS feedback with parents and
direction on comments they may include within the Focus for Development section of the
Annual Pupil Profile Report.
This report has provided a comprehensive overview of this year’s implementation of the
new assessment and reporting arrangements across Primary Schools in Northern Ireland.
Findings from this evaluation show that embedding such new arrangements has required a
large degree of additional work from teachers and Principals. Schools require further
assistance to ensure that they feel adequately supported and empowered to implement
these new arrangements.
Prepared by Research & Statistics
10
KW2031/0809
Limitations
Parents, Principals and teachers have indicated that they are less than satisfied about the
uncertainty about new transfer arrangements. This has impacted on findings obtained
within this evaluation as many were concerned that either InCAS or the Annual Pupil
Profile Report could or would be used to transfer pupils to post primary schools.
A second limitation of the research is the poor response rate to the Principal and teacher
questionnaires. It may be that those most dissatisfied with InCAS and the Annual Pupil
Profile Report completed and returned questionnaires to CCEA. This may have negatively
skewed results but is a common limitation of research.
Prepared by Research & Statistics
11
KW2031/0809
Recommendations
Before commencing the second roll out of InCAS and the Annual Pupil Profile Report there
are a number of issues documented within this report that should be addressed.
Communication
This evaluation suggests that:
•
Communications regarding the new assessment and reporting arrangements should be
re-evaluated. All communications on these arrangements should be shared between
all partners (CCEA, DE, C2K and CEM).
•
At the start of the next school year CCEA should inform schools which elements of the
Annual Pupil Profile Reports are statutory – it is imperative that such information is
specific to each Year Group.
•
CCEA should provide teachers with an ‘at a glance guide’ outlining what elements of
InCAS and the Annual Pupil Profile Report are statutory and flexible.
•
CCEA should communicate with parents on the new assessment and reporting
arrangements.
InCAS
This evaluation suggests that:
•
Further work should be carried out to determine whether or not additional computer
capacity may be required to support the use of InCAS within Primary schools.
•
CCEA should continue to empower schools to embrace InCAS and develop expertise
and confidence in both administering and analysing feedback produced from InCAS.
CCEA should also reassure schools of the reliability and validity of InCAS.
•
Teachers would welcome advice on how to share InCAS feedback with parents.
•
Further adaptations to InCAS are needed to meet the needs of pupils with Special
Educational Needs and those with English as an Additional Language.
Prepared by Research & Statistics
12
KW2031/0809
Recommendations continued
Profiles 7 Software
This evaluation suggests that:
•
Consideration should be given to whether Profiles 7 is the most appropriate report
writing software or if an alternative should be sought and trialled. Alternatively, Profiles
7 should be developed as a matter of urgency to ensure it better meets the needs of
schools completing the Annual Pupil Profile Report.
•
All teachers who plan to use the Profiles 7 report writing software should be provided
with appropriate training.
Annual Pupil Profile Report
This evaluation suggests that:
•
Schools should be advised explicitly what the purpose of the Annual Pupil Profile
Reports is and in particular its role (if any) in transfer arrangements.
•
Schools require development time to tailor comment banks and ensure all Annual Pupil
Profile Reports are personalised to individual pupils.
•
Principals, teachers, pupils and parents would like to see marks/grades to be included
within the Annual Pupil Profile Reports.
•
Consideration should be given to moving the ‘Focus for Development’ section towards
the end of the Annual Pupil Profile Report.
Prepared by Research & Statistics
13
KW2031/0809
2.0 Introduction
The implementation of InCAS computer-based assessments and the Annual Pupil Profile
Report commenced at the start of the 2007 academic year. This involves teachers
assessing and reporting pupil progress and achievements. It will be used to support and
inform decisions throughout a pupil’s time at school, providing information for;
•
Parents - about helping their child’s learning at home;
•
Teachers - as they plan learning and teaching; and
•
Pupils - about their achievements and how to improve.
InCAS Computer-based Assessments
InCAS statutory computer-based assessments are completed during the autumn term.
InCAS was developed by the Curriculum, Evaluation and Management Centre at Durham
University and is an adaptive computer-based assessment. Computer-adaptive
assessments are tailored to match a pupil’s ability. The assessment program generates
items depending on the pupil’s response to a previous number of items. The difficulty of
subsequent assessment items presented to the pupil depends upon their responses as
they proceed through the assessment. The adaptability of this assessment method means
that learners are not faced with the de-motivating experience of being unable to answer
many of the items (which is often the case with pencil and paper tests).
Prepared by Research & Statistics
14
KW2031/0809
The InCAS computer-based assessment consists of 11 modules. By the end of the
autumn term 2007, all Primary Schools in Northern Ireland were required to administer two
statutory assessments - Reading and General Math’s to Year 5 pupils to produce age
equivalent scores. The Reading age is achieved on the completion of the Word
Recognition, Word Decoding and Comprehension modules. The General Math’s module
generates the general Math’s Age. Teachers can make full use of the other seven
modules within the InCAS assessment tool (Spelling, Mental Arithmetic, Picture
Vocabulary, Non-Verbal Ability, Attitude to Math’s, Attitude to Reading and Attitude to
School).
The Department of Education determined 2007/2008 a transition year for the new
assessment and reporting arrangements. During this year outcomes from InCAS did not
have to be formally reported to parents. However, during 2008/2009 schools are required
to share the outcomes from the InCAS assessments at a parents meeting in the autumn
term for Years 5 and 6 and by 2009/10 this will be statutory for Years 4-7.
Annual Pupil Profile Reports
This year, revised annual reporting arrangements were introduced for pupils in Years 1
and 5. The purpose of the Annual Pupil Profile Report is to promote consistency in
reporting arrangements across schools in Northern Ireland.
CCEA’s Research & Statistics Unit is responsible for the evaluation of the implementation
of InCAS and the Annual Pupil Profile Reports. The purpose of this evaluation was to
provide all stakeholders (Principals, teachers, pupils and parents) involved in this
implementation with an opportunity to present their views and opinions of these new
assessment and reporting arrangements. Information collected from this evaluation will be
used to inform the future planning and development of the new arrangements.
Prepared by Research & Statistics
15
KW2031/0809
3.0 Method
The following section includes a summary of the methodology used to evaluate the final
phase of the implementation of the InCAS computer-based assessments and the Annual
Pupil Profile Report. A full breakdown of this year’s (2007-08) consultation can be found in
the Annex of this report.
For this final phase of evaluation to be as inclusive as possible, information was collected
from all stakeholders involved in the implementation of the InCAS computer-based
assessments and the Annual Pupil Profile Report. This encompassed Principals,
teachers, pupils and parents. The overall aim of this evaluation was to conduct a robust
and reliable evaluation of these new assessment and reporting arrangements which can
be used to inform and advise upon future implementation years.
Summary of Participants & Research Tools
Questionnaires were the main method of collecting data from stakeholders (Principals,
teachers, pupils and parents). However, a range of qualitative methods were also
employed to further explore and reinforce information collected from the Principal, Year 5
teacher and Year 5 parent questionnaires. In total CCEA’s Research and Statistics Unit
facilitated eight Principal workshops, ten Year 5 teacher workshops and 13 parent focus
groups. Qualitative consultation will be undertaken with Foundation Stage teachers next
year (June 2009).
Response rates to questionnaires and attendance rates at the workshops and focus
groups are detailed below.
Table 1
Response Rates to Questionnaire
Attendance rate at Workshop/Focus Gp
Principal
337
111
Year 1 Teacher
281
N/A
Year 5 Teacher
378
201
Year 1 Parent
3,631
N/A
Year 5 Parent
3,586
145
Year 5 Pupil
5,120
N/A
Prepared by Research & Statistics
16
KW2031/0809
4.0 Results
4.1
Results collected from Principal Questionnaires & Workshops
This section presents the results for each question from the Principal questionnaire.
Results are based on completed questionnaires from 339 respondents. Percentages
relate to the number of Principals who answered each question, which may be fewer than
the overall total of 339. A number of Principals reiterated comments throughout their
questionnaire; as a result some comments are repeated throughout this section of the
report.
Qualitative information obtained from eight Principal workshops attended by 111 Principals
have been analysed alongside the feedback collected from the questionnaire. The
purpose of the workshops was to collect in-depth information to reinforce and further
explore that collected from the questionnaire.
Prepared by Research & Statistics
17
KW2031/0809
Manageability of the InCAS computer-based assessments
Figure 1: How Manageable were the InCAS assessments to complete?
100
80.4
Figure 1 illustrates that the
% Respondents
80
majority of Principals felt that
60
the InCAS assessments were
manageable to complete
40
20
8.7
9.6
1.2
(89.1%, n = 287).
0
Very
Manageable
Not
Not at all
m anageable
m anageable m anageable
However, despite such positive findings, 64.9% of respondents (n = 204) encountered
challenges administering InCAS. The most common challenges Principals commented on
are summarised below.
•
Administration/set up was time consuming (33 comments)
•
Experienced general technical difficulties (28 comments)
•
Difficulties providing adequate supervision/sub-cover (23 comments)
•
Availability/allocation of computers (22 comments)
•
Computers crashing (20 comments)
•
Problems accessing the software (16 comments)
•
Difficulties adjusting volume (15 comments)
•
Problems with passwords (10 comments)
•
Managing the rest of a class while the assessments were administered
(10 comments)
Prepared by Research & Statistics
18
KW2031/0809
Workshop findings
Feedback from the Principal workshops reinforced all of the technical and user
difficulties Principals noted within the questionnaire. A large number of participants
within all of the Principal workshops believed that unlike the Post Primary sector,
Primary Schools do not have the computer capacity or ICT support for the
completion of electronic assessments. Many participants stated that teachers with
composite classes and schools without computer suites had particular difficulties
administering InCAS.
An additional 8,000 laptops have been installed in Primary Schools across Northern
Ireland. These together with the 5,000 assessment laptops distributed in 2007 mean
there are now over 28,000 devices available for use with InCAS. However,
attendees at the Principal workshops felt that despite schools receiving additional
laptops they still did not have a sufficient number of workstations to administer
InCAS. Many Principals highlighted that the laptops posed a number of problems for
schools. Principals within one particular workshop highlighted that setting up the
laptops for InCAS was extremely time consuming especially temporary computer
suites made up of a number of laptops. Such Principals stated that this created a
number of accommodation and health and safety issues for schools.
The majority of Principals within all of the Principal workshops believed that schools
had difficulties implementing InCAS. Principals also expressed concerns about the
impact on teaching time as a result of administering InCAS. One Principal stated
that InCAS “should have bedded into the school year and been a natural part of the
school year, but the fact is this would not have worked without the dedicated work of
Principals and teachers who have sacrificed all over the place to administer this.
[This year] other pupils have done without so that Year 5 pupils could complete
InCAS”
Prepared by Research & Statistics
19
KW2031/0809
Principals were asked in the questionnaire to document any managerial decisions made
for the completion of the InCAS assessment with Year 5 pupils. Managerial decisions that
Principals made are listed below.
•
Time-tabling (78%, n = 263)
•
Allocation of resources (68%, n = 228)
•
Deployment of staff (48%, n = 163)
•
Substitute cover / supervision for classes (15%, n = 51)
Just over 70% of respondents (70.7%, n = 205) indicated that they intend to put
procedures in place to support the administration of InCAS next year. The two most
common procedures amongst Principals include:
(1) Allocation of sub cover/in-house supervision (n = 46); and
(2) Timetabling of computer suites (n = 45). Other procedures are summarised below.
•
Utilising ICT skills of staff within school (15 comments)
•
Allocate time for teachers to administer and prepare (11 comments)
•
Plan to introduce a computer suite (10 comments)
•
Purchase more hardware (7 comments)
•
Remove pupils in composite classes not completing InCAS (6 comments)
•
Ensure tests are conducted in a quiet area (6 comments)
•
Plan meetings with all staff involved (5 comments)
A sizeable number of Principals (n = 34) stated that it is impossible for them to set up
procedures without additional funding for resources and staff release.
Prepared by Research & Statistics
20
KW2031/0809
Workshop findings
The majority of participants within all eight Principal workshops unanimously agreed that
schools require money to either release teachers to administer InCAS or to employ
someone to supervise pupils completing this electronic assessment.
A large number of Principals expressed concern at the prospect of two year groups
completing InCAS next year. The majority of Principals within two workshops stated that
unless InCAS is improved upon and schools are provided with the necessary support to
administer InCAS they will not permit their teachers to administer this assessment next
year.
Sharing InCAS Feedback with parents
Although not a statutory requirement this year 54.4% of Principals (n = 178) indicated that
their school shared InCAS feedback with parents.
Figure 2: How were results shared with parents?
% Respondents
100
94.4
80
Figure 2 illustrates that the majority
60
of schools (94.4%, n = 170) shared
40
InCAS feedback with parents
2.2
20
2.8
verbally.
0.6
0
Verbally
As a chart
As a table
Prepared by Research & Statistics
Other
form at
21
KW2031/0809
Respondents were asked if they felt the assessment feedback was meaningful to parents,
76% of Principals felt it was (n = 136). Such Principals felt that the feedback was useful to
parents for the following reasons.
•
Highlights children’s strengths and weaknesses (9 comments)
•
The age equivalent score was easy for parents to understand (6 comments)
•
Visual representation was useful i.e. graphs and charts (5 comments)
•
Reinforced teachers’ professional judgement (3 comments)
•
Useful when used alongside NFER results (3 comments)
Principals who felt InCAS feedback was not meaningful to parents provided the following
reasons for their answer.
•
The feedback contained too much jargon (8 comments)
•
The feedback was too harsh/blunt for SEN pupils (5 comments)
•
InCAS results were unclear to parents (4 comments)
Principals who chose not to share InCAS feedback with parents were asked to outline their
reasons for this decision, these have been summarised below.
•
Sharing feedback with parents was not statutory this year (42 comments)
•
Felt the results were unreliable (25 comments)
•
Teachers were not adequately trained to share feedback with parents (19 comments)
•
Teachers wanted to familiarise themselves with feedback this transition year
(16 comments)
•
Did not believe parents would understand results (12 comments)
•
Did not complete InCAS in time for parent interviews (7 comments)
•
Anticipated that feedback would be time consuming to explain (6 comments)
Prepared by Research & Statistics
22
KW2031/0809
Workshop findings
Information collected from all of the Principal workshops regarding the suitability of InCAS
feedback for parents was generally very negative. Many Principals stated that the
meaningfulness of this information varied greatly from parent to parent. The majority of
Principals stated that teachers were not confident interpreting the results and therefore
were reluctant to share results with parents. Many Principals expressed concerns about
the reliability of InCAS in comparison with existing assessments used within school. The
majority of Principals from three of the eight Principal workshops felt that the InCAS results
did not match with standardised test results.
A small number of Principals within all of the workshops believed that the feedback from
InCAS was very “blunt” for pupils with Special Educational Needs. One Principal stated
that when sharing feedback with parents whose child has Special Educational Needs “you
are reporting back that the child is so many years behind – what is that doing to the child
but giving them an educational kicking”.
Within all of the Principal workshops a small number of participants expressed concern
over the potential use of InCAS. Such Principals were dubious that InCAS was only to
inform teacher planning and learning. Many felt that InCAS results could and would be
used for benchmarking against schools and for school inspections – which one Principal
stated will “distort the whole rationale for InCAS as it could be used as a stick to beat
schools”.
From 2008/2009, schools are required to report the age-equivalent scores for Reading and
General Math’s to parents with children in Years 5 and 6. Principals indicated that next
year they plan to share InCAS feedback with parents using either a chart (46.1%, n = 118)
or a table (39.5%, n = 101).
Prepared by Research & Statistics
23
KW2031/0809
Was the InCAS feedback informative to teachers?
Principals were asked if they felt the feedback from InCAS informed teacher planning.
Whilst 71.7% of Principals (n = 220) believed the feedback from InCAS informed teacher
planning, 28.3% felt it did not. A small number of Principals who believed the feedback
from InCAS informed teacher planning, commented that it highlighted pupils’ strengths and
weaknesses (n = 29) and informed planning for both individual pupils and the whole class
(n = 16). Comments from Principals who felt InCAS feedback did not aid teacher planning
are summarised below.
•
Provided no additional information (32 comments)
•
Teachers do not have time to utilise the results in planning (10 comments)
•
Unexpected results suggest that results may not be reliable (8 comments)
•
Results from the General Maths module need to be broken down further (6 comments)
Was the InCAS feedback informative to parents?
58.9% of Principals (n = 162) felt that the feedback from InCAS did not inform parents of
how they can support their child’s learning at home. Such Principals felt that the feedback;
•
Required teacher guidance and explanation (16 comments);
•
Is difficult for some parents to understand (15 comments);
•
Did not provide any new information to parents (14 comments); and
•
Requires more detail on how parents can help their child improve (5 comments).
Prepared by Research & Statistics
24
KW2031/0809
Was the InCAS feedback informative to pupils?
Over 50% of Principals felt the results from InCAS did not inform pupils of their
achievements (n = 136) or how they can improve (n = 126). A small number of Principals
(n = 16) commented that pupils require teacher guidance for this information to be
meaningful.
Workshop findings
At all of the Principal workshops Principals’ views on the usefulness of the InCAS
feedback varied greatly. Those who deemed the feedback to be useful particularly liked
the instant feedback and felt it provided effective visual information for teachers and
parents.
Those who did not find the feedback useful believed it was non-specific, too general, not
diagnostic, limited and did not inform teachers or parents of remedial action they should
take.
Profiles 7 Report Writing Training
Two members of staff from every Primary School in Northern Ireland were offered training
in the use of Profiles 7 report writing software by C2K. The majority of Principals
(92.8%, n = 308) who completed the Principal questionnaire indicated that someone from
their school attended this training. A small number of Principals who indicated that no one
from their school attended the Profiles 7 training made the following comments.
•
Did not attend training as sub cover was not provided (8 comments)
•
Not offered this training (4 comments)
Nearly 70% of Principals (69.4%, n = 218) revealed that they attended the report writing
training themselves.
Prepared by Research & Statistics
25
KW2031/0809
Figure 3: How useful was the Profiles 7 training?
100
Principals who attended the
% Respondents
80
Profiles 7 report writing training
64.5
offered by C2K were asked to rate
60
how useful they found it. Figure 3
29.9
40
indicates that the majority of
20
3.7
1.9
0
Very useful
Useful
Not useful
Not at all
useful
respondents rated this training as
either very useful or useful (94.4%,
n = 202).
Despite the majority of Principals rating this training as useful, 42.5% (n = 111) of
respondents felt it was not adequate preparation for using the software. Such Principals
stated that;
•
All staff need training as it is too difficult for individuals to cascade training to other
teachers (34 comments);
•
More training is required as the session was too intensive (28 comments);
•
There was too much time between training and using the software (16 comments);
•
Further training is required on creating and developing comment banks (9 comments);
and
•
Technical difficulties with Profiles 7 were not mentioned at training (7 comments).
Workshop findings
The majority of participants within all of the Principal workshops indicated that someone
from their school attended the Profiles 7 training offered by C2K. The majority of
Principals felt that the C2K training was not sufficient preparation for using the software.
Principals stated that they were overloaded with information which was impossible for
them to remember when they were back in school. Many Principals also felt that there
was too great of a time delay between teachers attending the training and completing the
Annual Pupil Profile Reports. Principals stated that sub cover should have been provided
for this training and that teachers should not be expected to cascade such intensive
training to other teachers.
Online Tutorials
Prepared by Research & Statistics
26
KW2031/0809
CCEA provided online tutorials on the Northern Ireland Curriculum website to support
teachers’ use of the Profiles 7 report writing software. Only 15% of Principals (n = 48)
used these tutorials. A small number of respondents who did not use the tutorials
commented that they were not aware of them (n =19) or did not have time to use them (n =
13).
Nearly 90% of Principals who used the online tutorials deemed them to be useful
(89.6%, n = 43). However, only 43.8% of such Principals (n = 21) felt that the tutorials
alone were adequate preparation for using the software.
Workshop findings
Findings from the Principal workshops reinforce those collected from the questionnaires.
All attendees at the Principal workshops were unaware of the online tutorials on the
Northern Ireland Curriculum website. However, many Principals stated that even if they
had been aware of them teachers did not have time to use them.
Using the Profiles 7 report writing software
76.7% of Principals (n = 247) indicated that their school used the Profiles 7 report writing
software to complete this year’s Annual Pupil Profile Reports. Principals were asked to
indicate if they thought this software was user-friendly; 65.8% of respondents (n = 169)
believed it was.
However, despite over 65% finding the software user-friendly, 62.3% of Principals
(n = 162) experienced difficulties using it. The most common difficulties Principals
commented on are summarised below.
•
Sims crashed (34 comments)
•
Encountered difficulties loading Sims to the system (16 comments)
•
Unable to access Sims (10 comments)
•
Class details were missing or inaccurate (10 comments)
•
Editing was difficult (7 comments)
Prepared by Research & Statistics
27
KW2031/0809
Workshop findings
Principals within all of the workshops were very critical of the C2K training and time
required for teachers to familiarise themselves with the Profiles 7 software. Principals
were extremely angry and frustrated at the implementation of this software as they did not
feel it was user-friendly or fit for purpose. Three illustrated examples are given below.
¾ “Too complex for a simple objective”
¾ “Is this a cheap solution for the Department and an expensive solution for
schools?”
¾ “Costly for schools, teachers and also pupils – took their teachers away to do
something that didn’t give them enough back”
Figure 4: How manageable is Profiles 7 in comparison
to completing reports by other methods?
100
Figure 4 illustrates that 70.3% of
70.3
% Respondents
80
respondents (n = 196) found
Profiles 7 less manageable to
60
other methods they have used to
40
16.5
20
13.3
compile school reports.
0
More manageable Less manageable
No difference
Respondents were asked to comment on their response, these are summarised below.
•
Software is extremely time consuming (85 comments)
•
Time taken to complete reports affected work/life balance (77 comments)
•
Completing reports in Microsoft Word is more manageable (13 comments)
•
Schools require funding to provide non-contact time for teachers (11 comments)
•
Reports are impersonal and less informative than previous reports (9 comments)
•
Too many detailed areas to comment on (7 comments)
•
Time is required for schools to write their own comments (7 comments)
Prepared by Research & Statistics
28
KW2031/0809
Workshop findings
The majority of Principals within all workshops indicated that their school used the Profiles
7 software to complete Year 1 and 5 Annual Pupil Profile reports. Virtually all participants
stated that they did not find this software user-friendly. Many Principals felt that the
Profiles 7 software required a high level of ICT skills.
The majority of Principals were appalled at the amount of technical difficulties their staff
encountered when using Profiles 7. The most common problems commented upon by
participants were; editing and font changing (especially in preview mode) and SIMS
closing down at a specific time of the day. A small number of Principals were perplexed by
the fact that they did not receive a password for this software.
Virtually all of the attendees at the Principal workshops deemed the Profiles 7 software to
be less manageable than other methods they have used to complete school reports. A
large number of Principals felt that a word document would be much more user-friendly
and fit for purpose than the software provided by the Department of Education/C2K. Many
Principals deemed the Profiles 7 software to be under-developed and questioned the
degree of piloting prior to this year’s implementation.
Principals were particularly unhappy that schools were not provided with time
(sub-cover / closure days) to complete the new Annual Pupil Profile Reports and as a
result do not feel supported by CCEA or the Department of Education. A large number of
Principals stated that using the software has been extremely stressful for teachers and had
a detrimental effect on their work/life balance. Many Principals said that they had to keep
their school open until 9pm at night so that Year 1 and 5 teachers could complete reports
using the Profiles 7 software.
C2K provided dongles, which are similar to data keys to provide schools with access to
SIMS at home. It is planned that small Primary Schools will receive one dongle and larger
schools will be provided with two. Principals stated that two dongles per school is
insufficient and it is unfair that schools are expected to buy additional supplies. Many
Principals also stated that dongles are not the solution to writing school reports as
teachers should not be expected to compile reports in their own time. It is important to
note that these devices were never solely intended to be used for the completion of reports
at home.
Prepared by Research & Statistics
29
KW2031/0809
Principals who indicated that their school did not use the Profiles 7 software to complete
this year’s Annual Pupil Profile Reports were asked to document their main reason for
making this decision. The most common were that they were unable to complete reports
at home (n = 32) or because they found the software too time consuming (n = 23). Other
reasons Principals documented are summarised below.
•
Experienced technical difficulties with software (14 comments)
•
Word template is easier to use (12 comments)
•
Inadequate training (5 comments)
•
No time to create comment banks (4 comments)
•
Difficulties logging on (4 comments)
CCEA Comment Banks
93.5% of Principals (n = 301) indicated that they made use of the CCEA comment banks.
Respondents were asked to rate how useful they found the comment banks. Although
82.7% of respondents (n = 248) felt the comment banks were useful – the majority of
Principals (91.3%, n = 274) felt they could be improved upon. A large number of
respondents provided suggestions on how the comment banks could be improved, these
are listed below.
•
Comments are required for all areas of the curriculum (73 comments)
•
More comments needed on areas of weakness (47 comments)
•
More specific/individual comments (35 comments)
•
Greater variety to avoid repetitive reports from year to year (34 comments)
•
Make it easier to add/adapt own comments (14 comments)
•
Comments reflecting progression and levelling (13 comments)
•
Less educational language/jargon (13 comments)
•
Punctuation and grammar needs to be corrected within comment banks (7 comments)
•
Make comments more appropriate to each Key Stage (7 comments)
•
More general comments (6 comments)
•
Insertion of <forename> <he/she> (6 comments)
Prepared by Research & Statistics
30
KW2031/0809
Workshop findings
Findings from the Principal workshops reinforce those collected from the questionnaire.
Whilst the majority of participants understand the benefits of having comment banks for
teachers to refer to, they were dissatisfied with the quality of them. Virtually all Principals
agreed that the comment banks were too positive, bland, vague and impersonal.
Principals also stated that the comment banks did not cover the new areas of learning in
the curriculum. They felt this was something that teachers should not be expected to
develop as this is a particular area in which teachers require support. Participants also felt
that there was not sufficient choice within the comment banks and that the language used
was not parent friendly.
Annual Pupil Profile Reports
Figure 5: Format of the Annual Pupil Profile Reports
Foundation Stage
% of Respondents
100
72.2
80
Key Stage 1 & 2
71.2
Principals were asked to rate the
overall format of both the Foundation
Stage and Key Stage 1 and 2 Annual
60
Pupil Profile Reports. Figure 5
40
indicates that over 75% of respondents
20
15.4
6.2
16.5
6.5
rated both the Foundation Stage (n =
5.9 5.8
0
Excellent
Good
Poor
Very poor
241) and the Key Stage 1 and 2 (n =
240) Annual Pupil Profile Reports as
either excellent or good.
A small number of Principals who did not rate the format of the Annual Pupil Profile Report
highly made the following comments.
•
Reports are too long and detailed (25 comments)
•
Schools previous reporting format was better (10 comments)
•
Too many areas to comment on, some of which are overlapping (8 comments)
•
Would appreciate opportunity to include comments in other areas such as
play/punctuality (5 comments)
•
Disliked the ordering of the boxes (5 comments)
•
No grades or comparison to peer groups (5 comments)
Prepared by Research & Statistics
31
KW2031/0809
How informative are the Annual Pupil Profile Reports?
Figure 6: How informative are the Annual Pupil
Profile Reports?
Figure 6 shows that over 70% of
Foundation Stage
100
Principals felt that both the Foundation
Key Stage 1 & 2
Stage (70.1%, n = 213) and the Key
Pupil Profile Reports were either very
informative or informative. Over 28%
of the remaining respondents did not
% of Respondents
Stage 1 and 2 (71.6%, n = 159) Annual
80
deem either version of reports as
62.2
63.5
60
40
22
20
7.9
8.1
20.7
7.9
informative.
7.7
0
Very
Informative
Not
Not at all
informative
informative informative
Respondents were asked to comment on their answer. The majority of comments were
from Principals who did not feel the Annual Pupil Profile Reports were informative.
•
No better or worse than previous reports (15 comments)
•
Reports are impersonal (10 comments)
•
Reports give no idea of progression (6 comments)
•
Reports are too positive (5 comments)
•
Overload of information within reports (4 comments)
•
Grades are more informative than qualitative detail within Annual Pupil Profile Report
(4 comments)
•
Standardised tests are more informative (3 comments)
Prepared by Research & Statistics
32
KW2031/0809
Workshop findings
Findings from the Principal workshops on the format and content of the Annual Pupil
Profile Reports were significantly more negative than those collected from the
questionnaire. Virtually all attendees at the Principal workshops described the reports as
“bland, general, vague, too wordy and too positive”. A large number of Principals also felt
that the Annual Pupil Profile Report is not useful to parents as it is too wordy and does not
contain the information parents particularly want – grades/scores.
The majority of Principals deemed their previous school report to be superior to the Annual
Pupil Profile Report – and felt that implementing a standardised report has taken away the
professionalism of the school.
A small number of Principals stated that they were issuing a letter of apology with the
Annual Pupil Profile Report as they were very disappointed with them.
Prepared by Research & Statistics
33
KW2031/0809
Did the Annual Pupil Profile Reports provide Principals with an overall picture of
pupil’s performance?
Principals were asked to indicate if the Annual Pupil Profile Reports provided them with a
picture of pupils overall performance. Over 64% of Principals agreed that the Foundation
Stage Report (66.3%, n = 201) provided them with a picture of pupils’ overall performance
in Year 1 and the Key Stage 1 and 2 Annual Pupil Profile Report (64.8%, n = 142)
provided them with a picture of pupils’ performance in Year 5.
Principals were asked to comment on their answer. The majority of comments came from
those who were dissatisfied with the new reporting arrangements. Such respondents
commented that the Annual Pupil Profile Reports provided no additional information
(n = 17) and were too positive (n = 14). Other respondents felt that the reports should
include grades and scores from InCAS (n = 14). The remainder of comments are
summarised below.
•
States what pupils ‘can do’- no indication of what they cannot do or progression
(12 comments)
•
No comparison to peers (11 comments)
•
Reports are impersonal (8 comments)
•
Only informative due to the teacher’s personal comments (5 comments)
How manageable did teachers find the Annual Pupil Profile Reports to complete?
Principals were asked to indicate how manageable Year 1 and 5 teachers found the
Annual Pupil Profile Reports to complete. Just over 50% of Principals felt that Year 1
teachers (50.3%, n = 158) and Year 5 teachers (55.9%, n = 127) found the reports
manageable to complete. A large number of respondents who believed teachers found
the reports unmanageable to complete commented that the reports were very time
consuming to compile (n = 79). Others felt the process was particularly unmanageable
because teachers were unable to complete reports at home (n = 39) and because no
funding was provided for schools to buy sub-cover (n = 20).
Prepared by Research & Statistics
34
KW2031/0809
Workshop findings
Throughout all of the workshops Principals stated that this year’s implementation of the
Annual Pupil Profile Reports has had a detrimental effect on teachers’ work life balance.
Principals highlighted that the new Annual Pupil Profile Reports have dramatically
increased the amount of time teachers spend collating and completing reports. Principals
stated that teachers have found completing this new report extremely stressful, particularly
those who used the Profiles 7 software. Many Principals felt they had no choice but to
keep their school open until 9pm at night so that teachers could complete the reports on
the Profiles 7 software. A substantial number of Principals felt that the implementation of
this new reporting arrangement has “abused the good will of teachers”. One Principal
stated that:
¾ “Primary 1 and 5 teachers have undergone training for the revised curriculum
this year and if you asked them what has been the major impact of this year,
they will say report writing and InCAS, not that they learned how to teach the
areas of the revised curriculum .”
Were the Annual Pupil Profile Reports used within any other year groups?
Over 50% of Principals (n = >172) indicated that their school used the Annual Pupil Profile
formats to report to parents with children in Years 2, 3, 4 and 6. Over a third of Principals
(34%, n = 115) also used the Annual Pupil Profile Report for pupils in Year 7. Such
Principals stated that they implemented the Annual Pupil Profile Report into other year
groups as they wanted uniformity in both reporting and teacher workload (n = 24).
Prepared by Research & Statistics
35
KW2031/0809
Do the Annual Pupil Profile Reports inform measures to improve whole school
performance?
Principals were asked if they thought the Annual Pupil Profile Reports informed measures
to improve whole school performance. Only 27.4% of respondents (n = 62) believed that
the Annual Pupil Profile Report achieved this. Principals who felt the report did not inform
measures to improve overall school performance felt it did not accomplish this for the
following reasons.
•
Need grades to inform improvement (15 comments)
•
Does not provide any additional information (9 comments)
•
Not specific enough to do this (8 comments)
•
Reports are too positive (6 comments)
•
Difficult to identify whole school pattern from individual reports (6 comments)
•
InCAS is a better way to do this (4 comments)
Do the Annual Pupil Profile Reports inform teachers’ planning for learning and
teaching?
Less than 50% of Principals (n = 102) felt that the Annual Pupil Profile Reports inform
teachers’ planning for learning and teaching. Those who did not believe the report
informed teachers’ planning commented that the report does not provide teachers with any
additional information (n = 9) and that teachers find grades more informative (n = 4).
Workshop findings
All Principals felt that the Annual Pupil Profile Reports did not inform teacher planning.
Principals stated that annual reports are written in June which is the wrong time of year to
inform teachers’ planning. A small number of Principals also stated that the summative
nature of the annual report is not appropriate to inform teachers as they plan learning and
teaching.
Prepared by Research & Statistics
36
KW2031/0809
Do the Annual Pupil Profile Reports inform parents of how they can support their
child’s learning at home?
55.1% of Principals (n = 119) believed that the Annual Pupil Profile Report informs parents
of how they can support their child’s learning at home. A small number of such
respondents felt that the Focus for Development section within the report is particularly
useful for parents. Comments from Principals who believed the report does not inform
parents of how they can support their child’s learning are summarised below.
•
Report provides no additional information (11 comments)
•
Information is limited, and not specific or meaningful (11 comments)
•
Reports require face to face consultation (6 comments)
•
Reports are too positive - do not highlight weaknesses effectively (6 comments)
Workshop findings
Virtually all attendees at the Principal workshops firmly stated that the Annual Pupil Profile
Reports do not inform parents of how they can support their child’s learning at home. A
large number of Principals stated that the report does not provide enough relevant
information to aid improvement. The majority of Principals within all focus groups stated
that the Annual Pupil Profile Reports are too positive and do not provide parents with a
true reflection of their child’s performance at school. Principals also felt that the reports
were not parent friendly as they overload parents with detail containing educational
language. Principals felt that many parents would not understand the content of the
report.
The majority of Principals within two workshops felt that the Annual Pupil Profile Report
should include commentary on pupils’ behaviour.
Prepared by Research & Statistics
37
KW2031/0809
Do the Annual Pupil Profile Reports inform pupils of their achievements and how
they can improve?
Over 62% of Principals (62.9%, n = 139) believed that the Annual Pupil Profile Reports
informed pupils of their achievements. A small number of such Principals felt that this
information was clearly documented within the interest and strengths section of the report
(n = 3) and was encouraging to pupils (n = 3). Principals who felt that the report did not
inform pupils of their achievements made the following comments.
•
Contains no more information than previous school report (10 comments)
•
Pupils did not understand information (6 comments)
•
Information was too general (4 comments)
•
Year 1 pupils are too young to understand (3 comments)
•
Information was not personal enough to each individual child (2 comments)
•
Pupils have a better understanding of grades (2 comments)
•
Reports are for parents not pupils (2 comments)
Principals were also asked if they thought the Annual Pupil Profile Report informed pupils
of how they can improve. 56.3% of respondents (n = 120) felt that the report advised
pupils of how they can improve. Such respondents believed that the Focus for
Development section clearly informed pupils of how they could improve (n = 10).
Principals who felt the Annual Pupil Profile Report did not inform pupils of how they can
improve stated that the report did not provide pupils with any additional information (n =
17) and would require either the pupil’s teacher or parent to outline how they could
progress (n = 9).
Workshop findings
All Principals at the workshops felt that the Annual Pupil Profile Reports did not inform
pupils of their achievements and how they could improve. A large number of Principals
felt that pupils need either teachers or parents to discuss this with them.
Prepared by Research & Statistics
38
KW2031/0809
Additional Information from the Principal Workshops
Throughout all of the Principal workshops there was widespread criticism of the new
assessment and reporting arrangements. Principals felt that schools were not provided
with adequate support to implement InCAS or the Annual Pupil Profile Report. Principals
were particularly vocal about how unhappy and frustrated they were with the
implementation of the Annual Pupil Profile Report.
Principals expressed concern over the increasing requirement of schools to consult with
parents and have requested support to deliver this. Principals were also displeased that
CCEA was seeking parents’ views on the new assessment and reporting arrangements
during this transition year.
A number of Principals stated that this has been the worst year of their teaching career.
Principals asked for reassurance that the strength of feeling within the workshops would
be evident within the final evaluation report. However, the majority of Principals stated that
they were sceptical about what action would be taken following this consultation.
Prepared by Research & Statistics
39
KW2031/0809
4.2 Results collected from Year 5 Teacher Questionnaires & Workshops
This section presents the results for each question from the Year 5 teacher questionnaire.
Results are based on questionnaires received from 378 respondents. Percentages relate
to the number of teachers who answered each question, which may be fewer than the
overall total 378. A number of teachers reiterated comments throughout their
questionnaire; as a result some comments are repeated throughout this section of the
report.
Qualitative information obtained from ten teacher workshops attended by 201 teachers has
been analysed alongside the feedback collected from the questionnaire. The purpose of
the workshops was to provide in-depth information that will reinforce and further explore
data collected from the questionnaire.
Prepared by Research & Statistics
40
KW2031/0809
InCAS Computer-based Assessments
Manageability of the InCAS computer-based assessments
The majority of teachers (88%, n = 314) stated that they were confident at the prospect of
administering InCAS next year. A large number of such teachers stated that “having
administered [InCAS once they are] confident at doing it again” (n = 49). A small number
of teachers who were not confident at administering InCAS next year said that they found
this assessment time consuming to administer (n = 12) and would require further training
(n = 6).
45% of teachers (n = 154) indicated that they plan to introduce procedures to administer
InCAS next year. The most common procedures teachers plan to implement are
summarised below.
•
Layout of class when completing InCAS (19 comments)
•
Monitor pupils as they complete InCAS and make use of additional supervision if
available (18 comments)
•
Obtain access to more computers / laptops (16 comments)
•
Administer InCAS in smaller groups (11 comments)
•
Administer InCAS earlier in the term (11 comments)
•
Allow more time for administration (9 comments)
•
Plan to provide mice for laptops (8 comments)
•
Co-ordinate the use of computer suites with other teachers (7 comments)
Prepared by Research & Statistics
41
KW2031/0809
Workshop findings
Within all of the teacher workshops teachers’ views on the manageability of the
administration of the InCAS assessments varied greatly. Teachers’ opinions were largely
dependant upon a number of external factors which include: their ICT skills; class size;
availability of computer suites; number of computers within school, staff availability and
contingency plans they had in place.
Virtually all teachers said they encountered challenges administering InCAS. A large
number of challenges were the result of teachers administering InCAS for the first time
(e.g. problems with wireless access and passwords). Other technical challenges specific
to InCAS have been identified within earlier evaluations and will be rectified for the
2008/09 academic year (e.g. volume control).
Feedback from a sizeable number of teachers suggests that they felt they had not planned
or organised the administration of InCAS effectively. As a result, a large number of
teachers stated that they intend to put a number of procedures in place next year.
Procedures teachers plan to put in place next year are listed below.
•
•
•
•
Timetable when InCAS will be administered to pupils
Pupils will complete InCAS in quieter rooms to reduce distractions
Teachers plan to make use of the C2K and CCEA helpdesks
Teachers plan to use the pupil demo
A large number of teachers stated that both sub-cover and ICT support are required to
support teachers during the administration of InCAS – particularly for those with large or
composite classes. Feedback from the teacher workshops also suggests that some
schools do not have the computer capacity to deliver InCAS. One teacher stated that
“schools without a computer suite were really at a severe disadvantage”.
Teachers had a number of concerns regarding the roll out of InCAS in 2008/09. Teachers’
main concerns were regarding the lack of computers and the insufficient support and time
teachers have to administer InCAS. A small number of teachers were also concerned that
low test results from the InCAS assessments could damage teachers’ reputations.
A small number of teachers expressed concerns about the impact the InCAS assessments
has had on the curriculum time, as a large amount of time has been spent administering
assessments as opposed to teaching.
Prepared by Research & Statistics
42
KW2031/0809
InCAS feedback
Figure 7: Did InCAS inform planning for learning
and teaching?
Figure 7 illustrates that over 70% of
Year 5 teachers (n = 251) felt that
% Respondents
100
the feedback from InCAS informed
80
70.9
their planning for learning and
60
teaching. Comments from such
40
29.1
20
respondents are summarised below.
0
Yes
No
•
Highlighted pupils’ strengths and weaknesses (62 comments)
•
Confirmed and supported teachers’ professional judgements (42 comments)
•
The breakdown of InCAS feedback was useful (27 comments)
•
Feedback focused planning and teaching (20 comments)
Teachers who felt the InCAS feedback did not inform their planning for learning and
teaching made the following comments.
•
General Maths feedback requires further breakdown (17 comments)
•
Did not reflect pupils’ abilities (14 comments)
•
Pupils were assessed and grouped prior to receiving InCAS feedback (14 comments)
Prepared by Research & Statistics
43
KW2031/0809
Figure 8: Did the InCAS feedback inform parents
of how they can support their child’s learning?
Nearly 58% of teachers (n = 201) felt
the feedback from InCAS did not
support their child’s learning at
home. However, despite such
negative findings, the main reason
100
% Respondents
inform parents of how they can
80
60
57.8
42.2
40
20
0
teachers felt that the InCAS
Yes
No
feedback was not useful to parents
was because they chose not to share InCAS results with parents (72 comments). Other
teachers felt the feedback produced from InCAS was not useful to parents as they do not
have enough information about this assessment (16 comments).
Teachers who believed the InCAS feedback informed parents of how they can support
their child’s learning, made the following comments.
•
Strengths and weaknesses could be discussed with parents (30 comments)
•
Discussed InCAS results at parent / teacher meetings (23 comments)
¾ “ as a result was able to discuss areas for improvement and possible help”
•
Highlighted specific areas to focus on (20 comments)
•
Reinforced teachers’ professional judgement (10 comments)
Teachers were asked to indicate if the feedback produced from InCAS informed pupils of
their strengths and how they could improve. Only 48% of teachers (n = 169) believed that
the feedback from InCAS informed pupils of their strengths and even fewer teachers
(38.2%, n = 131) believed that the feedback informed pupils of how they could improve.
However, the main reason teachers felt InCAS did not inform pupils of their strengths or
how they could improve was because they did not share the results with pupils (n = 100).
Prepared by Research & Statistics
44
KW2031/0809
Workshop findings
Information collected from all of the teacher workshops regarding the usefulness of the
InCAS feedback was very mixed. Those who felt the feedback was useful particularly
liked that it highlighted pupils’ strengths and weaknesses. Other aspects of InCAS that
teachers liked included charts and graphs, breakdown of results and the fact that the
feedback informed class groupings. A small number of teachers also stated that the
feedback helped to identify pupils with dyslexia.
¾ “InCAS certainly supported some of my concerns with dyslexic children and I
was able to put three of them into IEPs with dyslexic concerns and now they
have been tested by the psychologist”
Many of the teachers who did not find the feedback from the InCAS useful felt that the
content was not in relation to standardised tests and that the feedback from the General
Maths module was not sufficiently broken down. A sizeable number of teachers also
queried the reliability of InCAS – “there were too many discrepancies that impacted on the
staff’s competency and confidence in the system”.
The majority of teachers who participated within the workshops shared InCAS feedback
with parents. A large number of such teachers felt guidance is needed to advise teachers
on how they share feedback with parents. One teacher stated that:
¾ “Teachers want to know how much feedback and what kind of language to
use, some parents challenged teachers on the statistics and where they
came from – therefore maybe a bit of advice on what kind of language needs
to be involved in feedback is required”
Teachers who did not share feedback with parents generally did not do so because it was
not statutory this year or because they had concerns regarding the reliability of the results
produced from InCAS. A small number of teachers also stated that sharing the age
equivalent scores of pupils whose age was significantly below their actual age would be
very distressing for parents.
Prepared by Research & Statistics
45
KW2031/0809
Profiles 7 Report Writing Training
Despite 90.5% of respondents (n = 333) indicating that someone from their school
attended the Profiles 7 training offered by C2K, less than 50% of respondents attended it
themselves (49.5%, n = 172). Respondents indicated who in their school attended this
training.
•
Year 5 teacher (113 comments)
•
Principal (112 comments)
•
ICT Co-ordinator (57 comments)
•
Vice Principal (31 comments)
•
Year 1 teacher (30 comments)
•
Assessment co-ordinator (19 comments)
Figure 9: How useful was the Profiles 7 training?
% Respondent
100
Teachers who attended the Profiles
80
7 report writing training offered by
66.3
60
C2K were asked to rate how useful
40
20
they found it. Figure 9 indicates that
25.7
6.9
the majority of respondents rated
1.1
0
Very
useful
Useful
Not useful
this training as either very useful or
Not at all
useful
useful (92%, n = 161).
Despite the majority of teachers rating this training as useful, 40.9% (n = 72) of
respondents felt it was not adequate preparation for using the software. Comments from
such teachers are summarised below.
•
Further training / support is required (10 comments)
•
The time span between training and using the software was too long (9 comments)
Problems commenced when attempting to use the system for the first time
(7 comments)
¾ “Unprepared for the errors that came up on the screen which was frustrating”
•
More time should have been allocated to get used to the system (5 comments)
Prepared by Research & Statistics
46
KW2031/0809
Workshop findings
The majority of participants within all of the teacher workshops indicated that someone
from their school attended the Profiles 7 training offered by C2K. The majority of teachers
felt that the C2K training was not sufficient preparation for using the software. Attendees
stated that all teachers should have received this training and that it was unfair to expect
teachers to cascade such technical training back in school. One teacher stated that “the
training was good for people who went but every teacher needed to attend it. Perhaps
more than one session is needed [per school]”. A large number of attendees also stated
that they had difficulties remembering the training when they returned to school.
A substantial number of teachers also highlighted that often the most relevant teachers
who were required to use the Profiles 7 software did not attend the training. The main
reason for this was because schools were not provided with sub-cover for the training and
as a result Principals decided to attend the training themselves or send other members of
staff as they saw fit.
Online Tutorials
CCEA provided online tutorials on the Northern Ireland Curriculum website to support
teachers’ use of the Profiles 7 report writing software. Only 8.4% of teachers (n = 31) used
these tutorials. A small number of respondents who did not use the tutorials commented
that they were not aware of them (n = 44) or did not have time to use them (n = 10).
Nearly 70% of teachers who used the online tutorials deemed them to be useful
(69.2%, n = 27). However, only 45% of such teachers (n = 18) felt that the tutorials alone
were adequate preparation for using the software.
Workshop findings
Findings from the teacher workshops reinforce those collected from the questionnaires.
The majority of teachers were unaware of the online tutorials on the Northern Ireland
Curriculum website. However, many teachers stated that even if they had been aware of
them they would not have had time to use them.
Prepared by Research & Statistics
47
KW2031/0809
Using the Profiles 7 report writing software
72.5% of Year 5 teachers (n = 264) used the Profiles 7 report writing software to complete
this year’s Annual Pupil Profile Reports. Teachers were asked to indicate if they thought
this software was user-friendly – 59.4% of respondents (n = 161) believed it was.
However, 64.1% of teachers (n = 162) experienced difficulties using the software. The
most common difficulties teachers commented on are summarised below.
•
SIMs kept shutting down/freezing/crashing “SIMs kept shutting down in the middle of
writing a profile - very frustrating!” (70 comments)
•
Issues saving / deleting / editing (19 comments)
•
Problems logging onto SIMs (14 comments)
•
SIMs not installed at all / or correctly on the computers (6 comments)
•
System very slow (6 comments)
•
Class names on system were incorrect (2 comments)
•
Problems with passwords (2 comments)
•
Spelling and grammatical errors in the comment editor (2 comments)
Figure 10: How manageable is Profiles 7 in comparison
to completing reports by other methods?
% Respondent
100
Figure 10 illustrates that the majority of
80
teachers (68.8%, n = 185) found
68.8
60
Profiles 7 software less manageable to
40
20
17.8
13.4
0
More
manageable
Less
No difference
manageable
other methods they have used to
compile school reports.
Respondents were asked to comment on their response, these are summarised below.
•
Extremely time consuming (91 comments)
¾
“I lost count of the number of extra unpaid hours needed in school – not to mention
my child minding fees”
•
Should be able to access Profiles 7 from home (47 comments)
Prepared by Research & Statistics
48
KW2031/0809
Workshop findings
Attendees at all of the teacher workshops were very critical of the Profiles 7 software. A
large number of teachers expressed high levels of frustration with the Profiles 7 software.
Teachers encountered a large number of problems with the software, particularly with
editing and saving, SIMS shutting down, the system crashing (and losing work as a result),
fonts changing, difficulties adjusting boxes and problems with the school server. A
sizeable number of teachers stated that their school had bought sub cover for a specific
day so that Year 1 and 5 teachers could write reports on the Profiles 7 software; however
they were unable to access SIMS on this specific day and the sub-cover was wasted.
Overall, the majority of teachers found the Profiles 7 software extremely time consuming,
labour intensive, and less manageable than previous methods they have used to complete
reports. Many teachers stated that completing the Annual Pupil Profile Reports had a
detrimental effect on their work/life balance.
¾ “We felt it only worked because we were prepared to make it work, putting in the
exact hours, which isn’t even including time for printing, reprinting and principals
looking at them and telling you extra things that had been overlooked. It is
scandalous as in some cases 1-2 reports were only being done in one day.”
¾ “We spent hours and hours, used exceptional closure days, days should be
provided by DENI, we shouldn’t have to use our exceptional closure days.
Further issue is that we weren’t allowed to take them home.”
¾ “The fact you couldn’t take the reports home was terrible, some teachers were
coming into school at 7am and leaving at 8pm”.
A large number of teachers within all of the teacher workshops thought that they were
statutorily required to use the Profiles 7 software to complete this year’s Annual Pupil
Profile Reports. When teachers were made aware that this was not the case they were
extremely angry and critical of communications from government agencies.
¾ “We were under the impression that we had to use SIMS to complete the Pupil
Profile, I found the experience of using SIMS very cumbersome and having to
stay late (until 9pm many evenings) and a weekend in order to complete these,
an absolute nightmare as we had to have them completed for a stipulated date
in mid May!”
Prepared by Research & Statistics
49
KW2031/0809
CCEA Comment Banks
93.7% of teachers (n = 341) indicated that they made use of the CCEA comment banks.
Respondents were asked to rate how useful they found them. Although 77.5% of
respondents (n = 268) felt the comment banks were useful – the majority of teachers
(91.6%, n = 316) felt they could be improved upon. A large number of respondents
provided suggestions on how these could be improved.
•
Wider variety of comments required to cover all areas of curriculum (146 comments)
¾ “Supply specific banks for all areas of the curriculum. We are all supposed to be
following the revised curriculum therefore all comments should be made available for
staff”
•
Too positive – need more negative comments (65 comments)
•
Comments need to be more personal and less generalised (53 comments)
•
Grammar and vocabulary needs to be improved (16 comments)
•
Consult with schools to add to comment banks / amend current one (13 comments)
•
Different comments for different year groups / pupil abilities (13 comments)
Prepared by Research & Statistics
50
KW2031/0809
Workshop findings
Findings from the teacher workshops reinforce those collected from the questionnaire.
Whilst the majority of participants foresee the benefits of having comment banks to refer
to, the majority of teachers were dissatisfied with the quality of them. Teachers described
the comment banks as “too positive, too general, repetitive and impersonal”.
Teachers made a number of recommendations and suggestions to improve them. A large
number of teachers stated that the spelling and punctuation within the comment banks is
incorrect and needs to be corrected. Respondents also recommended that comments
should include “beginning to” in addition to the “can do” comments. A sizeable number of
teachers felt that exemplary comments should be provided for the Focus for Development
section of the report. Teachers also highlighted that comment banks for two separate
levels can not be opened/used simultaneously and felt this should be programmed into the
Profiles 7 software for next year.
A large number of teachers incorrectly believed that the comments banks were statutory
and they were required to cut and paste comments into the reports. This is possibly one of
the reasons why teachers felt the reports were so impersonal and did not adequately
reflect pupils’ progression throughout the year. A small number of such teachers stated
that they had more or less written identical reports for every pupil within their class.
Annual Pupil Profile Reports
Figure 11: Format of the Annual Pupil Profile Reports
100
Teachers were asked to rate the overall
% Respondent
80
format of the Key Stage 1 and 2 Report.
67.4
60
Figure 11 indicates that over 70% of
40
20
respondents (71.2%, n = 247) rated the
22.2
3.7
Key Stage 1 and 2 Report as either
6.6
excellent or good.
0
Excellent
Good
Poor
Prepared by Research & Statistics
Very poor
51
KW2031/0809
A small number of teachers who did not rate the format of the Annual Pupil Profile Report
highly made the following comments.
•
Report is too “wordy” (45 comments)
•
There is a lot of overlap between boxes which should be combined (18 comments)
¾ “Focus for Development, The Arts, P.E., The World Around Us and Interests
and Strengths can all be reported on in one box at the end”
•
Disliked current format (15 comments)
•
Grades should be recorded within the report (9 comments)
•
Prefer previous school report (7 comments)
•
Focus for Development section would be better placed at the end of the report
(4 comments)
•
Interests and Strengths should be combined within the Optional Content box
(3 comments)
Workshop findings
Within the majority of the teacher focus groups a small number of teachers felt that the
Focus for Development section should be repositioned from the front page of the report to
the end. Such teachers felt that it was important that the report begins positively and
concludes with constructive advice on areas for development.
Did the Key Stage 1 and 2 Annual Pupil Profile Reports provide you with a picture of
the overall performance of individual pupils?
55.2% of teachers (n = 190) believed that the Key Stage 1 and 2 Annual Pupil Profile
Reports provided them with a picture of the overall performance of individual pupils.
44.8% of teachers (n = 154) disagreed and provided the following reasons for their
answer.
•
Reports are too positive – require realistic or some negative comments (42 comments)
•
Reports are too impersonal / vague (39 comments)
¾ “Too general, not really specific to the individual pupils”
•
Grade / results should be included within reports (27 comments)
•
Personalised comments from teacher are better than amending or adding to comment
banks (20 comments)
Prepared by Research & Statistics
52
KW2031/0809
Do the Annual Pupil Profile Reports inform measures to improve whole school
performance?
Teachers were asked if they thought the Annual Pupil Profile Reports inform measures to
improve whole school performance. Only 20.6% of respondents (n = 74) believed that the
Annual Pupil Profile Report achieved this. Teachers who stated that the Annual Pupil
Profile Reports did not inform measures to improve overall school performance felt it did
not accomplish this for the following reasons.
•
Results required (15 comments)
•
No more useful than previous school reports (11 comments)
•
Too bland, impersonal and difficult to interpret (11 comments)
•
Too positive (6 comments)
Do the Annual Pupil Profile Reports inform teachers’ planning for learning and
teaching?
Only 42.1% of teachers (n = 149) felt that the Annual Pupil Profile Reports informed
teachers planning for learning and teaching. Comments from those who did not believe
the report informed teachers’ planning are summarised below.
•
Standard assessment results provide teachers with more useful information
(20 comments)
•
Prefer format of previous school reports (13 comments)
•
Language and terminology used in reports is vague and difficult to interpret
(10 comments)
•
Teachers use other methods to plan for the future (7 comments)
Workshop findings
Participants within all of the teacher workshops unanimously agreed that the Annual Pupil
Profile Report is not informative for teacher planning. The majority of teachers felt that the
report is too vague and too positive. A sizeable number of teachers also stated that the
reports are impersonal, very similar and do not reflect pupils’ ability.
Prepared by Research & Statistics
53
KW2031/0809
Do the Annual Pupil Profile Reports inform parents of how they can support their
child’s learning at home?
63% of teachers (n = 218) believed that the Annual Pupil Profile Report informed parents
of how they can support their child’s learning at home. A small number of such
respondents felt that the Focus for Development section within the report is particularly
useful for parents (n = 56). Comments from teachers who believed the report does not
inform parents of how they can support their child’s learning are summarised below.
•
The Annual Pupil Profile Reports are no better than previous reports (19 comments)
•
Comments within reports are too positive (15 comments)
•
Language within reports is impersonal and difficult for parents to interpret
(13 comments)
•
Grades/scores should be included in the reports - “A lot of parents commented that
they didn’t know where exactly their child was academically. Many would like to know
if they are average, above average or below average. The profile didn’t allow for this
information” (12 comments)
•
The report does not highlight areas where children can improve (10 comments)
Workshop findings
The majority of attendees at all of the teacher workshops firmly stated that the Annual
Pupil Profile Reports do not inform parents of how they can support their child’s learning at
home. The main reasons teachers felt that the reports were not informative to parents
was because (1) they do not include scores / grades and (2) because they are not parent
friendly as they contain a lot of “educational jargon”.
The majority of teachers also stated that the Annual Pupil Profile Reports were too positive
and therefore may have given parents a false picture of their child’s ability/progress. One
teacher stated that they had used comment banks to write reports for several pupils who
were working at a level below their year group. This teacher stated that the report read
very positively as the comments she had inserted from the bank of statements were
positive and generally “can do” statements.
Prepared by Research & Statistics
54
KW2031/0809
Do the Annual Pupil Profile Reports inform pupils of their achievements and how
they can improve?
Nearly 80% of teachers (79.7%, n = 287) believed that the Annual Pupil Profile Reports
informed pupils of their achievements. A small number of such teachers (n = 32)
commented that the Key Stage 1 & 2 Pupil Profile Report is a very positive document that
focuses on the child’s strengths rather than problems.
Teachers who felt that the report did not inform pupils of their achievements made the
following comments.
•
Previous school report was more informative (13 comments)
•
Language vague and difficult to interpret (13 comments)
•
The comments were too positive (8 comments)
•
Grades/scores should be included within the reports (6 comments)
•
Does not inform pupils of their weakness (6 comments)
•
Pupils are already aware of their strengths (3 comments)
Teachers were also asked if they thought the Annual Pupil Profile Report informed pupils
of how they can improve. 69.7% of respondents (n = 242) felt that the report did advise
pupils of how they can improve. Such respondents believed that the Focus for
Development section provided pupils with this information (n = 49).
Workshop findings
All attendees at the teacher workshops felt that the Annual Pupil Profile Reports did not
inform pupils of their achievements and how they could improve. A large number of
teachers felt that pupils need grades / marks to inform them of their achievements and
areas in which they need to improve. Other teachers stated that the reports were not pupil
friendly as pupils do not understand the language used within them. A large number of
teachers also felt that the reports were too positive and as a result misleading to parents.
Prepared by Research & Statistics
55
KW2031/0809
Additional Comments
Respondents were given the opportunity to make additional comments at the end of the
questionnaire. The majority of comments highlighted concerns teachers had about the
Annual Pupil Profile Report. A large number of respondents stated that they found
completing the Annual Pupil Profile Reports extremely time consuming (n=117). 23
teachers stated that they would like to see grades/scores included within future reports,
one teacher commented:
“Our parents are very keen for their children to go to grammar school – these
profiles did not tell them how their children had achieved in the year”.
Other teachers indicated that the Annual Pupil Profile Reports were impersonal and vague
(n=21). One teacher commented:
“Children received bland, impersonal reports that left parents uninformed about their
progress in comparison to others”.
13 respondents preferred the format of their schools previous report (n=13).
Prepared by Research & Statistics
56
KW2031/0809
4.3
Results collected from Year 5 Parent Questionnaires & Focus Groups
This section presents the results for each question from the Year 5 parent questionnaire.
Results are based on questionnaires received from 3586 respondents. Percentages relate
to the number of respondents who answered each question, which may be fewer than the
overall total 3586. A number of parents reiterated comments throughout their
questionnaire; as a result some comments are repeated throughout this section of the
report.
Qualitative information obtained from thirteen parent focus groups attended by 145 parents
has been analysed alongside the feedback collected from the questionnaire. The purpose
of this is to provide further in-depth information to reinforce and expand upon data
collected from the questionnaire.
Prepared by Research & Statistics
57
KW2031/0809
InCAS Feedback
Parent Teacher Meeting
Over 90% of parents (90.6%, n = 3225) attended a parent teacher meeting about their
child this school year. At this meeting, 76.3% of parents (n = 2461) were informed about
the InCAS assessment their child completed in the autumn term. Parents comments
regarding the information they were provided with at this meeting are very mixed. Whilst a
considerable number of parents (n = 130) stated that they were satisfied with the
information they were provided with, many were not (n = 102). A small number of such
parents (n = 13) felt that teachers were unsure about the InCAS assessments and as a
result were unable to fully inform them about InCAS:
¾ “The teachers themselves seemed a little unsure of the aims of the assessment and
could only tell me what I already knew - that his Maths and English were age
graded by means of a computer test”
¾ “Teachers were very unsure of InCAS what the results would be used for and the
significance of testing”
¾ “A short explanation was given but we were unsure if this was part of a new transfer
process or not”
Figure 12: Were you informed of your child’s
results from InCAS?
Parents who attended the parent teacher
% of Respondents
100
meeting were asked if they were
80
60
informed of their child’s results from the
55
InCAS assessment. Figure 12 illustrates
45
40
that 55% of parents received their child’s
20
results from InCAS.
0
Yes
No
A small number of parents who received their child’s InCAS results commented that they
were pleased with them (n = 53) and felt they informed them of what their child should be
obtaining (n = 43 comments).
Prepared by Research & Statistics
58
KW2031/0809
A number of parents commented that they were not satisfied with the information they
were provided with regarding InCAS for the following reasons.
•
Would have liked to discuss the results in more detail (71 comments)
•
Had difficulty understanding the results (21 comments)
•
Uncertain of the tests validity (18 comments)
•
Felt results were meaningless without benchmark or measurement against the rest of
the class (12 comments)
•
Results did not reflect child’s ability (7 comments)
Figure 13: How were the results from InCAS
Shared with you?
Parents were asked to specify how
% of Respondents
100
80
teachers shared the InCAS feedback with
80.3
them. Figure 13 indicates that 80.3% of
60
parents (n = 1549) were verbally informed
40
of their child’s InCAS results. The
20
19.7
remaining 19.7% of parents (n = 379)
0
Verbally
received this information in writing.
In Writing
A sizeable number of parents who were informed of their child’s InCAS results verbally
said that they would have also liked a copy of the results in writing for their “own records”
(n = 83). A small number of parents stated that the verbal information they received at the
parents meeting was not explained and did not contain sufficient detail (n = 31).
Comments were also made from parents who received their child’s InCAS results in
writing. Whilst some parents deemed this information as clear and felt it provided a good
indication of how their child was developing (n = 12) others felt that they were not provided
with enough information (n = 9).
Prepared by Research & Statistics
59
KW2031/0809
Figure 14: Did the results from InCAS enhance
the information shared at the parents meeting?
Figure 14 illustrates that over 57% of
% of Respondents
100
parents (57.5%, n = 1097) believed
80
60
that the results from InCAS
57.5
40
42.5
enhanced the information shared at
the parents meeting. Such parents
20
commented that the results;
0
Yes
No
•
Identified their child’s strengths and areas they can improve upon (131 comments);
•
Were very detailed (26 comments);
•
Confirmed what teachers predicted (20 comments);
•
Enabled teacher/parent to compare children’s results (6 comments);
•
Were a positive addition to the parent meeting (5 comments); and
•
Complemented information already provided by the teacher (5 comments).
Two parents commented that the InCAS results:
¾ “gives a more accurate understanding of a child’s performance. It
pinpoints areas of ability and weaknesses in a straightforward manner
that all parents can understand”
¾ “gives a better reflection of the child and it is non-biased”
Comments from parents who did not believe the InCAS results enhanced the parents
meeting are summarised below.
•
Would like more information on purpose of InCAS assessments (41 comments)
•
Results were not explained in great detail (39 comments)
•
Computer test does not show a child’s ability/progress (22 comments)
•
Would like to know how InCAS is measured/benchmarked (17 comments)
•
Would prefer the assessment later in the year so that progress could be measured
(13 comments)
•
Unsure of the assessments validity (12 comments)
•
Information was difficult to interpret (11 comments)
Prepared by Research & Statistics
60
KW2031/0809
A small number of parents were concerned that children with less ICT skills were at a
disadvantage (n = 6). One parent commented that they “have many reservations
surrounding InCAS. I feel computerised testing requires a different skill set to written
assessment. I am particularly concerned about assessment of Mathematics and
Numeracy via InCAS. My child scored very highly but I remain concerned about the
process”.
Focus Group findings
Overall findings from the parent focus groups were very negative. The majority of parents
who participated in the focus groups did not feel fully informed about the purpose and use
of InCAS. With the current uncertainty regarding transfer procedures parents were very
concerned that InCAS could potentially be used as a method for transferring pupils to Post
Primary Schools. Such thinking has heightened many parents concerns about InCAS as
they believe there may be a “hidden agenda” with this computer-based assessment.
A large number of parents indicated that they distrust computer-based assessments.
Parents particularly queried where the data from this assessment is stored and what this
information could potentially be used for. A number of parents queried why so much
money was being spent on new assessment arrangements if it is not going to be used for
other purposes.
The majority of participants within the parent focus groups felt that schools were not fully
informed or trained to use InCAS. Parents were also discontent with the amount of time
teachers have spent on training and administering InCAS, which they believed was to the
detriment of pupil learning.
A sizeable number of parents had been alerted to problems that had occurred whilst pupils
were completing InCAS. Such parents were concerned about these and questioned
whether schools have the technology to administer this assessment. Parents were also
concerned about the inconsistencies between schools, stating that schools with a greater
number of teachers with ICT expertise and computers were at an advantage.
A sizeable number of parents questioned the validity of the InCAS feedback and felt that
standardised tests such as NFER had a greater level of reliability and validity. Many of
these parents stated that teachers are not confident with the results produced from InCAS.
Prepared by Research & Statistics
61
KW2031/0809
Focus Group findings continued
The majority of parents were concerned about the level of ICT skills children required to
complete InCAS. Such parents felt that children with less experience of computers were
at an unfair disadvantage. A large number of parents said they would like notice of when
children are completing the assessment to ensure that they have adequate rest prior to the
assessment and understand the importance of the “test”.
A sizeable number of parents who did not receive their child’s scores to InCAS felt that
schools were withholding information from them. A small number of parents who
requested their child’s InCAS results from their school stated that the school was very
reluctant to share this information with them. Such parents stated that this left them
feeling very frustrated and anxious.
Annual Pupil Profile Report
72% of parents (n = 2474) received information from their child’s school about the
introduction of the Annual Pupil Profile Report. A large number of parents indicated that
they received this information in either a letter (n = 176) or a school meeting (n = 103).
Figure 15: Format of the Pupil Profile Report
% of Respondents
100
Parents were asked to rate the format
80
of the Annual Pupil Profile Report.
54.9
60
40
Figure 15 illustrates that the majority of
parents rated this report highly (86.8%,
31.9
9.3
20
n = 3007). Comments from parents
3.9
0
Excellent
Good
Poor
Very poor
who rated the report highly are
summarised below.
•
A very detailed, comprehensive and informative report (156 comments)
•
The report was clear and concise and well presented (133 comments)
•
Very pleased with child’s report (74 comments)
•
Report provided a good indication of child’s strengths and weaknesses (49 comments)
•
Easy to understand and follow (18 comments)
Prepared by Research & Statistics
62
KW2031/0809
A large number of comments were also received from parents who did not rate the Annual
Pupil Profile Report highly, such parents felt that the report;
•
Should include grades (181 comments);
•
Was impersonal and the language felt prescribed (137 comments)
•
Should include a measure/benchmark of how their child is performing in relation to their
peers (74 comments).
•
Was not as good as the previous report (55 comments)
•
Difficult to interpret/understand (28 comments)
•
Was too wordy (27 comments)
•
Similar to the previous school report (14 comments)
•
Did not advise parents of how they can help their child progress (14 comments)
•
Was too positive (10 comments)
•
Not very detailed (7 comments)
One parent stated that the report was:
¾ “Uninformative – it seemed to be regurgitation of the curriculum stating what
our child had done and was able to do but not how well, the amount of help
required and how her knowledge, understanding and independence rated in
comparison to her age and other classmates”
Did you find the information within the Annual Pupil Profile Report easy to read and
understand?
The majority of parents indicated that they found their child’s Annual Pupil Profile Report
easy to read (94.8%, n = 3280) and understand (91%, n = 3080). A large number of such
parents (n = 203) said that the report was clearly presented and easy to understand - “the
language used was clear and concise. I would imagine that most parents could
understand it regardless of whether they were from an educational background or not. I
certainly was very happy with it”.
Prepared by Research & Statistics
63
KW2031/0809
A number of parents highlighted aspects of the Annual Pupil Profile Report they disliked.
•
Report should include marks/grades to identify the level children are working at
(117 comments)
•
The language used within the report was too formal (91 comments)
•
Report felt impersonal and prescribed (72 comments)
•
Report should include benchmark/measurement of how children perform in relation to
their peers (48 comments)
•
Preferred previous school report (32 comments)
•
Report did not contain enough detailed information on the child (24 comments)
•
Report should contain greater detail on how the child should improve (21 comments)
Comments from two parents are provided below.
¾ “The terminology used is educationalist. In parts did not appear to be
specific/individual to our child. Do not have a clear idea as to child’s
attainment in relation to curriculum or against peers”.
¾ “Yes, it is written in plain English but it is not clear how well or not my child is
doing. It is unclear precisely what knowledge and behaviour P5 children are
expected to possess/display and it is therefore impossible to assess my
child’s performance from the profile report. For example my child can
recognise simple 2D and 3D shapes. Is this what he was taught or were
there other shapes taught that he failed to name? Likewise my child can
describe some common landscape features and geographical conditions but
unclear if “some” was all he was taught and expected to know. My child has
improved or made progress in some areas however it is not possible to
measure how much he has improved, how he has improved and in
specifically what areas because explanation is too general”.
How informative are the Annual Pupil Profile Reports?
The majority of parents agreed that the Annual Pupil Profile Report informed them of their
child’s achievements (88.3%, n = 3067) and how their child could improve
(81.7%, n = 2790). One parent described the Annual Pupil Profile Report as “a very wellrounded report”.
Prepared by Research & Statistics
64
KW2031/0809
Many of the parents who indicated that the Annual Pupil Profile Report did not inform them
of their child’s achievements or areas for improvement provided reasons for their answer.
•
Not enough information on areas for progression (131 comments)
•
Parents would like report to include pupil grades/scores (113 comments)
•
Language in reports was too formal and vague (83 comments)
•
Reports should have more detail (61 comments)
•
There was no benchmark or measure of child’s progress (49 comments)
•
Reports were impersonal – did not provide an individual profile of the child
(26 comments)
Figure 16: Focus for Development
Parents were asked if the Focus for
Development section of the Annual Pupil
100
69.3
Profile Report informed them of how they
% of Respondents
80
can support their child’s learning at
60
30.7
home. Figure 16 illustrates that nearly
40
70% of parents (n = 2345) believed that
20
this particular section of the report
0
Yes
advised them of how they could aid their
No
child’s learning at home.
A large number of such parents (n = 242) highlighted that they found this section of the
report particularly useful as it identified specific areas they could focus on and encourage
their child’s learning.
Parents who did not find the Focus for Development section of the report useful stated
that;
•
It did not advise them of how they could support their child’s learning (155 comments);
•
Information within this section was limited (87 comments);
•
The language used was vague and difficult to interpret (56 comments);
•
This report did not say anything new about the child (24 comments); and
•
Their report did not contain this section (21 comments).
Prepared by Research & Statistics
65
KW2031/0809
What did you like most about your child’s Annual Pupil Profile Report?
Parents were asked to indicate what they liked most about their child’s Annual Pupil Profile
Report. The most prevalent responses from parents are summarised below.
•
Comprehensive and detailed overview of child’s progress (660 comments)
•
The report was clear, concise and easy to read (493 comments)
•
Report identifies strengths and weaknesses (242 comments)
•
Breakdown of each subject / format of report (209 comments)
•
‘Positive’ language teachers used in the reports (104 comments)
•
Allows the teacher to personalise/individualise the report more effectively than previous
reports (88 comments)
•
Focus for Development section was useful (55 comments)
Comments from two parents are provided below.
¾ “I found the report to be more informative, easy to read and well
presented. I feel this format of reporting is by far the best I have received
to date on my son’s education”
¾ “It provided constructive feedback about my daughter’s progress
indicating all round strengths not just from an academic perspective. It
clearly indicated a focus for improvement thus indicating where I can
support her development from home”
Prepared by Research & Statistics
66
KW2031/0809
What did you dislike about your child’s Annual Pupil Profile Report?
Parents were asked to indicate what they disliked about their child’s Annual Pupil Profile
Report. The most frequent responses are summarised below.
•
Did not include grades/scores to show child’s progress (664 comments)
•
Reports seemed impersonal in that they could ‘have been written about any child”
(212 comments)
•
The reports did not provide any evidence as to where the child is in relation to
class/peer average (209 comments)
•
The language used was formal and difficult for both parents and children to interpret
(202 comments)
•
There should be more information provided on areas in which the child could develop
and improve (122 comments)
•
Report took too long to read (57 comments)
Comments from two parents are summarised below:
¾ “The report does not give any grades or indication of academic achievement
so we can identify [our child’s] strengths and weaknesses. Nor can we see if
she is above or below average in her peer group”.
¾ “It was generalised, full of standard sentences drawn from a pool of
unbelievable blandness, designed not to offend or at another level inform”.
Focus Group findings
At the time of the focus groups virtually all parents had received their child’s Annual Pupil
Profile Report. Overall findings collected from the parent focus groups are very negative.
The majority of parents did not feel that they had been provided with sufficient information
on the new reporting arrangements. A large number of parents revealed that their main
reason for taking part in the focus group was because they thought the Annual Pupil
Profile Report was a replacement for the transfer test. With the current uncertainty
regarding transfer procedures parents were reluctant to believe that this report would not
be used for transfer purposes. Parents were particularly dubious about the Pupil Profile
Summary that their child will receive in Year 7.
Prepared by Research & Statistics
67
KW2031/0809
Focus Group Findings Continued
The majority of attendees at all of the parent focus groups were particularly dissatisfied
with three aspects of the Annual Pupil Profile Report. One of the largest criticisms of this
report was that it did not contain grades. Virtually all parents deemed grades to be an
essential component of a school report. Many parents stated that grades clearly inform
them of their child’s academic performance, behaviour and enable them to monitor their
child’s progress year on year. The majority of parents also stated that they would like a
measure of how their child is performing in comparison to their peers. Parents questioned
why the Department of Education made the decision to exclude grades from school
reports.
The second complaint concerning the Annual Pupil Profile Report was regarding how
impersonal and general the reports were. Parents felt the reports were overloaded with
bland detail that did not inform them of their child’s performance. A large number of
parents stated that it was evident that teachers had used comment banks to complete
reports as the report read like a “cut and paste exercise”. A small number of parents said
that they were aware that practically the same report had been completed for every pupil
within their child’s class.
The third aspect of the Annual Pupil Profile Report that parents were dissatisfied with was
regarding how “hyper-positive” the report was. Many parents stated that they found this
particular aspect of the report unhelpful and misleading. A large number of parents
revealed that they were extremely happy the first time they read the comments within their
child’s report; however they later felt extremely frustrated as the teacher had omitted
crucial information on areas in which their child needs to develop. Many parents stated
that the report is full of positive statements such as “can do”, “beginning to” and that it is
not clear from the report how their child is developing according to their age or to their
individual potential.
Many parents indicated that as the Annual Pupil Profile Report did not include grades, was
vague and contained too much positive commentary they had no choice but to try and
“decode” the report for important pieces of information, especially areas in which their child
needs to improve. As a result, many parents indicated that they are very unhappy with the
Annual Pupil Profile Report they received this year and prefer the schools previous school
report.
Prepared by Research & Statistics
68
KW2031/0809
Additional Comments
Respondents were given the opportunity to make additional comments at the end of the
questionnaire. Many of these responses have been stated before and incorporated into
the main body of results; however the responses are also summarised below.
Parents raised a number of concerns regarding the Annual Pupil Profile Report. A large
number of parents were dissatisfied that the report did not contain marks/scores (n=286) to
indicate a child’s ability. 103 respondents felt that there was no evidence as to where their
child was in relation to the rest of the class. A significant number of parents also
commented that the language used within the report was too formal and difficult to
interpret (n=100). Parents also described the comments within the report as impersonal
and formulaic (n=72). Parents were aware that teachers had devoted a lot of time to the
reports and were worried that it was at the expense of valuable teaching time (n=75). A
significant number of parents (n=69) felt that the report did not give them enough
information on how their children can improve. 87 respondents said that they preferred the
previous format of their child’s reports.
Parents were also concerned with certain aspects of InCAS. Some parents were
concerned that pupils with lower ICT skills are placed at a disadvantage (n=20). 15
parents felt that the InCAS assessments should be administered later in the school year
and 17 respondents stated that they would like more information on InCAS. One parent
commented that:
“If you tested my child at this stage [June] of P5 rather than the beginning of P5 you
would get a totally different result. I think any computer that dictates your child's
abilities is wrong. Children are people - different in the extreme, brilliant at some
things, not so good at others but to try and put them into tick box categories is
missing the point”.
A large number of parents requested more information on the transfer procedure (111
comments) and on the InCAS computer-based assessment (60 comments).
Prepared by Research & Statistics
69
KW2031/0809
4.4
Results collected from Year 5 Pupil Questionnaires
This section presents the results for each question from the Year 5 pupil questionnaire.
Results are based on questionnaires from 5,120 respondents. Percentages relate to the
number of pupils who answered each question, which may be fewer than the overall total
of 5,120.
Did you read your Annual Pupil Profile Report?
Just over 50% of pupils read their Annual Pupil Profile Report (n = 2739). In addition to
this, nearly 70% of pupils (69%, n = 3538) indicated that their parent read their report to
them and 1.7% of pupils (n = 86) stated that their teacher read it to them.
Figure 17: Did pupils find the Annual Pupil Profile Report
easy to understand?
100
% of Respondents
80
Pupils were asked if they found
their Annual Pupil Profile Report
60
easy to understand. Figure 17
50.2
40.7
illustrates that the majority of pupils
40
(90.9%, n = 4538) found their
20
6.8
report easy to understand.
2.3
0
Strongly
agree
Agree
Disagree
Strongly
disagree
¾ “I liked this years report better because I understood it better”
¾ “My Pupil Profile Report was much easier to understand than the old report. It
explained things much more deeply, and just got straight to the point”
A large number of pupils said that they liked the layout of the Annual Pupil Profile Report,
which they felt made it easier for them to understand (n = 91).
Prepared by Research & Statistics
70
KW2031/0809
¾ “I prefer this report as it is split into paragraphs and is easier to understand”
¾ “I think that the way the report is set out is easy to understand and shows the areas
I need to improve some of my work”
Figure 18: Did the Annual Pupil Profile report
inform pupils of their strengths?
100
Figure 18 indicates that 96% of
pupils (n = 4784) agreed that their
Annual Pupil Profile Report
informed them of their strengths.
% of Respondents
80
65.1
60
40
30.9
20
2.9
1
0
Strongly
agree
Agree
Disagree
Strongly
disagree
A large number of pupils said that they particularly liked that the Annual Pupil Profile
Report had a more positive approach than their previous school report (n = 94). One pupil
said this made them “feel very pleased about [themselves] and encouraged [them] to keep
working hard”.
Prepared by Research & Statistics
71
KW2031/0809
Figure 19: Did the Annual Pupil Profile Report inform
Pupils of how they can improve?
Pupils were asked if they felt their
100
Annual Pupil Profile Report informed
them of how they can improve. Just
% of Respondents
80
over 85.4% of pupils (85.5%, n = 4239)
agreed that the Annual Pupil Profile
60
44.2
41.2
Report informed them of how they can
40
improve. A large number of pupils
20
(n = 200) also commented that their
10.2
4.4
report made them aware of areas in
0
Strongly
agree
Agree
Disagree
Strongly
Disagree
which they need to improve. One pupil
stated:
¾ “I was really happy about it and happy about what my teacher said about me. There
used to be lots of hard words, but now there is not as many and it is like it’s made for
kids as well as parents. It made me understand properly the things I need to improve.
It was easy to understand.”
Figure 20: Did the Annual Pupil Profile Reports
provide pupils with useful information?
100
% of Respondents
80
65.1
Figure 20 highlights that 96% of
60
pupils strongly agreed or agreed
40
that their Annual Pupil Profile
30.9
Report provided them with useful
20
2.9
information.
1
0
Strongly
agree
Agree
Disagree
Prepared by Research & Statistics
Strongly
Disagree
72
KW2031/0809
A large number of pupils commented that their Annual Pupil Profile Report provided them
with a lot of useful information (n = 90). One pupil said that “the reports for Primary 5 are
great. Usually the reports say about Reading, Maths and English but this time there was
lots more”. A sizeable number of pupils also stated that they preferred the Annual Pupil
Profile Report to their previous school report (n = 76).
¾ “The Pupil Profile Report is better than our old report because there is more detail
added to it”.
Despite such positive findings, a large number of pupils highlighted a number of aspects of
the Annual Pupil Profile Report that they disliked, these are summarised below.
•
Would like test results / scores included in the report – “grades help me to understand
where I need to improve stuff like handwriting” (194 comments)
•
Wording within the report was difficult to understand – “there were too many big words
explaining the subjects” (181 comments)
•
Would like grades within the report (A,B,C / Average) (179 comments)
•
Preferred last year’s report (142 comments)
•
Report was too long – “there was too much writing and too many pages”
(83 comments)
Prepared by Research & Statistics
73
KW2031/0809
4.5
Results collected from Year 1 Teacher Questionnaires
This section presents the results for each question from the Year 1 teacher questionnaire.
Results are based on questionnaires received from 261 respondents. Percentages relate
to the number of teachers who answered each question, which may be fewer than the
overall total 261. A number of teachers reiterated comments throughout their
questionnaire; as a result some comments are repeated throughout this section of the
report.
Profiles 7 Report Writing Training
Despite 92.4% of Year 1 teachers (n = 232) indicating that someone from their school
attended the Profiles 7 training offered by C2K, less than a third of respondents attended it
themselves (30.8%, n = 74). The most common members of staff respondents who
attended this training are listed below.
•
Principal (85 comments)
•
P5 teacher (52 comments)
•
ICT Co-ordinator (46 comments)
•
P1 teacher (39 comments)
Figure 21: How useful was the Profiles 7 training?
Year 1 teachers who attended
% of Respondents
80
the Profiles 7 report writing
65.4
training offered by C2K were
60
asked to rate how useful they
40
found it. Figure 21 illustrates
29.6
that the majority of respondents
20
3.7
1.2
Not useful
Not at all
useful
rated this training as either very
useful or useful (95%, n = 77).
0
Very useful
Useful
Prepared by Research & Statistics
74
KW2031/0809
Despite the majority of teachers rating this training as useful, 54.7% (n = 47) of
respondents felt it was not adequate preparation for using the software. Comments from
such teachers are summarised below.
•
Teachers require more practice to familiarise themselves with software (9 comments)
•
All teachers require training in the software (3 comments)
•
Too much time is required for reading notes and information provided (3 comments)
•
Training was not satisfactory – more is needed (3 comments)
Prepared by Research & Statistics
75
KW2031/0809
Online Tutorials
CCEA provided online tutorials on the Northern Ireland Curriculum website to support
teachers’ use of the Profiles 7 report writing software. Only 4.4% of teachers (n = 11) used
these tutorials. A small number of respondents who did not use the tutorials commented
that they were not aware of them (n =41) others did not have time to use them (n = 7).
Seven of the 11 teachers who used the online tutorials deemed them to be useful.
However, only two teachers felt that the tutorials alone were adequate preparation for
using the software.
Using the Profiles 7 report writing software
70.9% of Year 1 teachers (n = 175) used the Profiles 7 report writing software to complete
this year’s Annual Pupil Profile Reports. Teachers were asked to indicate if they thought
this software was user-friendly – 56.6% of respondents (n = 103) believed it was.
However, 67.2% of teachers (n = 123) experienced difficulties using the software. The
most common difficulties teachers commented on are summarised below.
•
Software crashed / shut down which resulted in work being lost (55 comments)
•
Software was extremely time consuming (40 comments)
•
Difficulty editing profiles and saving changes (34 comments)
•
Could not preview profiles before printing them (16 comments)
•
Disproportionate amount of time taken to load software / log on (16 comments)
•
Software recurrently froze (15 comments)
•
Comments in banks were unsuitable (14 comments)
•
Unable to access software after 9pm (7 comments)
•
Unable to work on reports at home (6 comments)
Prepared by Research & Statistics
76
KW2031/0809
Figure 22: How manageable is Profiles 7 in comparison
to completing reports by other methods?
Figure 22 illustrates that the
% of Respondents
80
73.2
majority of teachers
60
(73.2%, n = 161) found the
40
Profiles 7 software less
manageable than other
20
14.8
12
0
More
manageable
Less
manageable
No difference
methods they have used to
compile school reports.
Many respondents highlighted why they found the Profiles 7 software less manageable,
these are summarised below.
•
Using the software is confined to school (84 comments)
•
Software is very time consuming to use (80 comments)
•
Comment banks did not provide for differentiation and were unhelpful (16 comments)
•
Previous annual reports were less time consuming and easier to complete
(11 comments)
•
Prefer to complete report on Microsoft Word / other software (8 comments)
•
School required sub-cover to complete reports on this software (5 comments)
Prepared by Research & Statistics
77
KW2031/0809
CCEA Comment Banks
92.8% of teachers (n = 233) made use of the CCEA comment banks. Respondents were
asked to rate how useful they found the comment banks. Although 80.9% of respondents
(n = 190) felt the comment banks were useful – the majority of teachers (92.5%, n = 223)
felt they could be improved upon. A large number of respondents provided suggestions on
how the comment banks could be improved, these are summarised below.
•
Comment banks are required for all learning areas (77 comments)
•
A wider range of comments are required to differentiate children’s abilities
(52 comments)
•
Comment banks need to be more specific / clear (37 comments)
•
A greater range of comments are required for Foundation Stage pupils (21 comments)
•
Comment banks need to be more personal (13 comments)
•
Different comments are required for each Key Stage / Year Group (9 comments)
•
Comments were too positive; do not provide a succinct view of the child (6 comments)
•
CCEA needs to work alongside schools to develop the comment banks (4 comments)
Annual Pupil Profile Report
Figure 23: Format of the Annual Pupil Profile Reports
Teachers were asked to rate
100
the overall format of the
66.4
% of Respondent
80
Foundation Stage Annual
Pupil Profile Report. Figure
60
23 indicates that over 70% of
40
20
23.5
respondents (71.9%, n = 171)
5.5
4.6
rated this report as either
excellent or good.
0
Excellent
Good
Prepared by Research & Statistics
Poor
Very poor
78
KW2031/0809
A small number of teachers who did not rate the format of the Annual Pupil Profile Report
highly made the following comments.
•
Contains too much detail for Year 1 pupils (42 comments)
•
Certain sections overlapped and could be amalgamated (14 comments)
•
Need to be able to adjust box sizes (7 comments)
•
Very impersonal (6 comments)
•
Too many boxes (4 comments)
•
Parents want levels / scores (4 comments)
•
Language is difficult for parents to understand - too much jargon (4 comments)
Did the Foundation Stage Annual Pupil Profile Report provide you with a picture of
the overall performance of individual pupils?
65.4% of teachers (n = 159) believed that the Foundation Stage Annual Pupil Profile
Report provided them with a picture of individual pupils performance. Just over a third of
teachers (34.6%, n = 84) disagreed and provided the following reasons for their answer.
•
Report was very general (19 comments)
•
Report was too positive and could be misleading to parents (18 comments)
•
Each report is too similar / impersonal (13 comments)
•
Comments had to be adapted to make them specific to individual children
(11 comments)
•
Teachers are already aware of how children are performing (10 comments)
•
Pupil Profile is no better than original school report (9 comments)
•
Does not provide a true overall picture of the child (6 comments)
•
Additional comments based on teachers professional judgement were more useful
(6 comments)
Prepared by Research & Statistics
79
KW2031/0809
Do the Annual Pupil Profile Reports inform measures to improve whole school
performance?
Teachers were asked if they thought the Annual Pupil Profile Reports inform measures to
improve whole school performance. Only 22.3% of respondents (n = 53) believed that the
Annual Pupil Profile Report achieved this. Teachers who felt the report did not inform
measures to improve overall school performance felt it did not accomplish this for the
following reasons.
•
Report focused on what the child could do and did not mention areas for improvement
(7 comments)
•
Comments are too general (4 comments)
•
Did not reflect whole school performance (3 comments)
•
To improve school performance standardised testing is required (2 comments)
•
Foundation stage comment bank was too broad (2 comments)
Do the Annual Pupil Profile Reports inform teachers’ planning for learning and
teaching?
Less that 50% of teachers (48.8%, n = 117) felt that the Annual Pupil Profile Reports
inform teachers planning. Comments from those who did not believe the report informed
teachers’ planning are summarised below.
•
Observations are taken throughout the year and are more helpful (9 comments)
•
There is no difference from the previous school reports (7 comments)
•
Depends on how the teacher uses the comment banks (5 comments)
•
Too time consuming for teachers to read (4 comments)
A small number of teachers who felt that the Annual Pupil Profile Report does inform
teacher planning commented that the ‘Focus for Development’ section of the report is
particularly useful for the next year’s teacher (n = 36).
Prepared by Research & Statistics
80
KW2031/0809
Do the Annual Pupil Profile Reports inform parents of how they can support their
child’s learning at home?
71.2% of teachers (n = 168) believed that the Annual Pupil Profile Report informs parents
of how they can support their child’s learning at home. A sizeable number of such
respondents felt that the ‘Focus for Development’ section within the report is particularly
useful for parents (n = 62). Comments from teachers who believed the report does not
inform parents of how they can support their child’s learning are summarised below.
•
Previous school reports provided the same information (10 comments)
•
One to one interviews inform parents of how to support their child (9 comments)
•
Depends on level of parental interest (7 comments)
•
Reports are not specific enough (7 comments)
•
Focus for Development section is too small (7 comments)
•
Insufficient information on areas the child needs to improve - too positive (4 comments)
•
Comments to give parents more direction on how to help would be useful
(4 comments)
Do the Annual Pupil Profile Reports inform pupils of their achievements and how
they can improve?
83.9% of teachers (n = 203) believed that the Annual Pupil Profile Reports informed pupils
of their achievements. A small number of such teachers commented that the ‘Interests
and Strengths’ section of the report clearly informed pupils of their achievements (n = 9).
A small number of teachers also stated that it is important to highlight pupils’ strengths
(n = 7).
Prepared by Research & Statistics
81
KW2031/0809
Teachers who felt that the report did not inform pupils of their achievements made the
following comments.
•
Pupils are too young to understand this report – it is too complicated (16 comments)
•
Parents inform pupils of their strengths (15 comments)
•
All reports inform pupils of their strengths (11 comments)
•
All comments are worded positively - gives a false impression (9 comments)
•
Children are already aware of their strengths - informed throughout the year
(7 comments)
•
Report over emphasises strengths leaving little scope for development (6 comments)
Teachers were also asked if they thought the Annual Pupil Profile Report informed pupils
of how they can improve. 71.2% of respondents (n = 163) felt that the report did advise
pupils of how they can improve. A small number of such respondents stated that the
Focus for Development section advised pupils of how they could improve (n = 27).
Additional Comments
Respondents were given the opportunity to make additional comments at the end of the
questionnaire. The majority of these responses have already been captured in the main
body of the report, however a further summary is provided below.
Teachers raised a number of concerns. The largest concern expressed by respondents
(n = 77) was regarding the large amount of time required to complete the Annual Pupil
Profile Reports. A number of respondents felt preparing reports was an additional source
of stress on teachers (n = 18) and that has reduced planning and teaching time (n = 17).
A number of Year 1 teachers stated that they would like the facility to complete the reports
from home (n = 40) and requested sub cover (n = 27) to allow for this option.
A small number of teachers stated that the reports were impersonal (n = 22) did not give a
clear indication of the difficulties children experienced (n = 15) or effectively detail their
progress (n = 12).
Prepared by Research & Statistics
82
KW2031/0809
4.6 Results collected from Year 1 Parent Questionnaires
This section presents the results for each question from the Year 1 parent questionnaire.
Results are based on questionnaires received from 3631 respondents. Percentages relate
to the number of respondents who answered each question, which may be fewer than the
overall total 3631. A number of parents reiterated comments throughout their
questionnaire; as a result some comments are repeated throughout this section of the
report.
Annual Pupil Profile Report
71.5% of parents of Year 1 children (n = 2465) received information from their child’s
school about the introduction of the Annual Pupil Profile Report. A large number of
parents indicated that they received this information from either a school meeting (n = 162)
or a letter (n = 81).
Figure 24: Format of the Pupil Profile Report
100
Parents were asked to rate the format
% of Respondents
80
60
of the Foundation Stage Annual Pupil
56.3
Profile Report. Figure 24 illustrates
40.3
that the majority of parents rated this
40
report highly (96.6%, n = 3467).
20
2.4
Parents who rated the report highly felt
1
the report was;
0
Excellent
Good
Poor
Very poor
•
Clearly laid out (405 comments);
•
Detailed and informative (323 comments);
•
Covered a wide range of topics that were broken down well (220 comments);
•
Easy to read (115 comments);
•
Highlighted strengths and weaknesses effectively (66 comments);
•
Personal to the child (35 comments); and
•
Provided a good breakdown of the year’s activities (32 comments).
Prepared by Research & Statistics
83
KW2031/0809
A large number of comments were also received from parents who did not rate the Annual
Pupil Profile Report highly, such parents felt that the report;
•
Contained impersonal/computer generated comments (68 comments);
•
Should provide a comparison to their child’s peer group (49 comments);
•
Contained too many words that were not informative (34 comments);
•
Should include grades (33 comments); and
•
Should highlight areas for improvement, not just strengths (28 comments).
Did you find the information within the Annual Pupil Profile Report easy to read and
understand?
The majority of parents indicated that they found their child’s Annual Pupil Profile Report
easy to read (98%, n = 3506) and understand (96.5%, n = 3344). A large number of
parents made positive comments regarding their child’s Annual Pupil Profile Report, these
are summarised below.
•
Report was well set out (170 comments)
•
Report included very detailed and useful information (155 comments)
•
Language was parent friendly (124 comments)
•
Information was accessible as subjects were commented on separately (67 comments)
•
All aspects of pupil development were covered (40 comments)
•
Gave a good understanding of the child’s progress (39 comments)
A number of parents highlighted aspects of the Annual Pupil Profile Report they disliked.
•
Too much educational terminology – difficult to understanding (67 comments)
•
Reports were impersonal and too general (57 comments)
•
Difficult to take in so much information (46 comments)
•
Provided no indication of the child’s stage of development (41 comments)
•
Should give a comparison to child’s peers (17 comments)
Prepared by Research & Statistics
84
KW2031/0809
How informative are the Annual Pupil Profile Reports?
The majority of parents agreed that the Annual Pupil Profile Report informed them of their
child’s achievements (96.5%, n = 3466) and how their child could improve
(88%, n = 3097). One parents described the report as “a very well rounded report”.
Many of the parents who indicated that the Annual Pupil Profile Report did not inform them
of their child’s achievements or areas for improvement provided reasons for their answer.
•
Report was too generic and vague (86 comments)
•
Report did not advise how children could improve their weaknesses (72 comments)
•
Report highlighted what a child is achieving not what they should be achieving
(55 comments)
•
Comments were too positive - no focus on weakness (46 comments)
•
Grades would highlight areas in need of improvement (24 comments)
•
Focus for Development box does not provide enough information (22 comments)
Figure 25: Focus for Development
Figure 25 illustrates that over
81.1
% of Respondents
100
81% of parents
(81.1%, n = 2839) believed that
80
the Focus for Development
60
section of the Annual Pupil
40
18.9
Profile Report advised them of
20
how they could support their
child’s learning at home.
0
Yes
No
A large number of parents stated that this section of the report clearly identified areas
which they should work on with their child at home (n = 534).
Comments from parents who did not find the Focus for Development section of the report
useful are listed below.
Prepared by Research & Statistics
85
KW2031/0809
•
It did not advise how child’s learning could be supported at home (122 comments)
•
Comments were too general and therefore not meaningful (98 comments)
•
This section was excluded from the report received (48 comments)
•
Information was very brief/limited (32 comments)
•
Information seemed to be for school rather than home use (18 comments)
Figure 26: Do you plan to discuss your child’s report with them?
86.5
The majority of parents
100
(86.5%, n = 3102) indicated
% of Respondents
80
that they plan to discuss their
child’s Annual Pupil Profile
60
Report with them. Many of
40
13.5
20
such parents stated that they
plan to discuss this report
with their child so that they
0
Yes
No
can;
•
Praise their achievements (355 comments);
•
Encourage their child’s learning (170 comments);
•
Positively inform their child of were they can improve (124 comments);
•
Inform their child of their own progress and development (84 comments); and
•
Discuss each topic individually (14 comments).
A sizeable number of parents who indicated that they do not plan to discuss their child’s
Annual Pupil Profile Report with them felt that the report was too complex for their child to
understand.
Prepared by Research & Statistics
86
KW2031/0809
What did you like about your child’s Annual Pupil Profile Report?
Parents were asked to indicate what they liked most about their child’s Annual Pupil Profile
Report. The most prevalent responses from parents are summarised below.
•
Well detailed and informative (719 comments)
•
Clear format that is easy to read (680 comments)
•
Detailed breakdown of the curriculum areas (603 comments)
•
Clearly indicates progress and development (350 comments)
•
Advice on home learning and areas of weakness (329 comments)
•
Easy to understand (258 comments)
•
Highlights child’s abilities and achievements (171 comments)
•
Positive language used (151 comments)
•
Focuses on areas beyond the academic (108 comments)
•
Personal to the child (170 comments)
What did you dislike about your child’s Annual Pupil Profile Report?
Parents were asked to indicate what they disliked about their child’s Annual Pupil Profile
Report. The most frequent responses are summarised below.
•
Need comparison to peers - below or above average (204 comments)
•
Impersonal and clinical (172 comments)
•
Grades would provide better information (132 comments)
•
Non-specific and general (119 comments)
•
Too long and complex for this age group (82 comments)
•
No indication given on how to aid learning at home (79 comments)
•
Unclear as to what level a child is achieving (67 comments)
•
Comments are overly positive and could be misleading (36 comments)
•
Jargon used is difficult to understand (33 comments)
Prepared by Research & Statistics
87
KW2031/0809
Additional Comments
Respondents were given the opportunity to make additional comments at the end of the
questionnaire. A large number of respondents felt that the Annual Pupil Profile Report was
detailed and comprehensive (n = 70) and clearly indicated the child’s progress (n = 53). A
number of parents also indicated that they liked the layout of the report (n = 43). Certain
parents felt that the new report format identified strengths and weaknesses (n = 18). A
small number of parents highlighted that they appreciated the time and effort devoted to
the reports (n = 14) with one parent commenting:
“I appreciate all the work the teacher puts into our child’s learning. The report gave
me an understanding of my child’s progress and development throughout the year.
The school provides great friendship and support towards our children and are
there for each parent’s concerns”.
However, parents raised a number of issues. Firstly, a large number of respondents were
dissatisfied that the Annual Pupil Profile Report did not contain grades/scores (n = 163),
which they felt made it difficult to gauge their child’s ability (n = 90). A significant number
of parents found the report impersonal (n = 86) and complex (n = 27) with an overuse of
formal, technical language (n = 29). Some parents (n = 73) were worried that valuable
teaching time was being lost due to emphasis placed on preparing the reports. A small
number of respondents also felt that the report was too positive (n = 26) and did not
provide a balanced picture for parents.
Parents indicated that they would like.
•
More specific information is on how they can support learning at home (65 comments)
•
More regular meetings with their teachers (28 comments)
•
Parents would like more regular reports throughout the school year to help monitor
their child’s progress (23 comments)
Prepared by Research & Statistics
88
KW2031/0809
5.0 Conclusion
This evaluation has identified a number of issues within the Primary Education sector.
Many Primary schools felt pressured from a combination of demands from government
agencies and parents. In some cases this is exacerbated by the perception that the new
assessment and reporting arrangements are being introduced at a demanding pace.
Feedback collected from earlier phases of evaluation on InCAS has been very positive.
The majority of teachers who participated within previous evaluations felt confident
interpreting feedback and deemed it to be consistent with their professional judgement.
Feedback collected from all stakeholders in June 2008 has been somewhat more
negative. However, problems teachers encountered completing the Annual Pupil Profile
Reports is likely to have escalated their criticisms and concerns about InCAS.
Furthermore the deployment and use of InCAS took place at the same time as a major
transition to a newly contracted C2K managed service in all primary schools. The majority
of concerns raised have been identified in previous evaluations and rectified for the
2008/09 academic year. However, one manageability aspect of InCAS that has not been
highlighted as strongly within previous research was that many respondents do not believe
that Primary Schools have the computer capacity to manage the delivery of InCAS as it
involves more year groups. 8,000 additional laptops have been installed in primary
schools across Northern Ireland. These together with the 5,000 assessment laptops
distributed in 2007 mean there are now over 28,000 devices available for use with InCAS.
However, despite this investment respondents stated that teachers with composite classes
and schools without computer suites have found delivering the InCAS assessments
extremely difficult. Many Principals felt that further ICT investment is required for such
schools and teachers need further support to administer InCAS.
Feedback from this evaluation suggests that some Principals and teachers are unclear
about the nature and status of the Annual Pupil Profile Report. The most widely held
misconceptions were that schools were required to use the Profiles 7 software to complete
reports and that the use of comments in the bank were mandatory. As a consequence
some teachers reported being stressed. In some cases comments were used well, with
parents reporting a positive reaction to the information they received about their children.
However less careful use of comments from the comment banks resulted in reports which
were considered by parents as overly positive, general and impersonal.
Prepared by Research & Statistics
89
KW2031/0809
Some schools report receiving different messages about requirements of revised
assessment and reporting arrangements from education stakeholders.
The implementation of new electronic assessment and reporting arrangements represents
significant change for many Primary Schools in Northern Ireland. CCEA provided schools
with a variety of support and guidance in the form of; training materials, telephone
helpdesk, a Pupil Profile section on the Northern Ireland Curriculum website, online
tutorials and an InCAS pupil demonstration. Feedback from this year’s consultation
indicates that some schools did not use the materials or follow published instructions.
Some teachers within the teacher focus groups felt that they did not have time to make
use of the resources provided.
Both Principals and teachers expressed high levels of dissatisfaction with the Profiles 7
report writing software. Feedback indicates that teachers and Principals believe this
software has significantly increased the amount of time teachers take to produce Annual
Pupil Profile Reports whilst reducing the quality of information. The majority of Principals
and teachers deemed this software to be extremely time consuming to use, underdeveloped and have questioned the degree of piloting prior to this year’s implementation.
Only one participant at the Principal and teacher workshops indicated that they intend to
use this software next year.
This evaluation indicates that parents are not entirely satisfied with the new assessment
and reporting arrangements. However, it is important to note that parent’s feelings have
been heightened by the current uncertainty over school transfer arrangements and their
lack of knowledge of InCAS and the Annual Pupil Profile Report. Some parents indicated
that they did not know that their children were being assessed at the beginning of the
school year.
A large number of parents, Principals and teachers expressed concerns that INCAS
assessment outcomes may be used as a method of selection at a future date.
A large number of parents were dissatisfied with the format and content of their child’s
Annual Pupil Profile Report. Although many parents recognised the work required by
teachers to produce this new report there were a number of aspects they criticised.
Prepared by Research & Statistics
90
KW2031/0809
Parents’ main criticism was that it did not contain grades/marks or a comparison with their
child’s peers which they deemed to be an essential component of any school report.
Parents described the Pupil Profile Reports as “impersonal, vague, over generalised” and
as a result were unclear about how their child was progressing. There was also
widespread concern among parents on how ‘hyper-positive’ the Annual Pupil Profile
Reports were. Many parents felt that the report did not provide them with a true account of
their child’s progress. One parent stated:
“[The Annual Pupil Profile Report] was written in politically correct language which
told me nothing. Please encourage teachers to write what they feel and not what
they think ought to be written. More teacher freedom please”.
Some teachers felt uneasy about writing what might be considered less than positive
comments in the reports.
Teachers also have requested advice on how to share InCAS feedback with parents and
direction on comments they may include within the Focus for Development section of the
Annual Pupil Profile Report.
This report has provided a comprehensive overview of this year’s implementation of the
new assessment and reporting arrangements across Primary Schools in Northern Ireland.
Findings from this evaluation show that embedding such new arrangements has required a
large degree of additional work from teachers and Principals. Schools require further
assistance to ensure that they feel adequately supported and empowered to implement
these new arrangements.
Prepared by Research & Statistics
91
KW2031/0809
Limitations
Parents, Principals and teachers have indicated that they are less than satisfied about the
uncertainty about new transfer arrangements. This has impacted on findings obtained
within this evaluation as many were concerned that either InCAS or the Annual Pupil
Profile Report could or would be used to transfer pupils to Post Primary Schools.
A second limitation of the research is the poor response rate to the Principal and teacher
questionnaires. It may be that those most dissatisfied with InCAS and the Annual Pupil
Profile Report completed and returned questionnaires to CCEA. This may have negatively
skewed results but is a common limitation of research.
Prepared by Research & Statistics
92
KW2031/0809
6.0 Recommendations
Before commencing the second roll out of InCAS and the Annual Pupil Profile Report there
are a number of issues documented within this report that should be addressed.
Communication
This evaluation suggests that:
•
Communications regarding the new assessment and reporting arrangements should be
re-evaluated. All communications on these arrangements should be shared between
all partners (CCEA, DE, C2K and CEM).
•
At the start of the next school year CCEA should inform schools which elements of the
Annual Pupil Profile Reports are statutory – it is imperative that such information is
specific to each Year Group.
•
CCEA should provide teachers with an ‘at a glance guide’ outlining what elements of
InCAS and the Annual Pupil Profile Report are statutory and flexible.
•
CCEA should communicate with parents on the new assessment and reporting
arrangements.
InCAS
This evaluation suggests that:
•
Further work should be carried out to determine whether or not additional computer
capacity may be required to support the use of InCAS within Primary schools.
•
CCEA should continue to empower schools to embrace InCAS and develop expertise
and confidence in both administering and analysing feedback produced from InCAS.
CCEA should also reassure schools of the reliability and validity of InCAS.
•
Teachers would welcome advice on how to share InCAS feedback with parents.
•
Further adaptations to InCAS are needed to meet the needs of pupils with Special
Educational Needs and those with English as an Additional Language.
Prepared by Research & Statistics
93
KW2031/0809
Recommendations continued
Profiles 7 Software
This evaluation suggests that:
•
Consideration should be given to whether Profiles 7 is the most appropriate report
writing software or if an alternative should be sought and trialled. Alternatively, Profiles
7 should be developed as a matter of urgency to ensure it better meets the needs of
schools completing the Annual Pupil Profile Report.
•
All teachers who plan to use the Profiles 7 report writing software should be provided
with appropriate training.
Annual Pupil Profile Report
This evaluation suggests that:
•
Schools should be advised explicitly what the purpose of the Annual Pupil Profile
Reports is and in particular its role (if any) in transfer arrangements.
•
Schools require development time to tailor comment banks and ensure all Annual Pupil
Profile Reports are personalised to individual pupils.
•
Principals, teachers, pupils and parents would like to see marks/grades to be included
within the Annual Pupil Profile Reports.
•
Consideration should be given to moving the ‘Focus for Development’ section towards
the end of the Annual Pupil Profile Report.
Prepared by Research & Statistics
94
KW2031/0809
7.0 Annex
METHOD OF CONSULTATION
The roll out of the Pupil Profile Process commenced this academic year with Years 1 and
5. CCEA’s Research & Statistics Unit has undertaken a full evaluation of the
implementation of InCAS and the Annual Pupil Profile Report. This year’s evaluation has
involved consultation with Principals, Year 1 and 5 teachers, Year 5 pupils, and parents
with children in Year 1 and 5. Consultation has been undertaken at various phases of this
academic year and a range of research methods have been employed to collect feedback.
Details of the consultation are provided below.
Phase 1 Consultation:
May 2007
All Year 5 teachers across Northern Ireland
(approximately 1500) were invited to attend awareness
raising training on delivering the Pupil Profile and using
InCAS computer-based assessments. A questionnaire
was used to collect Year 5 teachers’ views and opinions
of this training. A total of 1395 questionnaires were
completed and returned.
Phase 2 Consultation
October - November
Before administering InCAS to pupils, all Year 5 teachers
2007
across Northern Ireland were invited to attend training on
understanding InCAS assessment data. A total of 1425
teachers attended this training. Before the training
commenced 80.2% of teachers (n = 1143) completed a
questionnaire on their previous experiences with InCAS. The
aim of this evaluation was to obtain teachers views and
opinions of InCAS which would be used to inform the future
development of the software for 2008/2009.
On completion of the training 80.8% (n = 1151) of teachers
also completed an evaluation questionnaire based on the
days training on ‘Understanding InCAS Assessment Data’.
Prepared by Research & Statistics
95
KW2031/0809
November - December Following the teacher training on ‘Understanding InCAS
2007
Assessment Data’, teachers were required to administer the
InCAS assessments to their pupils by 31st December 2007.
Questionnaires were posted to all Year 5 teachers
(approximately 1500) and pupils (approximately 25,000) in
Northern Ireland. Questionnaires were completed and
returned by 37% of Year 5 teachers (n = 556) and 59% of
Year 5 pupils (n = 14,011) in Northern Ireland.
The aim of the questionnaires was to evaluate the success of
the implementation of the InCAS Assessments within Year 5
to date and to inform the development of the tool.
Phase 3 Consultation
June 2008
Schools are required to produce an Annual Pupil Profile
Report for all Year 1 and 5 pupils by the end of June 2008.
Throughout June, CCEA’s Research & Statistics Unit
consulted will all key stakeholders involved in this process:
Principals, teachers, pupils and parents. Questionnaires
were the main method of collecting data; however a range of
qualitative methods were also used. A sample of
stakeholders participated in research seminars and focus
groups. Details of these methods are documented below.
Seminars & Focus Groups
Throughout June 2008, CCEA’s Research & Statistics Unit
facilitated; eight Principal workshops, ten teacher workshops
and 13 parent focus groups. The purpose of the workshops
and focus groups was to collect rich in-depth information to
supplement data collected from questionnaires. Two
members of the Research & Statistics Unit facilitated
discussions at these consultations and a representative from
the Pupil Profile Implementation Team was present to answer
any questions that arose throughout the course of the day.
Prepared by Research & Statistics
96
KW2031/0809