An independently analysed
research trial of
Project X CODE
Ros Fisher,
Associate Professor,
University of Exeter
2
Contents
Executive summary............................................................3
Introduction.........................................................................4
About Project X CODE..................................................4
About the trial..............................................................4
The results of the trial........................................................6
Overall progress............................................................6
Sentence reading age.................................................6
Comprehension............................................................7
Phonics .........................................................................7
Children’s attitudes and confidence.......................7
How have schools used Project X CODE ?........................8
What do schools say about Project X CODE ?................9
What do TAs say about Project X CODE ?..................... 10
What do link teachers say about Project X CODE ?.... 11
What do parents say about Project X CODE ?............. 12
What do children say about Project X CODE ?............ 13
A look at six pupils’ performances............................... 15
Conclusion ........................................................................ 18
Appendix............................................................................ 19
An independently analysed research trial of Project X CODE
Executive summary
Introduction
Project X CODE is a structured reading intervention
programme published in 2012 by Oxford University Press
and part of the Project X reading programme aimed at
pupils in Years 2–4 who have experienced a phonics
programme but are falling behind in reading. As part of
the launch of the programme, Oxford University Press
decided to undertake a small scale trial. The trial took
place over 8–10 weeks in the summer term of 2012 in 13
schools from eight local authorities located in different
parts of England.
Views of school staff
Teachers and TAs who were involved with the
programme reported that:
•Pupils made good progress in reading
•Pupils’ reading strategies improved
•Pupils became more confident and motivated
as readers
•The texts are attractive and modern
•The teachers’ materials are helpful and easy-to-use.
Views of parents and pupils
Methodology
The trial was evaluated by a research design that
includes both quantitative and qualitative data.
Children were tested on an independent standardised
reading test both before and after the intervention
and case studies of comments from children, staff
and parents involved in the trial were collected.
Data were analysed and the report written by an
independent researcher.
Key findings
Progress
•Pupils made an average gain of 6.8 months in
sentence reading age.
•Pupils made an average gain of 8.7 months in
phonics age.
•These gains were found to be significantly greater
than would be expected as a result of chance.
•Pupils made an average gain of 30%
in comprehension scores.
•Pupils made an average gain of 20% in phonics
scores.
3 © Oxford University Press 2012
•Parents reported an increase in their children’s
motivation to read and that they talked about the
books at home.
•Children said they loved the books, they liked to talk
about them with their friends and that they had
helped with their reading.
Conclusion
The trial indicates that Project X CODE was
very successful in supporting pupils who were
experiencing difficulty in reading to make more than
expected progress in phonics, sentence reading and
comprehension. It also indicates that the programme
is motivating to otherwise reluctant readers. It was
found to have useful teaching materials that are
easy-to-use.
An independently analysed research trial of Project X CODE
Introduction
About Project X CODE
Project X CODE is a structured reading intervention
programme published in 2012 by Oxford University Press
and part of the Project X reading programme aimed at
pupils in Years 2–4 who have experienced a phonics
programme but are falling behind in reading. It was
designed to combine ‘systematic synthetic phonics,
comprehension development, motivational 3D design
and gripping stories to accelerate struggling readers’
progress so that children reach expected literacy levels
as soon as possible’.1 It is aimed to fit into a school’s
provision map for ‘lighter touch’ Wave 3 support
(children working either one-to-one or in a very small
group with a TA) or as Wave 2 provision (children working
in a small group with a TA).
Thus the programme addresses a group of children who
have caused concern to educationalists, politicians,
parents and employers: those children who do not
make the expected progress in reading. Despite
improvements in the standards of reading evidenced
by an increase in the number of children who achieve
NC level 2 by the end of Year 2, there remains around
16% of children who do not achieve these levels. Many
broadly aimed reading programmes as well as more
specific interventions have been produced to address
this minority.
Project X CODE claims to be based on research findings
that indicate the need for interventions to include a
time-limited approach with clear entry and exit criteria,
regular daily sessions, a clear pedagogical structure
with familiar strategies embedded, detailed session
notes, regular assessment, motivational materials and
attention to children’s self-esteem or view of themselves
as readers.2
Project X CODE is based on a series of adventure stories
set in a theme park, Micro World, with different zones.
Each zone has a different setting and theme park ride.
Micro World is controlled by a central computer called
CODE. Maintenance of the park is undertaken by small
robots called MITEs and each zone is managed by a
robot known as a BITE. These are different in each zone
and, when they become angry, they develop powerful
new capabilities. The adventures start on the opening
day of Micro World when the computer goes wrong and
the central characters, Team X, have to try to put things
right and thereby save the world.
1 Project X CODE Teaching and Assessment Handbook 1 p2
2 Unpublished summary of the intervention written by the series
editors: Di Hatchett and Maureen Lewis.
4 © Oxford University Press 2012
About the trial
As part of the launch of the Project X CODE intervention
programme, Oxford University Press decided to
undertake a small scale trial. The trial took place in 13
schools from eight local authorities located in different
parts of England: large and small; urban and rural. A
proportion of these schools were located in areas of
social deprivation with a larger than expected number
of children with special educational needs. The trial took
place over 8–10 weeks in the summer term of 2012.
The trial was evaluated by a research design that
includes both quantitative and qualitative data.
Children were tested on an independent standardised
reading test both before and after the intervention and
comments were invited from children, staff and parents
involved in the trial. Data were collected by schools and
sent to the publishers. Data sets were then anonymised
and sent to the researcher for analysis. All names used
in this report are pseudonyms.
Both sets of data have been analysed and the report
written by a researcher who is independent from the
authors and publishers of the programme. Nevertheless,
as with all research, certain limitations must be taken
into account. It is well established that interventions
may have a short term affect because they are new
and all those involved have much invested in the
programme. Their very newness provides a motivation to
teachers and pupils. In the case of this evaluation, there
is also the caveat that the research was designed by the
publishers of the intervention and data were collected
by schools which had been given resources to run the
intervention. This may have influenced their responses
in the case study reports.
It is also important to recognise that although some will
argue that the only robust form of research to evaluate
an intervention is the randomised control trial, this is
not universally agreed. In such research there is an
independent control group selected at random and large
enough to match the intervention group. It is argued
that only such research can provide reliable evidence
that any gains can be attributed to the intervention and
are not just a result of chance.3 However, such research
3 It is possible to produce convincing data without a control group
in reading interventions when standardised scores are available.
Standardised scores permit more reliable comparisons to be made
and allow effect sizes to be calculated. Unfortunately, because
schools in this study were free to choose which age group to use the
intervention with, some of which fell outside the range within which
standardised scores could be calculated, it has not been possible to
use standardised scores. Nevertheless, reading ages are provided for
some of the test scores and these allow ratio gains to be calculated.
An independently analysed research trial of Project X CODE
may well provide broad statistics but does not give
sufficient detail of local conditions or give information
about how the intervention worked. A design such
as is used here allows the voice of the participants
to be heard.
Before the introduction of the Project X CODE
intervention, four or five children in each class were
tested for reading skills using the Phonics and Early
Reading Assessment (PERA) test published by Hodder
Education. This test is designed for use with children
in their first three years at school. There are two parts
to the test: sentence reading and comprehension; and
a phonics test in three parts which involves reading
single words aloud, pronouncing non-words and
recognising words on a card. There are two versions
of the text which allows for retesting without risk of
children remembering what they read previously. The
tests provide a sentence reading age and a phonics age
which allows testers to judge how much more than the
normally expected progress children have made over
the period of the intervention. There are also scores for
comprehension and word accuracy, non-word accuracy
and word/non word recognition.
In addition to the reading and phonics test, schools
were asked to complete a case study record. This case
study record was in five parts:
•General information about the school
•Questions for the teacher or TA leading the
intervention to answer about pupils’ attitudes to the
programme and about the materials themselves
•Reflections from the link teacher on the programme
and how it has worked
•An interview with a single pupil or group of pupils
about their views of reading before the start of the
trial and their thoughts about the programme itself
•Information about parents’ views of the programme.
Table one: Summary of data collected
SCHOOL
NUMBER OF PUPILS INVOLVED
IN PILOT
PRE-TEST
April/May
2012
POST-TEST
July 2012
CASE STUDY
July 2012
1
five Year 2
Completed
Completed
Completed
2
four Year 3
Completed
Completed
Completed
3
four Year 2
Completed
Completed
Completed
4
five Year 2
Completed
Completed
Completed
5
eight Year 3
Completed
Completed
Completed
6
four Year 4
Completed
Different test used
Completed
7
four Year 2
Completed
No post-test given
No return
8
six Year 2; two Year 4
Completed
Completed
Children’s comments only
9
four Year 2
Completed
Completed
Completed
10
five Year 2
Completed
Completed
Completed
11
five Year 2; two Year 3; one
Year 4
Completed
Completed
Completed
12
five Year 3
Completed
Completed
Completed
13
three Year 2; three Year 3
Completed
Completed
Completed
5 © Oxford University Press 2012
An independently analysed research trial of Project X CODE
The results of the trial
Overall progress
Sentence reading age
The key data in any evaluation of an intervention
scheme are those that give evidence of progress. The test
results, even after the relatively short period of two and a
half to three months, do show very encouraging gains for
all pupils. As can be seen from the summary table two
below: nearly all pupils made some gains despite the
short period of the intervention. No child made no gain
in any of the items. Children showed an average gain of
6.8 months in sentence reading age and 8.7 months in
phonics age. There was a 30% increase in the average
comprehension score.
The test results
… show very
encouraging gains
for all pupils.
The two scores that allow some statistical analysis are
those in which a ‘reading age’ is given. Here the test has
been tested on a large number of pupils throughout the
country so that the test developers can calculate an
average score for children of particular age groups. This
then enables interventions without a control group to be
judged as to whether anything more than what would
normally be expected had been achieved.
A paired-samples t-test4 was conducted to determine
whether students had made more than expected
progress in sentence reading as indicated by the gain
in their sentence reading ages. There was a statistically
significant increase in sentence reading scores from the
pre-test age with an increase in the average sentence
reading age from 5.74 years to 6.25 years at the end of
the intervention. These scores reinforce the impressions
reported by the schools that pupils had made good
progress during the intervention. The full results can be
examined in the appendix.
Table two: Number of pupils making gains in PERA test items
Total pupils with
scores on this item
Number of pupils
with increased score
Number of pupils whose
scores did not increase
Average gain on
this item
Sentence reading age
59
46
13
6.8 months
Comprehension score
59
55
4
30%
Word accuracy score
61
49
12
3.75*
Non-word accuracy score
61
54
7
2.6*
Word/ non-word recognition
61
54
7
3.6*
Phonics total score
61
55
6
20%
Phonics age
60
55
5
8.7 months
*Depending on the test used, the total for these test items varied. Therefore it was not possible to give a percentage increase for the whole sample.
4 A paired-sample t-test is used to determine whether there is a statistically significant difference between the average values of the
same measurement made under two different conditions: in this case the difference between the reading ages at the start of the
intervention (pre-test) and at the end (post-test).
6 © Oxford University Press 2012
An independently analysed research trial of Project X CODE
Comprehension
55 of the 59 children who attempted this part of the test
made some gains in comprehension with many making
gains of six, seven or eight correct answers out of 10
possible. One child who had no correct answers in the
pre-test scored nine in the post-test. These findings are
confirmed by the case study reports in which schools
really stress the impact the reading programme had on
readers’ comprehension skills. Over the whole sample
there was a 30% average gain in comprehension scores.
It is also interesting to look at these scores in
conjunction with the number of children who did not
make gains in sentence reading age. If we look at the
first three columns of table two, we can see that the
largest number of pupils made no gains in sentence
reading and word accuracy. These two measures
would tend to go together: a child who can read words
accurately is likely to be able to read a sentence
accurately. The most interesting aspect of these three
columns is that only four children failed to make any
gains in comprehension and only one of these four was
one of the children who failed to increase their sentence
reading age. Thus, most of those who made no progress
in reading did make progress in comprehension,
suggesting an improvement in how they read if not
in the number of words they could recognise. The
phenomenon of children ‘barking at print’ is familiar
to many teachers of children who are not making
adequate progress in reading.
Phonics
A paired-samples t-test was conducted to determine
whether pupils had made a greater than expected gain
in phonics as indicated by the increase in their phonics
ages. There was a statistically significant increase in
phonics scores from the pre-test age to the post-test age
with an increase in the average phonics age from 5.58
years to 6.33 years. The full results can be examined in
the appendix.
One feature of Project X CODE is the introduction of
non-word reading. Pseudo words are incorporated into
the stories and reading these is used as a means of
assessing children’s skills at blending. Table two shows
that pupils made good gains in both non-word reading
and recognition. Overall the average gain in phonics
score was 10 out of a possible 50, giving a 20% increase
over the time of the intervention.
7 © Oxford University Press 2012
Children’s attitudes and confidence
Although evidence from the test results indicated
that nearly all children had made more than expected
progress in both sentence reading and phonics, it is
the increase in children’s confidence as readers and
the improvement in their attitude that is most in
evidence from the case study reports. Many schools
had particular stories or anecdotes to tell in order to
illustrate this feature of the intervention. Teachers
reported children coming up to them in the corridor and
enthusiastically telling them about the latest story.
Parents were also reported to be enthusiastic about
the programme. One school reported that the parent
of a Year 2 boy who had been really struggling with his
reading had enjoyed reading for the first time according
to his mum. She was delighted. Another school reported
a father saying: “they are a brilliant idea – never have I
seen him so interested in reading.” (School 2).
It is the increase in
children’s confidence
as readers and the
improvement in their
attitude that is most in
evidence from the case
study reports.
A Year 2 boy
… had enjoyed
reading for the
first time.
An independently analysed research trial of Project X CODE
How have schools used Project X CODE ?
Project X CODE is aimed at pupils in Years 2–4
delivered by a TA in small groups with a maximum
recommended size of four. Schools in the trial mainly
used Project X CODE four or five times in the week. One
school which had an additional group that only had
sessions two or three times a week found that these
children did not make as much progress as those who
had had the sessions every day. Schools reported
that they found the intervention was easy-to-use and
fitted well into the usual routine of the school.
The sessions were delivered by TAs most of whom
claimed to have received some training in phonics
teaching or the delivery of reading interventions. Most
TAs said they had received some training for phonics
teaching. Many claimed to have had specific training in
SSP (Systematic Synthetic Phonics) and most claimed
to have had some training in Letters and Sounds or
ELS (Early Literacy Support) – one was training to be
a teacher. Three schools gave no information but no
school reported using a TA for the intervention who had
not had any training in phonics teaching.
As with any such intervention, close liaison between the
class teacher and the TA delivering the programme was
seen as important. In some cases the teacher had very
little involvement beyond the initial selection of pupils.
However, most schools reported that they had managed
to liaise well and one teacher commented that they
found the materials and structured resources supported
this liaison. Some schools commented that they would
take more care with this aspect at the outset next time:
“This could have been better – there is a need for all
involved in the programme to liaise effectively. In
this case there were difficulties, in that there were two
people delivering the programme. Although a detailed
diary was kept, outlining the day-to-day progress, the
system needs to be streamlined – one teacher or TA
delivering the programme, closely monitored by the
class teacher.” (School 2)
Most schools have found the books to be very interesting
and motivating for children with pupils eager to read the
next story in the series. They also found them modern
looking and of a good length to keep children engaged.
They report that they promote a lot of discussion and
the prompts provided with the text mean that they
are very usable even with less experienced TAs. Some
schools have even taken some of the early books to use
as stories for Reception age children.
One school (School 5) did comment that they found
that pupils sometimes struggled with the language
used and needed extra explanations in addition
to the ‘Exploring vocabulary’ words included in the
planning materials.
Schools used the session notes and found them useful.
Most TAs found them easy-to-use but some made the
comment that it was helpful to have read them through
before the start of the session. Particularly helpful
were the text effects such as the use of italics to
highlight the key points.
“The session notes are very useful too – not overly
long and too detailed but with enough information
to deliver the sessions easily with little time needed
for preparation.” (School 11)
The intervention starts with a launch session designed
to familiarise the pupils with the background story to
the adventures in the reading books. It is intended to
‘build excitement and interest from the start’.5 This
appears to have been a great success with schools
reporting that it was very successful in generating
interest and motivation. In particular, the animation
aroused a lot of interest:
“We read the launch book and watched the
animation as a whole class. The children thoroughly
enjoyed it, all were engaged and had lots of questions
to ask. Most of the class have asked about taking
part and often ask the CODE group what they have
been reading about. This was a great way to launch
the scheme as it really excited the children.” (School 1)
We read the launch
book and watched the
animation as a whole
class. This was a great way
to launch the scheme as it
really excited
the children.
5 Project X CODE Teaching and Assessment Handbook 1 p49
8 © Oxford University Press 2012
An independently analysed research trial of Project X CODE
What do schools say about
Project X CODE ?
The programme also involves assessment materials
designed as, ‘easy-to-use systems for tracking and
assessing progress … built in at regular intervals, from
each session to end of band checks.’6 Not all schools
had used the assessment materials. This may have
been because they already had to administer the PERA
test twice in the term as well as other ongoing in-house
assessment. Most of the seven who did use them, said
they found them useful, although one TA commented
that they would have liked a link to Assessment for
Learning. Five schools found them very useful with
comments such as:
The comments written in the case study reports are
almost entirely positive. The progress made by pupils was
mentioned by many schools both from the test results
and observation of improvements made by the children
involved in the intervention. Many schools commented
that nearly all pupils had made some progress and a few
had made excellent progress. Teachers and TAs working
with the pupils commented also on improved reading
strategies being used by some pupils.
“Easy-to-use and provide good on-going assessment.
I used the end of band reading passages and took a
running record of each child’s attempt – gave a good
profile of their text reading ability – also carried out
the end of band progress check. The comprehension
checklist is a useful tool for teachers /TAs.” (School 2)
“It was noticeable how children began to change their
foremost strategy in solving unknown words – from
guessing, using the initial letter(s) to one of blending
right through the word. Further work is needed with
the more difficult phonemes but they have certainly
gained in this area.” (TA, School 2)
There is also software on a CD-ROM associated with
the programme designed for independent use. The
activities include phonic and comprehension tasks
as well as sentence structure, spelling and memory
activities. Some schools did not make much use of the
software beyond assessment purposes. Mostly lack of
time or access to computers was the reason given for not
using it, although one school had problems loading the
software onto their school computers. Those who did use
the software thought the animation great and said the
children loved it. One TA commented that the children
enjoyed using the clip art. Another loved the eBooks as
did all the children involved.
The most overwhelming response was to the question
as to whether schools had noticed any change in pupils’
attitudes to reading. All eleven of the schools that
completed this section of the case study report agreed
that children’s confidence and/or attitude to reading had
improved. A typical comment was:
“I used them [the eBooks] to model fluency and ‘story
voice’ to the children during lessons and children were
keen to listen and then imitate. They were also very
useful for catch-up lessons when the children were
absent and could be used to share the reading with
the individual child, particularly in higher reading
bands.” (School 5)
The comprehension
checklist is a useful
tool for teachers/TAs.
6 Project X CODE Teaching and Assessment Handbook 1 p2
9 © Oxford University Press 2012
“The children were extremely enthralled with the stories.
It was hard to stop them reading on. There was a real
sense of pride and excitement in their reading.”
(TA, School 9)
There were remarkably few negative comments and
these tended to relate to a particular child who had not
done well or very specific aspects of the programme. Two
schools reported that they felt the programme was more
appropriate for boys than girls, although others found
girls enjoyed it too.
“It was noticeable that the boys were more interested
in the sessions than the girls – the boys would ask
when they were doing CODE, and were disappointed
if they had to wait. The girls were more reluctant to
come to the sessions, although they became more
keen once actually there.” (TA, School 13)
Other more negative comments were offered in the
form of suggestions. One school found the vocabulary
in the later books needed quite a bit of introduction
as some words were unfamiliar. Many teachers
commented on the importance of finding the right
level for children to enter the programme. In the trial
this became a problem for one school where they
had selected four children to be involved who turned
out to be at different levels and they did not have the
staffing to create two groups.
An independently analysed research trial of Project X CODE
What do TAs say about Project X CODE ?
The TAs who have been involved in delivering the
intervention were overwhelmingly positive about the
success of the programme. With a very few exceptions
of individual children, TAs report that pupils were
keen to take part in the programme, keen to come
out of class and disappointed if the session had to be
cancelled for some reason.
“Attitude/engagement during lessons was good and
the children often complained when it was time
to end the lesson, disappointed that they couldn’t
continue to read on.” (School 5)
“During one week of the trial, a couple of CODE
sessions were missed as the children were involved
in rehearsing for their end of year performance. The
children were absolutely devastated and moaned
until we managed to fit in an extra session to catch
up!” (School 11)
The TAs also reported that they had been pleased
with the obvious progress made by pupils. Those who
commented on this referred to the PERA test or the
number of sub-levels or book bands that children had
progressed through. Many TAs picked out particular
skills that they felt had been developed through the
intervention, such as:
• fluency and using phonic knowledge more
consistently and accurately (School 1),
• willingness to answer questions (School 4),
• use of inference and deduction skills (School 10),
• word recognition ability to blend sounds for reading
(School 11).
One commented: “It has also provided opportunity
to work closely with these children and identify
individual needs such as development of independent
reading strategies, b/d confusions, unknown GPCs/
confusions and feed these back to the class teacher.”
(School 5)
The TAs who had worked with pupils reported
very positive impressions of the books used in the
programme. One very full answer came from the TA
in School 1:
“The books are very bright and colourful and the
layout makes it easy for the children to follow. The
children have become confident in identifying the
parts of the text that are non-fiction (for example,
the one about ‘Hungry plants’). We have used the
books for 1:1 reading (not as part of the trial) with
10 © Oxford University Press 2012
our weakest SEN children and they have enjoyed
them. The books are pitched well at the different
levels, however, due to the level of the children at
the start, the first zone would have been far too
simple. This resulted in us having to read these
books quickly to the children so that they knew
what had happened so far, before starting at the
appropriate zone for their reading ability.” (School 1)
The word work at the start of the books was found to be
very useful.
Each book contains a first text that is 100%
decodable and a second text that is 80% decodable.
One TA praised the authors of the decodable text,
commenting that it did not appear to be as contrived
as do some of these type of books (School 2).
TAs report that children find the books captivating:
“The books are fantastic. Well laid out with amazing
graphics. The children are captivated by these books.
More children want to read them – ones who are not on
the programme. Even the smaller things like the MITEs
being different colours in each zone appeals to the
children and they look out for the colours in each new
zone.” (School 3)
Overall, the TAs focus mainly on the enthusiasm
children have shown for the programme and its impact
on children’s confidence and achievement. To sum up,
one said:
“The most memorable points for me have [been] the
children’s understanding/comprehension from their
reading, as well as the development of their vocabulary.
This has been particularly shown from a little girl who
has EAL.” (School 3)
The books are
fantastic. Well laid out
with amazing graphics.
The children are
captivated by
these books.
An independently analysed research trial of Project X CODE
What do link teachers say about Project X CODE ?
The link teachers largely mirror what their TAs have
reported and share their enthusiasm. Most made
particular mention of how engaging pupils have found
the programme and how pleased they have been with
most pupils’ progress. Link teachers also made some
useful points about the running of the intervention:
“It has been a very exciting and engaging project.
Those children who have demonstrated reluctance
to participate in reading have increased their
engagement and confidence levels. Engagement
levels have been higher than in other interventions
adopted by the school.” (School 9)
The main points that they raise are more to do with how
they have used the programme in their school and how
they might make changes next time. They commented,
for example, on the importance of taking enough time
to set up the scheme to allow TAs to get to know the
materials and for the assessment of the pupils. Link
teachers also stressed the importance of close liaison
between the person delivering the programme and the
class teacher. Where this has happened this appears to
have worked well; one teacher even commented that it
had been influential in developing a closer relationship
between TA and class teacher. However, one link teacher
regretted that she had not spent more time in liaison
and thought that the pupils might have got even more
from the programme if she had done so. For example,
School 2 reflected:
“In this case there were difficulties, in that there were
two people delivering the programme. Although a
detailed diary was kept, outlining the day-to-day
progress, the system needs to be streamlined – one
teacher or TA delivering the programme, closely
monitored by the class teacher.” (School 2)
Where the link teacher was also the class teacher,
some mentioned that other children in the class
showed an interest in the stories and said that
they too would like to join the group. Others made
particular mention of specific children whose
engagement in the programme and back in class
after sessions showed pleasing improvement:
“One pupil, who was exclusively reliant on using
meaning and syntax cues, has now learnt to use and
apply his phonic knowledge to decode. An EAL pupil is
engaged by the special vocabulary in the programme,
and has begun to reflect it in her spoken language.
We have overheard CODE pupils recommending
Project X reading books to other children in their
class.” (School 3)
11 © Oxford University Press 2012
Some teachers emphasised the need for a dedicated
space to be set up for the intervention to store and
display the resources. School 11 did this and found
it useful:
“We have also established a permanent teaching area
for the CODE intervention group. This is in the corridor,
just outside the main classroom, with a display of
the work the children have been doing and some
of the characters and the map from the story. This
has made it easy for me to be part of some of the
sessions the children have been involved in. It has
also generated lots of talk from the other children
who want to know what is going on! The CODE group
love to come back into class and share their latest
adventures with the other children and we often
watch the eBooks on screen as a whole class, with
the CODE group leading our discussions about what
is going on.” (School 11)
An EAL pupil is
engaged by the special
vocabulary in the
programme, and has
begun to reflect it in
her spoken language.
We often watch
the eBooks on screen
as a whole class, with
the CODE group leading
our discussions about
what is going on.
An independently analysed research trial of Project X CODE
What do parents say about Project X CODE ?
Eight of the schools managed to report a conversation
with at least one parent about Project X CODE. Parents
were asked three questions:
•Have you noticed a change in your child’s attitudes
to reading?
•Have you seen the Project X CODE materials?
What did you think of them?
•Has your child talked about Project X CODE
at home?
Not all parents answered each question so only
a general impression can be gained. Also, not all
schools involved parents in the programme and at
least one school did not allow books to go home. All
parents questioned made positive comments about
the programme. Parents from four of the schools
spontaneously commented that they thought their
child had made progress in reading. One said:
“Yes. I have noticed a change in his reading. He is
reading things off the television and reading things
from the shops.” (School 3)
Parents in seven of the eight schools commented
on how their children’s attitudes to reading had
improved. Parents reported that their children were
willing to read ‘without being coaxed’.
“He’s keener to read – he’s reminded me that it’s
time to read and got his reading book out. He’s more
interested in being read to at night, and the story
holds his attention for longer.” (School 13)
Not all parents had seen the materials but those who
answered that question said that they liked them.
“They are really enjoying the materials. It isn’t
something they have to do … they really want to.”
(School 4)
These parents have also found that their children
talked about Project X CODE at home:
“She always talks about her lessons. She talks to me
about the ‘ask me about’ labels. She works through
her homework and then tells me about it.“ (School 3)
Another parent commented:
“He talks about the project every day and seems to
enjoy doing it.” (School 1)
12 © Oxford University Press 2012
Also, elsewhere in the data, it is reported that children
enjoyed taking the books home and then telling the rest
of the class about the reading they have done:
“Two children in particular have talked a lot about the
books at home, resulting in their parents regularly
coming into school and asking questions about the
trial so far. One of these children has other Project X
Xbooks
booksatathome
homeand
andhas
hasbeen
beenbringing
bringingthem
theminto
intoschool
and reading
them tothem
the class.
child
school
and reading
to theThis
class.
Thispreviously
child
would not have
confidence
to read in front
of
previously
wouldhad
notthe
have
had the confidence
to read
thefront
classofand
for known
regularly
the phrase
in
the was
classknown
and was
forusing
regularly
using
‘I can’t
read.”
(School
1) (School 1)
the
phrase
‘I can’t
read.”
Parents reported
that their children
were willing to
read ‘without
being coaxed’.
They are really
enjoying the materials.
It isn’t something they
have to do … they
really want to.
An independently analysed research trial of Project X CODE
What do children say about Project X CODE ?
Schools were given a brief interview schedule to conduct
with a child or group of children. Different schools
opted for different sized groups and some children
were interviewed individually and others as a group.
Also, some TAs wrote verbatim what children said while
others paraphrased. Some even just wrote a general
response for the whole group. This means that just a
general idea can be given as to children’s opinions of
the intervention, recognising as with the parents that
this is just a second hand report of what was said. TAs
were asked to ask children’s thoughts about reading
before the start of the trial and then to ask three other
questions at the end of the trial:
•What did you think of Project X CODE?
•Would you recommend the books to your friends?
•How do you feel about your reading now?
None of the children interviewed were reported to say
that they did not enjoy the programme or that it had
made them worse readers. About half the children
said that at the start of the programme they did not
enjoy reading or found it boring and said that after
the programme they were better readers and enjoyed
reading more. The other half said that at the outset
they enjoyed reading and that they still did at the
end. Some children had clearly given careful thought
to their answers.
One child at the start of the intervention had said that
s/he thought books boring and found them hard. At
the end of the time s/he said:
“My reading is marvellous because I know more words
because I practice them before we read the books. [S/He
means sound spotters and tricky words etc.] I like taking
harder books home to read now.” (School 1)
everybody and I have three cats at home.”
(School 11: 6-year-old girl)
“I like it when they defeat the bad guys. The
characters are cool – I want to be Max. I play Max on
the playground and try to get the BITEs.”
(School 11: 6 year-old-boy)
All but three children said they would recommend the
books to their friends. Amusingly two of three children
who said that they would NOT recommend the books
to friends said that this was because they were too
good and wanted to keep them to themselves!
“[I’d recommend] some of them, but not all. Some of
them I want to keep just for me – I don’t want anyone
else to read them.” (School 13)
Table three: Children interviewed
about Project X CODE
SCHOOL
NUMBER OF
CHILDREN
AGE
1
five
6- & 7-year-olds
2
one
7-year-old
3
four
7-year-olds
4
Group (no
number given)
6- & 7-year-olds
5
three
8-year-olds
one
7-year-old
one
6-year-old
two
8-year-olds
two
9-year-olds
6
Others commented:
“It’s a bit better. When I get stuck on my reading I
can sound out words – CODE helped me with that.”
7
zero
8
one
8-year-old
two
7.5-year-olds
three
6-year-olds
9
one
7-year-old
10
two
7-year-olds
11
two
6-year-olds
12
seven
7-year-olds
13
two
8-year-olds
three
7-year-olds
one
6-year-old
(School 5: 8-year-old boy)
“I think I have got better at reading and I know my
sounds better.” (School 11: 6-year-old girl)
Children mostly spoke positively about the books:
“It’s fun. Tiger is my favourite character. The Dragon
Quest zone is my favourite so far. The pictures are
fantastic. The sound checker page really helps me
before reading.” (School 3: 7-year-old)
“I think the books are really good … the stories are
exciting and the children had good adventures.
My favourite character is Cat because she helped
13 © Oxford University Press 2012
An independently analysed research trial of Project X CODE
There was just one school that reported less than
enthusiastic responses from the children. This was
School 8. These children all reported that they felt
positively about reading before the start of the trial and
still felt positive at the end of the trial. They also said
they quite liked the books but two of them preferred
Biff and Chip (Oxford Reading Tree). Also these children
seemed very preoccupied with book bands and were
concerned as they did not think that they were on
book-banded books.
One child in this school said they would not recommend
Project X CODE to their friends: “because they may get
bored as they are not moving up the book bands”.
14 © Oxford University Press 2012
The school commented that, although the books are
book-banded, these children were not told what colour
band they were on as they were all started on Yellow
band when it was realised that they were missing a
lot of the phoneme and grapheme strategies covered
in the early books. These children’s comments give
an interesting insight into the sensitivities involved
in working with pupils in an intervention and how
important their understanding of what matters
can be.
An independently analysed research trial of Project X CODE
A look at six pupils’ performances
In the following section six pupils’ performance during
the trial are examined in more detail. They have been
chosen not because of their test scores but mainly
because they were children who had something to
say about Project X CODE and they came from schools
which had sent in full sets of data.
Colin and Joe
School 5 is a medium size primary school in a shire county
with nearly 300 pupils including those in the nursery. They
ran two groups in the trial: one group of three and one group
of four. The original two groups of four were reduced to one
of four and one of three as one boy became confused with
the change of programme and it was felt to be better to omit
him from the intervention. The school found the intervention
to be a success with all children involved making some
progress in phonics and most improving elsewhere as well.
The average gain in sentence reading age in this school was
8.25 months and the average gain in pupils’ phonics score
was 12 out 50.
Colin was eight years old in a Year 3 class in School
5. His teacher assessed him to be at NC level 1B in
reading. Unlike most of the other children in the trial,
according to the PERA tests he had made little or no
measurable improvement in his reading. He is included
here because his own comments are interesting.
Although his progress does not show up in his reading
test performance, his own self-confidence and
awareness of reading skills seem to have improved.
He said that he had enjoyed the books and felt they
had helped him. This is an important pre-requisite to
making progress.
What did you think of Project X CODE?
“It’s fun and exciting because I like the different BITEs
and the different powers they have. I look forward to
finding out what the BITE is going to be.”
Would you recommend the books to your friends?
“Yes, because they’re fun.”
How do you feel about your reading now?
“It’s a bit better. When I get stuck on my reading I can
sound out words – CODE helped me with that.”
NB: All names used in this report are pseudonyms to respect the
anonymity of the participants.
15 © Oxford University Press 2012
Table four: Colin’s reading test results
Pre-test
Post-test
Sentence reading age
6y 6m
6y 6m
Comprehension score
6/10
8/10
Word accuracy
12/17
12/17
Non-word accuracy
4/9
4/9
Word/non-word recognition
21/24
22/24
Phonics total
37/50
38/50
Phonics age
6y 2m
6y 2m
Joe was seven and a half years old in the term of the trial.
He was in a Year 2 class in School 5. Joe’s reading level at
the start of the intervention was judged by his teacher to
be on the NC level 1C/1B borderline. On the PERA reading
test his sentence reading age was only four years and 10
months, nearly three years below what would be expected
for his age. Although his word recognition and phonics were
not so poor, his comprehension score on the pre-test was
zero. At the end of the term after being involved in Project X
CODE for four days a week, his sentence reading level had
gone up by nearly two years, his phonics age by 11 months
and all other scores had also improved. In particular, at
the end of this period he scored eight out of 10 on the
comprehension questions. He commented that Project X
CODE is: “fantastic because we get to read and we can see
what happens next”.
When asked how he felt about his reading at the end of
the intervention, he said: “Happy, because I like reading
CODE books and school books”.
Table five: Joe’s reading test results
Pre-test
Post-test
Sentence reading age
4y 10m
6y 6m
Comprehension score
0
8
Word accuracy
10/17
14/17
Non-word accuracy
3/9
8/9
Word/non-word recognition
18/24
23/24
Phonics total
31/50
45/50
Phonics age
5y 8m
6y 7m
An independently analysed research trial of Project X CODE
Umar and Mandy
School 11 is a very large, popular school located in
a mixed socio-economic area on the edge of a large
town in the midlands. Attainment on admission had
been declining for several years as the demographics
of the area had changed. The percentage of pupils
with learning difficulties appears to have stabilised
recently, having increased substantially from 3% to
15%. There has been a significant change to the ethnic
mix and the percentage of minority ethnic children is
rising annually (7% increase over the last five years).
Increasing numbers of children start school with poor
personal, social and emotional skills.
Project X CODE was very well received in School 11.
The TA commented:
“All the children have loved CODE! There was an
equally positive response from both boys and girls.
Their confidence in reading has greatly improved,
especially their ability to sound out and try to read new
or unfamiliar words. They are all now very keen to talk
about their reading and love telling the rest of the class
about their latest CODE adventure!”
The class teacher was also very pleased with how the
intervention had gone with two boys.
“It is hard to convey in words the excitement CODE has
generated in two of the boys from Year 2 in particular
– it needs to be seen first hand! They both love to read
now, which certainly wasn’t the case earlier in the year
and they believe in themselves as readers. One of the
boys in particular, rarely contributed ideas in class
about books we read, but now he is the first one to put
up his hand and is bursting to share his ideas. These two
boys also regularly have their own CODE adventures out
on the playground and it is a delight to see them having
so much fun based on their reading.” (Class teacher)
Umar was six years nine months at the start of the
trial. When asked how he felt about reading he said:
“I find it a little bit tricky … sometimes I forget my
sounds and I can’t read the words”.
His sentence reading age was seven months behind
that expected for his age and his phonics age was
10 months behind. In the ten and a half weeks
between the tests, he increased his sentence reading
age by four months and his phonics age by six
months. Other scores increased too, in particular his
comprehension score doubled.
16 © Oxford University Press 2012
When he was asked about how he felt about reading
after the intervention, he said:
“I think I have got better and better and better.
I can remember my sounds better….. my spelling is
getting better too …. I get four and five out of five
now! I love reading now. These books are so good
and help us with our tricky words.”
Table six: Umar’s reading test results
Pre-test
Post-test
Sentence reading age
6y 2m
6y 6m
Comprehension score
4/10
8/10
Word accuracy
9/17
13/17
Non-word accuracy
6/9
7/9
Word/non-word recognition
19/24
21/24
Phonics total
34/50
41/50
Phonics age
5y 11m
6y 5m
Mandy is a summer born girl in Year 2 of School 11.
As can be seen from table seven, Mandy’s scores have
all increased. Although she is still scoring below what
would be expected for her age, she has made good
progress across the range of skills tested. At the start,
when asked about her reading she said: “It is hard. I
keep forgetting my sounds and get them mixed up”. At
the end of the trial she spoke enthusiastically about the
books and how exciting she found the stories. She also
commented about her reading progress: “I think I have
got better at reading and I know my sounds better”.
Table seven: Mandy’s reading test results
Pre-test
Post-test
Sentence reading age
5y 9m
6y 6m
Comprehension score
5/10
9/10
Word accuracy
7/17
12/17
Non-word accuracy
2/9
6/9
Word/non-word recognition
13/24
20/24
Phonics total
22/50
38/50
Phonics age
5y 1m
6y 2m
An independently analysed research trial of Project X CODE
Jany and Adam
School 3 is an average size primary school on the
outskirts of a small, steadily growing town in the
south of England. The proportion of pupils attending
with learning difficulties and/or disabilities is well
above average. This includes pupils with moderate
learning difficulties, visual impairment, autism and
emotional and behavioural difficulties. Most pupils are
white British. A few pupils, particularly from Poland,
Latvia or Lithuania, speak English as an additional
language. The school found the intervention helpful
and were impressed by the progress made by the four
children involved. These children showed the highest
average increase in sentence reading age (15 and a half
months) and phonics score (an increase of 15 out of 50).
The TA delivering the intervention said:
“Since starting the programme, all children are
very keen to learn. The programme is exciting for
them. They love the books, the characters and the
adventures. As soon as they arrive for the lesson they
are keen and ready to learn.”
Jany is a seven year old girl from Eastern Europe
who was learning to speak English as an additional
language. She was seven years and two months at
the start of the intervention with a sentence reading
age of five years and five months and a phonics age
of five years and four months. In the just less than
three months between tests, Jany increased one year
over and above what would be expected in sentence
reading and eight months in phonics. Other scores
also rose in each area. The school found that the
intervention had particularly helped her vocabulary
and comprehension and they had noticed her using
some of the vocabulary in her day-to-day speech. At
the start of the trial she seemed to enjoy reading but
found it hard as she could not read to her mother at
home. She enjoyed Project X CODE and, at the end of
term said:
“It’s good because we all work together. The books are
fabulous. There are good drawings and we learn new
words. [Now I feel] very happy. It has helped me get
better. I understand what I am reading and how to
tackle harder words.”
17 © Oxford University Press 2012
Table eight: Jany’s reading test results
Pre-test
Post-test
Sentence reading age
5y 5m
6y 5m
Comprehension score
3/10
7/10
Word accuracy
4/25
8/25
Non-word accuracy
2/9
5/9
Word/non-word recognition
3/16
8/16
Phonics total
9/50
24/50
Phonics age
5y 4m
6y 0m
Adam was seven years and seven months at the start
of the trial. His sentence reading age was five years
and one month, his phonics age five years and five
months and he was only able to answer one of the
comprehension questions correctly. Despite this he did
feel quite confident about his reading. In the post-test
his sentence reading age had risen by two years to
seven years and three months, only six months below
his chronological age, and he was able to answer all
the comprehension questions correctly. His phonics
age also rose to six years and nine months. He spoke
enthusiastically about the programme after the trial.
What did you think of Project X CODE?
“Brilliant. I love reading the books and finding out
about the adventures. I like all the characters.”
How do you feel about your reading now?
“I feel confident. The books have helped me to get
better. The sound checker page with tricky words
really helped.”
Table nine: Adam’s reading test results
Pre-test
Post-test
Sentence reading age
5y 3m
7y 3m
Comprehension score
1
10
Word accuracy
9/25
12/25
Non-word accuracy
2/9
6/9
Word/non-word recognition
3/16
13/16
Phonics total
14/50
31/50
Phonics age
5y 5m
6y 9m
An independently analysed research trial of Project X CODE
Conclusion
• Project X CODE has clearly been very successful in the schools involved in the
trial. Substantial and significant gains in sentence reading and in phonics
have been made by nearly all pupils over the two–three month period of the
intervention.
• Teachers, TAs and parents who were asked all agreed that the programme was
modern and attractive. TAs and teachers who worked on the intervention mostly
found it easy-to-use.
• Teachers, TAs and parents who were asked felt that the intervention increased
most pupils’ motivation and confidence in reading.
• Children who were asked also spoke positively about the stories and the impact
of the programme on their reading.
• There is some indication that the programme works best when delivered daily.
• As with other interventions of this kind, evidence points to the importance of
good liaison throughout the intervention between class teachers and those
delivering the programme.
Most schools have
found the books to be
very interesting and
motivating for children
with pupils eager to read
the next story in
the series.
18 © Oxford University Press 2012
An independently analysed research trial of Project X CODE
Appendix
Paired-samples t-test using pre-test sentence reading age and post-test
sentence reading age
Paired-Samples Statistics
Mean
N
Std. Deviation
Std. Error Mean
Pair 1 Pre-reading Sentence Age
5.736620
56
.5536318
.0739822
Post sentence Reading Age
(years)
6.251488
56
.6529843
.0872587
Paired-Samples Test
Paired Differences
Pair 1 Pre-reading
Sentence Age –
Post Sentence
Reading Age
(years)
Mean
Std.
Deviation
Std. Error
Mean
-.5148679
.5939703
.0793726
95% Confidence
Interval of the
Difference
Lower
Upper
t
df
Sig.
(2-tailed)
-.6739341
-.3558016
-6.487
55
.000
A paired-samples t-test was conducted to determine whether students had made more than
expected gain in sentence reading as indicated by the difference between their pre- and posttest sentence reading ages. There was a statistically significant increase in sentence reading
age from the pre-test age (M=5.37, S.D= 0.55) to the post-test age (M=6.251, S.D= 0.65) t(55)
= -6.48, p= < 0.005 (two-tailed). The mean increase in sentence reading age was 0.515, with
a 95% confidence interval ranging from -0.674 to -0.356. The eta squared statistic (0.43)
indicated a large effect size.
19 © Oxford University Press 2012
An independently analysed research trial of Project X CODE
Paired samples t-test using pre-phonics age and post-phonics age
Paired-Samples Statistics
Mean
N
Std. Deviation
Std. Error Mean
Pair 1 Pre-Result Phonics Age in years
5.579167
60
.5224347
.0674460
Post Result Phonics Age (years)
6.329168
60
.4637076
.0598644
Paired Differences
Mean
Pair 1 Pre-Result
Phonics Age
in Years – Post
Result Phonics
Age (years)
-.7500017
Std.
Deviation
.5491406
Std. Error
Mean
.0708937
t
df
Sig.
(2-tailed)
-10.579
59
.000
95% Confidence Interval
of the Difference
Lower
Upper
-.8918597
-.6081436
A paired-samples t-test 1 was conducted to determine whether students have made more
than expected gain in phonics as indicated by the difference between their pre- and post-test
phonics ages. There was a statistically significant increase in phonics scores from the pre-test
age (M=5.58, S.D= 0.522) to the post-test age (M=6.33, S.D= 0.46) t(59) = -10.58, p= < 0.005
(two-tailed). The mean increase in phonics age was 0.75, with a 95% confidence interval
ranging from -0.89 to -0.61.The eta squared statistic (0.65) indicated a large effect size.
Acknowledgements
Oxford University Press is very grateful to the schools who took part in the trial.
Please note: The photographs of children in this report are for illustration purposes only.
They do not show children from the schools featured.
20 © Oxford University Press 2012