A RCT evaluation of The Letterbox Club
in Northern Ireland
Dr Karen Winter and Jennifer Mooney
[email protected]
[email protected]
Context
• Poor educational outcomes
• Measures: attendance rates; suspensions; exclusions;
attainment in tests, exams and formal qualifications
• Contributory factors: interplay of individual child; familial and
care related characteristics
• Disability; abuse; trauma; physical/emotional ill health; poor
familial relationships; multiple transitions; poor attachment;
stigma; labelling; lack of support for carers
Initiatives
• Legislation
• Policy and practice initiatives
• Macro level - structures and processes
• Micro level – direct to child interventions
• Concern with evidence, what works, cost
effectiveness
The Letterbox Club
•
•
•
•
•
•
•
•
Once monthly parcel for 6 months
May to October
Books, stationery, number games
Addressed to the child
Personalised letter
Interest level, not ability level
Aims
www.letterboxclub.org.uk
Contents
Available research
•
•
•
•
Letterbox Club began 2002
2003-2006 pilot work
2007 – national pilot funded by government
2008; 2010 evaluations by programme
designers in England and Northern Ireland
• Gains in reading and number skills
• 2011 independent evaluation data in Northern
Ireland – gains reported
• www.qub.ac.uk/cee
The gap
Can the gains in reading and number skills be
attributed directly to the Letterbox Club
intervention?
The gap
Comparison using standardised scores - limits
Importance of control group – identical, random
allocation, evenly balanced, variations accounted for
Essential if we are serious about addressing inequality
Next slides consider 3 elements to study design: the RCT,
logic model and the process evaluation.
Mismatch between inputs
Logic
model
and anticipated outcomes?
• Does not account for previous
research around reading
• Assumes a linear movement
• Hinges on feelings of ownership
• Assumes children do not have access to
materials
• Middle class deficit view of children in care
The RCT
Current
study study
RCT study participants
Flow chart
Data collected
Logic model
Measures
• Neale Analysis of reading ability
• Elementary Reading Attitude Survey (Garfield)
Other data
• Age
• Gender
• Type of placement
• Trust area
•
•
•
•
Placement moves
Carer information
Length of time in care
Siblings/ foster siblings
• Letterbox fun days
RCT challenges
•
•
•
•
•
•
Attitudes
Ethics
Collaborators
Fieldworkers
Results
Responses to results
Process evaluation
Logic model
• Interviews foster children and their carers
• Exploring:
• what actually happens at time of receipt of
the parcel;
• engagement with parcels;
• views on parcel content;
• and ongoing support with the materials.
The Future
Logic model
• Further Studies- nurture groups and school
based interventions
• Collaborations- with colleagues in Canada and
REES centre (Oxford)
• Development of the programme
Many Thanks!