INTERVENTIONS ARE NECESSARY BUT
NOT SUFFICIENT
Peter Tymms
Outline
•
•
•
•
•
A quick look backwards
The need for interventions
Two recent eye openers
Alternative perspectives
Conclusions
– For teaching
– For policy making
Historically
• Education was an evidence free zone
• Experience, opinion and good ideas ruled
• Two example examples
– Kenneth Baker
– Estelle Morris
Interview of Lord Baker with Giles Dinot on
18 February 2011
• Q: One criticism was that the National
Curriculum just “taught to test”; is there a
problem with that?
• A: I went to a Church of England primary school
during the war in Southport. I’ve still got the little
report books, and there they are, marks out of
100. I was quite clever in those days. I was being
tested every term. And I took it home and
showed my mother and father.
Estelle Morris: Education Secretary
When in office she did not base any action on research
but took considerable notice of the Daily Mail
The need for interventions
• Consider
– £500 million on the National Literacy Strategy with no
detectable impact
– Etc etc etc
• World wide recognition of the issue
– Campbell Collaboration
– USA
• What Works Clearing House
• Slavin’s Best Evidence
– John Hattie
– Tool kit
– CEM, Durham
Meta-analysis
• Example: use of digital technology to increase
literacy in middle schools. (2005)
Forest Plot
Points to note
•
•
•
•
Overall: ES=0.49
Variation of impact of interventions
Need to know more ….
But we now have a best estimate
First eye opener
• Slavin et al 2014
• Cooperative Learning in mathematics has
been shown to be effective in the USA
• Tried a large RCT in England
– And failed “most surprising”
• Tried again with a revitalised approach
– And failed again
Second eye opening
• Lemons et al 2014
• Five RCTs of the efficacy of KG Peer-Assisted
Learning on reading program.
• Over 9 years involving 2,591 students.
• Early results very positive and widely reported
• Most recent results
– Failed
• Team: “shocked and depressed”
Explanations
• Slavin et al
– “Differences between traditional teaching practices in
England and in North America”
– “Teaching methods proven to be effective in one culture
and system cannot be assumed to be effective in another.”
• Lemons et al
– “The changed context”. The bar had been raised in KG.
– “the change agent - a no-nonsense Chief Instructional
Officer”
– The controls were different
A Further Complication
• We live in a complex interactive world.
• Even deterministic worlds are not predictable
– http://www.math24.net/double-pendulum.html
• But patterns appear (strange attractors)
– https://www.youtube.com/watch?v=aAJkLh76QnM
• Error propagation is the cause and it is
summed up in the Dynamic linear model.
The nature of teaching
• Great teaching involves
– Planning
– The ability to ditch Plan A and go to Plan B
– Dealing with idiosyncratic
• children
• classes
• Integrating professional judgement with scientific
evidence
• The bottom line
– Academic progress
– Positive non-cognitive progress
Some features of policy making
• Making big decisions.
• John Maynard Keynes
– "When the facts change, I change my mind. What
do you do, sir?"
• But U-turns are bad news
– “The lady’s not for turning”
– Are “Reforms as Experiments” conceivable?
• Time scales are long
Propositions
• Teachers:
–
–
–
–
must have up-to-date evidence
must interact and collect feedback continuously
must not be held to account for processes
progress is the bottom line
• Policy makers should:
–
–
–
–
–
accumulate & generate evidence
run pilot projects for proposed policy initiatives
spend large sums only when with evidence
must monitor
be held to account for consequences