Measuring quality of non-market
output in education:
Approaches developed in England
Presentation for the joint OECD/ONS/Government of Norway workshop, October 2006
Deborah Garniss
What are we trying to measure?
► Education
seen as an activity
 output is volume of this activity provided
 seen as amount of teaching delivered
► So
output is number pupils actually taught
► Measuring quality of teaching delivered
► Difficult to measure directly
How can we measure it?
Assume a relationship between teaching, learning and attainment…..
Quality of
teaching delivered
Quality of
pupil learning
Exam attainment
School inspection ratings
BUT: ignores many of the wider outputs and outcomes of education
What do we need to do?
► Match
attainment to pupils taught in year
 but don’t have exams every year
 no measure of yearly attainment per pupil
► So
attainment not perfect measure
► But what attainment data do we have?
 final attainment at end of schooling
 Key Stage tests at ages 7, 11 and 14
A number of alternatives
► Final
output measures
 % passing threshold attainment level (5 good GCSEs)
 Average attainment scores over all exams
► ‘Intermediate’
measures
 Average attainment scores at end each Key Stage
► Added
value (cohort progress) measures
 Average progress made between each Key Stage
► Apportionment
measures
 Apportioning attainment to each year of schooling
Conceptual issues with attainment
► Changes
in attainment need accurately to
reflect changes in learning / teaching
► So strong assumptions about link between
attainment and quality
 x% Δ attainment = x% Δ quality
 zero attainment = zero quality
Practical issues with attainment
► Incomplete
measures of output quality
 measured at 4 points during 11 yrs schooling;
no data for other 7 years
 don’t all capture full range of attainment (at top
and bottom ends of distribution)
► Inconsistencies
over time
 changing scales for measuring attainment
 changing scope of final output measures
What do we conclude to date?
► Final
output measures
 Transparent and simple, related to what employers
look for from education system, can value in £ terms
 but ignore rest of education system, and long timelag before impact of policy shows up (up to 11 years)
► Added-value
measures
 Greater coverage across the system, and closer to
added-value in private sector
 but reliant on a number of assumptions, which need
further analysis
What difference do they make?
115
Current
110
Threshold
105
Average points
100
Simple added-value
95
Apportioned addedvalue
Ofsted inspection
90
85
1998 1999 2000 2001 2002 2003 2004
Is attainment the right approach?
► Argue




education is productive process
output more than just about activity levels
multiple outputs
tangible and observable (e.g. qualifications)
some less so (e.g. soft skills)
► More
going on in school than just teaching
► Output & quality in Nat Accts very narrow
► So any productivity estimates are partial