Key Stage 2 to 4: Progress Measures

Best 8 including English and mathematics Value Added – Key Stage 2 to 4
Definition
Best 8 including English and mathematics Value Added (“Best 8” VA) is a measure of
progress between Key Stage 2 and Key Stage 4. It uses KS2 prior attainment to estimate
subsequent KS4 attainment, in terms of a pupil’s capped best eight GCSE and equivalent
outcomes (plus a separate bonus for attainment in each of English and mathematics).
VA measures are a statistical means of assessing the relative effectiveness of a school and
measuring pupil progress.
Eligibility
Pupils are included in the “Best 8” VA model if:
 their Key Stage 4 attainment can be matched to their attainment at Key Stage 2;
 they have a KS2 average point score that is greater than zero;
 they do not have a disregarded outcome in all three KS2 tests/TAs;
 they attend a Maintained Mainstream school (including Academies and City Technology
Colleges) – see ‘Notes’ section for calculation of Special school VA scores.
All Maintained Mainstream and Special Schools will have a “Best 8” VA score provided they
have at least one eligible pupil.
Formula /
Method
Behind the “Best 8” VA measure sits a statistical model. This model generates an estimate of
attainment for each pupil in their best eight GCSE and equivalent outcomes (plus a separate
bonus for attainment in each of English and mathematics). The estimated KS4 outcome is
expressed as a point score, and is based on the performance nationally of all pupils with the
same KS2 prior attainment.
For further guidance on calculation of a pupil’s best eight GCSE and equivalent outcomes
and English/maths bonuses, see Appendix D and the Performance Tables guidance on ‘Point
scores for tests and examinations’:
http://www.education.gov.uk/performancetables/schools_10/documents.shtml
The VA score for a pupil is then calculated as the difference (positive or negative) between
the model’s estimate for pupils like them nationally and their actual KS4 attainment. The VA
score for a school is then calculated as the average VA score of all pupils in the school (with
a small adjustment based on number of pupils – see below).
The VA score for a particular pupil group (e.g. FSM pupils) in a school is calculated as the
average VA score of all pupils that belong to the pupil group in the school. Similarly, the VA
score for a particular pupil group nationally is calculated as the average VA score of all pupils
that belongs to the pupil group nationally.
The models use the pupil’s KS2 average point score (using fine grading). Teacher
Assessment data is substituted for any KS2 test in which a pupil scores a level 2, a B (‘Below
the level of the KS2 Test’) or an N (‘Not awarded a Test level’). A complete description of the
adjustment is presented in the table in the ‘Notes’ section.
The models produce coefficients to be applied to the pupil level KS2 prior attainment
variables described below.
For a pupil:
The estimated KS4 attainment of the pupil, E p , is given by:

 
E p  c  c1  KS 2 APS   c2  KS 2 APS 2  c3  KS 2 APS 3
 c4  ENGDEV   c5  MATDEV 
where:

KS2 APS
is the pupil’s KS2 average point score (APS)
KS 2APS 2
is the pupil’s KS2 APS squared
KS 2APS 3
ci
is the pupil’s KS2 APS cubed
is the difference between the pupil’s KS2 English
score and their KS2 APS
is the difference between the pupil’s KS2
mathematics score and their KS2 APS
are the coefficients from the VA model
c
is the constant from the VA model
ENGDEV
MATDEV
The VA score of the pupil, VAp , is then calculated as the difference between their actual
result and their estimate ( E p ), given by:
VAp  Ap  E p ,
where:
is the pupil’s actual KS4 “Best 8” outcome, given by:
Ap
Ap  C8  Be  Bm ,
where:
is the pupil’s point score in their capped best eight GCSE
and equivalent qualifications
is the pupil’s point score awarded as bonus for
attainment in KS4 English
is the pupil’s point score awarded as bonus for
attainment in KS4 mathematics
C8
Be
Bm
Note that VAp scores are centred around 0.
For a school:
The school’s “Best 8” VA score, VAs , is given by:


VAs  1000  S  VA p ,
where:
is the “shrinkage factor” for the school
is the average VA score for all eligible pupils within
the school, given by:
S
VA p
ns
VA p 
VA
p 1
ns
p
,
where:
is the number of eligible pupils in the school
ns
ns
VA
p 1
p
is the sum of the VA scores of eligible pupils within
the school
The “shrinkage factor”, S , is an adjustment which provides a better estimate for VA scores
for schools with small numbers of pupils in the calculation, given by:
S
B
W
B
ns
where:
is the national variance between schools
is the national variance within schools
B
W
For a pupil group (e.g. FSM) within a school:
The pupil group “Best 8” VA score for any school, VAg , is given by:
VAg  1000  VA pg ,
where:
is the average VA score for all eligible pupils that
belong to the pupil group within the school, given by:
VA pg
npg
VA pg 
VA
p 1
p
,
n pg
where:
is the number of eligible pupils that belong to the
pupil group within the school
n pg
n pg
VA
p 1
p
is the sum of the VA scores of eligible pupils that
belong to the pupil group within the school
Note a “shrinkage factor” is not applied to pupil groups within schools.
For a pupil group nationally:
The national “Best 8” VA score for a pupil group, VAG , is given by:
VAG  1000  VAPG ,
where:
VAPG
is the average VA score for all eligible pupils that
belong to the pupil group nationally, given by:
n PG
VAPG 
VA
p 1
nPG
p
,
where:
is the number of eligible pupils that belong to the
pupil group nationally
nPG
n PG
VA
p 1
p
is the sum of the VA scores of eligible pupils that
belong to the pupil group nationally
Note a “shrinkage factor” is not applied to pupil groups nationally.
Statistical
significance
For a school:
A 95% confidence interval is calculated around the school’s “Best 8” VA score, defining the
range of values within which we are statistically confident that the true value of the school’s
VA score lies.
The confidence interval, denoted LowCI s ,UppCIs , is given by the formula:
LowCIs ,UppCIs   VAs  CI s ,VAs  CI s  ,
where:
LowCI s
is the lower confidence limit for the school’s VA score
UppCIs
is the upper confidence limit for the school’s VA score
VAs
is the school’s VA score
CI s
is the size of the confidence interval for the school, given by:
CI s  1.96 
B W
B  ns   W
For KS2-4 “Best 8” VA, the national average of all maintained Mainstream school scores is
1000.

When a school has LowCI s > 1000, the school’s VA score is above average and the
result is statistically significant (denoted “Sig+”).

When a school has UppCIs < 1000, the school’s VA score is below average and the
result is statistically significant (denoted “Sig-”).

In the other case when LowCI s < 1000 < UppCIs , we cannot say with confidence
whether the school’s VA score is above or below average, and say the result is not
statistically significant.
See ‘Notes’ section for calculation of Special school confidence intervals.
For a pupil group within a school:
A 95% confidence interval is calculated around each pupil group VA score for the school,
defining the range of values within which we are statistically confident that the true value of
the pupil group VA score for the school lies.
The confidence interval, denoted
LowCI
g
LowCI
g

, UppCI g , is given by the formula:
 

,UppCI g  VAg  CI g ,VAg  CI g ,
where:
LowCI g
UppCI g
VAg
CI g
is the lower confidence limit for the pupil group VA score
for the school
is the upper confidence limit for the pupil group VA score
for the school
is the pupil group VA score for the school
is the size of the confidence interval for the pupil group VA
score for the school, given by:
CI g  1.96 
N
n pg
where:
VAg
is the school’s VA estimate for that pupil group
N
is the standard deviation of the VA scores for all eligible
pupils nationally;
n pg
is the number of eligible pupils that belong to the pupil
group within the school;
We then test for significance by comparing the range of the confidence interval to the
national VA score for the pupil group in Mainstream schools, VAG .

When a pupil group within a school has LowCI g > VAG , the school’s pupil group VA
score is above the national average pupil group VA score and the result is statistically
significant (denoted “Sig+”).

When a pupil group within a school has UppCI g < VAG , the school’s pupil group VA
score is below the national average pupil group VA score and the result is statistically
significant (denoted “Sig-”).

In the other case when LowCI g < VAG < UppCI g , we cannot say with confidence
whether the school’s pupil group VA score is above or below the national average
pupil group VA score, and say the result is not statistically significant.
We are also interested in how the pupil group within the school performs compared to all
pupils nationally, hence we also test for significance by comparing the range of the
confidence interval to the national “Best 8” VA Mainstream pupil average, i.e. 1000.

When a pupil group within a school has LowCI g > 1000, the school’s pupil group VA
score is above the national average pupil VA score and the result is statistically
significant (denoted “Sig+”).

When a pupil group within a school has UppCI g < 1000, the school’s pupil group VA
score is below the national average pupil VA score and the result is statistically
significant (denoted “Sig-”).

In the other case when LowCI g < 1000 < UppCI g , we cannot say with confidence
whether the school’s pupil group VA score is above or below the national average
pupil VA score, and say the result is not statistically significant.
When comparing a school’s pupil group VA score to the national pupil group average and the
national average for all pupils, It could be the case that the VA score is statistically
significant in one result but not in the other, or indeed “Sig +” in one and “Sig –” in the
other. For example, a school’s VA score for their disadvantaged pupils could be “Sig +”
compared with the national VA score for disadvantaged pupils but still “Sig-” compared with
the national VA score for all pupils. In other words, the school’s disadvantaged pupils are
performing significantly better than disadvantaged pupils nationally, but are still performing
significantly worse than the average pupil nationally.
Note: these tests of statistical significance for pupil groups are different to those previously
calculated for VA.
See ‘Notes’ section for calculation of Special school pupil group confidence intervals and
significance testing.
Return
Format
Inclusion
within
RAISEonline
Decimal
Places
Number
Interactive Report
tbc
One
Summary Report
tbc
tbc
Data Source
Notes
tbc
School and College Performance Tables Frequency of
Two times per year
data
publication
Calculation of KS2-4 “Best 8” VA scores – two times per year
The “Best 8” VA model is calculated twice each year, using the actual attainment of the
pupils in those schools eligible for VA. The measure included in the first release of
RAISEonline reports each year is based on data that has not been fully validated with
schools (known as ‘unamended’ data). The model and scores are then recalculated using the
validated, finalised data (known as ‘amended’ data). Due to the relative nature of VA scores,
all schools will experience some change in their scores between the two releases, even if
their results have not changed for amended data.
Calculation of Special school VA scores and confidence intervals
The estimated KS4 attainment ( E p ) for pupils in Special schools is based on comparison
with pupils of the same prior attainment in Mainstream schools. This means that their VA
scores are calculated based on the model coefficients ( c i and c ) derived from Mainstream
schools only.
Similarly, confidence intervals special schools and their pupils groups are calculated using
the values from the Mainstream school model. Comparisons are then made to Mainstream
school national averages (1000 for the school VA scores).
Adjustment made to KS2 test scores using Teacher Assessment (TA) data
If test score =
3-5
2
Use pupil’s fine grade score
If TA available
Award:
W=3
Level 1 = 9
Level 2 = 15
Any higher = use pupil’s
fine grade score
A,D,F,L,P,Z = Exclude pupil
If no TA available
Exclude Pupil
B, N
If TA available
A, M, Q, T, X
If no TA available
If TA available
Award:
W=3
Level 1 = 9
Level 2 = 15
Any higher = 15 (capped)
A,D,F,L,P,Z = Exclude pupil
Exclude Pupil
Award:
W=3
Level 1 = 9
Level 2 = 15
Level 3 = 21
Level 4 = 27
Level 5 = 33
Any higher = 33 (capped)
A,D,F,L,P,Z = Exclude pupil
If no TA available
Exclude Pupil
Notes on grade codes
A – Absent
B – Working below the level of the test
D – Disapplied
F – KS2 pupil not at end of KS2 and taking this subject in future years
L – Left
N – Not awarded a test level
M – Missing
P – Results for subject found in previous year’s dataset
Q – Maladministration
T – Working at the level of the tests but not able to access them
X – Lost
Z – Ineligible
Worked
Example
(1) Calculation of a pupil’s “Best 8” VA score
A pupil at the end of Key Stage 4 has the following attainment:
Surname
Forename
KS2 fine grade average points score
KS2 English
KS2 mathematics
KS4 English points (GCSE grade)
KS4 mathematics points (GCSE grade)
KS4 points in capped best eight GCSE and
equivalent outcomes
KS4 English bonus points
KS4 mathematics bonus points
Jones
Gillian
31.54
30.18
31.44
46 (B)
52 (A)
412
46
52
Gillian’s estimated ‘Best 8’ attainment is calculated by inserting the following values,
reflecting her KS2 outcomes, into the formulae given above for E p :
Notation
KS2 APS
KS 2APS 2
KS 2APS 3
ENGDEV
MATDEV
Description
KS2 APS
KS2 APS squared
KS2 APS cubed
KS2 English minus KS2 APS
KS2 mathematics minus KS2 APS
Pupil value
31.54
994.77
31375.10
-1.36
-0.10
The table below presents the values for 2012 ‘Best 8’ VA model coefficients:
Coefficient
c
c1
c2
c3
c4
c5
Applied to
Constant applied to all pupils
KS2 APS
KS 2APS 2
KS 2APS 3
ENGDEV
MATDEV
Coefficient
183.795446
7.148033
-0.130326
0.006702
4.237709
1.992549
Gillian’s estimated ‘Best 8’ attainment, E p , is then calculated as:

 
E p  c  c1  KS 2 APS   c2  KS 2 APS 2  c3  KS 2 APS 3
 c4  ENGDEV   c5  MATDEV 

 183.795446  7.148033  31.54  - 0.130326  994.77
 0.006702  31375.10  4.237709  1.36  1.992549  0.10
 183.795446  225.448961 129.644395  210.275920  5.763284  0.199255
 483.91 (to 2 decimal places, or d.p.)
Gillian’s actual ‘Best 8’ attainment is given by Ap  C8  Be  Bm  412  46  52  510 .
Therefore, her VA score is given by:
VAp  Ap  E p  510  483.91  26.09 (to 2 d.p.).
(2) Calculation of a school’s VA score
Let us then say that Gillian is one of 100 pupils in her school’s KS4 cohort, who gain
a range of ‘Best 8’ VA scores:
Pupil #
1
2

100
Pupil name
Gillian
Lindsay

David
Sum
VA score
26.09
-2.04

32.75
986.35
The next step in the calculation is to calculate VA p , the average ‘Best 8’ VA score for
all eligible pupils within the school KS4 cohort:
ns
VA p 
 VA
p 1
ns
p

26.08  2.04    32.75  986.35  9.864
100
100
(to 3 d.p.)
We next calculate the “shrinkage factor”, using ‘Best 8’ VA amended model values
for B (459.532268) and W (4354.730348):
S
B
459.532268

 0.913 (to 3 d.p.)
W 
4354.730348 
B
 459.532268 

ns 
100

Hence the final ‘Best 8’ VA score for this school, VAs , is given by:


VAs  1000  S VA p  1000  0.913  9.864  1009.006 (to 3 d.p.)
Note: We would publish this score as 1009.0, but retain the decimal places for this
example for illustrative purposes for the confidence interval calculation.
(3) Calculation of a confidence interval around a school’s VA score
Using ‘Best 8’ VA model values for B (459.532268) and W (4354.730348), we can
also calculate the size of the confidence interval for the school’s ‘Best 8’ VA score,
based on the 100 pupils in Gillian’s school’s KS4 cohort:
CI s  1.96 
B W
B  n s   W
459.532268  4354.730348
 1.96  6.307  12.362 (to 3 d.p.)
459.532268  100  4354.730348
We derive the confidence interval as follows:
 1.96 
LowCIs ,UppCIs   VAs  CI s ,VAs  CI s 
 1009.001  12.362 , 1009.001  12.362  996.6 , 1021.4 (to 1 d.p.)
Hence, as LowCI s < 1,000 < UppCIs , we cannot say with confidence whether this
school’s ‘Best 8’ VA score is above or below average, hence the school’s VA score is
not statistically significant either side of the national average.
(4) Calculation of a pupil group VA score within a school
Let us then say that Gillian is one of 30 ‘disadvantaged’ pupils (defined, for
Performance Tables purposes, as pupils who are either eligible for Free School
Meals or are children who are looked after) among the 100 pupils in her school’s KS4
cohort, who gain a range of ‘Best 8’ VA scores:
Disadvantaged
pupil #
1
2

30
Disadvantaged
VA
pupil name
score
Gillian
26.09
Ross
-16.44


Alison
12.16
Sum 347.41
We calculate the disadvantaged pupil group VA score for the school, VAg , by
calculating the average VA score of the disadvantaged pupils within the school, as
follows:
n pg
VAg  1000  VA pg  1000 
 1000 
VA
p 1
p
n pg
26.08  16.44    12.16  1000  347.41  1011.580
30
30
(to 3 d.p.)
Note: We would publish this score as 1011.6, but retain the decimal places for this
example for illustrative purposes for the confidence interval calculation.
(5) Calculation of confidence intervals around a school’s pupil group VA score
Referring back to the disadvantaged pupil group example, we can then calculate the
size of the confidence interval for the school’s disadvantaged pupil group VA score
using CI g :
CI g  1.96 
N
n pg
 1.96 
69.190055
30
 1.96  12.632  24.759 (to 3 d.p.)
We derive the confidence interval for the school’s disadvantaged pupil group VA
score:
LowCI
g
 
,UppCI g  VAg  CI g ,VAg  CI g

 1011.580  24.759 , 1011.580  24.759  986.8 , 1036.3 (to 1 d.p.)
As LowCI g < 1,000 < UppCI g , we cannot say with confidence whether the school’s
disadvantaged pupil group VA score is above or below the national pupil VA score,
and say this result is not statistically significant.
We can also test for significance by comparing the range of the confidence interval to
VAG , the national VA score for the pupil group in Mainstream schools. The “Best 8”
VA amended score for FSM pupils nationally, VAG , has been calculated to be 981.9.
As LowCI g > 981.9, we can say with confidence that the school’s FSM VA score is
above the national FSM VA score, and this result is denoted “Sig+”. However, as
LowCI g < 1000 < UppCI g , we cannot say with confidence whether the school’s FSM
score is above or below the national pupil VA score, and say this result is not
statistically significant. In other words, the school’s FSM pupils are performing
significantly better than FSM pupils nationally, but we cannot say whether the
school’s FSM pupils are performing better or worse than the average pupil nationally.
Further
Guidance
Last Modified
Changes
since last
version
N/A
15/01/2013
Updated with 2012 amended VA model coefficients