Feedback from CEM
Assessments:
Individual Pupil Records &
Predictions
Belfast, March 6th 2013
Neil Defty
Business & Development Manager
CEM
[email protected]
Baseline Assessment Data
• To inform professional
judgement
• To start a conversation
USING BASELINE DATA FOR TEACHING AND LEARNING
IF FEEDBACK DATA IS TO BE TRUSTED THEN PROBABLY:
•
The students understood the purpose of the
assessments
• And each student did their best at the time…….
• As a year group, how able are they?
• What strengths and weaknesses does the group have?
• How did each student perform?
Bands, percentiles, standardised scores…
C B
A
D
0
50
60
70
80
90
100
110
120
130
140
Standardised scores
Percentiles:
1
5
10
20
30 40 50 60 70
25
80
75
90
95
99
150
Alis: year 12 students, two baseline profiles
for the same school
Nationally, 25% in each band
Band Profile Graph: all MidYIS cohort
A school with a ‘completely average’ intake would
have 25% of pupils within each band.
Band D
Checking this graph each year will give you an
immediate overview of your intake.
Band A
Vocabulary: Percent in each Band
Year 7 MidYIS Test 2011/2012
100
90
90
80
80
70
70
60
60
Percent
Percent
100
50
37
40
20
25
35
26
30
20
17
21
20
10
17
10
0
0
D
C
B
Band D
A
D
Band A
90
80
80
70
70
60
60
Percent
90
40
33
35
30
A
Band A
50
40
30
29
21
20
B
Skills: Percent in each Band
Year 7 MidYIS Test 2011/2012
100
50
C
Band D
Non-Verbal: Percent in each Band
Year 7 MidYIS Test 2011/2012
100
Percent
50
40
30
Maths: Percent in each Band
Year 7 MidYIS Test 2011/2012
24
26
21
20
11
10
10
0
0
D
C
B
A
D
C
B
A
Proof Reading: Percent in each Band
Year 7 MidYIS Test 2011/2012
100
90
80
70
Percent
60
Skills: Percent in each Band
Year 7 MidYIS Test 2011/2012
100
50
40
38
26
30
19
90
20
17
80
10
70
0
D
40
30
100
29
24
26
21
C
B
A
Perceptual Speed & Accuracy: Percent in each Band
Year 7 MidYIS Test 2011/2012
50
90
20
80
10
70
0
D
C
B
A
60
Percent
Percent
60
50
40
30
30
24
23
23
D
C
B
20
10
0
A
Scores Standardised On A Nationally Representative Sample of Schools
Standardised Scores
Sort Data By...
Mean Score: 100
Standard Deviation: 15
Average
100.6
95.9
105.3
99.2
98.1
St. error
1.1
1.1
1.1
1.1
1.1
Band
St. Score
MidYIS Score
Band
St. Score
Skills
Band
St. Score
Non Verbal
Band
Maths
St. Score
Band
St. Score
Vocabulary
On another day, with 95% certainty, the score of 100.6 (+/- 1.1 x 2)
• would not be higher than 100.6 + 2.2 = 102.8
• would not be lower than 100.6 – 2.2 = 98.4
100.6
95.9
105.3
99.2
98.1
93.7
102.2
1.1
1.1
1.1
1.1
1.1
1.1
1.1
134
95
99
93
116
93
92
102
A
C
C
C
A
C
C
B
119
97
104
94
99
95
83
102
A
C
B
C
C
C
D
B
116
88
112
120
118
120
112
112
A
D
A
A
A
A
A
A
87
85
110
91
84
113
94
108
D
D
B
C
D
A
C
B
130
95
101
93
109
93
87
102
A
C
B
C
B
C
D
B
101
76
97
80
102
119
80
106
B
D
C
D
B
A
D
B
83
93
114
99
78
107
102
107
Band
St. Score
Band
Proof Reading PSA
St. Score
Band
St. Score
MidYIS Score
Band
St. Score
Skills
Band
St. Score
Non Verbal
Band
Maths
St. Score
Band
St. Score
Vocabulary
D
C
A
C
D
B
B
B
Stanine 1
2
4%
50
60
3
4
5
6
7
8
9
7% 12% 17% 20% 17% 12% 7%
70
80
90
100
110
120
Standardised Test Score
4%
130
140
150
Individual Pupil Records (IPRs)
(See
www.cem.org/midyisiprbooklet
for report: “Using MidYIS
Individual Pupil Records to
Inform Teaching and Learning”)
• To start a
conversation
Individual Pupil Record
Date Of Birth:
Sex:
Vocabulary
Maths
Non-Verbal
Skills
MidYIS Score
75
Student Number:
Form:
24/12/97
F
Band
Stanine
Percentile
Standard
Score
B
5
53
101
B
5
52
101
C
5
46
98
C
5
46
99
B
5
52
101
Nationally Standardised Scores with 95% Confidence Band
Nationally Standardised Scores
150
140
130
120
110
100
90
80
70
60
50
Vocabulary
Maths
Non-Verbal
Skills
MidYIS Score
Sex:
Form:
F
Vocabulary
Maths
Non-Verbal
Skills
MidYIS Score
7C
Band
Stanine
Percentile
Standard
Score
C
4
27
91
B
6
68
107
B
6
75
110
D
2
10
81
C
5
45
98
Nationally Standardised Scores with 95% Confidence Band
Nationally Standardised Scores
150
140
130
120
110
100
90
80
70
60
50
Vocabulary
Maths
Non-Verbal
Skills
MidYIS Score
INSIGHT Pupil IPR
Speed Reading
Text Comprehension
Passage Comprehension
Overall Reading
Number & Algebra
Handling Data
Space, Shape & Measures
Overall Mathematics
Biology
Chemistry
Physics
Overall Science
Vocabulary
Non Verbal
Skills
Overall Ability
Standard
KS3
Score
Equivalent
Band
Stanine
Percentile
B
B
6
5
69
60
107
104
6c
5a
C
5
45
98
5b
B
5
59
103
5a
D
3
19
87
4a
B
D
6
4
61
23
104
89
6a
5c
C
4
31
93
5b
A
A
A
7
9
8
82
96
89
114
127
118
6a
7a
7c
A
8
93
122
7b
B
B
A
6
6
8
65
70
92
106
108
121
C
4
36
94
Overall Science
Physics
Chemistry
Biology
Overall Mathematics
Space, Shape & Measures
Handling Data
Number & Algebra
Overall Reading
Passage Comprehension
Text Comprehension
a
8b
c
a
7b
c
a
6b
c
a
5b
c
a
4b
c
a
3b
c
2a
N
Speed Reading
KS3 Level
IPRs in KS3 Levels
KS3 Equivalent Levels with 95% Confidence Band
NATIONAL AVERAGE
The code:
j: reduces size
k: infills
m: inverts
“Predictions”
How is a ‘prediction’ generated?
*
** *** ******************* A*
* ** *** ******************************** A
* ** *** ********************************* *** ** * B
C
GRADE
GRADE
* ** *** ************************************ *** ** *
* ** *** ******************************** **** ** *
* ** ***************************** *** ** *
* ** *** ******************* ** * *
* ** *********** ** *
50
100
BASELINE SCORE
150
3 key points are:
• The higher the baseline score the higher
the final grade
• Any one grade is achievable from a range
of baseline scores
• From any baseline score, a range of
grades are possible
How is a ‘prediction’ generated?
Subject National
trend line
*
(regression line)
50% on or above the
trend line
* ** *** ******************* A*
** *** ******************************** A
* ** *** ********************************* *** ** * B
‘PREDICTION’ (expected grade)
* ** *** ******************************** **** ** *
* ** ***************************** *** ** *
* ** *** ******************* ** * *
50% on or below the
* ** *********** ** *
trend line
BASELINE SCORE
C
GRADE
GRADE
* ** *** ************************************ *** ** *
2008 GCSE Maths (representative sample)
9
8
7
Result
6
5
4
3
2
1
0
0
20
40
60
baseline score
80
100
‘Predictions’…...are based on
Average performance by similar
pupils in past examinations
The problem with the word ‘prediction’ is…?
An alternative is ‘expected’ grade
Predictions
6.6 points = A/B
4 points = D
Trend line
FACTORS THAT WILL INFLUENCE
RELIABILITY OF PREDICTIONS:
•
•
•
•
•
•
•
•
•
•
Knowledge of student
Parental support/home life
Peer influences/social life
Student attitude, interest, language
Expectations of staff
Department/institution ethos
Resources
Quality of teaching and learning: pace of lessons
Understanding how children learn………
And the reliability of the predictions......
Result
Correlation = 1
29
Correlation = 0
Correlation = 0.7
CORRELATIONS
SUBJECT
ART and DESIGN
BIOLOGY
ECONOMICS
ENG LIT
FRENCH
HISTORY
MATHS
MUSIC
PHYS EDUC
PSYCHOLOGY
GCSE
to A2
0.54
0.69
0.65
0.71
0.6
0.69
0.6
0.65
0.65
0.62
MIDYIS( Yr 7)
subject
to GCSE
ART
0.49
BIOL
0.56
ENG LANG
0.7
FRENCH
0.64
HISTORY
0.63
MATHS
0.74
MUSIC
0.57
The graph below shows the middle 2/3 of some subject trend lines
MidYIS Year 7 to GCSE 2011
Value Added Analysis
Art & Design
8
Biology
7
English
GCSE Points Score
6
5
French
4
History
3
Mathematics
2
1
50
60
70
80
90
100
MidYIS Test Score
110
120
130
140
150
Science
Individual Chances Graph for Student 4- GCSE English
MidYIS Score 101 MidYIS Band B
Teacher's Adjustment : 0 grades / levels / points
45
39
40
35
Percent
30
Prediction/expected grade:
5.1 grade C
27
25
20
19
Most likely grade
15
9
10
5
5
0
0
U
G
1
1
0
F
E
D
Grade
C
B
A
A*
Not a label for life
...just another piece of information
• The Chances graphs show that, from almost any
baseline score, students come up with almost any
grade - - -there are just different probabilities for
each grade depending on the baseline score.
• In working with students these graphs are more
useful than a single predicted or target grade
• Chances graphs show what can be achieved:
– By students of similar ability
– By students with lower baseline scores
MidYIS Year 9 Test Academic Year 2011/2012
Predictions based on 2011 Regression Analysis
to place school at 75th
percentile of value added
results
Predictions & Chances Graphs to GCSE subjects
based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of
the method used to make these predictions
Points To Grades
Adjust
GCSE Point Predictions
Teacher's
Adjustment
(Grades)
Display Legend
4.7 5.8 6.1 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.6
3
F
91400042
108
B
4.6 5.6 6.0 6.1 5.4 6.1 5.5 5.4 5.6 5.7 5.2 5.4
4
F
91400043
101
B
4.2 5.0 5.6 5.7
4.8 5.7
Geography
A
French
111
English Literature
9140005K
5.1 5.9
English
F
Cust
Drama
2
CEM Id
Design & Technology
4.4 5.3 5.8 5.9
Form
Chemistry
B
Sex
Business Studies
105
Forename
Biology
9140005J
Surname
Art & Design
Additional Applied Science
F
Generate File for
MIS Import
Additional Science
MidYIS Band
1
Prior Value
added
Student
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
MidYIS Score
Upper Quartile
of Schools
5.3 5.2 5.4 5.5 5.0 5.1
5.0 5.0 5.1 5.3 4.7 4.7
MidYIS Year 9 Test Academic Year 2011/2012
Predictions based on 2011 Regression Analysis
Predictions & Chances Graphs to GCSE subjects
based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of
the method used to make these predictions
Points To Grades
Adjust
GCSE Point Predictions
Teacher's
Adjustment
(Grades)
Display Legend
4.9 6.0 6.3 6.5 5.8 6.5 5.9 5.8 6.0 6.1 5.6 5.8
3
F
91400042
108
B
4.8 5.8 6.2 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.6
4
F
91400043
101
B
4.4 5.2 5.8 5.9
5.0 5.9
Geography
A
French
111
English Literature
9140005K
5.3 6.1
English
F
Cust
Drama
2
CEM Id
Design & Technology
4.6 5.5 6.0 6.1
Form
Chemistry
B
Sex
Business Studies
105
Forename
Biology
9140005J
Surname
Art & Design
Additional Applied Science
F
Generate File for
MIS Import
Additional Science
MidYIS Band
1
Prior Value
added
Student
0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
MidYIS Score
Reset
Adjustments
5.5 5.4 5.6 5.7 5.2 5.3
5.2 5.2 5.3 5.5 4.9 4.9
MidYIS Year 9 Test Academic Year 2011/2012
Predictions based on 2011 Regression Analysis
Predictions & Chances Graphs to GCSE subjects
based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of
the method used to make these predictions
Points To Grades
Adjust
GCSE Point Predictions
Teacher's
Adjustment
(Grades)
Display Legend
4.7 5.8 6.1 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.6
B
4.6 5.6 6.0 6.1 5.4 6.1 5.5 5.4 5.6 5.7 5.2 5.4
4
F
91400043
101
B
4.2 5.0 5.6 5.7
4.8 5.7
Geography
A
108
French
111
91400042
English Literature
9140005K
F
5.1 5.9
English
F
3
Cust
Drama
2
CEM Id
Design & Technology
4.4 5.3 5.8 5.9
Form
Chemistry
B
Sex
Business Studies
105
Forename
Biology
9140005J
Surname
Art & Design
Additional Applied Science
F
Generate File for
MIS Import
Additional Science
MidYIS Band
Or insert own values
and click Adjust
1
Prior Value
added
Student
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
MidYIS Score
Upper Quartile
of Schools
5.3 5.2 5.4 5.5 5.0 5.1
5.0 5.0 5.1 5.3 4.7 4.7
MidYIS Year 9 Test Academic Year 2011/2012
Predictions based on 2011 Regression Analysis
Predictions & Chances Graphs to GCSE subjects
based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of
the method used to make these predictions
Points To Grades
Adjust
GCSE Point Predictions
Teacher's
Adjustment
(Grades)
Display Legend
B
4.6 5.9 7.4 6.8 5.4 6.8 5.5 6.3 6.9 6.5 5.9 6.8
4
F
91400043
101
B
4.2 5.3 7.0 6.4
4.8 6.4
Geography
108
French
91400042
English Literature
F
5.1 6.6
English
3
Cust
Drama
4.7 6.1 7.5 7.0 5.6 7.0 5.7 6.5 7.1 6.7 6.1 7.0
CEM Id
Design & Technology
4.4 5.6 7.2 6.6
A
Form
Chemistry
B
111
Sex
Business Studies
105
9140005K
Forename
Biology
9140005J
F
Surname
Art & Design
Additional Applied Science
F
2
Generate File for
MIS Import
Additional Science
MidYIS Band
1
Reset
Adjustments
Student
0.0 0.3 1.4 0.7 0.0 0.7 0.0 0.9 1.3 0.8 0.7 1.4
MidYIS Score
Upper Quartile
of Schools
5.3 6.1 6.7 6.3 5.7 6.5
5.0 5.9 6.4 6.1 5.4 6.1
MidYIS Year 9 Test Academic Year 2011/2012
Predictions based on 2011 Regression Analysis
Predictions & Chances Graphs to GCSE subjects
based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of
the method used to make these predictions
Adjust
A
A/B
B
A
B/C A/B
A
A/B
B
A
A/B
C
A/B
A/B
B
B/C
B
2
F
9140005K
111
A
C/D
3
F
91400042
108
B
C/D
4
F
91400043
101
B
D
Cust
B/C
A
A/B B/C
C
B
B
Geography
B/C A/B
A
C/D B/C
CEM Id
French
A
B/C
B
Form
English Literature
B/C
A
105
Sex
English
Business Studies
A
B A*/A
9140005J
Forename
Drama
Biology
B A*/A
F
Surname
Design & Technology
Art & Design
C
MidYIS Band
A/B
1
Generate File for
MIS Import
Student
A
MidYIS Score
Reset
Adjustments
0.0 0.3 1.4 0.7 0.0 0.7 0.0 0.9 1.3 0.8 0.7 1.4
Additional Science
Upper Quartile
of Schools
Additional Applied Science
Teacher's
Adjustment
(Grades)
GCSE Grade Predictions
Chemistry
Grades To Points
Display Legend
A/B A/B B/C A/B
Prediction/expected grade:
6.4 grade A/B
Most likely grade
MidYIS Year 9 Test Academic Year 2011/2012
Predictions based on 2011 Independent Sector Regression Analysis
Predictions & Chances Graphs to GCSE subjects
based on Year 9 Independent Sector standardised scores
Please refer to the Information Sheet for details of
the method used to make these predictions
Points To Grades
Design & Technology
C
6.1 6.8 6.7 6.0 6.7 6.1 6.4 6.4 6.5 6.4 6.1 6.5 6.2
3
F
91400042
90
C
6.0 6.7 6.6
5.9 6.6
4
F
91400043
81
D
5.5 6.5 6.1
5.4 6.1 5.6 5.8 5.9 5.9 5.9 5.4 5.8 5.6
German
94
Geography
9140005K
French
F
5.7 6.4
English Literature
2
Cust
English
5.8 6.6 6.4
CEM Id
Drama
D
Form
Chemistry
87
Sex
Business Studies
9140005J
Forename
Biology
Additional Science
F
Surname
Art & Design
MidYIS Band
1
Generate File for
MIS Import
Student
Classical Civilisation
GCSE Point Predictions
MidYIS Score
Display Legend
5.8 6.1 6.1 6.1 6.2 5.7 6.1 5.8
6.0 6.3 6.3 6.3 6.3 5.9 6.3 6.0