Validation, Annual Review and
Periodic Subject Review
at the University of Northampton
Dr Anne Craven,
Head of Quality and Academic Partnerships
25 April 2014
Validation
External Context
• The University is required to meet the
expectations set down in the QAA’s UK Quality
Code for Higher Education
• Of particular relevance are:
– Chapter B1: Programme design, development
and approval (October 2013)
– Chapter B8: Programme monitoring and
review (October 2013)
• QAA Higher Education Review due to take place
in 2015 – we will need to provide evidence that
we do this
Purpose of Validation
• To assess the quality and standards of proposed
new programmes (particularly the curriculum)
• To ensure that the new programmes align with
the University’s strategic direction (i.e. Raising
the Bar, Ashoka-U and the move to Waterside
• To ensure that the new programmes align with
the QAA Subject Benchmark Statements, PSRB
requirements and the needs of employers
• To ensure that the Programme Team is
appropriately equipped to deliver a high quality
student experience
Underlying Principles
• Validation is underlined by the principle of external
peer review – absolutely essential
• Student engagement: student Panel members,
student focus groups and evidence of student
involvement of the programme development
• The importance of the appropriate consideration of all
‘Education with Others’ provision
• Strategic fit and direction with the University’s
strategic plan (particularly in relation to employability,
social enterprise and internationalisation)
• Proportionality and responsiveness
• Accountability – ultimately Senate is responsible
Key Validation Issues
• This is not an exhaustive list:
– Evidence of a real market for the programme and its fit
with the strategic direction of the School and University
– The programme design and curriculum (academic level,
progression between levels, diversity of assessment
strategies, coherence of modules and the Award Map
– The provision of appropriate resources (staffing, books,
on-line resources, laboratories, specialist equipment etc)
– The preparation of students for employment
– The quality of the student experience overall (i.e. access
to Personal Academic Tutors, support for international
students etc)
First Steps
• School based planning process
• Development approval
– University Management Team
– If collaborative provision: approval of both institution and
programme
– Throughout the process
• Due Diligence
• Assessment of risks
Validation Processes
•
•
•
•
•
•
•
•
•
•
From 2013-2014 the majority of validations will be held ‘online’ – using an iterative process
TUNDRA 2 used as a forum to conduct ‘on-line’ consultation
Still a robust consideration of a rationale and curriculum
documentation
An External Academic Advisor to provide advice to the
Programme Team from the earliest opportunity
An External Academic Reviewer to review the final
documentation and assure that best practice has been carried
out
There will still be an internal Chair to facilitate the process
A QAP Officer will monitor the process
PSRB-related validations might still require an ‘event’
Student-focus groups will be used to obtain the student view
All Panels to have an External Industry Advisor
Role of Panel members
• To provide their peer academic expertise (academic Panel
members) to positively critique the provision being
considered
• To identify areas that need further exploration (i.e. issues
that may need addressing to ensure the best quality
student experience)
• To identify examples of good practice and enhancement
that can be disseminated more widely
• To confirm that the standards are equivalent to other
programmes at their own institutions (externals)
• To ensure that the programme will meet the needs of
employers and enhance the employability of students
• Above all to be ‘Critical Friends’ – constructive not
confrontational
Questions to Ask at
Validation (some examples)
• Some examples of areas to explore:
– The variety of assessment strategies (any innovative
forms?)
– Evidence of progression between levels (Learning
Outcomes)
– What is the strategic fit of the programme?
– What is the strategic direction of travel?
– How will the employability of students be enhanced?
– Is there a real and sustainable market for the students?
– Will there be adequate resources (books, journals, staff
etc) to provide a quality experience?
– How will the student voice be heard?
Other
Considerations
• Lifelong Learning (PT study, timetabling,
equivalent qualifications)
• Accreditation of prior learning
• Internationalisation; collaborative provision
Sources of Information
• Quality Assurance Agency website:
– www.qaa.ac.uk
• Quality and Academic Partnerships:
– Contact [email protected]
– The Validation Handbook being amended to
reflect introduction of ‘on-line’ validation
processes from 2013-2014
Annual Review
Annual Monitoring
• Refocused on the ‘module’ and
the ‘programme’
• Efficient and risk-based
approach
• Refocus to action planning
• Philosophy of continuous
improvement
• More intelligent use of
institutional data
• Integration with ‘Raising the
Bar’
• Ownership of quality at the
individual staff and team level
Where next? Role of Programme Leaders in quality enhancement and
student experience. Module evaluation (EvaSys). Northampton Online
Framework. Programme of Auditing NILE sites.
The Dashboard
B/M
Red <80%
Red <C
Total
Total Pass
Pass %
% Pass
Indicator
STD
77
66
86%
v
B-
u
EYS1012
STD
76
71
93%
v
C
v
EYS1013
STD
76
74
97%
u
C+
w
EYS1013
STDCY
28
28
100%
u
C+
w
EYS1014
STD
76
67
88%
v
C+
w
EYS1110
LC
15
15
100%
u
C+
w
EYS1110
LS
23
23
100%
u
C
v
EYS1110
STD
26
26
100%
u
C
v
EYS1110
STDB
27
27
100%
u
C+
w
EYS1111
LC
15
15
100%
u
C+
w
EYS1111
LS
23
23
100%
u
C+
w
EYS1111
STD
27
26
96%
u
C+
w
EYS1111
STDB
26
26
100%
u
C+
w
EYS1112
LC
15
14
93%
v
B-
u
EYS1112
LS
23
23
100%
u
B-
u
EYS1112
STD
26
26
100%
u
C+
w
EYS1112
STDB
27
27
100%
u
C+
w
606
577
95%
HE Level
Module
Session Code
4
EYS1011
HE Level 4 Total
B/M 90%
01/08/2011
FIELD OF EARLY YEARS
Av Pass
Grade
C+
Pass Grade
Indicator
The Dashboard –
Comparability of Session Codes
Module av.
Indicator
HRM3011
DL
HRM3011
STD
1
Module av.
Indicator
0%
43
40
0%
93%
Average
(arrow
compares
field av.)
B-
25
58%
47%
B-
Module av.
Indicator
Average
(arrow
compares
field av.)
Module av.
Indicator
29%
Module av.
Indicator
Module av.
Indicator
LEI3006
DL
2
2
100%
B+
LEI3006
DL02
1
1
100%
C-
0%
LEI3006
DLBN
5
2
40%
D+
0%
LEI3006
DLH
31
17
55%
C
0%
LEI3006
DLH02
11
9
82%
D+
0%
LEI3006
JAN
90
75
83%
C
23
26%
LEI3006
STD
173
150
87%
C+
60
35%
173
150
87%
C
60
35%
2
100%
Dashboard
• Red = below threshold. This requires a commentary and a
SMART action to remedy and improve.
• Green = exceeding benchmark. This may (where
appropriate) require a commentary and a SMART action for
the dissemination of good practice.
• Amber = below threshold but normally relates to small
numbers (less than 10). This does not require immediate
action but a continuing trend will attract an action for
improvement.
• Black = between threshold and benchmark– no action
required.
Periodic Subject Review (PSR)
External Context
• The University is required to meet the expectations set
down in the QAA UK Quality Code for Higher Education
• Of particular relevance to PSR are:
– Chapter B1: Programme design, development and
approval (October 2013)
– Chapter B8: Programme monitoring and review (October
2013)
• QAA Higher Education Review (HER) due to take place in
2015 – the University will need to provide evidence that we
do this
Purpose of PSR
• To allow Schools and Subject Teams to assess the
continued ‘health’ and appropriateness of existing
programmes
• To ensure on-going alignment with QAA Subject Benchmark
Statements, PSRB and employers needs
• To assess the future strategic direction of the programmes
- particularly in relation to Raising the Bar and Ashoka-U
• To test that there are appropriate resources/strategies in
place to ensure the effective management of student
learning opportunities
• To identify examples of good practice and enhancement to
disseminate to the wider University
• To ensure that any Education with Others (EWO)
arrangements remain appropriate
• To ensure that public information is correct and accessible
Underlying
Principles
•
PSR is underlined by the principles of both external and internal peer review –
absolutely essential
•
The importance of student engagement:
•
Recent graduate and (from 2014-2015) a current student on all PSR Panels
•
Panels meet with a representative sample of students during the event
•
The importance of the appropriate consideration of all ‘Education with Others’
provision
•
Strategic fit and direction (and sustainability)
•
Currency/coherence of the provision since the last PSR (taking into account
changes since the last PSR)
•
Ensuring staff research informs the curriculum
Key PSR Issues
• This is not an exhaustive list:
• The appropriateness of the strategic direction of the
programmes and their fit with Raising the Bar etc
• The currency of the programmes - particularly in
meeting employers, PSRBs and students’ needs
• The identification of actions arising from the Annual
Review process and student surveys/evaluations and the
extent to which these are being carried out
• The on-going appropriateness of ‘Education with Others’
provision and arrangements (comparability with on-site
provision)
• Ongoing development of the programmes – ‘futureproofing’
PSR Panel
constitutions
• Panel Chair - normally an Executive Dean or Deputy
Dean from a School outside of the Subject area
• Three internal Panel members (two from other
Schools outside of the Subject area)
• Employer representative (may be a representative of
a Professional/Statutory/Regulatory Body)
• Recent graduate from the same Subject area
• From 2014-2015: a current student from a ‘pool’ of
trained student reviewers (will be from outside of the
Subject area)
• An Officer – normally from within Quality and
Academic Partnerships
PSR Documentation
•
Self-Evaluation Document (previously a Critical Appraisal) – a critical
reflective document, identifying both good practice and areas to address
•
Curriculum documentation – Programme Specifications, Module
Specifications and Award Maps
•
Subject Team CVs
•
Library and Learning Services Supporting Statement
•
BIMI data (e.g. enrolment, retention, achievement, NSS)
•
Links to indicative reading lists, Programme and Module Guides on
TALIS/NILE
•
Marketing materials
•
External Examiner reports
•
Supporting EWO documentation (Memos of Co-operation)
Questions asked at PSR
(some examples)
•
Assessment strategies – sufficiently varied and innovative?
•
Evidence of progression between academic levels (demonstrated by
the Learning Outcomes)?
•
What does trend analysis from the data such as enrolments and
retention reports reveal?
•
What is the strategic fit of the programmes?
•
What is the strategic direction of travel?
•
What is the KIS data telling potential students about the programmes
•
Is there evidence of student feedback being addressed?
•
How does staff research inform the curriculum?
Sources of
Information
• Quality Assurance Agency website:
• www.qaa.ac.uk
• Quality and Academic Partnerships:
• Contact [email protected]
• Periodic Subject Review Handbook
Thank you; any questions?