Senior Verifier Event
26th August 2014
Welcome
Senior Verifier Event
Matthew McCullagh
Quality Manager
Past, Present and Future





Housekeeping - Lowden
New Approach to QA
Improvements & Benefits
This session– staying same but improving practice
Moving forward 2015/16
Senior Verifier Event
Welcome
Recap of previous session
Approval Guidance
Tea/Coffee Break
Workshop 1 – Quality Assurance Criteria
ASV Comments
Referencing Evidence
Electronic Evidence
Lunch
Decision Logs and Standardisation
Moving forward - session 2014/15
Tea/Coffee Break
Workshop 2 – Senior External Verifier Role
Plenary
Close
Senior Verifier Event
Juliette McGinley
QA Logistics Manager
Re-cap of previous session
SQA’s Approach to QA Events
 880 Visit Plans received in session 2013/14
Session
Completed
Visits
Ongoing Suspension
of
Certification
Not
Running
2013/14
1677
75
127
327
2012/13
749
-
51
293
SQA’s Approach to QA Outcomes
2013/14
 1513 compliant SQA’s Approach to QA visits
 164 non-compliant SQA’s Approach to QA
visits, of which 121 now compliant
Significant
Strengths
Strengths
Some
outweigh
Strengths
Weaknesses Some
Weaknesses
Weaknesses
outweigh
Strengths
Significant
Weaknesses
1513
21
5
3
135
Criteria Compliance Levels 2013/14
Criterion
2.3
2.4
2.8
3.2
4.1
4.2
4.3
4.5
4.6
4.7
4.8
6.6
Impact
Low
High
High
Medium
Medium
High
High
High
High
High
Medium
Low
Green
1242
1308
1645
1653
1609
1608
1587
1660
1663
1670
1656
1664
Amber
30
33
20
10
60
63
81
14
8
5
9
4
Red
1
2
3
0
4
3
4
1
0
0
4
0
Quality Framework and NQ Visits
Status Completed
Visits
Not
Running
Holds
12/13 13/14 12/13 13/14 12/13 13/14
Type
Visiting NQ
183
379
176
72
9
14
Visiting
HN/SVQ
1461
245
1041
22
48
10
QA Management System
 1752 visits allocated and recorded using QAMS
 Senior Verifiers provided feedback from teams
QAMS Feedback - Positives
The ability to log
in and check own
and also read (as
Senior EV) team
reports is helpful.
It allows team
support and
discussions at an
early stage.
The system is
straightforward
and simple to use
during a visit. The
facility to
complete reports
electronically is
useful.
When you have to
update an Evidence
Report you only get to
see the Amber or Red .
All team said
the prepopulation of
centre
information.
The electronic
submission
relieves me of
data protection
issues as it is
always with
SQA. Also, it is
so much easier
to use than the
old report
I think the system is brilliant
and the best so far having
worked with SQA for nearly
twenty five years now! I can't
think of any improvements at
this point but can give more
feedback as we progress
with it.
QAMS Feedback - Concerns
The requirement to
create the report on line
remains an issue at
certain times and in
certain parts of the
country where there is
very limited or very slow
internet, or is time
restricted with
insufficient time to
complete the report.
Layout of
report as it is
too “ boxy “
and does not
read in a
natural flow.
It needs to be
reported that
the "scribble"
form is not in
the exact order
of the actual
report.
I would like more
save buttons.
Continually having to
scroll up to save is
infuriating and I have
had the system crash
on me several times
thus losing the data
recently added.
Please could all information be
gathered together in one place?
Moving from EV+ to QAMS, to the
other locations and processing paper
allocations is very frustrating.
QAMS Feedback – Improvements
Some built in/fully
integrated guidance
notes on completion of
comments section would
be helpful. An App would
also be good.
Have visit
plans
integration
added so that
further details
can be
automatically
generated.
The ability to
access
previous
reports to refer
to, especially
in cases of
previous holds
etc.
Spellcheck
/grammar check
facility such as
that found on
word to help
highlight
problems.
It would be useful if the
scribble pad was an exact
copy of the report then cut
and paste would simplify
the reporting process.
I strongly agree on the issues of
consolidating EV+ and QAMS. I would
go further to say that information,
forms and paperwork, as well as
allocations and reports, do seem to be
quite scattered across SQA sites and
complete centralisation would be a big
move forward for Verifiers, particularly
new ones.
QAMS Feedback
67 SEV contacted for feedback, 33 responses to date
 Most positive features
– Pre-populated centre information
– Ability for Senior EV to view all team’s reports at all
times
– Auto calculates Outcome Rating
– Can see at a glance what the non-compliant criterion
are
QAMS Feedback
 Most common improvements suggested
– Auto-save/more Save buttons
– Spell Check (use Firefox or most up to date IE)
– One system for all QV activity/removal of EV+
– Streamline scribble pad with online report (in
progress)
Senior Verifier Event
David Pirnie
Lead Verifier – HN & VQ
Qualification Verification
Qualification Verifier practice – a general update
 Approval Guidance for Qualification Verification.
 Qualification Verification queries from Session
2013/2014.
 Quality Assurance Exercise – seeking your feedback.
Approval Guidance Qualification
Verification
Approval Guidance follows this format:
 Quality Assurance Criteria by section, for example,
Resources, Candidate Support.
 Evidence required from the applying centre in relation to
the Quality Assurance Criterion in each section.
 Support guidance in relation to each Quality Assurance
Criterion.
Quality Assurance Criterion
Resources
2.4 Records must be maintained to provide evidence that the centre has
sufficient competent staff who have the necessary qualifications,
occupational experience and understanding to support the assessment and
internal verification of qualifications being offered in the centre.
Application Evidence
 B4.1: Staff Details: for each staff member listed, you need to complete the
supporting document: ‘Staff Qualifications and Experience’ listing your staffs’
qualifications (full titles rather than abbreviations), telling us where and when
they were gained and a brief outline of the experience they have in relation to
the qualification/units they will be assessing and/or verifying.
 Original qualification certificates or copies should be
provided at the approval stage for assessors and verifiers
of SVQs/QCF Units.
Support guidance
HN Unit, Assessors and internal verifiers need to be competent in the
subject/occupational area to a level appropriate to the qualification.
 This means providing evidence of experience and current practice in
line with the qualification approval being sought. In addition, a relevant
qualification in the subject area would be best practice but not essential
unless a specific requirement of the unit specification or assessment
strategy.
 HN assessors and internal verifiers also need to be competent in
assessment (or internal verification) of the type involved in the
qualification. This means providing experience of current assessment/
verification practice in line with the qualification requirements.
 A relevant qualification in assessment/verification
would be best practice but not essential unless . . .
Approval Guidance - features
 Where necessary, information is grouped separately for
HNs, SVQs and QCF Units.
 Guidance more detailed in terms of the
information/evidence being sought at the approval stage.
 We would like you to discuss with your groups in the
October/November QV Group Standardisation Meetings.
Quality Assurance Guidance
 A minor update of this guidance and the Qualification
Verification Guidance will be made for Session
2015/2016.
 Feedback on these guidance documents can be
communicated through the QA Appointee Website AT
ANY TIME.
Example of queries processed and
turned into FAQs this session
 Is there still a requirement for the external verifier to
sample the work of the internal-verifier-candidate for
L&D11 as there was with V1?
 Can you give some more examples of what is meant by
‘Good Practice’?
 If I achieve QCF Assessor/Verifier qualifications, would
they be recognised if I then went on to
assess SVQs?
FAQs location
 FAQs currently found on the QA Appointee
Website under QV Queries.
http://www.sqa.org.uk/qaappointee
 The same Website should be used for posting
any queries you/your team would like a
response to concerning Qualification Verification
requirements/guidance.
Queries highlighted in QV Reports
last session
 Red applies when absolutely no evidence is available.
 At times different judgements are made relating to
Criterion 6.6.
 Clarification needed on when to complete ASV & QV
boxes.
 Criteria 4.1, 4.2 & 4.3 – ensuring comments match each
criterion.
 Assessor/IV table at times incomplete.
Senior Verifier Event
Workshop 1
Quality Assurance Criteria
Senior Verifier Event
Barbara Irvine
Quality Enhancement Manager
SQA’s Approach to Quality Assurance
Quality Enhancement Managers
Five QEMs working geographically across Scotland





Sarah McRoberts – South West
Barbara Irvine – South East
Brian Stokes – Glasgow, Argyll & Bute
Pamela Hosey – North of Scotland
James Burns – Central Belt
SQA’s Approach to Quality
Assurance: The Four Processes
 Systems Approval
 Qualification Approval
 Systems Verification
 Qualification Verification
Criteria for Systems Verification






Management of a Centre – 12 (all)
Resources – 5 (5/7)
Candidate support – 7 (7/8)
Assessment and Internal verification – 5 (5/9)
External assessment – 0
Records and Data management – 8 (8/9)
Total of 37 criteria to be reviewed during these visits
Qualification Verifier Reports
Referral to ASV section
 What is the purpose of this box?
 What type of information should you be putting in this
box?
Information the QEM is looking for
Any issues you may identify as centre having problems
with a particular system in the centre, for example




internal verification system
Issues with system for candidate induction
Issues with appeals and/or complaints policy
Issues with partnership arrangements
Senior Verifier Event
Donnie Carthew
Senior External Verifier
Referencing Evidence
Senior Verifier Event
Welcome Back
Senior Verifier Event
Gavin Rees
Senior External Verifier
QV Standardisation Decision Record
Decision Log
 Used since 2012
 About 8 entries long
Decision Log
 Content
– Title
– Discussion
– Agreement
– Date
 General points/Specific details
 Running Text Document
 <link to log>
Decision Log
 Why
– Standardise approaches to assessment where
practitioner approaches vary
– Helps centres meet national standard
– Agree seriousness of issues
– Helps get new verifiers up to speed quickly
– Can be incorporated into IAR where appropriate
– Inform future unit/ASP development
Decision Log
 How
– Ask the team to keep a record of any
discussion arising from experience as
practitioners, visits or prior verification
– Chat through the issues at annual meeting
– Encourages discussion amongst experienced
practitioners
– Write up after the event
Decision Log
 Conclusion
– Help centres
– Help team keep track and agree approaches
– Establishes consistency and national
standard
– Reduce risk of dispute
– Not onerous
Senior Verifier Event
Juliette McGinley
QA Logistics Manager
Moving forward – session 2014/15
Moving Forward





QA Management System
Induction for appointees
Senior Without Portfolio – Pilot update
Standardisation Meetings
Appointee Website
 Workshop 2 - Senior Verifier role review
QA Management System
 IT review of feedback
 Improvements
 Timescales
Induction – 3 Stage Process
 Stage 1 - Pre Induction Day Course Work
 Stage 2 - Induction Day
 Stage 3 - Post Induction Day Case Study Work
Stage 1 - Induction
5 Modules of e-learning:
 Module A - Qualification Verification Practice
 Module B - Roles and Responsibilities
 Module C - Assessment: The purpose, principles and
practice
 Module D - Internal Verification: Purpose Roles and
Practices
 Module E - Quality Assurance Criteria
Final Multiple Choice Assessment – passmark 75%
Stage 2 - Induction





Prepare for Qualification Verification
Planning for Qualification Verification
Carry out Qualification Verification
Communicate Qualification Verification Decisions
Completing Qualification Verification Report
Stage 3 - Induction
 Sign off from Stage 2
 Case Study to complete a QV Visiting report and visit
plan
Year 1
 Senior Verifiers will be notified of new appointees after
successful sign off at stage 2
 Case Study will be reviewed internally however each
Senior Verifier will be notified of the outcome of induction
and case study
Senior Without Portfolio (SWP)
Senior Without Portfolio Piloted 2013/14
 9 SWP in total
 Mentored 104 Singleton appointees
 21 Double bank visits
 55 required full support
Senior Without Portfolio - The role explained
Overall
Identify and agree individual support needs
Organise support needs where required
Provide advice, support and guidance in
relation to qualification verification
Monitor progress
Accompany on double bank visit/mentor role
Senior Without Portfolio - The role explained
Planning
 Familiarisation with templates and documents
 Review of centre allocations and visit dates arranged
 Reminders for candidate lists
 Advice and guidance given on how to complete the
visit plan
 Visit plans being reviewed prior to issue to centre
Senior Without Portfolio - The role explained
Feedback from EVs with SWP
Reports being reviewed prior to submission to
SQA
Suggestions to enhance report
Clarification sought on points being made to
remove ambiguity
Senior Without Portfolio - The role explained
Continued Support
 Assistance with EV+
 QA Appointee Website
 Assist with arranging alternative mentor
 Use of scribble pad and manual calculations
Mentors
 To assist appointees with the move to SQA’s approach
to quality assurance, SQA has 7 appointees who
undertook mentor activity for session 2013/14
 Mentors carried out 44 Double Bank Visits
Standardisation Meetings
 No EV update this year
 All Saturdays in October and first Saturday in November
block booked in Optima for singletons
 QA Logistics Team support at events
 Qualification Team support at events, where applicable
 Final submission date is Wednesday 19th September for
booking event
 Complete online TARE forms
Standardisation Meetings
Dates
Saturday 4th October 2014
Saturday 11th October 2014
Saturday 18th October 2014
Saturday 25th October 2014
Saturday 1st November 2014
Standardisation
 Online request for Standardisation Meetings – must
come from SEV
 Annual Standardisation Meetings will be scheduled
in line with QAMS to allow for training
 Standardisation Meeting Minutes and/or Decision
Logs must be submitted to [email protected] within
10 days of meeting
 Fees and expenses will be paid upon receipt of
Standardisation paperwork
Final Updates
 New communications area under construction – users
should subscribe to be alerted to new content
 QV Calculator “App” now available on website.
Encourage EVs in groups to use this
 Online Standardisation Meeting Request
 Online TARE form also under construction. Users
should select either Approval or Verification visit type
which will direct the content to correct department
automatically
 IAR Report Deadline date 29th August 2014
China Verification Activity
Activity schedule
Visiting Verification
 January 2015
 March 2015
 September 2015
Graded Unit Central Events
 May 2015
 July 2015
Senior Verifier Event
Workshop 2
Senior External Verifier Role
Workshop Activity
Senior Verifier Role - feedback
As a Senior Verifier with SQA, your responsibilities will eventually cover:
 The monitoring of Qualification Approver/Verifier practice (this currently only applies to central
verification)
 The intention, moving forward to Session 2015/2016, is for Senior Verifiers to be well placed to
fully meet this key responsibility
Group discussion
At present, the monitoring of Qualification Verifiers mainly focus on the Senior Verifier reviewing the
contents of QV Reports. However, in order to widen this to include the monitoring of Qualification
Verifier practice (practice includes preparing for, planning, carrying out visits, making decisions,
communication, report writing and contribution to team standardisation), there needs to be thought
given as to how monitoring activities will be organised to ensure practice is monitored effectively.
Please discuss in your group the processes/resources you would like to see being put in place to
support you with these monitoring activities. The main points of the discussion will be recorded by a
facilitator assigned to your group.
Discussion time: 30 minutes