SINGAPORE WORKFORCE SKILLS QUALIFICATIONS SYSTEM (WSQ)
GUIDE TO CONTINUOUS IMPROVEMENT REVIEW (CIR)
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 1
Contents at a Glance
Introduction
Part I:
Part II:
Part III:
Annex A :
Annex B :
Annex C :
Annex D:
Annex E :
Content
Overview of Quality Assurance Framework
Continuous Improvement Review (CIR)
– Essential Elements of Review
Roles and Responsibilities of Approved Training Organisation
(ATO)
Steps-by-Steps Process for ATO
Self-assessment Checklist
2 Year Plan
CIR Presentation by ATO
CIR Outcome Measures
Pg
4
8
17
24
29
38
40
42
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 2
Introduction
The field of adult training is about providing quality courses to cater to the learning
needs of the adult. A quality course would be able to enhance the learning
experience of the adult and contribute to the adult in more ways than we can ever
imagine. Some examples of the contribution would be 1) increase employability, 2)
improve skills set, and 3) innovate work processes.
However, as an Approved Training Organisation (ATO), providing quality courses
are just one of the many responsibilities you are expected to do. It is really a case of
simple logic. To provide quality output i.e. quality courses, you need to have quality
input.
What is quality input you might ask?
To answer the question, we strongly recommend you to explore this guide. We
assure you that it will be a meaningful learning journey, and you will be able to take
away more than you can imagine.
The bottom-line is that this guide serves more than just preparing your organisation
for the Continuous Improvement Review (CIR). It aims to guide your organisation in
establishing a strong quality Continuous Education and Training (CET) infrastructure
system within your organisation.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 3
Part I
Overview of
Quality Assurance Framework
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 4
In this part …
Part 1 deals with the WSQ quality assurance framework and the components
existing within the framework. On the application side, this part introduces the
actions required by any organisation embarking on the journey of becoming a WSQ
Approved Training Organisation (ATO). It is a brief introduction of the different
processes involved. The objective of Part 1 is to allow you to understand the journey
you have taken to becoming an ATO and examine the processes that you would
have gone through before the CIR.
Part I
Overview of
Quality Assurance Framework
In This Part
 Introduction to the Quality Assurance Framework – What it means to you
 The Process within the Framework
The Concept of WSQ That You Just Have
to Know
If you are a WSQ ATO, the term WSQ would not be unfamiliar to you. From the
moment you have chosen to be a WSQ ATO, you should have a basic
understanding of WSQ. However, if it happens to slip your mind at the point you are
reading this guide, we are providing you with a quick note to refresh your memory.
The Singapore Workforce Skills Qualifications (WSQ) is a robust and integrated
continuing education and training system. Designed to build industry competencies,
it is founded on international best practices and validated by industries and
employers. WSQ caters to adult workers who have widely diverse training needs; it
offers a wide range of certification and qualification ranging from Certificate to
Graduate Diploma.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 5
Quality assurance provides the backbone for the WSQ. A strong quality assurance
framework is important to build up public confidence in the WSQ.
So as a provider under the WSQ framework, you are recognised by the public to be
providing courses that are based on a robust quality CET system.
It is thus a joint responsibility between yourself and SSG to continuously contribute
and improve on the robustness and quality of the CET system.
Mechanism to contribute and improve
WSQ quality
SSG has developed the quality assurance framework as a mechanism to uphold the
quality of WSQ. In addition, this framework serves to continuously improve the
quality of WSQ.
The quality assurance framework is made up of two processes:
i)
Pre-delivery Approval – There are 2 stages of accreditation under the
pre-delivery approval, 1) the organisation approval accreditation, and 2)
the course accreditation. The objectives of both accreditations are
similar; that is to ensure that organisation has the capability to delivery
courses that satisfy the competency requirements of the WSQ and is
accessible to workers through appropriate delivery modes and
resources. The process also ensures that the course will be delivered
by a quality training organisation using qualified adult educators. Upon
approval, the following status will be conferred:(a)
An organisation approved to deliver stipulated training
and/or assessment services under the WSQ is known as an
Approved Training Organisation (ATO);
Henceforth the term Approved Training Organisations or ATOs will
be used to refer to WSQ accredited organisations.
At this stage, we would assume that you have gone through the above mentioned
process and the 2 stages within the process. If you have not, we recommend you to
visit this link http://www.ssg-wsg.gov.sg/content so as to better understand the
details behind the accreditation.
However, do not stop your reading of this guide. It is still relevant for you to
understand the types of mechanism SSG is adopting to ensure the quality of the
WSQ frameworks.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 6
Moving on, the second process under the quality framework is none other than the
theme of this guide – Continuous Improvement Review (CIR).
ii)
Continuous Improvement Review (CIR) – Upon approval (organisation
and course), every ATO is to maintain and upkeep the standard of
delivery in training and assessment. In addition, every ATO is to pursue
continuous improvement to uplift their quality of design and delivery.
SSG will periodically validate the internal quality assurance system and
capability of each ATO under CIR.
It is important to note you are not taking this journey alone. Throughout the journey,
SSG will assist ATO to develop their capabilities in designing and delivering of
quality training and assessment under WSQ.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 7
Part II
Continuous Improvement Review (CIR)
– Essential Elements of Review
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 8
In this part …
Here is where you learn about the essential criteria of review under the Continuous
Improvement Review (CIR) framework. We will begin by explain the purpose of the
CIR. From there, we will look in the areas which we will review during the CIR. Along
the way, you will find out about the specific components that we are evaluating under
each of the essential criteria.
Part II
Continuous Improvement Review (CIR)
– Essential Elements of Review
In This Part
 The Purpose of CIR
 The Essential Elements of Review
Continuous Improvement Review
Continuous Improvement Review is one of the extremely important processes in
the quality assurance framework that SSG puts in place to maintain and improve the
quality of WSQ. CIR enables SSG to discern the gaps in quality pertaining to WSQ.
As an ATO, you could use CIR as a quality check tool to improve your internal CET
structure. It is thus valuable to understand the criteria in the CIR system so as to
maximise the usage of it to improve your organisation’s capability and quality as an
ATO.
Why CIR?
The main objectives of CIR are:
i) Inculcate the practice of seeking continuous improvement in the delivery of
training and assessment;
ii) Identify best practices and role models amongst the ATOs; and
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 9
iii) Nurture ATOs to be SSG’s partners in providing quality continuing education
and training.
Therefore, CIR is really a process for both SSG and the ATO to critically reflect on
the practices adopted by the ATO. Such reflections provide opportunities for the ATO
to refine its CET system, and thus contributing to the overall improvement of WSQ.
Essential Criteria of Review
All ATOs are subjected to audit on the following essential criteria for CIR:
1. Course Quality Assurance System
i) System to ensure courseware and assessment are aligned to WSQ,
pedagogically sound and contextualised
ii) System to manage profiling of learners to ensure that the course(s) is fit
for purpose
iii) System to manage training activities
iv) System to manage assessment activities
2. Plan-Do-Check-Act (PDCA) System
i) System to ensure PDCA is implemented regularly for the
comprehensiveness of training, assessment and management system
3. Outcome Evaluation System
i) System to monitor and analyse L1 outcomes
ii) System to monitor and analyse L2 outcomes
iii) System to monitor and analyse L3 outcomes
iv) System to monitor and analyse L4 outcomes
4. Adult Educator Management System
i) HR system to ensure most qualified adult educators are recruited and
inducted
ii) HR system to monitor, appraise and manage performance of adult
educators
iii) HR system to train and develop adult educators
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 10
5. WSQ Related Administrative System
i) Learner support system
ii) System to ensure accurate and timely submission of assessment
results
iii) Prior to 3 November 2014, WSQ certificate printing and distribution
system. Post 3 November 2014, to ensure availability of WSQ e-Cert
and its distribution (if applicable)
iv) Maintain up to date profile in Skillsconnect
6. Viability of Organisation
i) Governance and management of organisation
ii) Financial health of organisation
7. Free from Breach of Terms and Conditions
i) Records of organisation free from breach of SSG terms and conditions.
System-Evidence-Outcome Approach
SSG is adopting the system-evidence-outcome (SEO) approach in the evaluation of
the CIR criteria. The focus on the SEO approach is because there is strong
correlation between sound systems and the quality of outcomes. It has to be reiterated here that the continuation of your approval status/es is/are subjected to the
outcome of SSG’s scheduled CIR audit which would be graded, based on the
scoring matrices for S-E-O.
Definition of S-E-O
1. Systems: a set of interacting or interdependent entities forming an integrated
whole, giving a macro perspective. Components within the system generally have
interconnectivity i.e. functional and structural relationships.
2. Evidences: a thing or things helpful in forming a conclusion or judgment: for
instance, the broken window was evidence that a burglary had taken place.
3. Outcomes: refers to the way something turns out; result; consequence.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 11
To ensure that you are on the same page as us, we have provided an example of SE-O institute for courseware development. (Please see pg. 13 for the given example)
You have to note that in the given example, the system is represented by a process
flow chart. You can represent your system in many forms e.g. a standard operating
manual, a policy guide etc.
The key in developing a comprehensive system is how concise and systematic the
information is presented. To achieve this, you first have to set your organisation
SMART (Specific, Measurable, Agreed upon, Realistic and Time-based) targets
when drafting any system at the outset.
Succinctly, each system devised by you should address the following 5 questions:
i)
When
ii)
Who
iii)
What
iv)
Why
v)
How would each process within the system
would the system be devised and implemented; i.e., effective
start date of system, frequency of system implementation.
would be involved in the system; i.e., the personnel who is
directly involved in either the overseeing or implementing of the system
approach/tools would the system used; i.e., the specific
approach/tools that the system would be using in its course of
implementation.
did ATO have this system; i.e., the rationale/objective of the
system.
be evaluated and follow
up on; i.e., any necessary follow up actions/evaluation are to be
addressed accordingly in the system.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 12
Example of S-E-O
System: Process flow for Courseware Development
Courseware
Development Team *
Industry Information, Market
Research, Course Evaluation
Forms, Companies’ Feedback
Conduct research and survey the market and
customers’ needs
Perform thorough study of the course objectives and assess
capabilities such as trainer’s skills, facilities, equipment
(Refer Note 1)
Costing
Perform simple costing to evaluate if the course is
profitable
Profitable
No
Case Closed
Yes
Courseware Developer
(Refer Note 2)
Develop the Course Programme Design and Overview,
Trainers’ Guide, Lesson Plan and Course Materials as
per WSQ standards (1st draft to be submitted within 1
month to the Courseware Development Team)
WSQ Programme Design and
Overview, Trainers’ Guide,
Lesson Plan,
Course Materials
Courseware
Development Team
Submit to Courseware Development Team for
approval
No
Approved by Team
* Team consists of Principal, Training
Manager and Trainers
Course Developer to rework
based on Team’s directions
(See Note 3)?
Yes
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 13
Continue on pg 2 of 2
A
A
Training Manager
Brief the appropriate trainers on the training concept
before trainers proceed to conduct the new course.
Proceed to conduct the course
Trainers
Course
Evaluation
Form
(Trainee) – Form 1
Course
Evaluation
Form
(Trainer) – Form 2
Post-Course Profiling – Form 3
Perform design validation by analysing course
evaluation forms / interviewing trainees and conducting
post-course profiling for 3 sessions to gather
participants’ profiles/feedback on the new course
Changes in course
design?
Yes
Follow up by Courseware Developer to
amend and update the relevant
documents
and
affected
staff
accordingly
No
Proceed with next course
Note 1
For any deficiencies of trainer’s
skills, suitable trainers shall be
sent for training or training could
be outsourced to qualified parttime trainers.
Training facilities shall be properly
planned.
Any new training
equipment/products required can
be bought, pending the costing
calculation.
Note 2
New course developers will have
to attend pre-development briefing
conducted
by
the
Course
Development Team. The minutes
of the briefing and the attendance
must be recorded and compiled for
future reference.
Note 3
For WSQ programmes, any new
courses or revisions done to
training
materials
must
be
submitted to SSG for approval
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 14
It is always useful to enhance each flowchart and all the supporting form/s with
document names, document numbers, effective dates and version control tracking
mechanisms.
As the scoring for the CIR criteria is based on 3 components – system, evidence and
outcome, having a comprehensive flowchart means that the ATO would score for
system.
The Evidence in this specific example would be the followings:
- Industry Information, Market Research, Course Evaluation Forms, Companies’
Feedback, etc.
- Course Evaluation Form (Trainee) – Form 1, Course Evaluation Form (Trainer) –
Form 2, Post-Course Profiling – Form 3, etc.
The Outcomes in this specific example would be the followings:
- Good L1 scores as trainees find the course relevant;
- Repeat corporate clients being served (as the organizations know what they are
getting if they are requesting for the same training the second time).
Please note that the under this specific example, the ATO has to devise Forms 1, 2
and 3 as well as document the Industry Information, Market Research, Companies’
Feedback, etc as evidence of deployment of the system so as to score points under
the evidence matrix.
Likewise, outcome/s would have to be presented as well-collated L1 scores by
course or trainer on a periodic (e.g., quarterly, half-yearly, etc) basis. While the list of
clients served, especially established corporate clients or walk-ins through words of
mouth, etc if available would be useful in securing points in the outcome matrix.
In addition, it is important to note that there should be
clear and concise documentation for all the criteria under
the S-E-O. Any S-E-O without documentation will affect the
CIR evaluation outcome.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 15
You have come to the end of Part II of this guide. At this point, you should have a
general idea of the seven essential criteria of review in the CIR:
i) Course Quality Assurance System
ii) Plan-Do-Check-Act (PDCA) System
iii) Outcome Evaluation System
iv) Adult Educator Management System
v) WSQ Related Administrative System
vi) Viability of Organisation
vii) Free from Breach of Terms and Conditions
You should also have a general understanding of the S-E-0 approach adopted by
SSG in the evaluation of the essential criteria. However, if you still have doubts in
your mind, you can always email or drop a call to the officer-in-charge of your
organisation to seek their advice.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 16
Part III
Roles and Responsibilities
of ATO
In this part …
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 17
In the third part of the guide, we will be talking about the preparation requires by an
ATO when being notified of a CIR by SSG. We will talk about the things that you are
required to prepare after you have received the CIR notification from SSG, and the
things that you are required to do consistently as an ATO.
Part III
Roles and Responsibilities
of an ATO
In This Part





The Roles and Responsibilities of an ATO
Scope of CIR
ATO Continuous Improvement Efforts
Outcomes of CIR
Fees and Feedback
The Roles and Responsibilities
Together We Play a Part
–
It is important to note that you do not journey alone when you are an ATO. SSG will
provide the necessary assistance to aid you along the way. Thus, before the CIR,
during the CIR, and even after the CIR, SSG seeks to work closely with you to
improve your overall quality as an ATO. Some of the aids that SSG would provide to
you in building your quality are:
i)
ii)
iii)
iv)
v)
Validate the internal quality assurance of ATOs
Build up the competencies of the ATOs through feedback
Issue CIR audit report for CIR audit conducted
Highlight ATO’s strengths and areas for improvement
Facilitate ATO’s participation in capability development initiatives and
workshops
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 18
However, as an ATO, you need to have the correct attitude in building up your own
quality. As the saying goes, “Your Attitude Determines Your Altitude”, you would
have to put in effort to improve your own ability and quality as an ATO. Some of the
things you are recommended to do constantly are:
i) Practice internal quality assurance
ii) Ensure Plan-Do-Check-Action (PDCA) system is in place and is
diligently making continuous improvements to meet the required quality
or for better quality, especially in areas where required by SSG from
previous CIR audits conducted
iii) Ensure quality of accredited courseware and adult educators are
consistent with actual training and/or assessment
iv) Ensure evidences of all systems implementation, and dates of training
and assessment are available for audit when requested
You have learned about the criteria of review and the S-E-O that SSG used in CIR
evaluation. It is time to look into the activities that you are required to do on the day
of the CIR.
Scope of CIR
In part II of the guide, we have gone through the essential criteria of review, and the
expected responsibilities of an ATO. We will now move on to look into the specific
things you will be required to do when you received the CIR notification from SSG
and on the actual day of the CIR.
When you received the CIR notification email from SSG (one month before the CIR
date), you are required to response to the notification email within 5 workings days
providing the following:
i) A series of available dates and time for the CIR
ii) A training and/or assessment schedules
iii) A self assessment checklist (see Annex B)*
iv) A 2 year plan (see Annex C) (Applicable to only ATO offering public
courses)*
*This is applicable only for public ATOs and is to be provided within 2 weeks
from the date of the notification
Upon receiving your response with the attached documents, the officer-in-charge of
your CIR will then get in touch with you to finalise on the CIR date and time based on
the series of dates and time provided by you. The officer may also request more
information based on the 2 attached documents that you are required to submit.
Once the date and time of the CIR is finalised, you can look forward to the actual
CIR date and prepare the necessary presentation and materials for the CIR.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 19
According to the stated essential criteria in part II, you will be required to make a
presentation based on each criteria. The presentation is recommended to be kept
under 15mins (see Annex D for the breakdown of the presentation process). You
need to have the supporting evidences ready for verification during the CIR audit.
The review covers the following:
i) Showcase of all the implemented systems based on the essential
criteria of review e.g. relevant SOP manual
ii) Specific evidence of each implemented system, e.g. training
observation checklist with requirements clearly communicated to all
adult educators, or learners’ profiling checklists in determining
contextualisation of courseware, etc.
iii) Outcome of each implemented system, e.g. systematic documentation
of all CVs and qualifications of qualified adult educators (against
declaration submitted at accreditation), or assessment results with
analysis to reflect L1 and L2 outcomes, etc.
It is important that you prepare the relevant evidences and supporting documents in
advance before your scheduled CIR audit by SSG. The reason of the preparation is
really quite simple - you do not want to be running around getting the documents and
indirectly reflecting the possibility of an inadequacy in your administrative system.
Your Continuous Improvement Efforts
Specifically, ATO is required to practice continuous improvement in its ongoing
delivery of training and assessment under WSQ. Evidence of these efforts should be
available or demonstrated in the form of PDCA through any of the following:
i) Quality of design and delivery of courseware, e.g. regular reviews to
ensure currency, for continuous improvement, contextualisation to be
relevant to a particular industry or target audience etc;
ii) Quality of trainers, assessors and curriculum developers, e.g. training
and development plan and implementation, performance management
system etc.;
iii) Assessment Management System, e.g. regular reviews and
improvements made to internal quality assurance system particularly
for the management and conduct of assessments, ranging from how
assessments are designed and conducted to ensure the four principles
of assessments – reliability, fairness, validity and flexibility as well as
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 20
positive and negative features experienced in conducting the
assessments, and refinements made, assessment validation etc.;
iv) Administrative System (supporting training and assessments for
accurate and timely records submission to SSG, ensuring availability of
WSQ e-Cert to trainees);
v) Tracking of outcomes, e.g. end-of-course evaluation, attrition rate,
passing rate, surveys conducted etc.
You can refer to Annex A for the step-by-step of CIR for ATO to have an overview of
the CIR processes.
Outcomes of CIR
Possible Capability Development Initiatives
From the CIR pilots conducted, SSG has collated some common areas of
improvement arising from findings of the ATOs piloted. These are taken into
consideration for capability development initiatives that are being planned. Some of
the initiatives in the pipeline include the conduct of regular seminars to equip ATOs
with the knowledge to professionalise internal practices, which can be in the form of
monthly briefings, one-to-one consultations, or sharing of best practices and
outcomes of CIR in groups. ATOs can also look forward to the development of a
guidebook to assist them to set up a training system and implement continuous
improvement practices.
The CIR will further provide information and evidence that will enable ATOs and
SSG to identify areas of strength as well as areas in need of focused attention for
planned improvement in the short, medium and long term. With these needs
identified, SSG can then design suitable interventions to assist the ATOs.
Continuation of Approval Statuses
In addition to capability development, the CIR replaces the need for application by
ATO for renewal or extension of WSQ statuses, i.e. the CIR determines if the
approval status of the ATO should be continued.
The continuation of approval statuses underpinned by the results of CIR is guided by
the CIR Outcome Measures. Other considerations determining the continuation of
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 21
approval statuses besides the periodic CIR are also guided by the CIR Outcome
Measures.
In the event that the ATO's WSQ status is not renewed arising from the results of
CIR, the ATO can appeal to SSG on the outcome of the CIR. The ATO need to
submit an appeal letter to SSG, Deputy Director (Quality Management Division)
stating the background and nature of the appeal together with the reasons and
supporting evidence as to why the original outcome decision should be reversed or
with a non-refundable fee of $374.50 (inclusive of GST) within 2 weeks from the
revoke/termination notification letter issued by SSG.
The complete CIR Outcome Measures is represented in Annex E.
Fees
The CIR fees payable are as follows.
S/No. Type of Application
Period
Charges (Inclusive of GST)
1.
Effective from 1 Oct 2014
$1,605.00
*
Continuous
Improvement Review
(CIR)*
Note: With reference from CIR first activity date
Feedback
SSG welcomes any feedback to seek continuous improvement and fine tune the CIR
system. Please send your comments to the Feedback Portal https://portal.ssgwsg.gov.sg or [email protected]
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 22
Schedule 1: Qualification Requirements for
WSQ Adult Educators as at 1st October 2015
WSQ AE Role
WSQ Trainer and/or
Assessor
WSQ Curriculum Developer
WSQ Master Trainer 5
(applicable for In-house
WSQ ATOs only)
Public WSQ ATOs 1
At least 80% 3 with ACTA
qualification or equivalent
In-House WSQ ATOs 2
At least 80% 3 with WTP or
equivalent
100% with DACE
qualification or equivalent
At least one must attain
ACTA 4 qualification or
equivalent
Not applicable
At least one WSQ master
trainer with ACTA
qualification or equivalent
1
Public WSQ Approved Training Organisations (ATOs) are training providers that conduct WSQ
training and/or assessment for the public.
2
In-house WSQ ATOs are companies/organisations that conduct WSQ training and/or
assessment internally within their organisation for their own staff only.
3
Proportion of total unique AEs deployed for each course.
4
Individuals who are performing WSQ Curriculum Developer responsibilities for in-house WSQ
ATOs and who enrol for the 5th version of ACTA (ACTA v5) are advised to further their learning
by completing the 2 curriculum development modules (Develop Curriculum and Instruction for
Adult Learning Programme + Develop and Review Competency-based Assessment) under the
Diploma in Adult and Continuing Education Version 2 (DACE v2).
5
The WSQ Master Trainer and WSQ Curriculum Developer for the in-house WSQ ATO can be
the same person.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 23
Annex A: Step-by-Step Process of CIR for ATO
Step
Action Needed
Step 1:
Remarks
Documentary Evidence
ATOs are scheduled for CIR
and notified by SSG through
email 1 month before the CIR
Receipt
of
CIR Notice
Upon receipt of SSG’s CIR
notification, ATO is to respond
to SSG with the completed selfassessment checklist (Annex B)
and a 2 year plan (Annex C)
(Applicable to only ATO offering
public courses)
ATO is also to indicate a series
of available dates and time in
the response to the CIR
notification
For areas which ATO does not
have a system or not
implemented yet, ATO is to
submit an action plan providing
details on:
SSG Officer will evaluate the
ATO’s readiness for the CIR
According to CIR
schedule
Training and assessment
schedules
SSG Officer advises on
allocated documentation
submission schedule and/or
range of duration for possible
site audit and possible
training/assessment
observation sessions.
SSG Officer will finalise the
CIR date and time with the
ATO based on the provided
series of dates and time from
the ATO
a) Gaps
b) Follow-up actions to
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Timeline
Page 24
ATO is to respond
to SSG within 5
working days upon
receipt of CIR
notification
Step
Action Needed
Remarks
Documentary Evidence
address gaps
c) Time needed to
implement changes.
ATO
is
to
provide
training/assessment schedules
and anticipate a period (and not
a stipulated date and time)
when the site visit would
possibly be conducted
Step 2:
Preparing for
CIR Audit
ATO to prepare a 15-min
presentation outline and
coverage advised in Annex D
ATO to get ready required
evidences and supporting
documents and when informed
about the site audit, ATO to
avail facilities for site audit.
SSG-QMD will go down to
ATO’s training site to conduct
CIR site audit.
Standard CIR checks to
include :
1. Course Quality
Assurance System
2. Plan-Do-Check-Act
(PDCA) System
3. Outcome Evaluation
System
4. Adult Educator
Management System
5. WSQ Administrative
Related System
1. SOP manual/flow chart
etc, of the various
systems
2. CVs and evidence
showing qualifications
of trainers, assessors
and curriculum
developers declared
3. Process of review by
ATO to ensure
currency of
courseware, relevance
to industry, target
audience and
contextualisation
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 25
Timeline
Step
Action Needed
Remarks
Documentary Evidence
6. Viability of
Organisation
7. Free from Breach of
Terms and Conditions
8. Evidence-ofImplementation of the
Various System
9. Outcome(s) of the
Various System
4. Administrative System
– Show evidences of
accurate and timely
assessment
submission and
ensuring the availability
of WSQ e-Cert to the
learners
5. Outcomes tracking,
e.g. attrition rate,
passing rate, surveys
conducted etc
6. All evidences of
implementation of the
various systems
7. Documentation of
outcomes of all the
various systems
Supplementary evidence
sources or supporting
documents which may be
requested for the CIR
audit are:
1. Course files (course
materials submitted at
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 26
Timeline
Step
Action Needed
Remarks
Documentary Evidence
Timeline
point of course
application, includes
Assessment Plan,
Slides, Participants’
Handouts and Trainer
Guide)
2. Course attendance
records
3. Training and
assessment records
Step 3:
During site
audit
15 minutes
ATO
presentation
Site audit and MR interview
by The management
representative should be
present during the site audit
Conduct audit of training
premises (if applicable)
Step 4 :
Training/
assessment
Observation
ATO’s
slides
and The site audit would
handouts for presentation. normally not take
more than half a day
All SOPs related to the
implemented systems
ATO to avail
training/assessment sessions
for SSG Officer’s observation
Step 5 :
Go through areas (strengths
and weaknesses) highlighted by
After the audit SSG Officer
SSG Officer will
brief provider and
discuss follow-up
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 27
Step
Action Needed
Remarks
Documentary Evidence
Timeline
actions if necessary
SSG Officer to
prepare CIR report
Step 6 :
Study findings in the report.
Receive CIR
audit report
and outcome
letter
Take note of rectifications and
timeline in CIR outcome letter
and take corrective actions
CIR outcome
measures (shown in
Annex E) will apply
– where ATO has
cleared, taken
action or failed to
take corrective
actions within
stipulated time/does
not respond
* Timeline for CIR cases is 50 working days from the date of the last CIR activity.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 28
Annex B: Self-assessment Checklist
Proposed
CIR Date:
Name of ATO:
Name of Management Representative (MR):
Contact Details of MR:
(Email)
(Tel No)
(O)
(Hp)
Section A
Are you able to show evidence for all the items in the checklist? If your answers are all ‘Evident’ in Section A, your organisation is likely to be ready for a CIR
audit. Please note that the list of evidence suggested for each area is NOT EXHAUSTIVE.
Area of Review 1 – Course Quality Assurance System
Criteria
Interpretation of Criteria
Implemented System
Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
1.1) System to ensure
courseware and assessment are

System in place to manage developers
-E
-E
-E
- NE
- NE
- NE
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 29
aligned to WSQ, pedagogically
sound and contextualised


1.2) System to manage profiling
of learners to ensure that the
course(s) is fit for purpose


1.3) System to manage training
activities

1.4) System to manage
assessment activities

Process in place to conduct internal
evaluation and validation of
courseware to ensure alignment to
WSQ standards
-E
-E
-E
- NE
- NE
- NE
System in place to facilitate
contextualisation of courseware to
ensure its currency and relevancy, etc
-E
-E
-E
- NE
- NE
- NE
Process in place to ensure systematic
screening/profiling of learners, e.g. did
the learner meet the prerequisite of
the course:
1. literacy level
2. numeracy level
3. required assumed
knowledge/skills, etc
-E
-E
-E
- NE
- NE
- NE
Process in place ensure to only learners
with the appropriate aptitude/attitude
are enrolled into the course
-E
-E
-E
- NE
- NE
- NE
System in place to continuously
monitor and manage the training
processes to ensure:
1. it is in accordance to industry
practices i.e. ensuring quality and
currency of training
-E
-E
-E
- NE
- NE
- NE
System in place to continuously
monitor and validate the assessment
processes to ensure it is in accordance
to principles of assessment i.e., valid,
-E
-E
-E
- NE
- NE
- NE
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 30
reliable, fair, and flexible, and rules of
evidence
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 31
Area of Review 2 - Plan-Do-Check-Act (PDCA) System
Criteria
Interpretation of Criteria
Implemented System
Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
2.1) System to ensure PDCA is
implemented regularly for the
comprehensiveness of training,
assessment and management
system

Demonstration of establishment in
PDCA. This may include:
1. Organization management
2. Training management
3. Assessment management
-E
-E
-E
- NE
- NE
- NE
Area of Review 3 – Outcome Evaluation System
Criteria
Interpretation of Criteria
Implemented System
Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
3.1) System to monitor and
analyse L1 outcomes




Clear articulation of indicators, this is to
include trainee satisfaction rate.
Trainee satisfaction rate minimally on
these 4 areas:
1. Training Programme
2. Course Material
3. Trainers
4. Facilities and Logistical support
Analysis of obtained data
Follow up action(s) on analyzed data
-E
-E
-E
- NE
- NE
- NE
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 32
3.2) System to monitor and
analyse L2 outcomes
3.3) System to monitor and
analyse L3 outcomes
3.4) System to monitor and
analyse L4 outcomes

Clear articulation of indicators, this is to
include
1. Passing rate
2. Attrition rate
Analysis of obtained data
Follow up action(s) on analyzed data
Clear articulation of indicators on
impact of transfer of learning, this is to
include:
1. Extent of candidate’s behaviour
and/ or capability improvement
as indicated by candidate
2. Extent of candidate’s behaviour
and/or capability improvement
as indicated by employer
Analysis of obtained data

Follow up action(s)on analyzed data

Clear articulation of indicators on
impact of training on business, which
may include:
1. volumes,
2. values,
3. percentages,
4. timescales,
5. return on investment,
6. other quantifiable aspects of
organisational performance, for
instance; numbers of complaints,
staff turnover, wastage, noncompliance, quality ratings,
accident ratings, zero-defect
ratings, growth, retention, etc.




-E
-E
-E
- NE
- NE
- NE
-E
-E
-E
- NE
- NE
- NE
-E
-E
-E
- NE
- NE
- NE
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 33
Analysis of obtained data
Follow up action(s)on analyzed data
Area of Review 4 – Adult Educator Management System
Criteria
Interpretation of Criteria
Implemented System
Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
4.1) HR system to ensure most
qualified adult educators are
recruited and inducted
4.2) HR system to monitor,
appraise and manage
performance of adult educators
4.3) HR system to train and
develop adult educators



System in place to recruit and induct
qualified adult educators
-E
-E
-E
- NE
- NE
- NE
System in place to manage the
performance of adult educators
-E
-E
-E
- NE
- NE
- NE
System in place to facilitate adult
educators ongoing professional
development
-E
-E
-E
- NE
- NE
- NE
Area of Review 5 – WSQ Related Administrative System
Criteria
Interpretation of Criteria
Implemented System
Evidence of System
Implementation
Evident (E) / Not Evident (NE)
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 34
Outcome of System
Implementation
5.1) Learner Support System in
place

5.2) System to ensure accurate
and timely submission of
assessment results


Learner support system that covers:
1. Course advisory
2. Progression pathway
3. Course schedule
4. Fees and refund policy
5. Retest/Appeal procedures
6. Career advisory (if applicable)
7. Placement support (If applicable)
Assessment data are submitted in
accordance to prescribed timeline
within 2 weeks from the date of the
assessment/within 1 month from the
date of assessment*, unless otherwise
approved
Submission of results must be accurate
*applicable only to In-house ATO
5.3)
System to ensure the availability
of WSQ e-Cert to the learners
and is distributed (if applicable)

WSQ e-Cert is available according to
agreed service timeline:
1. e-Cert must be available to trainees
within 3 weeks from the date of
the assessment/within 5 weeks
from the date of assessment*
2. Upon request from trainee (after
assessment result has been
uploaded), WSQ e-Cert will be
printed and distributed.
3. Follow up with trainee upon receipt
of
email
notification
on
unsuccessful attempt to reach
trainee.
-E
-E
-E
- NE
- NE
- NE
-E
-E
-E
- NE
- NE
- NE
- NA
- NA
- NA
-E
- NE
- NA
-E
- NE
- NA
-E
- NE
- NA
-E
- NE
- NA
-E
- NE
- NA
-E
- NE
- NA
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 35
4. Full
qualifications
(including
transcripts) are printed and
distributed within 3 months from
date of assessment
*applicable only to In-house ATO
5.4) Maintain up to date profile
in Skillsconnect

Profile information in Skillsconnect
must maintain the most up-to-date
information. This include:
1. Organization profile
2. Course profile
3. Adult educators profile
-E
-E
-E
- NE
- NE
- NE
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 36
Section B
If answer to any question below is ‘Yes’, please elaborate under Remarks column.
S/N
Questions
1
Are there changes in the management level personnel for the past one year?
2
Is there any major instability in the organisation’s financial health for the past
one year?
3
Has your organisation violated any WSQ terms and conditions?
Yes
No
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 37
Remarks
Annex C: 2 Year Plan
(Applicable to only ATO offering public courses)
ATO is required to submit proposal on the training and assessment delivery on a 2-year
timeline, this is to ensure that the ATO have a clear mission and vision on their involvement
in WSQ. The proposal must indicate the following:
i) Identified one specific WSQ framework, e.g., Security WSQ;
ii) Intended scope of delivery within the framework, e.g. bundling of SOAs for
the 2 licensing modules at the Certificate in Security Operations (CSO) level,
full qualification of the CSO, etc;
iii) Target market and proposed clientele mixed
iv) Existing clientele and new clientele over the past year who dropped out, eg.,
trainees who had attained at least 3 GCE ‘O’ Levels including English
Language or its equivalent; and trainees with language difficulties and
cultural adjustment issues (ascertained through screening);
v) Proposed marketing and outreach strategies, eg. advertising in print media
and newspapers, course previews and career fairs at recruitment centres, etc;
vi) Intended annual headcount to be trained, eg. at least 25 trainees for the 2
licensing modules at the CSO level, (or 50 Statements of Attainments [SOAs])
per month; and
vii) Proposed innovative learning and assessment strategies
viii) Risk assessments by the ATO. These include:



Financial Risk
Operations Risk
Strategic Risk
To facilitate the submission of the proposal by the ATO, ATO can use the provided template
in the following page. ATO is advised to provide between 1000 and 1250 words for the
proposal.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 38
Template for 2-year plan
Applicants are advised to provide between 1000 and 1250 words for the proposal.
i. Specific WSQ framework
Identified existing WSQ framework, and/or new WSQ framework(s) with intention of embarking
into.
ii. Scope of delivery
Intended scope of delivery within the framework (existing and/or new). Indicate the WSQ modules
e.g. interact with guest, deliver service excellence, workplace safety and health.
iii. Target market and proposed new clientele-mix


The trainees and the intended proportion Eg., training only Singaporean and local PRs.
Target Corporate Clients (indicate clienteles’ names).
iv. Existing clientele


The existing ratio of Singaporeans/PRs trainees in relation to foreign trainees.
Existing Corporate Clients.
v. Proposed outreach strategies
Proposed marketing and outreach strategies, eg. advertising in the print media and newspapers,
course previews and career fairs, etc.
vi. Estimated annual headcount to be trained and projected number of SOAs
Eg. at least 25 trainees for the 2 licensing modules at the Customer Service Officer level, (or 40
Statements of Attainments [SOAs]) per month.
vii. Proposed learning and assessment strategies
Eg. Small group tutorials cum buddy scheme for learners with low learning retention ability and
compact disks (CD) would be deployed for intensive trainee revisions on the practical
performances before assessments at the centralised assessment centre.
viii. Risk assessments by the applicant
Eg. Security ATOs would face a declining no. of trainees should their clients convert to in-house
training or install the following IT infrastructure:
- Bio-metric access and manpower scheduling system;
- Remote CCTV surveillance system; and
- Visitor management system.
The ATO would hence face a challenge to meet the initial proposed training target/s. ATO to
suggest and elaborate how to mitigate this risk.
ix. Remark/Other Comments
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 39
Annex D: CIR Presentation by ATO
The Continuous Improvement Review (CIR) will commence with a
presentation by the ATO outlined as follows. The presentation can be in the
form of a powerpoint presentation or briefing notes. The Management
Representative (MR) must be present to conduct the presentation or provide
clarifications. The presentation or brief notes will be part of the documentary
evidences of the CIR. Evidences can include the following:
1.
Course Quality Assurance
i) Course files (course materials submitted at point of course application,
includes Assessment Plan, Slides, Participants’ Handouts and Trainer
Guide)
ii) Summary page of courses with updated changes and what
were the specific affected materials.
iii) E.g. 1 “Interact with Guests” - Lesson plan has been
contextualised.
iv) E.g.2 “Prepare for Service” – Assessment plan has been
amended.
v) Tracking mechanism used by ATO
2.
Plan-Do-Check-Act (PDCA)
i) Demonstration of establishment in PDCA.
3.
Outcome Evaluation
i) Update on actions taken based on organisation audit or previous CIR
ii) Continuous review and feedback mechanisms in place and
implementation
iii) System to monitor and analyse L1 to L4 outcomes with detailed
analyzed of collected data
4.
Adult Educator Management
i) Performance management of trainers/assessors
ii) Training and development of trainers/assessors
iii) Description of supervision mechanism for trainers
and
assessors under supervision
5.
WSQ Related Administrative System
i) Learner support system that covers course advisory, progression
pathway, course schedule, fees and refund policy, retest/appeal
procedures, career advisory, placement support (if applicable)
ii) System in place to ensure accurate and timely submission of
assessment results
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 40
iii) System in place to ensure availability of WSQ e-Cert and distribution (if
applicable).
iv) Course training and assessment records
6.
Viability of Organisation
i) Update on governance and management of ATO
ii) ATO’s financial health for the past one year
7.
Free from Breach of Terms and Conditions
i) Records of ATO for complaints or compliments received by ATO or on
ATO
*Presentation should be kept to 15 mins.
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 41
Annex E: CIR Outcome Measures
Result of CIR
Outcome
Not Addressed Within
Advised Timeline
A) Cleared CIR with no
action required
Ad-hoc CIR
(outside of CIR
Not Addressed After
Reminder
Continuation of approval status with new validity
period, with or without new applications
Not Addressed Within
Advised Timeline
Not Addressed After
Reminder
Reminder letter with
rectification within 5 working
days
Revoke of WSQ ATO
status.
schedule)
B) Triggered by Other
Factors
i) Complaints with
action required
ii) Misrepresentation
iii) Inaccurate or late
report submission
iv) Breach of SSG
Terms and
Conditions
Issued by: Quality Management Division
SkillsFuture Singapore
Version:
1.16
Effective: 7 October 2016
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 42
Version Control
No
Date
Changes Made
Version No.
1
31 May 07
Fee revised to $315 (pg. 6)
1.2
2
21 Sep 07
Fee revised to $321 (pg. 6)
1.3
3
5 Jan 09
Revision of areas of review
1.4
Added Essential Elements of Review
(pg.12)
Added System-Evidence -Outcome
Approach (pg. 13)
4
1 Aug 09
Added Annex B: Self-assessment
Checklist (pg. 25)
1.5
Removed NSRS
Added Example of S-E-O (pg.13)
Added Annex C: 2 year plan (pg.35)
5
1 July 10
Added CIR Grade (pg.40)
1.6
Revised submission timeline for self
assessment checklist and 2 year plan
(p.19)
Added criterion 1.2 into “Course
Quality assurance system”
Revised Outcome Measures, i.e.
“Revoke of WSQ ATO status” under
column “Not Addressed After
6
1 Sep 10
Reminder” (p.39)
1.7
Revised timeline for issuance of
certificates
7
4 Nov 11
Added Appeal Process (p.21)
1.8
8
13 Sep 12
Revised webpage link (p.6)
1.9
1 Oct 13
Fee revised from 1 October 2013 to
$353.10 and $1,605 on a yearly basis
(page 22)
9
1.10
Based on Internal Quality Assurance
(IQA) report, corrected grammatical
and typo errors:
10
5 Nov 13
Page 9 From “one of the extremely
important process” to “one of the
extremely important processes”
1.11
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 43
Page 9 From “Identify best practices
and role models amongst the ATO” to
“Identify best practices and role
models amongst the ATOs”
Page 10 From “All ATO is subjected
to” to “All ATOs are subjected to”
Page 11 From “free breach of SSG
terms and conditions” to “free from
breach of SSG terms and conditions
Page 18 From “building you quality” to
“building your quality”
Page 19 From “the office-in-charged”
to “officer-in-charge”
Page 23 From “ATO is to responding
to SSG” to “ATO is to respond to
SSG”
Page 25 From “All evidence” to “All
evidences”
Based on Internal Quality Assurance
(IQA) report, the irrelevant examples
are removed:
Page 35 Part iii Removed “training
only Malaysian learners”
Page 35 Part iv and v Removed
“Malaysian”
Page 35 Part vii Removed “Compact
Disks (CD) would be deployed for
intensive trainee revisions on the
practical
performances
before
assessments at the centralized
assessment centre, Polwel Cooperative Society Limited”.
Page 21, under ‘Continuation of
Approval Statuses’, changed from
‘Director (Quality Assurance Division)’
to ‘Deputy Director (Training Partners
Development Division)’
11
14 July 2014
1.12
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 44
12
25 September 2014
Page 39, under ‘Issued by’ changed
from ‘Quality Assurance Division’ to
‘Training Partners Development
Division’
Fee revised to $1,605 with effective
from 1 October 2014 (page 22)
1.13
Added the Post 3 November 2014
requirements for WSQ e-Cert to the
following pages: 11, 20, 25 and 33.
13
30 October 2014
1.14
Update to the Annex E, page 40:
Removed the word ‘grading’ from the
title and removed CIR outcome B.
14
11 December 2015
Inserted Schedule 1 for Qualifications
Requirements
for
WSQ
Adult
Educators (page 24)
1.15
Change the WDA logo to SkillsFuture
Singapore logo. Revise the footer of
the document to SkillsFuture and
adjust the version and effective date
accordingly
Replace “WDA” with “SSG” in the
whole document
15
7 October 2016
Change
TPDD
to
Quality
Management Division (QMD)
1.16
Update the feedback portal address
and SSG-WSG website address
Delete all content on certificate
issuance process prior to 3 November
2014 which is outdated
© SkillsFuture Singapore
SSG-WSQ-CIR-GUIDE Effective date: 7 October 2016 version 1.16
Page 45